When I started covering crime as a reporter for small newspapers in the 1980s, I was assigned to walk to the police department lobby each morning and look through all of the previous day’s police reports, clipped to a board on the counter, containing all the details laid out for anyone to see. We were able to report to the community each day on the major events in town – to explain why people heard sirens, or saw a smoke plume.

By the 1990s, the clipboards were moved out of the lobby, so we asked at the counter to see them. Then we were told we had to review them with the sergeant on duty. Then we were told we couldn’t see them – we had to ask the police what they felt was newsworthy. Then we were told to submit a public records request, and wait for days or weeks – if we got them at all.

For decades, journalists and civic activists have lamented the increasing secrecy of government – the times, they were denied government information, particularly from public records requests. Reports have shown secrecy getting worse at the federal, state and local government levels.

But those were usually anecdotal reports of problems. Now, there is data that brings those refusals into focus and which provides a fuller picture of government agencies hiding their work from the public they ostensibly serve.

Openness benefits people and society

The stakes, and potential ramifications for everyday people, are significant.

Access to government records helps people research their family history, identify quality schools for their children, monitor the cleanliness of their drinking water, background-check their online dates, and hold their local town officials accountable.

And there are clear benefits: Open records are proven to lead to less sex-offender recidivism, fewer food service complaints, increased trust in government institutions and reduced corruption.

Stanford University professor James Hamilton calculated that for every dollar spent by newspapers on public records-based journalism, society realizes benefits worth US$287 in lower taxes and saved lives.

Less transparency year after year

My analysis of government agencies’ compliance with public records laws through 37,000 federal Freedom of Information Act, or FOIA, requests submitted through the nonprofit MuckRock.com shows that a decade ago, if you asked the federal government for a public record, you might get it about half the time – which isn’t great. Today, you might get it about 12% of the time, and the trend is steadily downward.

The trend is similar though less uniform among state and local governments: You might receive what you ask for two-thirds of the time in Idaho or Washington state, but only 10% of the time in Alabama.

Every year in mid-March, since 2005, national Sunshine Week has promoted the right of people to acquire public records and attend public meetings. The Joseph L. Brechner Freedom of Information Project at the University of Florida, where I am the director, has conducted research and education about access to government information for nearly 50 years.

Our research indicates that U.S. government secrecy has never been so prevalent.

Increasing secrecy isn’t tied to any particular president or regime. The administration of President Barack Obama, who declared on his first day of office his intent to be the most transparent president in history, was slower to respond and less likely to release information than George W. Bush’s administration.

President Donald Trump’s administration was more secretive than Obama’s, and transparency continues to slide under the Biden administration.

Data tells a piece of the story

According to annual data collected by the U.S. Department of Justice, federal agencies have become more secretive over the past decade:

• The prevalence of people getting what they asked for through FOIA requests declined from 38% of the time in 2010 to 17% in 2022.

• In 2010, about 13% of the time, federal agencies would reply to FOIA requests by saying they couldn’t find records pertaining to the request. By 2022, the rate of that type of response had increased to 21%, which officials often attributed to outdated record management systems incapable of keeping up with the massive amounts of electronic records, particularly emails.

• Backlogs, where requests languish beyond the 20-day legal requirement for completion, have nearly doubled since 2010, from 12% of total requests to 22%. The average number of days it takes to process simple requests, which require little staff time and a smaller volume of records, has doubled since 2014, from 21 days to 41 days, according to Justice Department reports.

• While some secrecy is necessary to protect national security, the Government Accountability Office reported that the use of FOIA Exemption (b)(3), which allows federal agencies to deny records if another law makes the information secret, has more than doubled during the past decade, even though the number of requests only increased by a third. That includes denying people’s requests about properly withheld intelligence information. But it also includes refusing to release information on topics of great public interest, such as defective consumer products and employment discrimination cases.

Even if agencies grant requests, they present other obstacles. A.Jay Wagner of Marquette University and I surveyed 330 people who requested records in the U.S., finding that high fees to copy documents discourage people, such as journalists, nonprofits and members of the public, from seeking information in the public interest. And some agencies’ public information officers obstruct public access to information. They limit access to the people and documents most important for government transparency and accountability.

Research-based solutions

Just as researchers have identified secrecy spreading through the government, recent studies offer ideas for possible cures.

Independent oversight offices with enforcement power, such as in Connecticut, Ohio, Pennsylvania and more than 80 nations, provide private citizens an alternative to litigation. Instead of having to hire a lawyer to sue the government for what you are entitled to, the independent agencies will review your case, make a determination and force the government to provide you the information.

The federal FOIA Advisory Committee, working since 2014, has provided 52 recommendations for Congress and federal agencies to improve transparency in the United States, crafted from experts and researchers. A subcommittee I co-chair for the current term is close to finishing its assessment of how well the recommendations have been implemented, with results to be released in May 2024. Our preliminary assessment indicates that there is a lot of work left to do, and that Congress and government agencies have ignored many of the recommendations.

David Cuillier has received funding from the John S. and James L. Knight Foundation for research in access to information, and is former president of the nonprofit National Freedom of Information Coalition. He is currently director of the Joseph L. Brechner Freedom of Information Project, a nonpartisan organization at the University of Florida dedicated to research and education in freedom of information since 1977. The Brechner FOI Project coordinates Sunshine Week to educate the public about their right to know.