Public Records & Open-Source Research: Find What's Actually Out There
Section 7 of 14

11 min read Updated

Somewhere in a federal agency right now, a document exists that could change everything — an inspector general report, a contract amendment, an internal email about why a program got quietly defunded. That document is yours to request. The law says so. The catch is that most people have no idea how to ask for it in a way that actually works.

The Freedom of Information Act — almost always abbreviated to FOIA, pronounced "foy-uh" — is a 1966 federal law that gives any person the right to request records from federal executive branch agencies. Not just journalists. Not just lawyers. Any person. The Department of Justice's guide to FOIA[1] makes clear that citizenship isn't even a requirement — foreign nationals can file requests too. That single fact reframes what it means to do public-records research. You are not petitioning the government for a favor. You are exercising a statutory right.

Understanding that distinction takes most of the anxiety out of the process. Here's what makes the difference between a request that gets answered and one that disappears into a backlog for three years.

Start with what FOIA actually covers, because the scope is narrower than people assume. The law applies to the executive branch — the departments, agencies, and commissions that report, ultimately, to the White House. That means the EPA, the FDA, the Department of Defense, the FBI, the SEC, the FTC, and hundreds of other bodies. What it does not cover is Congress, the federal courts, or the President's immediate staff in the White House Office. State and local governments have their own sunshine laws — and those will get their own treatment shortly. But at the federal level, if the agency has an acronym and it answers to the executive, FOIA reaches it.

There's a common misconception worth addressing here: many people assume FOIA is only useful for getting records about other people or organizations. In fact, some of the most powerful FOIA requests are for records about how agencies make decisions — internal memos, policy guidance documents, contracts, and communications that reveal not what the government did, but why. That's often the more interesting layer.

Now for the mechanics. Every covered agency is required to maintain a FOIA office, and most of them have online portals where requests can be submitted directly. The FOIA.gov portal[2] — run by the Department of Justice — lists every federal agency's FOIA contact and links to their individual request portals. This is the right place to start. You pick the agency, you navigate to its specific portal, and you file there. Filing with the wrong agency doesn't kill your request, but it wastes weeks or months before a referral gets sorted out.

The request itself is simpler than people expect. There's no magic legal language required. A FOIA request doesn't need to cite specific statutes, use Latin phrases, or be written by an attorney. What it does need to do is describe the records you want with enough specificity that a government employee who knows the agency's filing systems could actually locate them. That last clause is the key. You're not describing what you want to learn. You're describing what records would contain the answer.

Consider the difference between these two framings. A request that says "all records related to the agency's response to the 2023 water contamination event in East Palestine, Ohio" is doing better than one that says "information about how the EPA handles environmental disasters." The first describes records. The second describes a topic. The Reporters Committee for Freedom of the Press's FOIA guide[3] emphasizes this distinction repeatedly: agencies respond to record descriptions, not research questions. Think about what a responsive document would look like — a memo, a contract, an email chain, a spreadsheet — and describe that document rather than the underlying question you're trying to answer.

Date ranges help enormously. "All emails between the agency's regional director and the contractor from January 2022 through December 2023" is far easier to fulfill than the same request without dates. Specific offices or individuals narrow the search. File types matter too — if you want emails, say emails. If you want inspection reports, say inspection reports. The more surgical your description, the faster the response and the lower the chance an agency will claim your request is too broad or burdensome to process.

There's a fee structure to know about. FOIA requests are not always free, though in practice many are. The National Security Archive's FOIA requester guide[4] breaks down the three fee categories: search fees, duplication fees, and review fees. Journalists and news organizations get preferred treatment — they can be charged duplication costs only, not search or review. Educational and non-commercial researchers get similar treatment. Commercial requesters pay the full freight. If you're an individual filing for personal use, you generally pay duplication fees but not search fees. And here's the thing most people don't know: you can — and should — include a fee waiver request in your initial submission. The standard is whether disclosure is "likely to contribute significantly to public understanding of the operations or activities of the government." That's a somewhat elastic standard, and agencies sometimes deny fee waivers on first pass, but asking costs nothing.

The timeline question is where things get messy. The law says agencies have twenty business days to respond. What "respond" means in practice is often just an acknowledgment that the request was received and a tracking number assigned — not the documents themselves. The FOIA Project at Syracuse University's TRAC database[5] tracks agency response times, and as of recent years, backlogs at major agencies like the FBI and State Department run to years, not months. The FBI's backlog has at times stretched past five years for complex requests. That's not a reason not to file — it's a reason to file early, to file specifically, and to know your options when things stall.

This is where MuckRock becomes genuinely useful. MuckRock[6] is a nonprofit platform that lets you draft, file, track, and publish FOIA requests — both federal and state — all in one place. It maintains a public database of requests and responses, which means before you file your own request, you can search whether someone else has already gotten the documents you're looking for. That search alone saves enormous time. MuckRock also generates agency-specific cover letters and handles fax submissions to agencies that still require them, which — and this is not a joke — some agencies do. The platform charges modest fees for some services, but the request database is publicly searchable for free. MuckRock's own documentation on how to use the service[7] describes it as having processed hundreds of thousands of requests across thousands of agencies. For anyone doing regular public-records research, it's infrastructure, not a luxury.

Now for the part that frustrates everyone: exemptions. FOIA has nine statutory exemptions — nine categories of records that agencies can withhold, in whole or in part. Worth knowing all nine at a high level, because the most common ones will appear in every denial letter you ever receive.

Exemption 1 covers classified national security information. Exemption 2 covers internal personnel rules and practices. Exemption 3 catches records specifically protected by other federal statutes — and this is a broad bucket, because many statutes explicitly shield certain agency records from disclosure. Exemption 4 protects trade secrets and confidential commercial information — expect this one when requesting records related to government contractors. Exemption 5 is the one journalists and advocates fight about most: it covers "inter-agency or intra-agency memorandums or letters" that would not normally be available in civil litigation. In plain English, that's the deliberative process privilege — the government's ability to protect the internal back-and-forth of how decisions get made. Agencies invoke Exemption 5 liberally, and courts have periodically pushed back. Exemption 6 protects personal privacy in personnel and medical files. Exemptions 7 through 9 cover law enforcement records, records about financial institution oversight, and geological exploration data — those come up less often in most research contexts.

The critical point about exemptions is that they're not all-or-nothing. Agencies are required to release non-exempt portions of a document even when exempt portions exist — that's the concept of segregability. A heavily redacted document isn't a lost cause. It still shows you what categories of information existed, what offices were involved, what the date range of relevant activity was. Experienced FOIA researchers treat redacted documents as a roadmap for follow-up requests, not as a dead end.

What happens when a request is denied, partially fulfilled, or simply ignored? This is the appeal ladder, and most requesters never climb it — which is exactly why agencies can afford to be aggressive with initial denials.

The first rung is the administrative appeal. Every FOIA denial must include information about the right to appeal within the agency. You have ninety days to file. The appeal goes to a different person than the one who made the initial determination — in theory, a more senior official conducting a fresh review. The Department of Justice's FOIA appeals guidance[8] notes that administrative appeals succeed surprisingly often. Agencies sometimes reverse initial denials at the appeal stage, particularly when the appeal letter specifically argues which exemptions were improperly invoked and cites relevant case law. You don't need to be a lawyer to do this, but you do need to be specific. Saying "I appeal this denial because you got it wrong" accomplishes less than saying "the deliberative process privilege of Exemption 5 does not apply to this record because it constitutes a final agency policy statement, not a predecisional draft, as established in Renegotiation Board v. Bannercraft."

The second rung is OGIS — the Office of Government Information Services, housed within the National Archives. OGIS[9] serves as a mediator between requesters and agencies. It's not an enforcement body; it can't order an agency to release records. But it can facilitate dialogue when requests are stalled or when appeals have failed, and it publishes reports on agency compliance that create some reputational accountability. Filing an OGIS complaint is free and takes minutes.

The third rung is federal court. Filing a FOIA lawsuit is a real option, and it's less exotic than it sounds. Federal district courts have jurisdiction over FOIA cases, and the government pays attorney fees when requesters substantially prevail. That fee-shifting provision means FOIA litigation is economically viable for advocacy organizations and media outlets. The Knight First Amendment Institute at Columbia University[10] has litigated landmark FOIA cases and publishes its legal filings publicly, which makes useful models for researchers interested in how FOIA arguments get constructed. For individual researchers not affiliated with an organization, litigation is a longer shot — but knowing the option exists matters, because agencies also know it exists, and sometimes an appeal letter that mentions potential litigation accelerates agency action.

State-level sunshine laws deserve equal attention, because for a huge category of research — local government, state agencies, municipal contracts, public schools, police departments — state law is the operative tool, not federal FOIA. Every state has some version of an open-records law, though the names vary. You'll hear "sunshine law," "open-records act," "right-to-know law," and "public-records law" used interchangeably.

The variation between state laws is genuinely significant. Some states are among the most open-records-friendly jurisdictions in the world. Florida's Government-in-the-Sunshine Law, codified in Chapter 119 of Florida statutes, is frequently cited as one of the broadest in the country — Florida's Office of Open Government[11] describes it as a strong presumption of openness, with records being public unless a specific exemption applies. Texas has a similarly robust framework with short response deadlines. At the other end, states like Arkansas and Mississippi have historically had weaker enforcement mechanisms and broader exemptions.

The Reporters Committee for Freedom of the Press maintains the Open Government Guide[3], a state-by-state breakdown of every jurisdiction's open-records and open-meetings laws. This is the reference to bookmark. It covers the response timeframes, fee structures, available exemptions, and appeal pathways for all fifty states and the District of Columbia. Before filing a state records request, fifteen minutes with this resource will tell you what to expect.

A few patterns hold across most state laws that are worth knowing. State requests generally go directly to the record-holding agency or office — often a specific department within city hall, a county clerk, a school district's records office. Unlike the federal system, there's usually no centralized portal. Response times at the state level are often shorter than federal FOIA timelines — many states require response within five to ten business days, though "response" again may mean acknowledgment rather than production. States vary widely on whether they charge for search time, and some states require agencies to create records — not just produce existing ones — while most don't.

One genuinely underused technique at the state level: open-records requests for records that federal agencies have already shared with state counterparts, or that state agencies have generated in connection with federally funded programs. A state environmental agency that receives EPA grants, for example, likely holds records that overlap with federally filed reports — and the state version may be easier and faster to obtain than the federal one. Parallel filing at multiple levels is a legitimate strategy that experienced investigators use regularly.

Back to the practical. When a request is going to take a while — and many will — there are things to do while waiting. First, check FOIA reading rooms. Agencies are required by law to post frequently requested records online, and some agencies are better than others about maintaining these. The FBI's Vault[12] is a famous example: it contains thousands of previously released documents, organized by subject, and searching it before filing a FOIA is just common sense. The CIA, the State Department, the Defense Department, and most major agencies have similar repositories. Second, use FOIA logs. Agencies are required to log all requests they receive, and these logs — which are themselves obtainable via FOIA — tell you what other researchers have asked for and what's been released. MuckRock has aggregated many of these logs and makes them searchable. Third, look for FOIA litigation databases. Court decisions interpreting FOIA exemptions are public record, and the Department of Justice maintains a FOIA case law database[13] that tracks significant decisions — useful when constructing an appeal.

A note on strategy that doesn't get said enough: narrower requests get faster responses. This seems counterintuitive — you want everything, so ask for everything. But broad requests trigger broader searches, larger fee estimates, and longer review times. Experienced FOIA practitioners often file a series of targeted requests rather than one omnibus one. Start with something narrow enough to get a quick response. The documents you get back will often reveal new offices, new date ranges, new document types you didn't know existed — which then inform the next, better-targeted request. Think of the first request as a probe, not the whole mission.

The last thing worth knowing is how FOIA research compounds with the other record types covered elsewhere in this course. A federal contract obtained through USASpending.gov tells you what was paid and to whom. A FOIA request for the procurement file — the competitive bids, the evaluation memos, the technical proposals — tells you why. Property records show who owns a building that houses a federal contractor. A FOIA request for the lease file tells you what the government paid for the space and whether there were any irregularities in the award. Court records from PACER show that a company was sued by a former employee. A FOIA request to the relevant federal agency might surface a simultaneous investigation. The records don't just coexist — they cross-validate and extend each other. That compounding effect is what separates a thin background check from a real investigation.

Knowing how to file, how to appeal, and how to use tools like MuckRock and reading rooms turns the law from an abstract right into a practical one — a door that opens with the right key. The question now is what's waiting behind the financial disclosure door, where campaign contributions, lobbying records, and federal spending data live in publicly searchable databases that most researchers haven't found yet.

Sources cited

  1. The Department of Justice's guide to FOIA justice.gov
  2. The FOIA.gov portal foia.gov
  3. The Reporters Committee for Freedom of the Press's FOIA guide rcfp.org
  4. The National Security Archive's FOIA requester guide nsarchive.gwu.edu
  5. The FOIA Project at Syracuse University's TRAC database trac.syr.edu
  6. MuckRock muckrock.com
  7. MuckRock's own documentation on how to use the service muckrock.com
  8. The Department of Justice's FOIA appeals guidance justice.gov
  9. OGIS archives.gov
  10. The Knight First Amendment Institute at Columbia University knightcolumbia.org
  11. Florida's Office of Open Government myfloridalegal.com
  12. The FBI's Vault vault.fbi.gov
  13. FOIA case law database justice.gov