People assume the law arrives as tidy rows: a title number, a section number, an amendment year, all clean and typed. It does not. US statutory data is public, which is a gift, and it is also messy in ways that only show up once you have loaded tens of millions of sections and watched them break your code. Title numbers that are not numbers. Amendment years set in the future. Fields that are simply missing on whole categories of rows. Category labels that drift as the corpus grows.
None of this is anyone's fault. Fifty-plus jurisdictions publish their codes in their own formats, on their own schedules, with their own quirks, and a federal code sits alongside them with different conventions again. The interesting engineering question is not how to make the data clean. It is what your product does when the data is dirty and you cannot fix it at the source.
Our answer is a rule we hold across the whole statutes surface: a dirty field must degrade gracefully, never crash the response, and never quietly produce a confident wrong citation. This post walks through the specific defenses that enforce that rule, all of which live in the code that serves our US statutes API today. There are two small snippets, simplified for readability, because the decisions are easier to see than to describe.
Why "just cast it to an integer" is a trap
Start with the field that looks the simplest: a title number.
If you have only ever seen the US Code, a title is a number from 1 to 54. So you type it as an integer, and every USC row serializes fine. Then a state row arrives.
Alabama's Criminal Code is Title 13A. Not 13. 13A. Alphanumeric title identifiers like that one are not typos or corruptions to be scrubbed out. They are the real, official, canonical identifiers, and if your product renders "13" instead of "13A" it has told the lawyer the wrong thing.
A strict integer field does not just render the wrong value here. It refuses the value entirely, and one rejected value inside a list response can fail a whole page of results rather than the single row. One unusual but completely legitimate identifier should never be able to turn a search into an error for everyone else, and that is exactly the failure our data model is designed to prevent.
So we do not assume a title is a number. On every result-facing model, the title number is a lenient string that accepts any scalar the corpus hands it and emits text. "26" stays "26". "13A" stays "13A". The section renders correctly either way, and no single alphanumeric title can crash a list of a hundred results.
There is a deliberate asymmetry worth calling out. On the way in, when a user filters a search by USC title, we still constrain that input to an integer between 1 and 54, because a federal-title filter genuinely is numeric and a malformed filter should be rejected loudly. On the way out, when we are reflecting whatever the corpus actually stored, we are permissive, because our job there is to faithfully show real data, not to argue with it. Strict where we control the input, forgiving where we are reporting reality.
A number that is not really a number
The title-number problem generalizes.
All across scraped and aggregated legal metadata, a value that should be a whole number arrives as something else.
A year comes in as the string "2024".
A count comes in as the float 3.0.
A field that was never populated comes in as an empty string, or as the literal None.
Occasionally a "year" field contains something that is not a year at all.
We route these fields through one lenient integer type, used everywhere a count or a year is populated from loose upstream data. Simplified, it is this:
# Simplified from our LooseInt coercion.
def loose_int(value):
if value is None:
return None
if isinstance(value, int):
return value
if isinstance(value, float):
return int(value) if value.is_integer() else None # 3.0 -> 3, 3.5 -> None
if isinstance(value, str):
s = value.strip()
if not s:
return None # "" -> None, not a crash
try:
return int(s) # "2024" -> 2024
except ValueError:
try:
f = float(s)
return int(f) if f.is_integer() else None
except ValueError:
return None # "13A", "iv", "N/A" -> None
return None
In plain English: a clean integer passes through, a numeric string becomes an integer, an integral float becomes an integer, and anything that cannot honestly be a whole number becomes None instead of raising.
The point is the last line.
When a field is genuinely garbage, we would rather show nothing than either crash the endpoint or invent a value.
A missing count is a small, honest gap in the UI.
A crashed endpoint is an outage, and a fabricated count is a lie.
The amendment year that had not happened yet
Here is the one that matters most for legal trust, because it touches what a lawyer reads and relies on.
Statute pages show a "last amended" year. It is a freshness signal, and lawyers use freshness signals to decide whether they are looking at current law. So the amendment year has to be right, or at least never confidently wrong.
Scraped amendment data is not reliably right. Ingest pipelines pull years out of source-credit text and citation histories, and sometimes what they pull is not an amendment year at all. Sometimes it is a sunset date or an effective date that lands in the future. The failure we refuse to ship is a section that proudly announces "Last amended 2029" because a scraper mistook a future effective date for an amendment. A future amendment year is worse than wrong. It undermines the exact trust the field exists to build.
We put a guard on this at the point of display, on top of whatever cleanup happened at ingest. Two facts bound a real amendment year. Federal law was not amended before the Constitution existed, so no federal amendment year predates 1789. And nothing has been amended in a year that has not arrived yet, so nothing exceeds the current year. Simplified, the sanitizer looks like this:
# Simplified from our amendment-year sanitizer.
MIN_YEAR = 1789 # federal floor: no federal statute predates the Constitution
def clean_amendment_years(years, stored_last_amended):
this_year = now().year
clean = sorted(
{int(y) for y in (years or [])
if y is not None and MIN_YEAR <= int(y) <= this_year},
reverse=True,
)
if clean:
last = clean[0] # newest surviving year
elif stored_last_amended and MIN_YEAR <= int(stored_last_amended) <= this_year:
last = int(stored_last_amended) # fall back only if it is plausible
else:
last = None # better to show nothing
return clean, last
In plain English: we throw out any year that is impossible, keep the surviving years newest-first, and re-derive "last amended" from the cleaned list rather than trusting whatever scalar the row happened to store. We only fall back to the stored value when it is itself plausible. If nothing survives, the field is blank, and blank is the honest answer. This guard runs on the section detail page, the amendments endpoint, and the list rows, so the same impossible year cannot slip through on one surface after being caught on another.
One honest wart, found while writing this post. That 1789 floor is federal reasoning, and it is running over a corpus that is federal plus fifty states. Colonial and early-state law predates the Constitution, and some of it is still on the books: Virginia's Statute for Religious Freedom was enacted in 1786. A pre-1789 state amendment year is rare, but it is not impossible, and our floor would silently null it. That is the exact failure this post is about, the confident discard of real data, sitting in our own sanitizer. The correct fix is to scope the 1789 floor to federal rows and give state rows a floor tied to that state's founding, and it is now on the list. We are leaving the wart in this post rather than quietly editing the code first, because the point of the post is that dirty-data guards are judgment calls, and a judgment call you never revisit is just a bug you have not met yet.
The identifier that contains almost every punctuation mark
There is a quieter kind of dirtiness in the identifiers themselves.
Every section in our corpus has an act ID, and it is tempting to validate it against a clean pattern: letters, digits, underscores, hyphens. Then you look at what the real identifiers contain. Around 470,000 rows carry a period in the ID, from Florida and many other states. Thousands carry commas. Federal regulation IDs carry parentheses. Some carry spaces, colons, brackets, semicolons, asterisks, a section symbol, even an en-dash.
Each of those is a legitimate act ID for a real provision. A tidy validation pattern would reject the provision as if it did not exist, which is the worst possible outcome for a research tool: a real section, invisible. So instead of enumerating a pattern that will always be one ingest quirk behind reality, we allow anything except the two things that are actually dangerous, path-traversal characters and control bytes, and we bound the request with a length cap and the downstream database lookup. The default is inclusion, because a real identifier we cannot render is a real answer we cannot give.
Enum drift, and the difference between failing loud and failing quiet
The last category is subtler. As the corpus grows, the set of category labels drifts. A new kind of source gets ingested, a legacy row has a null where a newer row has a value, and code that assumed a fixed set of labels starts making quiet mistakes.
We handle this two ways, and the split is intentional.
Where a wrong label would be dangerous and we want to catch it in development, we are strict on purpose. The corpus-type on a search result is constrained to a known set of values, so a backend regression that emits a brand-new label fails validation loudly instead of silently mislabeling a state statute as a federal regulation on the result card. Failing loud, in the right place, is a feature.
Where a label is merely missing, we infer it safely rather than guess.
Legacy state rows sometimes have no stored corpus-type at all.
Rather than let any code path fall back to "assume federal," which is the entire class of bug we set out to kill, we route through one helper with a firm guarantee: a known category maps to its type, an unknown category on a row we know is from a state resolves to "state," and only a genuinely unclassifiable row falls through to an explicit default.
There is no hidden else that quietly calls something federal.
A state provision never masquerades as a federal one just because a field was blank.
Why a buyer should care
Every defense above has the same shape. A field is dirty, we cannot fix it upstream, and the product has exactly three options: crash, fabricate, or degrade honestly. Crashing turns one bad row into an outage. Fabricating turns one bad row into a confident wrong citation, which in legal work is the failure that ends in a sanction. Degrading honestly turns one bad row into a small, visible gap and keeps everything around it correct.
Every defense in this post is the same fork drawn once. A dirty value we cannot repair upstream reaches serialization, and the model has three ways out.
The lenient title string, the loose integer, the amendment-year sanitizer, the permissive act-ID pattern, and the corpus-type inference are all the green path made concrete for one field. The strict input filters and the constrained result enum are the deliberate exception: where we control the input or a wrong label would be dangerous, we route the crash back in on purpose, because a loud failure in development is cheaper than a quiet one in front of a lawyer.
We chose the third option everywhere, and we chose it in code you could point at, not in a policy paragraph. That is the whole argument. A legal answer is only as trustworthy as its worst field, and the messiest fields are the ones a demo never shows you.
Test any legal AI yourself
You do not need our code to check whether a tool respects the mess in the underlying data. You need a few odd corners of real statute and five minutes.
-
The alphanumeric-title test. Ask any tool to pull up Alabama Title 13A, the Criminal Code, or any section inside it. A tool that models the law honestly shows you 13A. The tell of a tool that assumed titles are numbers: it shows "13," silently drops the section, or errors.
-
The impossible-freshness test. Find a section and ask when it was last amended. Then sanity-check the year. If a tool ever shows you a "last amended" date in the future, its freshness signal is decorative, and you should not trust any freshness signal it shows you.
-
The missing-field test. Browse to an obscure state provision, the kind with sparse metadata. A tool that degrades honestly leaves thin fields blank and keeps the rest correct. The tell of a tool that fabricates: every field is suspiciously full, including the ones the source never populated.
-
The weird-identifier test. Open a section whose citation includes a subsection or unusual punctuation. A tool that handles real identifiers resolves it. A tool that over-sanitized its inputs tells you a real provision does not exist.
If a tool passes these, someone on the other side has spent time in the same mess we have. If it fails any of them while still looking confident, you have learned that its polish is hiding the data, not handling it.
