Every table in one of our production Postgres databases is UNLOGGED.
If that box loses power, if the kernel panics, if a container gets a kill -9, Postgres comes back up and truncates all eight tables to empty.
Ten million documents of body text, plus tens of millions of metadata and citation rows, gone, on purpose.
This is not a gap in our operations that we are quietly working on. It is the design, we chose it deliberately, and we would choose it again. This post is the reasoning, including the one condition under which the same choice would be reckless.
The short version: durability is a feature you pay for, and you should only pay for it when you are the source of truth. For a derived dataset, durability is a cost, not a virtue.
What the database actually is
We work in legal. This particular database is a self-hosted mirror of a large public bulk dataset of public-domain legal text, roughly ten million documents of it, that we rebuild from the upstream publisher's own bulk files. We mirror only the handful of tables our product actually reads.
The important property is not the subject matter. It is this: every byte in that database exists somewhere else, for free, in a form we can re-download on demand. We did not author it, we do not mutate it, and no user writes to it. It is a read-only cache with a query engine bolted on.
The engine is Postgres 18.4 with ParadeDB's pg_search, which puts a Tantivy BM25 index inside Postgres.
BM25-first, no vector index, because on long legal text BM25 beats zero-shot dense retrieval in the evaluations we trust, at recall-at-5 of 63.63% against 61.28%.
It runs on one dedicated box: four cores, 62 GiB of usable RAM, 452 GB of usable disk on NVMe in RAID 1.
That last figure is the constraint that makes the rest of this post interesting. The document-body table alone is TOAST-heavy and carries roughly 150 GB of text. On a 452 GB disk, headroom is not abundant, and 180 GB of it is not something you have lying around.
What UNLOGGED buys, in actual numbers
A LOGGED Postgres table writes everything twice. Once to the write-ahead log, so a crash can replay it, and once to the heap. That WAL traffic is the price of durability, and for data you authored it is a bargain.
Turning it off bought us four things.
About 180 GB of peak disk.
Converting these tables to LOGGED means ALTER TABLE ... SET LOGGED, which rewrites the entire heap and WAL-logs all of it.
The transient space that operation needs is space we do not have on a 452 GB disk that is already holding 150 GB of text plus its indexes.
Saving disk was the smaller win. UNLOGGED kept the whole architecture viable on the hardware we chose.
About three hours of conversion time. That is three hours we would spend, every refresh, converting data whose entire value proposition is that we can re-download it.
Roughly 2x faster writes.
No WAL means the bulk load runs about twice as fast.
On a periodic refresh cadence, that compounds.
It is also what lets us turn off synchronous_commit, fsync, and full_page_writes during the load without lying to ourselves about the risk, because there is nothing in the database whose loss would cost us more than a re-download.
And the conversion itself costs about 10x.
This is the one that settles it.
The Postgres wiki is direct about SET LOGGED: it rewrites the heap and WAL-logs everything, and the practical cost is around ten times the cost of staying UNLOGGED.
We evaluated it as one option among many during the load-tuning research and it came out on the reject list alongside pg_bulkload and pgloader.
Against all of that, the thing we bought is crash survival. So the only honest question is: what is crash survival worth here?
The math that makes the answer easy
A crash costs us roughly two and a half to three hours of rebuild, and it is a rebuild we already know how to do, because it is the identical procedure we run every quarter when the upstream dataset publishes a new dump.
That is the whole argument, and it turns on a detail that is easy to miss. The recovery path is not a special emergency procedure that lives in a document nobody has run. It is the normal refresh path. We exercise it four times a year on a schedule. When a crash forces us onto it, we are running well-worn code, not a disaster plan whose first real execution is during the disaster.
Compare that with what durability would have bought. Instant recovery instead of a three-hour rebuild, in exchange for 180 GB of peak disk we do not have, three hours of conversion per refresh, half our write throughput, and roughly 10x on any conversion. Given a crash rate on a single dedicated box measured in events per year, we would be paying that tax continuously to avoid an outage that happens rarely and costs three hours when it does.
The tax is certain. The benefit is probabilistic and small. That is not a close call.
The condition that makes this reckless
Here is where I have to be careful, because "just make it UNLOGGED" is bad advice delivered without its precondition, and the precondition is doing all the work.
UNLOGGED is only defensible for data you can regenerate from a source of truth that is not you.
The moment this database holds a single byte of user-generated state, the argument collapses. A user's bookmark, a saved search, a comment, an annotation, a "last viewed" timestamp: none of that exists in the upstream bulk files. If it lives in an UNLOGGED table, then a crash does not cost three hours of rebuild. It costs data that no procedure on earth can bring back, because there is nowhere to bring it back from.
So the rule is written into our runbook in capital letters, and it is not a suggestion:
If you create user-generated state, comments, bookmarks, anything of that kind, put it in a SEPARATE LOGGED database.
This is what makes the choice sound rather than lucky. The UNLOGGED database is not "our database with durability turned off." It is a specific, bounded thing: a derived, read-only, reproducible cache. Its boundary is a load-bearing part of the design. The day someone adds a user-writable table to it, that boundary breaks silently, the argument in this post becomes false, and nothing in Postgres will warn you.
That is the real risk, and it is a discipline risk, not a technical one. Which is exactly why it belongs in a blog post and not just a config file.
The interaction we did not anticipate
There is one place where UNLOGGED bites harder than the arithmetic suggests, and it is worth naming because it is the kind of thing you only find by operating the system.
pg_search's index-build loop does not check Postgres's interrupt status.
This is documented upstream as ParadeDB issue #2211.
In practice it means a BM25 CREATE INDEX cannot be canceled once it starts.
We confirmed the entire ladder: pg_cancel_backend returns t and the backend keeps burning CPU.
pg_terminate_backend returns t and the backend keeps burning CPU.
kill -INT at the host PID level is ignored.
The only thing that stops it is kill -9 or a container restart.
Which crashes Postgres.
Which truncates every UNLOGGED table.
Which throws away the entire bulk load.
So UNLOGGED removes the escape hatch from an already-uncancellable operation. If you launch a long index build with a bad spec, you have exactly two options: wait it out and rebuild afterward, or lose everything and start over.
We do not treat this as an argument against UNLOGGED. We treat it as a requirement to validate index specs before launching them, which is a good practice regardless, and which I wrote up separately in Cutting a BM25 Index Build From Four Hours to Ninety Minutes. But it is the clearest example of how a durability decision does not stay inside the durability box. It changes the cost of unrelated mistakes elsewhere in the stack. Anyone reproducing this pattern should know that before they find out the way we did.
Making the loss detectable
A choice like this is only safe if the system knows when it has happened. An empty database that keeps answering queries with zero results is far worse than one that admits it is empty, because a search engine returning nothing looks identical to a search engine that found nothing.
So the detection is a triviality by design:
SELECT count(*) FROM small_reference_table;
-- Under 100 means the tables are empty. Post crash. Reload.
That reference table is the smallest thing in the database, a few thousand rows. It is either fully there or it is gone. There is no partial state to reason about, because a crash truncates all UNLOGGED tables together, not some of them.
That check belongs in the API shim's health endpoint, returning 503 when the tables are empty rather than serving confident emptiness, and triggering the reload. This is the part of the design that turns a scary property into an operable one. UNLOGGED data loss is total, immediate, and trivially detectable, which are three good qualities in a failure mode. Compare it with the failure modes that actually hurt: partial, silent, and discovered weeks later.
The tradeoffs, stated plainly
Here is what this choice actually costs us.
A crash means a rebuild, and rebuilds are not instant. Roughly two and a half to three hours from empty to serving. During that window our API shim returns 404s, and that is a wart we have not fixed. A 404 says "no such case," which is a lie: the case exists, we simply cannot serve it this hour. The health check above returns 503 because 503 is the honest code, and the data path should be saying the same thing. Writing this post is what made the inconsistency obvious, which is its own small argument for writing things down. A second box and a cutover would remove the window entirely. We have not built one, and the honest reason is that the rebuild window has not yet cost us more than the second box would.
We own the operations. No managed service is going to page itself when this box reboots. There is no support contract. The runbook is us.
The refresh cadence is the source's, not ours. The upstream dataset republishes on a regular cadence. Our mirror is exactly as fresh as the last dump and no fresher, and a full refresh is another three-hour window. For the retrieval this corpus feeds, that cadence is fine. If we needed same-day freshness, this architecture would be wrong and the UNLOGGED argument would need re-examining, because a rebuild you run daily is a very different proposition from one you run four times a year.
The boundary needs guarding forever. Every future engineer who touches this stack has to understand why this database is different from every other database they have worked on. That is an ongoing cost paid in documentation and review attention, and it does not go away.
Backups are deferred, not free. We have a backup path staged that would let us recover without re-pulling the full compressed source, and we have not activated it, because re-downloading the upstream bulk files is currently cheaper than maintaining a backup we would use once. That is a defensible call today and it is exactly the kind of call that stops being defensible quietly, as bandwidth costs or dataset size change.
The generalizable thesis
Strip out the specifics and one principle survives.
Durability protects data whose only copy is yours. For derived data, it is overhead wearing durability's clothes.
Ask one question of any dataset you are about to make durable: if this vanished right now, could I regenerate it, and what would that cost?
If the answer is "no, it is gone forever," then durability is not negotiable and you should pay whatever it costs. If the answer is "yes, in three hours, from a public source, using a procedure I already run on a schedule," then durability is buying you the difference between a three-hour rebuild and an instant one, and you should price it accordingly.
This is not a Postgres insight. It is the same reasoning that says a cache does not need to be backed up, that a read replica does not need its own disaster-recovery plan, that a derived search index does not need point-in-time recovery, and that a build artifact does not belong in a snapshot. Every one of those is a derived dataset, and every one of them has a source of truth somewhere else. When you replicate durability onto them, you are paying twice for one guarantee.
What we did was follow that reasoning to the place most people stop short of.
The uncomfortable step is not the logic.
It is watching SELECT count(*) return zero after a reboot and knowing that is the system working correctly.
The engineering that makes that comfortable is not the UNLOGGED flag. It is everything around it: a rebuild path that is the same path as the routine refresh, a detection check that cannot be ambiguous, a health endpoint that fails loudly rather than serving empty results, and a hard boundary saying user state lives in a different database.
Get those four right and durability becomes what it should be. Not a virtue you signal, but a line item you can price.
