So, I was looking at my smartctl report yesterday, and my secondary NVMe drive-- the
2 terabyte one I bought literally eight months ago-- is already showing six percent
wear.
Six percent.
That is...
that is INSANE for a drive that mostly just holds some code repositories and some
local tools.
And then I remembered I had Jellypod running in the background,
which led me down this rabbit hole of looking at Codex.
Wait, six percent in eight months?
That- that's like what, over a hundred terabytes of writes already?
What is Codex doing to that drive?
Exactly.
It is.
It's the Codex CLI version zero point one four two point zero,
and the Desktop version twenty-six point six one six.
They are absolutely shredding storage.
If you run iotop or look at Activity Monitor, you see this sustained,
non-stop write pressure.
It's hum-- hum...
it's hovered around four to five megabytes per second.
Every single second it is running.
Do the math on five megabytes per second over a full day, and you are looking at
nearly four hundred and thirty gigabytes of writes a day.
Four hundred gigabytes a day?
That is...
that's a death sentence for a consumer SSD.
Most of those drives only have a Terabytes Written...
uh, TBW rating of maybe six hundred terabytes total.
You'd burn through the entire warranty in what, a year and a half?
Less than a year if you leave it running twenty-four seven.
And the culprit is actually incredibly specific.
It's the logging subsystem.
Specifically, there's a rust module-- codex_api::sse::responses-- that is dumping
raw Server-Sent Events websocket frames directly into a SQLite database.
It's writing to a file called logs_2.sqlite in your local application data
directory.
And the developers forgot to put a throttle on the TRACE level logs for these SSE
streams.
Every single heartbeat, every workspace update, every tiny cursor movement event
gets written to disk instantly.
Wait, why is it writing raw websocket frames to a local SQLite db anyway?
Is that just poor design or are they trying to build some kind of local audit log?
Because SQLite, by default, is going to do a full write-ahead log write,
a WAL flush, for almost every single transaction unless they've configured it
specifically to hold things in memory.
It seems to be a debug feature that made it into production build profiles by
accident.
They are using SQLite in WAL mode, yes, which means you have two files:
logs_2.sqlite and logs_2.sqlite-wal.
And because the SSE connection is constantly screaming with data from the server,
the WAL file is just getting hammered with these tiny, sequential,
synchronous writes.
It is the absolute worst-case scenario for flash memory wear amplification.
Okay, so until they push a hotfix-- because who knows how long that will take-- we
need a way to stop this.
I mean, we can't just let our drives burn.
Can we just intercept the write?
What happens if we write-protect the file, or is there a cleaner way?
If you just make logs_2.sqlite read-only, the client actually crashes on startup.
It expects to be able to initialize the database schema.
But there's a really elegant workaround using SQLite triggers.
If you open the database directly using the sqlite3 command-line tool,
you can create a trigger that intercepts the inserts before they actually touch the
disk and just discards them.
Ah, interesting!
So you let the client think it's writing, but the database engine itself just drops
the payload?
How do you actually write that query?
Right.
So first, you have to find the database.
On macOS, it's usually under ~/Library/Application Support/Codex/logs_2.sqlite.
You run sqlite3 on that file.
Then you run a DDL command: CREATE TRIGGER block_sse_spam BEFORE INSERT
ON logs BEGIN SELECT RAISE (IGNORE) ; END;.
What this does is, every time the Codex client tries to run an INSERT INTO logs
query, the trigger runs first, executes RAISE (IGNORE) , which silently aborts the
insertion without returning an error to the host application.
The client thinks the write succeeded, but nothing gets written to the WAL file.
That is incredibly clever.
But...
wait, what happens when they release an update?
If the update runs a database migration, isn't it going to drop or fail because of
that trigger?
Or what if they change the table structure?
That is the big caveat.
If the next version of Codex tries to run a migration that drops or alters the logs
table, the migration might fail, or it'll just drop our trigger and start writing to
disk again.
So it's a temporary patch.
If you want a more robust solution that doesn't touch the database schema,
you have to use a RAM disk.
Right, offload the entire log directory to volatile memory.
On Linux, you could just symlink the log directory to /tmp or use tmpfs.
On macOS, you'd have to create a small RAM disk using hdid and format it as HFS+,
then symlink the Codex log folder to that mount point.
If it writes five megabytes per second to RAM, who cares?
RAM doesn't have a TBW limit.
Exactly.
You just run a startup script that creates, say, a 512-megabyte RAM disk,
mount it, and let Codex go wild.
The only downside there is that your logs are wiped every time you reboot.
But honestly, who actually needs historical TRACE logs of their editor's websocket
connection anyway?
It's completely useless data for ninety-nine percent of users.
Yeah, I think I can live without my cursor-movement telemetry surviving a system
crash.
I'm going to set up that SQLite trigger right now before my drive loses another
percent of its life.
Good catch on this one, Ethan.
Definitely.
Let's keep an eye on the next few release notes to see if they finally throttle this
thing.
Alright, talk to you next time.