It's really dumb that in order to check the basic correctness of database operations, you have to actually start a database server. Examples in the world of postgres include:

• clorinde (opens new window), reads raw sql query text and generates corresponding fast rust code to call it with full type-safety. It must have a running postgres server to prepare the query statements against.
• migra (opens new window), diffs two postgres schema sources against each other and produces the "diff" sql to make them equal. Must have a running postgres server to run sql sources against to perform its reflection.
• postgres_migrator (opens new window), a project of mine that helps manage and generate postgres migrations from a "declarative" sql schema. It uses migra and so inherits its need for a running postgres server.

clorinde cleverly uses temporary docker or podman containers for its postgres server, but this still introduces a dependency on either docker or podman, all the crates needed to talk to postgres (perhaps asynchronously), and a lot of overhead for the server itself and the IPC to talk to it.

In many environments this slog is tolerated, since the speed of a dev loop only ruins the lives of coders, rather than users or investors. We might not like it, but we routinely accept slow devtools and slow CI jobs with annoying-to-set-up dependencies.

But this isn't necessary at all! Any database could, with just a bit of design discipline and forethought, make its runtime parsing and type and permissions checking modules usable without needing to start a whole database process. Doing so would unlock a ton of innovations in performance, safety, and ergonomics, such as:

• Null tracking: In a post-rust/typescript world, any programming system that doesn't have rigorous null tracking (opens new window) is unacceptable. Our databases shouldn't make us constantly guard against impossible nulls or be surprised by ones we thought were impossible.
• Flagging destructive queries: If we can statically determine whether an operation could mutate or destroy data, we could have a form of statically guaranteed sandboxing.
• Static permissions checks: If the analyzer were configured with or could infer the assumed role/group of commands, it would be possible to check if all the commands are actually allowed, or warn about the possibility of a rejection if it can't be fully known.
• Tracked error warnings: There are many other situations where it's possible a database operation could fail, perhaps because it could violate a data constraint or a row-based security check. If something like flow effects were part of the checker we could propagate these possibilities to higher layers to handle or prevent them. And wouldn't it be nice to be able to reject any functions that executed dynamically constructed queries (opens new window)?
• Better typing in general: A database with an independent parser and type checker could evolve those layers much more actively. Maybe we could even someday have such wonders as dependent typing (opens new window)) in our databases!

If we had these benefits and others, I'm convinced our software world would be much safer and less annoying. Databases are the most stubborn hideout of clunky 80s abstractions, unsafe stringly-typed (opens new window) glop, and null values 😡. I personally am tired of it. A software system can only be as safe and expressive as its least powerful component, so that means we can't really make truly great full-stack systems without fixing databases.

But what to actually do?

Originally this post was going to be an announcement I was beginning a "Typescript for postgres" (opens new window) project. I was thinking I would build a static analyzer layered on top of postgres in the same way Typescript layers on top of javascript.

But after finishing the first preliminary stages of the project (just struct definitions (opens new window) and reflection functions (opens new window) for the postgres catalog tables (opens new window) in order to use postgres as a test oracle (opens new window)), I'm honestly wondering if I should continue. It would be such a big project, and in lots of ways it would feel so duplicative. Why make a new analyzer when the postgres checking internals should just be pulled out? Why layer an analyzer on top of a type system that sucks to begin with?

But on the other hand... (opens new window) am I really going to make a whole new database??? That's an even bigger project!!! Unless I could use an off-the-shelf storage engine to quickly build a proof-of-concept... 🙃 So you can see I'm internally waging the endless war between committing to a pragmatic but unsatisfying compromise, vs a multi-year effort to build a revolutionary clean design (opens new window).

The only thing I'm sure of is that databases should have independent static checkers, and the actual type systems of databases shouldn't suck. Companies/projects like gel (opens new window) or typedb (opens new window) are interesting but just don't feel quite right yet, so I'm going to keep looking for the right thing for me to spend my time on next.

Perhaps you have an idea that could improve the robustness and expressiveness and ergonomics of databases? I'd love to hear from you!