Skip the Orchestrator: Why Postgres Alone Can Run Durable Workflows
Durable workflow systems like Temporal, Airflow, and AWS Step Functions rely on a central orchestrator to dispatch steps to workers and checkpoint their results to a data store. DBOS argues this architecture is needlessly complex. Since the whole point of durable execution is checkpointing state to a database, the database can do the coordination itself — application servers poll a Postgres workflows table, dequeue work using row locks, and write step outputs back as checkpoints. Integrity constraints catch duplicate execution attempts, and any worker can recover another’s in-flight workflows from the last committed step.
Collapsing the orchestrator into Postgres inherits decades of operational maturity for free. Scalability and HA reduce to Postgres scalability and HA, which are well-trodden ground via vertical scaling, sharding, CockroachDB, streaming replication, and managed multi-AZ offerings. Observability becomes ordinary SQL over the checkpoint tables rather than a bespoke query layer over a key-value store. The reliability and security story also tightens: an external orchestrator is an extra single point of failure that touches sensitive workflow data, while a Postgres-only design adds no new infrastructure to harden if the app already depends on Postgres.
Read the full article
Continue reading at Hacker News →This is an AI-generated summary. Read the original for the full story.