RC RANDOM CHAOS
engineeringrant

Why Nobody Reads Your Documentation

Documentation stopped being true and the system had no mechanism to detect the loss. This is not a story about documentation. Not really.

· 7 min read

There is a particular kind of knowledge that exists only on paper. It was true once, possibly, at the moment someone wrote it down. Then the system changed and the document did not, and a gap opened between what was documented and what was real. Over time, the gap widened. Nobody noticed because nobody was checking. And nobody was checking because, at some point, everyone had quietly agreed that the document was not a source of truth. It was a source of comfort. This is not a story about documentation. Not really. It is a story about what happens when systems optimize for the appearance of knowledge rather than knowledge itself.

The Artifact Problem

Documentation began as transmission. Someone knew something and tried to preserve it for the next person. The writing existed in service of the knowledge. What changed was scale. As organizations grew, documentation became a process requirement rather than a communication act. Templates appeared. Review gates appeared. Audit requirements appeared. The system demanded artifacts, and artifacts were produced. But the system never demanded accuracy. It demanded existence.

This distinction matters more than it appears to. When the requirement is existence, the cheapest way to satisfy it is to produce something that looks correct. A template filled in. A procedure listed. A diagram drawn. Whether the content reflects reality is a separate question, and one that the system asking for the artifact has no mechanism to verify.

Over time, the people producing documentation understood this implicitly. They learned that a document is reviewed once, at creation. After that, it sits. Nobody returns to check whether the steps still work, whether the architecture diagram still reflects the architecture, whether the runbook still matches the system it describes. The artifact passed its one test, the moment of creation, and was never tested again.

This is how you end up with documentation that is simultaneously complete and useless. Every section filled in. Every template satisfied. And none of it true.

The Decay Curve

A document begins to decay the moment it is written. Every change to the system it describes introduces a divergence. A renamed field. A removed endpoint. A shifted responsibility. A dependency that was swapped out three months ago. Each divergence is small enough to overlook. Together, they transform the document from a description of reality into a historical fiction that happens to be formatted like a technical document.

The trajectory is consistent. For the first few weeks, the document is roughly accurate. People reference it. It helps. Then a change ships without a corresponding documentation update. Then another. Within six months, the document describes a system that no longer exists in quite the way it suggests. Within a year, it describes something that may never have existed, because the accumulated drift has compounded into a picture that was never true at any single point in time.

The people closest to the system know this. They stopped reading the documentation months ago. They read the code. They ask a colleague. They search the repository. They trace the actual execution path. These are rational responses to an information source that has demonstrated, through repeated experience, that it cannot be trusted.

The documentation did not fail because people are lazy. It failed because the system that produced it had no feedback loop. There was a process for creating documents. There was no process for maintaining them. And a document without maintenance is a document with an expiration date that nobody bothered to print.

The Compliance Mirror

This pattern is not confined to software documentation. It shows up anywhere a system requires evidence of understanding without verifying that the understanding exists.

Compliance frameworks produce the same artifact problem at institutional scale. An organization demonstrates that it has documented security controls, written incident response procedures, published data handling policies. Auditors review the documents. The documents reference the correct control names. The correct frameworks are cited. Checkboxes are checked.

Whether anyone in the organization actually follows those procedures is a separate question. Whether the documented procedures would work during an actual incident is yet another. The audit process is not designed to answer either. It is designed to verify that artifacts exist. And they do.

Governance follows the same curve. Policy documents are written with care, reviewed with rigor, published with ceremony. Then the organization changes and the policy does not. New systems are adopted that the policy does not address. Old systems are retired that the policy still references. Over time, the policy describes an organization that no longer exists, addressing risks that have been replaced by entirely different risks. But the policy is on file. The artifact persists. The knowledge has evaporated.

Dashboards exhibit the same failure mode. A dashboard is built to surface the health of a system. For a time, it reflects reality. Then the system changes. Metrics shift. What mattered six months ago is no longer the signal that matters now. But the dashboard remains, displaying numbers that everyone glances at and nobody questions. The dashboard becomes a ritual. A thing you look at because you are supposed to look at it, not because looking at it tells you something true.

In each case, the mechanism is identical. An artifact was created to represent knowledge. The artifact persisted. The knowledge did not. And the system measuring compliance, or governance, or operational awareness could not tell the difference.

The Incentive Structure

The reason this pattern perpetuates is not mysterious. It follows directly from what organizations choose to measure.

Producing a document is visible. Maintaining a document is invisible. Writing a runbook gets acknowledged. Updating a runbook six months later, when nobody asked you to, when there is no ticket for it, when it will not show up in any sprint review, does not. The incentive structure rewards creation and ignores maintenance. So creation happens and maintenance does not.

This is compounded by the fact that documentation debt is silent. Unlike code debt, which eventually produces errors, failed builds, or user-visible bugs, documentation debt produces no signal at all. The document sits there, looking exactly the same as it did when it was accurate. There is no test suite for documentation. There is no compiler that rejects a stale procedure. The failure mode is invisible right up until the moment someone depends on the document and discovers it has been lying to them.

Organizations respond to this by adding more process. Documentation review cycles. Ownership assignments. Freshness dates. These interventions address the symptom without touching the cause. The cause is that the system values the existence of artifacts over the accuracy of artifacts. Until that changes, every process layered on top will eventually succumb to the same drift. Reviews will become rubber stamps. Ownership will become nominal. Freshness dates will pass unnoticed.

The deeper problem is that accuracy is expensive to verify and impossible to automate in the general case. You cannot write a test that confirms a paragraph of prose still matches the system it describes. Verification requires a human who understands the system to read the document and compare it to reality. That is expensive. That is slow. And it produces no visible output. The economics favor letting it slide.

The Deeper Problem

The documentation problem is not really about documentation. It is about the broader tendency of systems to produce artifacts as proxies for the things those artifacts are supposed to represent, and then to optimize for the proxies.

Compliance documents become proxies for security. Policy documents become proxies for governance. Architecture diagrams become proxies for understanding. Documentation becomes a proxy for knowledge. Over time, the proxy replaces the thing it represents. The system measures proxies because proxies are countable, auditable, demonstrable. The underlying qualities, security, governance, understanding, knowledge, are none of those things.

Nobody reads the documentation because the documentation stopped being a path to understanding. It became an artifact that satisfies a process. And the people who need understanding have learned, through direct and repeated experience, that the artifact and the understanding separated a long time ago.

The system never noticed because the system was not measuring understanding. It was measuring artifacts. And by that measure, everything looked fine. It always looks fine. Right up until someone needs the knowledge the artifact was supposed to contain.

By the time someone reaches for the document in a moment that matters, the knowledge it was supposed to contain is already gone. What remains is the shape of understanding. The thing itself disappeared much earlier.