The role of documentation in an agile world

The Agile Manifesto values “working software over comprehensive documentation”. This has led to some to believe that documentation has little value in a modern software development workflow - that’s a mistake. Let’s look at what role documentation can play in an agile team.

When software was written back in the waterfall days we put a big emphasis on requirements and design documents. These were created before any code was written, and would be used to guide the implementation. Engineers were tasked with putting into practice what the documentation laid out.

Many may remember first hand the issues these sorts of projects would be plagued by:

• Requirements are vague, and not even properly understood by stakeholders
• The proposed design is not feasible to implement
• Constraints and problems are discovered during implementation that the documentation did not account for

In hindsight, it is unsurprising that this model of working caused people to react negatively to the idea of documentation.

Not the what, but the why and how

Documentation should not be a tool to blindly drive software development towards an unclear outcome. Instead of describing work to be done, your developer docs can provide valuable context that is not available from just looking at the code or machine-generated API descriptions.

While tools like Swagger and modern language toolchains (Rust and Elixir are prime examples) are great for describing the what of a project they fall short in explaining the bigger picture.

What if you want to know why a service was created? How does the flow of data through the system? Or what is the process for adding another HTTP route to this application? A class diagram or auto-generated API docs won’t be able to answer these questions - it requires context that someone needs to write down.

What is useful documentation?

So, what do we want to document? The exact answer will vary from team to team, but a general guideline to follow is that your docs should be written with a clear purpose in mind.

If you are a developer, consider what you’d want to know when coming to a project that hasn’t been touched in 18 months. Here are a few questions you can ask yourself:

• How do you set up your development environment?
• How is the project deployed and monitored?
• Why was the project originally created, and who did it, and what were the requirements?
• What are key concepts you need to be aware of when diving into the codebase?
• Why is the project structured like it is? Were some shortcuts taken? Why?
• How does a consumer of this service interface with the system?

Each one of these questions can be a section in your documentation.

Having this type of context available can get you up to speed quickly. And in the case where the original creators of the project are no longer available, it can save you months of forensic work on a foreign codebase.

If you are a team lead, you may want to consider:

• What documents are available to a new engineer to get them up to speed?
• Is there a service you manage that everyone is afraid of touching? Can we start documenting how it works?
• Do you have an service where only one engineer understands it? Can we document it to improve our bus/lottery factor?

This is where documentation can help. You don’t need 50-page engineering manuals, but having targeted, purposeful documentation can help you manage technical debt and allow your teams to move faster.

Documentation is never done

Your docs are never complete (in stark contrast with the old waterfall model). As long as your code is changing, your docs needs to change with it. In fact, the moment your documentation stops being trustworthy, it quickly loses most of its value.

Documentation is like a test suite - if it starts degrading, you can’t rely on it any more

This is why it’s important to invest in a process for keeping your documentation up to date. Be deliberate, and agree as a team that this is something you want to do.

Agile documentation is continuous documentation

Documentation is not against the spirit of agile. It is something that teams can do continuously to stay nimble. By incorporating continuous documentation into your development cycle, you help prevent losing knowledge when developers leave, make onboarding faster, and reduce technical debt.

Spend some time writing what your future readers will need. Your fellow developers, PMs, and team leads will thank you next time they are tasked with working on a new system they’re not familiar with.