Simple Agile Stepwise Refinement

By Brian Lucas

The one is for everyone who has the opportunity of giving a helping hand and a kind word to someone in need… and does so!

“Just do what must be done. This may not be happiness, but it is greatness.”

George Bernard Shaw

I am often asked what documentation I recommend on agile projects. My answer changes based on the situation, but it normally follows a simple pattern of stepwise refinement. I have written about this in my blog Keeping Agile in the post Turning Damnable Iteration into Elegant Stepwise Refinement in Agile. But I did not address flow clearly enough and subsequently received so many questions from readers, it was necessary to revisit the subject.

In agile documentation is minimal, yet not any less important. Because you use fewer words and artifacts to define something, those you use become more important!

I recommend strongly practicing stepwise refinement in agile development. I use a very specific interpretation of Niklaus Wirth’s stepwise refinement that embraces some top-down design concepts.  See the diagram below:


Every project, regardless of whether it is a software development effort or laying a rug at your church, should begin with a project charter. A simple (one page) project charter contains a short, clear and inclusive narration of what you are going to do and why you are going to do it (include motivating factors in this justification). It also defines any available budget and budgetary constraints. Any time or resource restrictions are noted as well.

From this charter, create a simple project definition in some repository (likeMicrosoft’s TFS or something more agile friendly). The project definition gives you a place holder for organizing all releases of the effort together in one spot.

Now define several releases for the effort. Three being the optimal number. Remember, you don’t have to get these perfect. You can refine them later. Concentrate on getting done what is critical to the basic functionality first. Have each release span about 3 chronological months. Thinking in terms of bitesize chunks is critical to keeping deliverables flowing and adhering to the Pareto principle.

While you are developing these releases, create storyboards, in some wireframing tool, that define the basic scenarios of use needed in each of the releases. Note each subsequent release will probably consume storyboards from a previous release. When you are writing the product and release definitions, use language that your end user or consumer would use and understand. These definitions should become the basis for marketing. And if you were so unforgivable as to not have the marketing and sales people involved earlier, now is the time you must do so.

Break down each release into a set of epics or very high level user stories that have more to do with flow and high level features, rather than individual functions. Note each subsequent release will probably consume epics from a previous release.

You are now ready to define the individual atomic functions necessary to implement each epic. Express these as well worded user stories that follow the Gherkin language and adhere to the INVEST principle. Make sure each user story has clearly derived acceptance criteria associated with it. They must be understood by the product owner, developer and tester alike. From the acceptance criteria the dedicated testers should develop specific test cases. These should then be automated, wherever possible, in a set of test scripts to promote ease of full regression testing.

You are now ready to go into release planning, followed by iteration planning. Generally, it works best if you have three iterations, each 4 weeks long for each release. Ideally, the last week being a hardening off time where no new development takes place only bug fixing.

In each step of the way, you can go back and make revisions to a higher level expression. For example, when you are defining the releases, you will gain greater insight into the overall product definition and might need to go back and add to, delete from or modify the language used there. Nothing is written in stone as the old saying goes. You should be taking in new knowledge all the time and leaning as you go, resulting in a better product.

A few final tips:

  1. Practice economy of writing. Never use 10 words when 5 will do as well.
  2. Practice uniformity of language. Use the same terms throughout with everyone. The best terms are the ones the user will recognize.
  3. Don’t get hung up on any one step. Often a subsequent step will aid you in understanding a previous one. So move on, then go back and correct.

If you are new to agile or even experienced some form of orderly progression like this will help you organize your efforts and keep them on track. Let me know what works for you!  Till next time, Keep Agile!

About Brian Lucas

In his life, Brian Lucas has been a coach, farm worker, forester, health care advocate, life guard, general contractor, mechanic, mixologist, musician/singer (in a rock group), salesman and teacher. Brian has worked as a project manager, technical marketer, methodologist, manager, software architect, systems designer, data modeler, business analyst, systems programmer, software developer and creative writer. These efforts include over a hundred hi-tech initiatives in almost every business and industrial sector as well as government and military projects. Among them, he designed and developed a quality assurance system for the first transatlantic fiber optic communications network, a manufacturing system for a large computer manufacture’s seven manufacturing centers, a data mining system for steel production, an instrumentation system for cable systems, defined requirements for government’s information systems and designed and developed human performance management systems. Brian has educated and mentored many over the years, designing programs to discover and develop talent. He has also lectured extensively to a variety of audiences. Brian is currently devoting as much time as possible to the innovation of business agility and human capital management along with the next generation of agile software development. As an amateur theoretical physicist he is working on joining general relativity and quantum mechanics through a multidimensional time corollary on string theory and negating the uncertainty principle with Louis de Broglie’s wave/particle hypothesis. He is also an avid blue-water sailor and wilderness backpacker. He enjoys billiards, boxing, chess, cooking, famous battle reenactments and war gaming, fencing, flying, gardening, horseback riding, martial arts (particularly Ninjutsu), philosophy and psychology, playing musical instruments (7 so far), poker, rapid-fire target shooting, reading (he tries to read a new book every night), painting with oils, scuba diving, skiing and recently writing novels.
This entry was posted in Agile and Strategic Planning, Agile for Beginners, Agile for Software Development, Agile in the Enterprise, Uncategorized and tagged , , , . Bookmark the permalink.

4 Responses to Simple Agile Stepwise Refinement

  1. Benjamin Raub, CSM - PMP says:

    Brian: I just started reading your blog. This is an excellent overview of how to tie user stories to greater meaning in scrum. We have struggled with capturing meaning from early in a project and incorporating it in the user stories. Your technique puts this information is the most appropriate places and enables it to influence the user stories so they are better organized. Best of all it is SIMPLE! Do you have any examples you would be willing to share? I realize this is asking a lot. Thanks for a great post! :Ben

    • Brian Lucas says:

      You can email me your questions Ben. I can’t share an actual project in the blog, but I will try to find the time to create a sample that I will post. -Brian Lucas

  2. Karl Nevers, CSM says:

    This is similar to what we have been doing for years and it works fine. No need to make it more complicated.

  3. Gloria Koch says:

    Nice explanation Brian. I love your blog and hope you post more along this practical advice when following scrum.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s