Looking for the right way to track the design’s delivered value

I am going to talk about contracts, working with internal and external teams, delivered value, accepted stories and I don’t know what else. It’s a packed post, where I suspect I will end up wrapping up 5 years of attempts to apply agile methodologies to design.

To whomever should decide to read it, I warn you that you will need 15 minutes with good concentration.
I will start from here.

Deliver business value

A few days ago, Fabio asked: “What does it mean to be agile?”
Gabriele Lana replied:

Deliver business value at regular and sustainable pace. The definitions of "value" and "sustainable" are context dependent and are where you can screw up big time.

If we want to be agile, we (designers and frontend developers) are requested to design ux, ui and visual and build a product or a digital service front-end in order to deliver business value. After all, the talk I gave with Ale at the Agile UX Camp and then again at the Agile Day in 2012 was just about this.
I will take a moment to clarify the perimeter enclosing the projects to which I will refer to:

• Agile contracts
• Internal or external teams
• Exploration, execution, measurement

1. Agile contracts

In 2011, Michele signed the first agile contract by e-xtrategy, where the price of our work is measured and paid based on the accepted iterations, i.e. the group of user stories agreed, planned, released and accepted by the client during an iteration.
In 3 years I have had the chance to see and live how the team changes the approach to the project based on this type of contract. In this post I will refer to those projects where I am involved and that are ruled by an "agile" contract.

2. Internal teams and external teams

By internal teams I mean teams who have defined their relationship with the client in a way aligned to or dependent on ours and who use the same working method. With these teams we succeed in organizing iteration including both our design deliverables as well as the development ones.
By external teams I mean teams who have defined their relationship with the client in a way independent of us and who can use methods different from ours. With these teams we get in touch regularly to discuss doubts, next steps, design hypotheses, etc., but we do not know what they are doing, with which criteria, and when they will release their deliverables. In practice, in the projects that we follow with external teams, we can only track our iterations, i.e. those related to the ux design process.

3. Exploration, execution, measurement

Experience teaches us that, in every project, the first iterations are more explorative, while the following ones are more executive. But at each iteration (based on the “precept” and the contract) we should provide a deliverable as complete as possible.
When we work with what I have defined as “internal team”, it happens that almost always we succeed in planning an iteration in such a way that everybody has business value to deliver.
During the exploratory phase the team researches and surveys. For instance, designers could run qualitative and qualitative analyses, define players and scenarios, sketch and wireframe an epic story… While developers could setup and structure all working environments, services, evaluation of external systems with which integration will be needed etc. All of us (designers and devs) constantly exchange views with the client on the strategic goals of the project.
At the end of these activities, the team will write down the stories that will be released. Then it weighs them, i.e. it grades each story, so as to give the lower score to the less complex and risky story and the higher score to a very or too complex or risky story. So far we have used scores from 1 to 8, based on the Fibonacci scale (therefore 1, 2, 3, 5 and 8). Iteration after iteration, accepted story after accepted story, there is a number that starts to stand out, which in jargon is called velocity and it is calculated by making the average of the scores delivered by the accepted stories. Thanks to the velocity, we can monitor the project trend and predict the delivery of the future iterations.
The measurement phase is usually carried out at the beginning, during the exploration, in order to collect information, make decisions and define which are the metrics that we will be monitoring; and again towards the end of the executive phase, in order to set the metrics and define the next steps.
Let’s get to the point.

User stories, epics and their limits

Stories are one of the key tools of the methodology. They have the benefit of forcing the team to refer to the people to whom the digital product we are making is aimed.
For the time being, the most effective method that we have found in order to write stories is the following:

• We link the name of a project macro-epic to a deadline (for example: cart);
• We enter the first exploration activities to be carried out within the chosen deadline (for example: research with the users, various analyses, sketches and wireframes, hypotheses on data that we want to collect after the release);
• Once the macro-epic exploration activities are accepted, we write the execution stories and we schedule them within the first deadline.

All of this works out very well.
What does not work are the “tools” (stories, epics, chores, milestones, releases, …) that we are using in order to include the activities that we will be carrying out in the iteration, because:

• The stories, as they should be written based on the “precept”, can be referred only to the execution phase of the project, not the exploration neither the measurement ones;
• On some projects, we include users tests, metrics analyses, etc. in the workflow making up stories and chores on Pivotal, in other projects they are part of the project milestones. This constant lack of consistency is symptomatic.
• How can we describe, share and track moments of values delivery such as workshops introducing to the design vision?

A story written as follows:

“as a product owner I want to see some sketches explaining how the cart works”

in my opinion, it simply is not a story.

In a few words, the analysis and explorations phase of one or more epic stories cannot be represented by stories, unless forced. Moreover they move the focus to the deliverable and go through the approval of a sketch, a wireframe or a visual. Which reminds me too much of a “Seen it. Print it”.

What’s left?

We had the chance to test that stories written in this way do not work, by comparing two projects similar by methods and efforts. The only difference is how teams are involved: in one project the team is all internal, in the other one the internal team designs and codes the frontend while development is assigned to an external team.
In the first project, in spite of the inconsistencies between the exploratory and measurement stories versus the execution ones, iterations remain fairly balanced, due to the fact that development works in parallel to the design and the delivered value is measured in relation to the fulfillment of the user’s needs as well as in relation to the business goal.
On the contrary, in the second project, we have followed the approach to write the stories as if the development team were all internal, with the result that we could not figure out when the opened stories were ready for release. After the first 8, 9 iterations, we realized that velocity and tracking of delivered value were highly affected, making us formerly unable to close the iteration and issue an invoice (NB: we have not had any problem with the client, but that’s not the issue).
After a quick retrospective, we decided to change approach and use the stories in order to describe exploratory activities as previously discussed, therefore from the point of view of the product owner and not of the person who will be using the system. The result is that we end up with a project unbalanced towards the business goals and that, ironically, does not describe the users’ needs.

The egg or the chicken?

I cannot figure out if we have an issue with agile contracts and the bond of the stories approval; or if it is an issue of methodology and tools chosen to describe the value of the activities that we perform.
For me this is a tough dilemma on how to track design activities linked to an agile contract as those that I have explained at the beginning of the post. A dilemma that hides a nice opportunity.

How to trace the design’s business value?

The agile’s manifesto was created by engineers and computer experts. In spite of this, even now, when I read it once again, I find myself agreeing with the 4 simple principles on which it is based. But the tools and the methods that we are currently using are indeed engineer and computer-like. My conclusion is that we need to find a designer’s method to track the design’s business value. A method that would allow for releases by iteration, that could be described and tracked with tools similar to stories, but that cannot be stories.
Furthermore, I strongly suspect that the bond of payment by accepted iteration is not applicable to design, other than by stretching the tools that we are currently using.

I’d better stop here.