Prototyping in Agile Environments

In my blog post a couple of days ago I wrote about prototyping and technical debt. After some discussions with my colleagues about the value of prototyping I want to refine my statements a bit further.

What is a prototype?

First of all, what is a prototype? Wikipedia states "A prototype is an early sample, model or release of a product built to test a concept or process or to act as a thing to be replicated or learned from". There are several types of prototypes which basically breaks down to the two dimensions:

• Horizontal vs. Vertical Prototype
• Evolutionary vs. Throwaway Prototype

 To help understand the relations and implications of the various kinds of prototypes, have a look at the following diagram. (The diagram assumes, that front-end (UI) aspects are on the upper side while backend aspects (logic, processing, persistence, etc) are on the lower side and time flows from left to right.)

Basically, a prototype should answer questions to the solution to be built. Therefore, an intrinsic value of a prototype is to help reduce a kind of risk by giving those answers.

Horizontal and Vertical Prototypes

A horizontal prototype may help to reduce functional risk, so the customer can validate if the solution to be developed meets her requirements. A vertical prototype may help to reduce technical risks as it proves that certain architectural or technical decisions are viable. A prototype may also be combination horizontal and vertical aspects, but typically should focus only one of these, for example verifying if certain UI components can be integrated together in the UI framework focuses more on vertical than horizontal aspects.

Nevertheless, in the lean world, most types of a prototype are waste (throwaway prototypes) or potential waste (horizontal, evolutionary prototypes). In order to reduce waste, efforts of producing a prototype that will not become part of the solution should be minimized. The prototype should focus only on reducing risks and giving answers to a predefined set of questions. In other words it should have a narrow and defined goal.

Technical Debt and Technical Credit 

Two aspects of prototyping in agile environments are technical debt and technical credit.  The latter being a concept you do not read often in the agile literature, but which exists if you think the concept of technical debt to the end.

The following diagram depicts the concept of technical credit and is a variation of the first diagram.

It differs from the first one in three areas:

• a vertical throwaway prototypes does not necessarily have to start at the front-end. It could actually cover any part of the technical stack.
• a horizontal evolutionary prototype may also cover other layers of the software, not only UI, and therefore may produce "Technical Credit" as the opposite of technical debt. 

Technical Debt

As it is obvious, a horizontal prototype that is part of the solution produces a lot of technical debt in the first place, especially if the prototype targets at the UI. The customer perceives the solution as finished, but it is dysfunctional from a technical perspective, and most of the feature promises are broken until the technical debt is worked off.

Technical Credit

The opposite of technical debt is technical credit. Technical credit is generated, when something is developed and delivered in advance, that does not provide any immediate and perceivable business value. An example could be an architecture refactoring that provides a simple extension mechanism so that business functionalities can be developed more rapidly. This could cost more in advance than is perceived as returned business value but could cost less afterwards compared with the business values additional functionalities provide. But still there is a great risk of producing waste, for example if the requirements change.

Functional Debt and Functional Credit

The other dimension of debt and credit is the functional. A project starts with a certain amount of functional debt - the product backlog, which is worked off continuously. Further, the requirements engineering discipline refers to the Kano model of success factors:

• Basic or Threshold Attributes - if they are missing, the solution is not accepted at all
• Performance Attributes - some attributes could be missing, but the more are implemented the better
• Excitement Attributes - the customer does not even know they want those. They could compensate for missing performance attributes and may help to exceed the customer expectations (making her a more happy customer).

Functional Debt

Functional debt are features that are implicitly or explicitly (Basic and Performance Attributes) required but are not yet delivered. Working off the functional debt of the backlog is exactly the concept of a vertical, evolutionary prototype, and therefore an intrinsic mitigation strategy in agile processes to reduce functional risks of overall project and maximize the work not done according to the principles of the agile manifesto.

Functional Credit

Functional credit on the other hand is generated, when Excitement Attributes are delivered. Functional credit may help to compensate for functional debt (missing Performance Attributes) or even more help to exceed the customer expectation by raising your functional balance above 0.

Nevertheless, concentrating on generating functional credit - in the meaning of developing something that is not implicitly or explicitly required - poses the risk of producing waste. For example, you cannot guarantee for sure the developed feature is an Excitement Attribute. The customer may even get upset that you do not provide what has was agreed on, but still it could be a risk worth to be taken - depends on the situation.

Theory applied

How will all this theory impact our daily work? Basically, it should help you to make informed decisions and know of the implications of using one concept to your project.

The following diagram depicts a possible situation in a project where the above discussed aspects have been taken into consideration.

The diagram shows:

• a limited horizontal evolutionary prototype with a limited scope that anticipates what has to be developed in the next iteration. This could support marketing operations by showing what the customer could expect soon and prove that it's vaporware.
• a lightweight horizontal throwaway prototype (the bar smaller than in the diagrams above) to support the vision (using a wireframe) that goes beyond the evolutionary horizontal prototype to mitigate functional risks
• maximized amount of work not done
• several proof-of-concept lightweight vertical throwaway prototypes (bars are more narrow) that cover combined the entire technical stack to mitigate technical risks.

Further, the diagram shows several iteration increments with growing perceivable value but also with growing technical debt (which might occur in every project). The next diagram depict how the iteration scope may have looked like (yellow), how it should look in an ideal world (green) and how it should look to reduce the technical debt again (red). The latter effectively depicts to the approach of producing only something of little business value but with a big (estimated) effort.

Of course it's better to avoid technical debt in the first place, but when you have created it you have to deal with it. The Disciplined Agile Delivery framework lists 11 strategies of dealing with technical debt.

Concluding Thoughts

There might be occasions where any kind of prototype has it's value and helps the project. But if you choose a certain type of prototyping approach in your project, keep the implications in mind. Always try to reduce waste and technical debt and maximize the work not done, while minimizing the risks.

Good practices for prototyping should include:

• Reduce functional risk without producing technical debt with a wireframe mockup (horizontal, throwaway prototype)
• Reduce high technical risks with a limited proof-of-concept throwaway prototype (horizontal or vertical). Define a set of questions to be answered by the prototype to limit the production of waste. (i.e. "is the architecture feasible and matches our criteria?", "does the framework to be used matches our needs?", "are the performance criteria achievable?", "does the solution integrate into the environment?" etc.). Such a prototype is always a tradeoff between waste and risk reduction and requires thorough considerations.
• Avoid producing a horizontal, evolutionary prototype. But if you do it, limit it's scope to limit production of technical waste. Anticipate only a small and manageable number of iterations (i.e. 1-3) and define a strategy along with the prototype creation how to work off the technical debt. Make it transparent.
• Do vertical, evolutionary prototyping whenever possible.
• Have a strategy to deal with technical debt.

tl;dr
The world of prototyping is not black and white, it all depends on the situation.

Social Bookmarking

Over the holiday season I sought for a solution to keep my bookmarks between my private and my working computer in sync. Although Firefox comes with a synchronization feature, I aimed more to a solution based on social bookmarking. Back in the days when working for IBM Software Services for Lotus I used social bookmarking on the internal platform - guess what - Lotus Connections (now IBM Connections), which had a good browser integration (an IBM internally available extension, I don't know if that's now part of the product or a free or paid asset). However, the combination of persisting my bookmarks and sharing my (tagged) links with the community was a great added value to my work as it significantly decreased the time to find specific information, especially in the field I was working in and in the network of colleagues that worked in the same field.
So the requirements for my sought-for social bookmarking solution were set:

• Browser integration being a substitute for Browser Bookmarks
• Synchronization between different working stations
• Support for Tagging, to ease search for information 
• Possibility to share links with colleagues and friends
• Possibility to have private links

I remembered, that there had been a social bookmarking site, if not THE social bookmarking site, called del.icio.us that had a good browser integration. So I started using it. But my experience with it was not that good and I guess it has its origin in the history of the site. Delicious was bought by yahoo, went through years of stagnation before it was sold again to AVOS, the company of the founders of youtube. The browser integration developed by yahoo doesn't seem well supported anymore (or at the moment?), especially the synchronization between browser and website brakes frequently and you have to sign out, delete the local bookmarks and sign in again (the sign-in of the browser extension is done via the website previous.delicious.com, that says everything). So I ended multiple times with editing bookmarks in the extension that were not persisted on the website or editing bookmarks on the websites that were not only not synchronized to the browser, but overwritten again with the data from the browser. On top of that, I don't find the web UI not very user friendly, it's overly simplistic and not very intuitive. So I sought for an alternative.

I took StumbleUpon into consideration, but it's poor browser integration (incompatible with NoScript plugin) was a no-go. Further it focuses more on news than knowledge sharing and has no concept of private links - everything is public. Though I find the idea behind stumble upon generally intriguing, it's not what I was looking for.
Google Bookmarks fell short a good browser integration and Evernote although allowing to capture merely everything is not really a social bookmarking site and therefore lacks some basic functionality.

After reading the good blog post regarding "10 Alternatives to Delicious.com" I tried out Diigo. And I must admit, that's exactly what I was looking for, even more. It comes with a browser plugin that allows to easily capture new bookmarks and also display your bookmarks in the bookmarks toolbar based on custom filters on the captured bookmarks. Bookmarks can also be organized into lists that can be filtered too. Further, the plugin has a sidebar to quickly search all your captured bookmarks. For me Diigo is a complete replacement for the browser bookmark management and helps me easily to keep in sync with different computer, including smart phones, as it also provides a neat app.
But Diigo is not only a social bookmarking tool. It goes well beyond that, its a knowledge working tool. With Diigo you can add sticky notes to web pages, highlight passages, comment on them, capture images, capture parts of a web page as image and share everything of that with the public or a group of people, and of course keep them private as well. So whenever I capture a bookmark, I can also highlight what was important for me from that page or why I looked up a certain information. When working in a group it helps, sharing certain information with each other.

There may be other alternatives in the web as well, but having found Diigo, I found a solution that works for me, so I stick with that.

tl;dr
Want to do social bookmarking and looking for an alternative to delicious? Try Diigo. It's a knowledge working tool.

Technical Debt and Prototyping

Currently I am preparing for my IREB Requirements Engineering Certification and I read (and already knew before) that in Requirements Engineering creating a prototype is a method and tool to validate requirements.
A prototype should always answer a beforehand defined set of questions. Further, there are two basic types of prototypes.

1. Throwaway Prototype - the prototype is not used further in the development process apart from being a technical example.
2. Evolutionary Prototype - the prototype itself is continuously developed further until it becomes the final (or a shippable) product.

The question is, what role does prototyping play in agile development?

Let's first have a look at the dimensions of a prototype. The wikipedia article about prototyping describes the two dimensions of a prototype as

• vertical (in german also known as "Durchstichprototyp"), in the meaning of a slice of the system to be developed, including frontend, business logic and persistent.
• horizontal, in the meaning of an almost complete implementation of a single layer of a multi-layered architecture. I guess, the most common example of a horizontal prototype is a UI prototype. Also common are interface stubs.

I would assume, that vertical prototypes are closer to the principles of agile development, as they are more "something that is done" than a horizontal prototype.

Why is that?

In agile the concept of "technical debt" is known as a measure of design or software quality, or better, the lack of it. There are some strategies to deal with it, as described in the DAD's "11 Strategies to deal with technical debt".  Further, agile or scrum, aims at producing increments that provide value to the customer, in most of the cases, the "value" reflects some additions or changes to the UI, that the customer can validate relatively easy.

A vertical prototype is a potentially shippable or consumable piece of software in the meaning of agile. It is functional and it is done. Maybe it needs some refactoring, but basically the amount of technical debt is relatively low. One could even say, that producing an evolutionary, vertical prototype is doing agile (if done right).

On the opposite, a horizontal prototype produces only an artifact on one single layer of the software, which is in most cases the UI. The customer can validate, if the prototype reflects the requirements. But, the prototype itself provides no real value, as it is a dysfunctional piece of software, as all the stuff behind the UI is not implemented. In other words, a horizontal UI prototype is a huge pile of technical debt. The more complete the prototype is, the higher is the debt.
If a evolutionary, horizontal prototype is the starting point of the implementation (i.e. the product of a pre-project or the first iterations), it is relatively difficult to plan backlog entries for the further iterations with increments that "provide value" from a customer perspective. Ways to deal with it could be:

• deliver only small increments that add value (i.e. slight adjustments to the UI) but that take more time than one would assume they would take. For example changing the background color takes a week. 
• define the disabling of elements of the UI as added value and focus on only one element that is actually implemented, and the continue as ususal. This means actually to make the evolutionary prototype a throwaway or least major parts of it.
• call a spade a spade and track it as chores that provide no actual value but a reduction of the technical debt and the stakeholders have to bite the bullet for getting several iterations with no added value

My suggestion would be to use either horizontal throwaway or vertical evolutionary prototypes.

The throwaway protoype - especially UI prototypes - could enrich the requirements descriptions as it provides a clearer vision, how the system should look like and partially how it behaves. Creating this prototype should be the job of the product owner or her assistant's. Wireframe models are a good example of such a prototype. The prototype can be decomposed into smaller user stories or tasks the development team implements, while keeping the entire prototype as a detailed version of a vision.

The evolutionary prototype is actually developed during iterations of the development process and it should always be of the vertical dimension which ensures, that every iterations produces something that is done.

Concluding, I would make the three suggestions:

• if using a prototype as part of the requirements definition (job of the PO), use a horizontal throwaway prototype. I'd recommend a wireframe model.
• creating a vertical evolutionary prototype is developing the product in an agile or at least iterative way
• avoid creating horizontal evolutionary or vertical throwaway prototypes as they produce high technical debt or are unfinished work, which is bad.

tl;dr

	horizontal prototype	vertical prototype
throwaway prototype	good done by PO as part of requirements	bad, use only to demonstrate general feasibility to reduce high risks (proof-of-concept), produces waste
evolutionary prototype	bad, produces high technical debt, avoid if possible	good, actual potentially shippable increment, done by development team