Alpha stage: testing hypotheses

Alpha is an experimental stage. It’s an opportunity to use prototypes to work out the right thing to build.

In Alpha stage you test the hypotheses that you reached at the end of Discovery stage. As you progress through Alpha, you will produce new hypotheses as you discover new things about the users and service.

You’re not validating what users like or dislike. You are finding out how well prototypes meet the actual needs of users.

Meeting the Digital Service Standard

You must follow the service design and delivery process to meet Criteria 3: Agile and user-centred process of the Digital Service Standard.

The Digital Service Standard guides teams to build services that are simpler, clearer and faster.

The team you need in Alpha stage

The same team that worked in the Discovery stage should continue to work in Alpha stage. This keeps the existing empathy, context and momentum.

You may need to add some other roles and capability in Alpha stage:

• business analyst
• technical architect
• delivery manager or similar (for example, scrum master).

What to do in Alpha stage

At the start of Alpha stage, you should have a good understanding of the user journey and know what the major stages of the user journey are.

The team will keep improving the user journey map as their understanding of the journey grows. You should work in an agile way, iterating regularly so that you can learn quickly.

When the team has some certainty about the journey of users, create a user story map. This will help you define the minimum viable product (MVP).

In Alpha stage, you:

• take your hypotheses from Discovery stage and create some high-level concepts
• continue to develop a vision for the service using your user journey and user story map
• create storyboards to see possible solutions
• create prototypes to test hypotheses with users — you should explore hypotheses with different kinds of prototypes, including paper and HTML
• complete the stage by defining the minimum viable product

Keep the decision register

The decision register that you set up in Discovery becomes even more important in Alpha. It contains the evidence that guides what you will build, including:

• an explanation of the problem
• empathy map
• insight map
• results of affinity mapping
• pain points
• hypotheses
• concepts for solutions
• decisions about changes in direction or why work on some solutions was stopped
• user feedback and actions taken

The register allows you to trace why you decided on a particular solution. It should be saved and stored appropriately within your agencies archives so that when Alpha is over, it is discoverable to others in your agency and can be used for future reference.

Identify service performance metrics

In Alpha stage you will need to decide which metrics you use to track performance so that you can meet Criteria 11 of the Service Standard; Measuring Performance.

To do this you should:

• have an analyst as part of your team (or available to your team) so you can start working out how you’re going to measure service performance
• use your hypotheses to help you define your goals and benefits
• develop a measurement plan that starts to define what metrics your service will report on
• clearly define the start and end points for your service
• engage with stakeholders to raise their awareness of your measurement plan and to decide the process for signing off and publishing the metrics.

Activities for each sprint

Each sprint, the team should:

• hold a short planning meeting to prioritise user stories for the sprint
• conduct user research with at least 5 to 6 users who are representative of actual users of the service
• capture learnings and insights from the research findings
• report back on the prototypes and user research to stakeholders in a showcase or shareback
• have a team retrospective to reflect on the sprint and capture improvements

This aligns with Scrum methodology and helps keep the team focused on delivering something of value to users every sprint.

Test with different kinds of prototypes

The main activity in Alpha stage is to test the hypotheses with real users. The best way to do this is by building and testing prototypes.

Alpha prototypes are like a proof of concept. When you want to know how people will interact with a service, you test how people will interact with the service. It would be wasteful and pointless to produce a finished product for users to test.

A prototype should not be functionally complete, have full end-to-end transactions or integrate with any working back-end systems. They exist to help you to test your understanding of the service. They will show if you have included the most important and meaningful steps the users take when going through the service.

Test with users using paper or other low-fidelity prototypes first. This allows you to test assumptions and ideas quickly and cheaply before the team gets attached to any solutions. 

The team tests which of these prototypes (if any) are best to carry forward. Once tested, they are iterated based on feedback and tested again.

As you learn from your user research, you can work up to testing with more interactive, higher fidelity prototypes.

What prototypes need

A good prototype has enough information for users to interact with and can be quickly adapted or thrown out.

Don’t spend too much time building the prototypes, as you will keep throwing them away and building new ones based on user feedback.

It’s completely fine (and strongly recommended) to throw away the prototypes you build. You may choose to re-use some design patterns or components when you build in Beta stage, but never confuse a prototype with a minimum viable product.

Always remember what you’re testing for and who you’re designing for.

Low fidelity prototypes

Some examples of low-fidelity prototypes include:

• storyboards
• card sorting.

Low fidelity prototypes are an excellent first step as they are inexpensive, can be quickly updated or changed, and are disposable.

High fidelity prototypes

There are many tools on the market to help build interactive, high fidelity prototypes. These allow you to sketch in code, using HTML, CSS and JavaScript. Software like Axure, Omnigraffle or Balsamiq can be hard to use in a fully multidisciplinary working environment.

Others work like web-builder apps and allow a person to drag and drop buttons, images and sequence out a user experience path. Some teams have even used slide decks with basic animation features to simulate an online service.

Whatever you choose to build an interactive prototype, remember the following:

• show a seamless user experience along the ‘happy path’
• include content and data that looks real
• respond with alerts and feedback in the right places
• make it look and feel like a real digital service
• use familiar design patterns — such as the Australian Government Design System or New South Wales Government Design System.

There are some known downsides to high fidelity prototypes:

• users are more inclined to comment on superficial parts of a service (colour, design, layout) rather than the content (Rogers, Preece, and Sharp, 2011)
• they take longer to make and to update than low-fidelity prototypes
• they may give users a false idea of how the final service will look or perform.

For these reasons high fidelity prototypes are usually introduced once low fidelity tests have been explored and the team has a stronger idea of how a service will work.

Read on for more advice on high fidelity and interactive prototypes.

Tools and technology for interactive prototypes

Technology should not be a blocker in prototyping. Choose tools for quick experimentation and rapid validation of the hypothesis.

If you decide to include some interactivity to simulate a working service (for example, capturing user details on a form) don’t use server-side technology for this functionality (for example, Java, .NET or Ruby). You want to keep the technology simple so it’s quick and easy to update your prototypes.

Use client-side technologies (for example, Web Storage API) combined with a JavaScript framework (for example, jQuery) to quickly store and retrieve information for the user’s session. Make it easy to reset and start again at the end of a user research session.

Use a static-site generator

You can use a static-site generator (for example, Jekyll).

A static-site generator allows you to keep your prototype simple, with just HTML, CSS and JavaScript, but allows you to easily share templates and style sheets across multiple pages.

It is also easy for designers to make changes directly to the prototype, without having to wait on a developer to make changes.

Use a version control system, like GitHub, to store the code for your prototype so that everyone in the team can work across the same copy and collaborate on changes.

Deploy the prototype to a cloud service so that you can easily share it around the team and with your stakeholders (for example, Amazon Web Services).

If you configure the service to automatically deploy whenever changes are made (sometimes called ‘creating a build pipeline’), then everyone will always be able to access the most up-to-date version.

Don’t support every browser in the prototypes

Users are likely to be using the prototype in a controlled environment, such as a user research lab or on a team member’s computer. This means that you need to support fewer browsers than a public website.

Support the latest version of each major browser (Chrome, Firefox, IE and Safari). Support a specific browser or version if a specific group of users or stakeholders depend on it.

Use semantic and accessible markup across prototypes.

As part of the user research test your prototypes with users who have accessibility needs. Test with screen reader software and other assistive technology.

Mobile first

More users are accessing government services using mobile devices than ever before. If you design and build for mobile first, you can test prototypes in a more realistic context.

To design for mobile you should make a simple screen, with only one or 2 things on each page. This means that users with larger screens will also have an easier experience.

Define the minimum viable product

By the end of Alpha, you will have a developed the user story map in depth.

The team should agree on a line across the user story map. What is above the line should reflect what is most important for the user and what the team can achieve. This will inform your minimum viable product (MVP). This is the simplest thing you can build that meets the user need.

The MVP will set out the scope of what is built in Beta stage. The research and testing you have been doing in Discovery and Alpha will help you make choices about the technology you will use to build it.

Caption: Everything above the dotted line will be in the minimum viable product.

How you know Alpha stage is finished

At the end of the Alpha stage, you should have:

• user stories — they should relate to the overall user journey rather than being tied to individual features
• analysis on the user needs research you have done
• a plan for your Beta stage and a less detailed plan for your Live stage
• a decision on the metrics you will use and a measurement plan for how to capture them
• a basic working system with limited functionality that you can demonstrate to users
• an understanding of legacy systems you need to replace or integrate with
• a decision on whether to progress to Beta stage
• a decision on your minimum viable product.

Moving on to Beta stage

Your team should feel confident that you are proposing a service that will work in Beta stage. Make sure your timelines, scope and vision match the budget, team, and resources you have for the next stage.

If you need a business proposal for Beta stage start on this as early as possible. You can start it as soon as the team is confident it can and should move to Beta. This gives you time to get approvals and avoids the team losing momentum.

If you think you may need a new domain name, make sure you understand the gov.au domain name policies and processes.

Hold a final demonstration of your Alpha stage prototype with the leaders for your service. Use the demo to show what you have achieved in Alpha stage and explain your vision for the Beta stage.

Hold a final retrospective and make a backlog of epic user stories for the Beta stage.

Deciding not to move on to Beta stage

You may decide to end your service work in Alpha stage rather than continue into Beta stage.

For example, you may find that:

• user needs for the service are already being met by another government service or the private sector
• it costs too much to build the service based on the number of people who will use it
• there’s a better non-digital solution
• adapting or developing another service is a better solution.

It’s still a successful Alpha stage if you decide not to move on to Beta because it means you have learned more about the problem, and you won’t waste time and money.

Deciding to start a new Alpha stage

At the end of Alpha, you may decide to start a new Alpha stage or even Discovery stage because you work out you need to focus on different things. For example, you might find a completely different user need that you want to explore through further research.