How user research changes during design and delivery
The kind of user research you do changes as you move through the service design and delivery process.
The needs and goals of the users define the service. You will uncover these needs and goals through your research in the Discovery stage and build on them through the other stages.
In Alpha and Beta stages you'll be testing prototypes, and in Live stage you will be researching to find out how you can improve the working service.
You will go from doing more generative research (making sure you are designing the right thing) in Discovery to more evaluative research (making sure you are designing it the right way) in the later stages.
In all stages:
- do research with a broad range of users, including those with access needs and low digital skills
- think about the whole user experience (including offline), not just the product you are delivering
Meeting the Digital Service Standard
You must do user research when designing and building your service to meet the following criteria:
- Criteria 1: Understand user needs
- Criteria 3: Agile and user-centred process
- Criteria 9: Make it accessible
- Criteria 10: Test the service
- Criteria 12: Don’t forget the non-digital experience
- Criteria 13: Encourage everyone to use the digital service
The Digital Service Standard guides teams to build services that are simpler, clearer and faster.
User research in Discovery stage
Before you start to design anything, find out about your users. This will help you to understand how the service you’re designing can meet their needs.
Watch how users do things now and what problems or barriers they face. This is called contextual qualitative research. Don't use market research for this: find the users' real problems by observation.
Interview and observe frontline service staff (if your agency has them).
This will help you to understand:
- limitations of the current service
- how staff explain some of the complexities
- undocumented workarounds that people use to get things done
Map the user journey
Create a user journey map of the experience of the people who want to do the task that brings them to your service.
Do research to understand the real user journey. It's more important than the journey the business would like to think users have.
Examine existing data
Look at existing data. This might include analytics, back-office workflow and support logs.
This can reveal new issues and quantitative evidence that supports qualitative insights.
Review previous user research
To avoid bias, review previous user research after your own primary research.
No matter how much research has been done before, the team needs to do its own research. This will help team members build a deeper understanding as well as empathy for the user and their needs.
What you'll find out in Discovery
From these activities you should get:
- an understanding of the problem your service is trying to solve for the user
- a list of users’ touchpoints as they try to achieve their goal — often expressed as a journey map, service map, mental model or alignment diagram
- sets of needs and task models for different types of users
- insights about users that you’ve gained from analysing the research
- a list of research gaps or limitations of the current research and opportunities for further research
- descriptions of user groups and the characteristics that are significant for this service
You’ll have done enough research when you understand:
- the different kinds of people who use your service
- what users needs from your service
- how people with support and access needs interact with your service
Combine profiles with real stories
Descriptions of users are often expressed as user profiles or personas.
We prefer to use profiles combined with actual stories from users the team has met. This tends to be more effective than developing personas. Empathy for an actual user is likely to be more authentic than empathy for a persona.
Keep on discovering
Discovery research doesn't just happen in the Discovery phase. Ongoing research helps teams to understand the evolving needs of their service.
This represents all of the empathy we built through Discovery with the real users. It’s a great way to introduce new people to the project. We used red post-its to flag pain points in user journeys and blue for where our research validated existing research.
— Service designer
User research in Alpha stage
In the Alpha stage, try lots of different approaches to meet user needs and find out which approach works best.
This is an important stage and you should make sure you have budget for it. The Alpha stage will help you reduce the risks around:
- design — making sure you get the right scope for the service
- business processes — finding out if government can build the service
- technical capability — being sure you can integrate the service and make it secure
Work out what works, not what's popular
Be prepared to park any ideas that do not meet user needs. You may also find that different elements of an idea works, sotake what is working and combine it with other ideas. Remember to document the different ideas you have tested to refer to later. This helps the team remember what has been tested and what has not.
Do task-based testing so that you can understand which version is most effective. We care about what works, not what people tell us they prefer.
You will know that something works best because people will find it easy to complete the tasks. They will understand what they are doing and won't make mistakes.
Keep iterating and testing
Once you have a prototype (even if it’s a paper one) you can start to test it with users. You can see whether your ideas will meet user needs and find out if the prototype is usable. You can then use this insight to iterate your design and test again with users.
Keep on interviewing users
You can combine testing your prototypes with user research interviews. This will help you to continue to deepen your understanding of user needs.
Park any ideas and prototypes if they don't meet user needs.
User research in Beta stage
There are two different research stages in Beta:
- researching as you build the beta service
- researching with users of a working beta service
1. Researching as you build the beta service
You can do a similar style of research as you were doing in Alpha as you build the beta service. But you are aiming to refine a solution for launch.
Do task-based usability testing with a wide range of users.
You should decide on hypotheses in your research. These are ideas about how you think the design will meet a user need.
Use a structured approach to evaluation. This will help you to be clear about what is working well and where there is more work to be done.
Make sure you do accessibility testing with people who have access needs. Commission an accessibility review before you put the service into a working beta.
2. Researching with users of a working beta service
Keep doing what you were doing before but add:
- analytics (including reporting KPIs to dashboard)
- interviews / shadowing of real users
- live intercepts
- A/B testing (comparing 2 versions of a web page to see which performs better) or multivariate testing — think about whether the sample size will be useful to test your beta service
- feedback from frontline and operational staff
- face-to-face and remote usability tests to find usability and accessibility issues
- an accessibility audit to uncover accessibility issues and get recommended fixes
- private or public beta tests of the end-to-end service with real users, including support options
- web analytics and back-office data to measure service performance
- surveys or follow-up interviews to collect detailed feedback from service users
What you'll find out researching in Beta
From these activities you’ll typically learn:
- more about how different kinds of users experience your services
- the usability and accessibility issues you need to fix
- ways to improve your service
User research in Live stage
When your service reaches the Live stage, research helps you to make sure it is still meeting user needs. It also helps you make decisions on improving the service.
You can learn more about your users’ needs through:
- reviewing web analytics and back-office data to measure service performance
- surveys or follow-up interviews to collect detailed feedback on your service
- interviews and visits to get a deeper understanding of any problems users tell you about
- face-to-face and remote usability tests to find issues with new or changed features
- A/B testing on new and changed features
What you'll find out researching in Live
These research activities will usually give you:
- deeper understanding of how different kinds of users experience your service
- insights into usability and accessibility issues and how to fix them
- ideas for ways to improve your service in future
How often to research in Live
The frequency and depth of your research once you reach Live stage depends on what you are trying to find out. For example, if you were looking to add a new component to your service, you would start Discovery again.
If you are going to decommission your service, find out how people use it first. This will help you to understand the gap that will be created when the service is gone.