Staged assessment process
All services within the scope of the Digital Service Standard (the Standard) will be assessed against the Standard. Depending on the service, this will follow either the staged assessment process or the in-flight assessment process.
The staged assessment process uses 3 assessments against the Digital Service Standard across the stages of the service design and delivery process.
- Alpha assessment takes place at the end of the Alpha stage. The service must pass criteria 1 to 3 of the Standard and show progress against other criteria to progress to the Beta stage.
- Beta assessment takes place during the Beta stage. The service must pass all criteria of the Standard before it can be released as a public beta.
- Live assessment takes place at the end of the Beta stage. The service must meet all criteria of the Standard before it can be released as a live service.
Each assessment is a conversation between the digital delivery team and a panel of assessors who are experts in their field. The team provides a brief overview of the service including a live demonstration. The assessors will then ask the team questions about how the service meets the Standard.
Assessments take from 2 to 4 hours, depending on the complexity of the service and the stage of development.
Determine if you can follow the staged assessment process
You can choose between an in-flight assessment or staged assessment if your service is:
- new or being redesigned and has / will have less than 50,000 transactions per year (low volume)
- an information service.
All DTA-led assessments on high volume transactional services will be in-flight assessments.
Plan for assessment
It’s essential to embed the Standard in the way you work from the beginning. It will help you to build a better service from the start. Tools are available to help you track how your service meets the Standard in a progressive way.
The DTA recommends that you share your progress against the Standard as part of your weekly showcase. Then when it comes time for an assessment you’ll have a good basis of artefacts and activities from which to draw upon to show how you meet the Standard.
In preparing for an assessment, the service manager needs to be confident that they can show how the service meets the Standard. The service manager should be able to satisfactorily answer the questions asked by the assessment panel.
If a service does not pass an assessment the service manager should seek a re-assessment. The service will be assessed against the criteria that were not passed. When the service is undergoing a re-assessment, the assessment panel will work through all previous recommendations. The service manager should ensure that all recommendations have been addressed before scheduling a re-assessment.
Before you arrange an assessment, it’s recommended that the team runs a mock assessment to identify any potential issues.
Arrange an assessment
When the service manager is satisfied they can demonstrate the service meets the Standard, they should contact their Digital Transformation Coordinator (DTC) to arrange an assessment panel and schedule an assessment date. The assessment panel should be independent of the digital delivery team.
The service manager will need to provide the DTC with a pre-assessment brief. The brief should include:
- description of the service, including
- the assessment stage - Alpha, Beta or Live
- who the users of the service are
- what user needs the service aims to meet
- the scope of the service for the assessment stage
- whether this service is intending to replace existing services or complement them
- a link to the service or prototype, including any information needed to access it; such as login details and dummy account data
- a high-level technical architecture diagram of the service, if appropriate.
The DTC should inform the DTA Standard team, by emailing email@example.com, of the planned assessment. The DTA will track all assessments and ensure the self-assessment team has the support it needs to conduct the assessment and report its outcome.
Attend the assessment
An assessment will consist of a discussion between the panel, the service manager and their team. The discussion is expected to take up to 4 hours.
The questions will be directed to the service manager, but the service manager can defer to other team members. Often these will include a technical architect, a designer, a user researcher or a delivery manager, but that’s up to you. The discussion should focus on what has been done, not what will be done.
Assessment conversations will generally follow a structure:
- Overview (5–15 minutes) – the service manager will explain
- what the service is
- reasons why it is being developed
- who the users are
- how it will meet their needs.
- Demonstration (20–45 minutes) – the service manager will demonstrate the service as close as possible to what will be made live, as appropriate to the stage. The assessors will ask questions during the demonstration.
- Review (up to 3 hours) – the assessors will go through the Standard focusing on how the service has met each criterion.
Assessment report is published
The service manager and the DTC will receive the outcome of the assessment (pass or not pass) and a report summarising the assessment within a few working days. The report will include recommendations with concise feedback on the service. The DTC will provide a copy of the report to the DTA to publish in the assessment reports section of the DTA site.
Follow the recommendations
There are 2 possible outcomes from an assessment: pass or not pass.
Passing an assessment means the service team can continue to progress through the stages of the service design and delivery process. For example, when a service passes a Beta assessment, the service can go public with Beta branding applied. When the service passes a Live assessment the Beta branding will be removed.
If a service does not pass an assessment the service should not progress beyond its current stage of development. To assist service teams, the assessment panel’s report will provide recommendations on the actions the service manager might undertake to ensure the service will meet the Standard.
Depending on the stage of development, the service manager should arrange a re-assessment. The re-assessment is only required for those criteria that were not passed.
If the service does not pass an assessment twice the DTC is encouraged to seek assistance from the DTA.
Last updated: 5 May 2016