The forum also has the potential to make the ATO more accessible and responsive to the changing needs and demands of Australian taxpayers and demonstrates the ATO’s ability to innovate and to encourage voluntary participation within the tax and superannuation systems. The conversations on the forum are already influencing the language used and guidance provided on ATO’s digital channels.

It should be noted that the Assessment panel were brought in when the project was already in the Public Beta stage. To support the assessors, the team have provided detailed advice and evidence aligned to the Discovery and Alpha stages and has worked collaboratively with the Assessment panel throughout the development of this report.

The Assessment panel have awarded Pass for Beta, however recommends not proceeding to Live until:

• the automation of data transfer between the platform host and ATO systems has been enabled (criteria 4)
• the identified accessibility issues have been resolved (criteria 9)

Areas of good performance

The team have consistently demonstrated their ability to be flexible, adaptive and responsive within a changing environment. Their professionalism, commitment to success, responsiveness and determination to support innovation within the ATO is a credit to the organisation.

Additionally the team have consistently demonstrated their strong commitment to user centred design through significant user research, testing and iterative processes. The teams’ knowledge and expertise in communication and marketing has been of significant benefit to this initiative.

The team have placed a strong emphasis on ensuring the forum meets WCAG 2.0 AA accessibility requirements. Accessibility issues have been identified as a key barrier for certain user groups attempting to utilise other community forums already in the marketplace.

Criteria 1: Understand user needs 

In response to external research of our client’s Social Media experience with the ATO, the ATO undertook extensive research into online community forums. Research revealed users of these non-government hosted forums and sites were not able to substantiate the source or confirm if the information provided was accurate. An analysis of content on these non-government forums also revealed that up to 20% of community provided responses were incorrect or inaccurate.

Initial research also revealed the demographics of a community forum were difficult to define as they were dependent on the topic and forum. Ongoing research throughout the Alpha and Beta stages has helped the team to define the target user groups for an ATO hosted forum.

The team demonstrated a clear understanding of the issues and user needs, how these needs were met and how this research has influenced subsequent design iterations.  People from a range of backgrounds participated in usability testing and the team worked with Vision Australia to assist in this as an example.

The team continues to engage with users using the site intercept survey to understand the experience of the user through successful interactions, call deflection and satisfaction.  Going forward this will help to identify any pain points or areas where more formalised research may be required.

Criteria 2: Have a multidisciplinary team 

The project team consisted of team members within ATO and, as a result of purchasing a COTs product as a SaaS, vendor IT staff to provide technical advice specific to the COTs product. IT technical advice and support was also provided by ATO staff (e.g. architect) to ensure integration across ATO systems, data stores and cloud services.

General knowledge subject matter experts have been engaged throughout the project. Tax and superannuation subject matter experts have also been engaged on an as needs basis to review and/or respond to highly complex postings and questions. The team also engaged relevant Client experience experts throughout the process as soon as new user groups were identified. 

The team demonstrated their knowledge and understanding of the user experience and key stakeholders. Details of an induction pack were provided for new team members as they were engaged. This included team principles, practices and collaborative processes.
The team has built their knowledge of the product that supports the service, is capable of managing the product and will continue to have responsibility for the product as it transitions to business as usual.

Criteria 3: Agile and user-centred process

The project was initially run using waterfall methodologies. During the Alpha stage this approach was changed to use agile practices. 

The team were able to demonstrate their use of agile methodologies which included daily stand-ups, scrums and 2 week sprints. Agile tools and processes included online tools, a physical Kanban, retrospectives, showcases, backlogs, user stories and features.

The team were flexible in their approach to feedback and changing circumstances which in turn demonstrated their commitment, motivation and drive to achieve a positive outcome for the user and for the ATO.

Criteria 4: Understand tools and systems 

The team have demonstrated a very strong understanding of how to operate and use the platform chosen to host the community forum. They have also demonstrated a good understanding of how this integrates with other front end social media tools (e.g. Facebook, Twitter, etc.) and with other government agencies that use the same platform.  

The team demonstrated an understanding of procurement processes needed to engage the supplier. Part of the decision making process included consideration to the number of agencies currently using this tool within Federal Government.

The team were also able to demonstrate how this service integrates with other ATO systems, data stores and cloud services. 

The team confirmed that all data captured by this tool is owned by ATO. Additionally, all branding, images, etc. are also owned by ATO. Intellectual property around the actual platform and software is owned by the supplier.

Recommendation for go-live is to enable the automation of data transfer between the platform host and ATO systems.

Criteria 5: Make it secure 

As the platform is externally hosted, the technical aspects of IT security predominantly sit with the external provider.  The platform supplier has a range of IT security measures to support data security and privacy. ATO has conducted an assessment on the tool, data integration with ATO systems and relevant security measures to ensure compliance with ATO IT security and audit logging requirements. The recommendations resulting from this review have been implemented or are planned for future releases. ATO IT security have also highlighted where additional IT security engagement will be required based on backlog features. 

To support the privacy impact assessment, the team obtained legal advice to ensure interactions comply with legal requirements, including privacy laws. All legal recommendations have been implemented including having users agree to site terms of service and prompts throughout the site reminding users to not share sensitive information (e.g. TFN). 

The team have also implemented a risk mitigation plan to track and manage risk.  This includes the moderation, reviewing and modifying of responses to ensure users sensitive information is removed from the site as early as practical. 

Criteria 6: Consistent and responsive design 

The team demonstrated an iterative and responsive design process, showing how the project content and design elements were modified based on user feedback, and how they ensured alignment to whole-of-government and ATO style guide and branding requirements.

Plain English language has been used throughout the site. Additionally, processes have been incorporated that limit/reduce the chance of ATO responses being verbose, legalistic or use technical language. The team demonstrated how the responses are in line with the language used by contributors and align to WCAG 2.0.

The team have demonstrated how the platform can be accessed using multiple devices including mobile devices and also web browsers. This includes different operating systems and versions.

Criteria 7: Use open standards and common platforms

The use of common platforms by the team was demonstrated throughout the Alpha and Beta stages where high fidelity prototypes were created using open standards and common services. 

The project team looked at a number of open standard products and commercial off the shelf products (COTs products) to determine the appropriate platform. The platform chosen was an extension of an existing licence by the ATO and is used by several other Federal government agencies.

Criteria 8: Make source code open

As a commercial product has been chosen, the ability to make the source code open is restricted as per the licencing agreement.

Criteria 9: Make it accessible

The team have demonstrated the depth and breadth of testing undertaken including testing on multiple devices, browsers, users with differing levels of digital skills and ability, and cultural and linguistic backgrounds. Initial testing identified several areas where the platform did not meet WCAG 2.0 requirements. These recommendations have resulted in enhancements to the product and working with the platform supplier to ensure the forum meets WCAG 2.0 AA. 

The vendor has provided a plan to resolve most of these issues prior to go-live. This will be reassessed as part of the live assessment.

Criteria 10: Test the service

The team demonstrated how they conducted end-to-end user testing and have iterated the service throughout the Alpha and Beta stages using a variety of methodologies. This included testing internal and external users using scenario based tests, mock-ups, prototypes and in the Beta environment. The team demonstrated how they quickly responded to feedback throughout the Alpha and Beta stages and during testing cycles.

Scalability, load and performance testing has been conducted by the platform supplier. Performance testing once data transfer is automated will need to occur prior to going live. 

The team have outlined the business continuity plans and contingency plans in preparation for go-live. This includes roll-back options.

Criteria 11: Measure performance

The number of users or uptake continues to increase towards what is needed for the site to become self-sustainable. 

Due to the interactions that the community has with tax and super, monthly reporting does not accurately reflect ATO Community forum’s performance. For example, we would expect unique visitor numbers to spike post 1 July as this is a peak tax reporting cycle.

In addition to DTA’s KPI’s, the project team are capturing other metrics relevant to an information site and are also capturing metrics for annual cycle comparison.

Criteria 12: Don’t forget the non-digital experience 

The team have demonstrated their understanding of users’ digital and non-digital experiences for the primary and secondary user groups. They have also demonstrated how the service compliments existing digital and non-digital channels. The ability for this channel to seamlessly pass the user to other channels is beyond the capability of this initiative. Evidence has been provided on separate initiatives currently being investigated/delivered to support an integrated omni-channel experience.

A comprehensive communication and engagement strategy has been developed for go-live.  

Criteria 13: Encourage everyone to use the digital service 

The team have developed a comprehensive communication and engagement strategy targeting internal and external users. This strategy outlines how the ATO will support users moving to the digital channel and promotion of the service.

The team have provided details of measurement of success metrics and associated timeframes. They have also provided details of ongoing analysis that will occur post go-live to verify if the proposed retirement of an existing service can be achieved. 

Get involved with the ATO Community

Metrics

Community Forum Performance Dashboard