Trade Mark Assist — Live assessment
Trade Mark Assist is an interactive tool, using machine learning algorithms, designed to educate and assist self-filers, in particular small and medium enterprises through the initial stages of the trade mark application process.
Trade Mark Assist has demonstrated sufficient evidence that the service reached a high level of quality in accordance with the Digital Service Standard. Given the body of evidence, and the expected trajectory of the work, it is the assessment team’s opinion that the service is ready to go live.
Not withholding areas of good performance, improvements have been identified collaboratively with the delivery team and included here as recommendations for ongoing consideration and actioning.
This assessment leveraged the comprehensive work from previous assessments, particularly the beta gateway, in gaining understanding and building a baseline for existing Digital Service Standard adherence levels. As a result, emphasis was given to understanding improvements and changes in the various criteria since the beta assessment.
Being a live assessment, the key question we set out to address is whether the service is ready to go live. As such, we directed many of our questions to understand not only the shape of the service at this stage, but expected trajectory post go-live, especially in the context of shifting from a development mindset to an operational mindset.
Criterion 1: Understand user needs
The team demonstrated a clear, high priority user pain point in the current trade mark application process – the ability for users to accurately choose the applicable goods and services for their trade mark. The team also demonstrated a user-centred approach to prioritisation and improvements.
Evidence shows good coverage of user demographics including trade mark domain knowledge, age, location, company size, accessibility, cultural diversity, literacy (ICT and general) and remoteness. Over 350 respondents have been involved in user research activities. Functionality has been implemented in the beta to allow users to provide feedback on the service. This feedback will be used to validate the service against user needs and to support ongoing enhancements.
Following on items from the previous assessment, the team demonstrated that they continue to use user needs to guide improvements: following user feedback, a new integration channel was developed with particular attention given to user experience which was identified through analysing user feedback as an area of weakness. Their success in understanding user needs is reflected in the reported high user satisfaction and positive feedback.
Recommendations for criterion 1
Ensure transition to operations (post go-live) includes handing over ongoing monitoring of user feedback and website analytics measuring user adoption and use
Criterion 2: Have a multidisciplinary team
The team structure remained consistent since Beta. Post go-live, the current development team will transition its responsibilities to the operational Digital Futures team. This team provides product ownership for IP Australia’s customer facing systems and has existing knowledge and understanding of the users’ needs. The teams are co-located which will aid in the handover. The team has considered and planned for the transition including handing over iterative delivery processes.
Following on items from the previous assessment, the team included strong representation from the business area that the service is targeting and it is clear how the new service will be incorporated in the business area.
Recommendations for criterion 2
Ensure the operations team (post go-live) has capacity to cover all disciplines existing in the current team.
Criterion 3: Agile and user-centred process
The team is continuing to make good use of agile tools and techniques. The team will leverage the IP Australia Automated Decision Making Framework in establishing governance for the new service.
Criterion 4: Understand tools and systems
Following on items from the previous assessment, the team has secured senior sponsorship to fund long-term contracts with the technology vendor, reducing operation risk post go-live. Sponsor also contributes to funds allocation by the organisation for post go-live costs of the operational team.
Recommendations for criterion 4:
Consider completing a decommissioning plan for existing systems (including technologies) that the new service could take over.
The team has plans to expose management of rules through API in order to reduce maintenance costs and efforts for their service over its lifespan. In this context, consider reducing technical debt associated with the management of rules by adopting an API based corporate rules service.
Criterion 5: Make it secure
All the data accessible via TM Assist is publicly available and there is no confidential data stored in the system. Security measures were developed in light of this understanding.
The service considered security controls from the get-go incorporating a security advisor from the ICT area in the design and development phases of the product to ensure alignment with best practice. The service leverages AWS services and security controls have been applied in accordance with IPA’s security policies.
Security consideration also includes iterative patching & upgrading for the underlying technology, in accordance with the product’s Agile methods.
A small number of issues raised at penetration testing for the production version have been addressed to the satisfaction of the IP Australia Information Technology Security Advisor.
Criterion 6: Consistent and responsive design
Content developed by the team is reviewed by all relevant areas of IP Australia. Content was developed in accordance to accessibility standard and complies with WCAG 2.0 AA.
Following on items from the previous assessment, the team applied a literacy checker tool to its developed content assisting in achieving a “green light” Grade 8 literacy level.
Criterion 7: Use open standards and common platforms
The team followed industry good practice with regards to leveraging a range of open platforms.
All components used for the development and implementation of Trade Mark Assist are available under permissive (i.e. Apache) licenses.
Criterion 8: Make source code open
The team has chosen a source code repository with wide adoption in the open source community – GitHub. Access to the repository is limited to the team due to sensitive business information and rules captured as code which are used to train the ML components of the solution. As a result, the team has made a decision to expose the machine learning components of the solution to third party developers through APIs. Efforts to accomplish this are being planned for future iterations of the service.
Criterion 9: Make it accessible
The team engaged an external party to undertake independent accessibility audits for compliance with Web Content Accessibility Guidelines 2.0 (WCAG 2.0). The service has been directly tested by multiple users requiring assistance to interact digitally.
Recommendations for criterion 9
Consider including events, thresholds or situation in which changes to the current service would require completing a new accessibility report. This would assist the operations teams post go-live in identifying potential work implications should a change be needed that may impact existing accessibility levels.
Criterion 10: Test the service
In following recommendations from previous assessment, the service now includes increased volumes of automated testing to the extent they can be developed. Due to the nature of the service which makes use of ML components, comprehensive end-to-end automated tests are not always practical.
Recommendations for criterion 10
It is noted that options for automated testing of machine learning technology is currently immature/limited. It is therefore recommended to enable a monitoring process to periodically evaluate whether this area has reached sufficient maturity to benefit the service.
Criterion 11: Measure performance
The team has integrated a web analytics framework into their service, to assist in monitoring usage. KPIs reported as at the time of the assessment include user satisfaction, digital take-up, completion rate and cost per transaction.
Recommendations for criterion 11
Consider including threshold triggering a service “kill switch”, i.e., a threshold in which we would consider the service to not provide sufficient value and at which point it should be considered for decommissioning.
Criterion 12: Don’t forget non-digital experience
The team has worked with IP Australia Call Centre to ensure that they are ready to support users who need assistance to interact digitally. The team has liaised with the Call Centre to ensure that sufficient information is available for non-digital users who may prefer to interact with IP Australia over the phone. This provides a mechanism for non-digital users to obtain Trade Mark Assist insights via the Call Centre.
Criterion 13: Encourage everyone to use the digital service
The service is being promoted in accordance to a plan, through Industry events, new articles, product demonstrations and IP Australia’s website.
Recommendations for criterion 13
The existing promotion plan is of high quality emphasising service introduction. It is recommended to also consider long term promotion and marketing activities post initial service penetration.