Trade Mark Assist — Beta assessment
Trade Mark Assist is an interactive tool, using machine learning algorithms, designed to educate and assist self-filers, in particular small and medium enterprises through the initial stages of the trade mark application process.
Self-filers are trade mark applicants without legal or trade mark attorney representation. The tool provides information that is tailored to a user’s circumstances and proposed trade mark. The Trade Mark Assist tool will benefit users by guiding them to make better decisions early in the application process, reduce complexity, reduce common applicant errors and improve the quality of trade mark applications.
The two primary capabilities this service aims to deliver are:
- improved Goods and Services (G&S) classification – a smart search and suggest function to improve the accuracy of G&S selected and reduce issues related to G&S classification
- fact checking – incorporating pre-defined business rule tests that are based on the Trade Mark Act 1995 as well as basic mandatory fields tests. The observations of the tests will be provided in real time to educate users before completing and submitting an application (reducing the likelihood of an adverse response).
Overall, Trade Mark Assist is progressing well and evidence shows there is reasonable justification and detail to move to a public Beta release for user feedback and further iteration as necessary.
Criterion 1: Understand user needs
The team demonstrated a clear, high priority user pain point in the current trade mark application process – the ability for users to accurately choose the applicable goods and services for their trade mark.
Evidence shows good coverage of user demographics including trade mark domain knowledge, age, location, company size, accessibility, cultural diversity, literacy (ICT and general) and remoteness. Over 350 respondents have been involved in user research activities. Functionality has been implemented in the beta to allow users to provide feedback on the service. This feedback will be used to validate the service against user needs and to support ongoing enhancements.
Recommendations for criterion 1
Ongoing validation of the service against user needs
Criterion 2: Have a multidisciplinary team
The core delivery team comprised a good mix of skills, with a product manager empowered to make majority of key decisions. Skill sets are shared across a number of resources, reducing the impact of membership changes.
The vendor delivery team is not co-located with the core delivery team. Although this poses some challenges in collaboration, the team are using online collaboration platforms to effectively manage communication and delivery across the two locations.
User research and usability testing activities have been overseen by an experienced researcher (product manager) and conducted by third-party consultancies. Team members demonstrated strong user empathy, based on research observation and active engagement with feedback channels.
Recommendations for criterion 2
The core team members remain throughout service delivery — that is Beta to Live. This will help keep organisational knowledge and research findings with the service.
Criterion 3: Agile and user-centred process
The team is making good use of agile tools and techniques.
Services must continue to improve and adapt to changing user needs and feedback. The team demonstrated their ability to quickly deploy new functionality to the service during Alpha and Beta. Once the service is fully integrated into the trade mark application process, enterprise release processes may impact deployment frequency. The team should collaborate with the relevant areas within the organisation to remove any impediments.
Recommendations for criterion 3
Collaborate and work with areas to remove impediments for deploying new functionality frequently once service is Live.
Criterion 4: Understand tools and systems
The service utilises an ongoing vendor relationship. This vendor has considerable domain knowledge and has implemented innovative machine learning solutions as part of the service. The team have indicated that intellectual property for the service is vested with IP Australia.
Recommendations for criterion 4
Plan for service operation and improvement beyond go-live, taking into consideration the possibility of a change or cessation of vendor arrangement.
Criterion 5: Make it secure
All the data accessible via Trade Mark Assist is publicly available, there is no confidential data stored in the system. The service considered security controls from the get-go incorporating a security advisor from the ICT area in the design and development phases of the product. The service leverages Amazon Web Services and security controls have been applied in accordance with IP Australia’s security policies.
Criterion 6: Consistent and responsive design
The team described how service content is reviewed by all relevant areas of IP Australia. It is recommended that the content is reviewed and tested to ensure it is as simple as possible (at an appropriate literacy level, usually grade 7 or 8).
Recommendations for criterion 6
Consider automated tools to assist with content literacy level.
Criterion 7: Use open standards and common platforms
The Trade Mark Assist solution has been built using open platforms where possible. No re-usable government platforms met the needs of the solution.
Criterion 8: Make source code open
The team have chosen a source code repository with wide adoption in the open source community – GitHub. Currently access to the repository is limited to the team. It is recommended that the team make this repository open to the public. It is noted the team are actively working with the vendor to achieve this recommendation.
Recommendations for criterion 8
Continue working with the vendor and ITSA to assess the suitability of making the code base available to the public.
Criterion 9: Make it accessible
The team engaged an external party to undertake independent accessibility audits for compliance with Web Content Accessibility Guidelines 2.0 (WCAG 2.0). The service has been directly tested by multiple users requiring assistance to interact digitally.
Criterion 10: Test the service
Recommendations for criterion 10
The team should further explore opportunities for automated testing in the deployment process. It is noted that options for automated testing of machine learning technology may be immature/limited.
Ensure ongoing testing with real users is being done as the service is refined.
Criterion 11: Measure performance
The team have integrated Google Analytics’ HEART framework into their beta service, which positions them well to meet criterion 11.
Feedback on the alpha service has been received from over 250 users. The team demonstrated how the service has evolved based on priority feedback. For example, the team will link the service to the trade mark application process. This will allow direct correlation between use of the new service and improvements in goods and services selection in the application.
Recommendations for criterion 11
Verify that the metrics captured position the service for reporting on the DTA Performance Dashboard.
Criterion 12: Don’t forget non-digital experience
The team may want to consider improvements to other channels to reduce the errors associated with goods and services selection. For example, improvements to trade mark application form guidance or website content, based on lessons learned during content refinement for Trade Mark Assist. Such improvements will help meet this criterion.
The team have worked with the Call Centre to ensure that they are ready to support users who need assistance to interact digitally.
Recommendations for criterion 12
Continue working with the broader set of Trade Mark services (search, application) to ensure users who are unable to interact digitally can receive equivalent information.
Criterion 13: Encourage everyone to use the digital service
The Trade Mark Assist service is promoted via the Alpha/Beta service page on the IP Australia website. IP Australia promotes these new services through its social media channels (Twitter, Facebook, LinkedIn).
Recommendations for criterion 13
Continue to promote the Beta service to encourage take up and feedback.