Seen but not clicked – lowering rankings to improve search

Have you looked at your web analytics report and wondered what your users really want?

A line drawing to demonstrate one search result moving up and another moving down.

The simple answer is they are already telling you, however services like Google, Bing and Baidu store this information. Observatory analysis of shows around 75% of our users are coming from search engines. A unique aspect of search is that people use plain English to find the services that meet their needs. You can use this information to understand what your users are trying to do. Luckily, these search engines offer these insights for free through their webmaster tools.

What are webmaster tools?

Webmaster tools are free services provided by most search engines. They are a rich source of data on how websites perform and how users access them.

Their reports provide:

  • search queries used to get to a specific page
  • the page’s position on the search results
  • the number of impressions — how many people viewed the result
  • the click-through rate — the percentage of users who clicked though from an impression
A screenshot of Google Search Console demonstrating the sort of data it can display. This shows the Performance report for the top 4 queries for in the form of a line graph with a data table underneath.

Caption: The Performance report shows the top 4 queries for in the last 6 months.

Why use them?

These services capture your users’ intent through the questions that they ask. It is a passive data collection technique and you and your users do not need to do anything more. It is also cheaper than other participatory user research methods.

However, it compliments other forms of user research rather than replacing them. You can create a holistic view of your service by combining these quantitative and qualitative methods.

How we use webmaster tools

Our hypothesis was that a significant number of users were coming to looking for visa information. They were then leaving to find that information at was an unnecessary step in their user journey.

We used the connection between Google’s BigQuery and Search Console and our whole-of-government Google Analytics 360 contract to find out why.

  1. We identified the common immigration-related search terms people used to arrive at
  2. We noted the pages people visited after using these search terms.
  3. We identified where our users were flowing from those pages.
  4. We visualised this through a Sankey diagram.

This diagram showed how users flowed through to their final destination.

A complex diagram showing how traffic flows between a selection of nodes on the environment. Flows begin from, and shows the various points users land at on their way to their goals.

Caption: This diagram shows the flow of traffic from left to right, starting at, moving through deeper content, or nodes, and finally various endpoints. Green nodes are webpages, blue nodes are webpages and red nodes are other pages. Select it to see a larger version.

Our results

The experiment showed that our hypothesis was correct. Search algorithms tend to favour established brands, so search engines consider results from more reliable than results from more specific agency websites. Users looked for general visa information on, and then moved quickly to Home Affairs websites. This suggests that is an unnecessary step in this user journey.

Our response was to do the opposite of what most sites do. We reversed our search engine optimisation to lower the ranking of This allows other websites with more current and accurate content to appear higher in search results, giving them a greater chance of being selected. It took about 3 months for the Google search algorithms to recognise the change and noticeably affect search results.

We also noticed that:

Our analysis showed users searched for very sparse phrases, such as “Australia Visa” or “Australia Immigration”. This makes it challenging for search algorithms to show the best search results. Agencies can take various approaches to solve this problem, but they need to invest in search engine optimisation and analyse their own data before deciding.

Other agencies can apply the methodology we used to other user research or policy evaluation problems. We’ve published a guide on optimising search engine results. We also have a web analytics guide, currently in Beta, to help product and service owners incorporate analytics into their agile lifecycle.

About the Observatory

The Observatory already has 32 partner agencies covering 1.167 billion interactions each month, and over 14 billion data points in the past year. We run free Google Analytics 360 training for agencies to help them get the most out of the whole-of-government Google Analytics 360 subscription. If you run a government service and want to know how the Observatory can help you improve it, email us at