Searchlight
Description
This is a tool which is designed to evaluate the quality of on-site search functionalities for web shops. The evaluation is based on 4 key metrics:
- Speed: How fast does the search show results?
- Visibility: How easy is it for users to find the search bar on the screen?
- Relevance: How well do the search results match the user input?
- Resilience: How well does the search handle mistakes from the user?
These metrics are compiled into a comprehensive report which can then be used in outbound sales efforts.
Background
foobar Agency had recently made the decision to widen the sales funnel for outbound sales. My task was to work with the lead sales development manager to identify pain points, and to see if there could be a way to solve them. The first thing we noticed, was that we needed to focus outreach efforts on one particular problem that we at foobar Agency are specialists at solving. We elected to use on-site search as that problem.
Next came the hard part: how do we decide what a "good" search is? And how can we collect and present data which makes a convincing argument to a potential client that we can help make their search "good" according to those standards?
In the end, we chose 4 key metrics to measure: speed, visibility, relevance, and resilience.
The solution
The tool that I delivered consisted of two main components: a Chrome Extension that can interact with a web shop to perform a large set of test searches and save the results to the backend, and a web platform for storing, compiling and exporting clean and usable reports using foobar Agency branding.
The Chrome Extension leveraged large language models to generate lists of potential search terms for a target web shop. It then performed the searches and recorded the speed of the search, the number of total results, and the titles of the resulting products. It then compared the product titles to the original search term to assess the relevance of the results (also using AI). It also performed the same searches with intentional spelling errors to see if there would be any significant change in the search results to measure our "resilience" metric. All of this data got saved into a database for later retrieval by the web platform.
The web platform consisted of the database, and a full-stack Svelte-kit application which presented the data from each web shop as a cleanly-designed report complete with visual graphics and pre-approved messaging to explain the data (in English and German).