PlanOut, which is Facebook’s set of tools for running online multivariate tests, makes it easy to run simple and complex field experiments. Facebook shared an open-source version of this framework, which is written in Python. We wanted to be able to use PlanOut to run experiments for our search engine, which is written in Go, so we wrote a Go Language interpreter. We’ve open-sourced this interpreter to make it possible to everyone to use PlanOut for applications written in Go – check it out here .

Designing tests for our search engine

We run multivariate tests to improve search relevancy and adjust the parameters our search engine uses to order, boost and filter search results. While there are a number of tools available, we selected PlanOut because it allows us to craft statistically sound experiments, we can integrate it anywhere in our stack, and we have control over the integration.

Some of the first tests we will run are around filtering or boosting certain search operators, like date and location. For example, are we providing better results if we boost results with a certain date range, or if we only show the most recent search results?

After we set up our testing parameters, we can randomly segment users, send them to different sets of search results, and then evaluate the both sets of search results based on click-through rates, search relevancy, and other key metrics.

If you’re interested in learning more about how we’re building our search engine, we’ve written more about it in these posts:

* The Science of Crawl Part 1: Deduplication of Web Content
* The Science of Crawl Part 2: Content Freshness

URX is hiring savvy engineers to build our search engine. View our job listings here.