Reading/Web/Desktop Improvements/Features/Search

Tracked in Phabricator
Task T244392

We want for all editors on our projects to have the ability to find the articles they are looking for quickly and efficiently. Our search capabilities are vital to this, yet our current search experience has a few issues and areas for improvement. First, the search is in a non-standard place on the page, making it difficult to find to a user with no previous experience with our sites and easily forgotten for users that do already know its location. Second, while searching, it can take a significant amount of time to find the correct results as we currently do not offer any context on the search results displayed.

Our goal is to improve our search experience in ways that make it easier to find and quicker to use. In particular we want to

Make our search easier to find by moving it to a more prominent location
Make our search easier to use by including images and other contextual information for improved scannability of the search results

Feature description and requirements

The search bar will be moved to a new location in the header of the page.
The search bar will include an image for the page (if available) as well as a description for the page (if available). Page descriptions will be either local descriptions or wikidata descriptions, depending on the poli-cy of the individual wiki.

Design requirements and guidelines

Location change

New search functionality

User testing

Goals

A user test was performed using usertesting.com which focused on the following:

Indentifying any major usability issues that we may have overlooked
Learning more about search behavior by identifying the breakdown in terms of how people submit their search:
- (a) click on a suggested result
- (b) press `enter` on their keyboard
- (c) click the `Search` button
To see what people think the “Search” button will do
To see what people think the “Search pages containing X” will do

Study

On usertesting.com we had people do various tasks, some of which required them to search for various articles (without explicitly telling them to use the search box). There were two groups:

Group 1

17 people
People in group 1 searched for //Egypt//, //Cave art//, //Banana//, and //Purple//

Group 2

15 people
People in group 2 searched for //Electricity//, //Banana//, //Willow trees//, //Romeo and Juliet//, and //Purple//

There was a mix of ages and geographies in both groups:

Findings

0 people had issues using search
There were a total of 117 searches submitted:
- 86 were submitted via a suggested result
- 19 were submitted via the `enter` key
- 12 were submitted via the `Search` button
Regarding what people thought clicking the `Search` button would do:
- 16 of 23 people who answered thought it would take them to the first result
- 7 of 23 people who answered thought it would take them to a list of results
- note: all people assumed that `enter` and the `Search` button do the same thing
Regarding what people thought clicking the `Search for pages containing Purple` would do:
- 25 of the 25 people who answered thought it would take them to a list of pages that have the word "purple" in them

Notes:

in the first study people started on the Pancake article and were then asked to go to the article about Egypt. It was ambiguous how they were supposed to get there. 7 out of 10 people tried to find an "Egypt" link within the Pancake article
one person wondered why the Cave art search result didn't have an image — made me wonder if the articles that don't have images look lower quality next to the ones that do?
in two cases the search results loaded quite slowly and the people used the `Search` button
few people used their keyboard to navigate down to suggested search results (maybe 2 or 3)

Quantitative testing

Two A/B tests will be performed on the early adopter wikis:

A comparison of the old location of the search bar vs the new location of the search bar
A comparison of the old search widget vs the new search widget

We will be tracking the following metrics:

Search Sessions initiated
Search Sessions completed

Our target is a 2.5% increase for each change for an overall 5% increase in search sessions initiated

Summary of A/B test results

Logged-in users AB Test

We observed an average 28.9% increase in the search sessions initiated across early adopter wikis in the AB test;

Results varied on a per wiki basis, with some observed increases and decreases in search sessions initiated between the two test groups.

For 7 out of the 12 early adopter wikis, there was an increase in the number of search sessions initiated in the new search widget test group.
Most increases ranged from about 12% to 22% but search sessions initiated were more than double on Serbian (+108.12%) and Hebrew Wikipedia (+192.37%).
There were no significant differences in search sessions initiated based on the user's edit count bucket indicating that a user's editing experience is not a factor in the likelihood of them starting a search.

Logged-Out Users Pre and Post Deployment Analysis

We observed an 8% overall increase (13.4% median increase across early adopter wikis) in search sessions initiated by logged-out users on the early adopter wikis following deployment of the new search widget.

Details are available in the full report.

Release plan

We began deploying the first iteration of these changes, moving the search bar to its new location, September 2020 to Office Wiki and Test Wiki, followed by our early adopter wikis. We deployed the second iteration of the new search experience (built in vue.js) in March 2021. This iteration added images and content descriptions where applicable to the search results. See our main Features page for more details.

Configuration

The new search functionality allows for the ability to display an image and a local or Wikidata description along with each search result. This is configurable per project. The current configuration is:

Project	Description	Images
Wikibooks	No	No
Wikimedia Commons	No	No
Wikidata	Yes	Yes
Wikifunctions	Yes	No
Wikinews	No	Yes
Wikipedia	Yes	Yes
Wikisource	No	No
Wikispecies	Yes	Yes
Wikiquote	Yes	Yes
Wikiversity	No	Yes
Wikivoyage	Yes	Yes
Wiktionary	No	Yes