This is the latest in a series of posts explaining the decisions we make that affect our users, as well as the results of those decisions (positive or negative).
Dribbble’s mission is to help professional designers earn a living doing work they take pride in. The primary objective of everything we do — every new feature, policy, and campaign — is to maximize the number of users searching, interacting, and transacting through our platform.
To execute our strategy, we’re introducing new transactional features (coming soon) and radically improving key facets of the user experience, namely search. Search is critical to stimulating the interactions between users that will ultimately become transactions.
Work is already underway to improve our search UI/UX, ranking algorithms, infrastructure, and human moderation. Below, I’ll share some early results of these efforts.
For context, a Dribbble user can discover content (“Shots”) a number of different ways:
- They can place a search (“Shot Search”). FYI, website traffic from Google (which is substantial) often lands directly on a search results page.
- They can browse a feed of design work that’s receiving lots of engagement from other users (“Popular”). FYI, a logged-out visitor to our homepage will see the same feed.
- If logged in, they can browse a feed of uploads from designers they have chosen to follow (“Following”).
- They can browse a feed of recent design work handpicked by our moderators (“New and Noteworthy”).
It’s important to note that most of the action happens in search, not feeds:
Search results account for over 80% of the content actually seen by users (“Shots Seen”). In other words, users are far more likely to discover content by placing a search, than browsing a feed. Consequently, most of the actual engagement with content (“Interactions”) also happens through search.
While this data is unambiguous, most of the qualitative feedback we receive relates to the contents of the Popular Feed. I suspect that designers who share their work on Dribbble over-index on the Popular Feed because that’s where “Inspiration” (in our primary navigation) links to. In any case, we’re focusing our resources on improving search because we expect that work to be far more impactful than optimizing the feeds.
To that end, we developed a new search algorithm which we recently released as an A/B test. We expected the new algorithm to generate more relevant search results than the previous version and increase user engagement with content. As an aside, over one million users place at least one search on Dribbble every single day so we were able to gather an immense amount of data during the test.
The new algorithm outperformed the previous across most metrics, including:
- +6.3% Shots Seen
- +0.5% Shots Clicked
- +1.9% Profiles Clicked
- +16.9% Likes ??
- +9.7% Saves ??
We were very encouraged by the improvement in Likes and Saves which are two of our strongest indicators of search relevance. That said, we observed a 44.9% decline in the number of distinct designers whose content was engaged with. While it’s not unusual for there to be some tension between relevance and diversity in search results, we’ll continue to calibrate this dimension of the algorithm.
To help visualize the differences between the two algorithms, below are some examples of the results generated for our most common search queries:
“Dashboard” (most common search query):
Previous
New
“Portfolio” (4th-most common search query):
Previous
New
“Table” (8th-most common search query):
Previous
New
“Calendar” (17th-most common search query) is a good example of the relevance/diversity tradeoff:
Previous
New
To be clear, I’m not suggesting the new algorithm yielded more relevant results for every search query (it almost certainly didn’t). However, across tens of millions of searches by millions of different users, the new algorithm generated a statistically significant increase in user engagement. Based on these results, we released it to everyone.
FYI, the search queries placed by our users follow a very long tail distribution. While the examples I shared above are among the most common, none accounts for more than 0.5% of searches this year.
I’ll wrap up by sharing that we’ve already introduced a newer version of the algorithm which we’re testing against the winner of the A/B test described above. Frankly, there are many, many hypotheses we plan to test so it’s unlikely that 100% of our users will see the same search results anytime soon.
Given that the ranking signals we use (and the weight we give them) will be a moving target, the most reliable way designers can maximize engagement with their content is to:
- Share compelling, original design work.
- Provide thoughtful and accurate metadata (titles, descriptions, and tags).
In upcoming posts, I’ll share some other work underway to improve Shot Search as well as our work around Designer Search (which is how users can discover specific designers available for work).
