Creating LLMs and algorithms to detect trends amongst billions of data points

Working with a team of data scientists, I helped guide their LLM and algorithm design through customer needs and domain research and the creation of prototyping environments for testing.

B2B SaaSEnterpriseData scienceSocial listening

Senior product designer & researcher
2021 – 2025

The Figma prototype of Brandwatch’s Trends feature, animated to show its interactions.

Background

Who is Brandwatch?

Brandwatch is a leading social analytics platform that allows businesses and their analysts to gain insights from consumers’ social media activity. These insights are used by global, enterprise businesses to help plan and measure their marketing strategies and product innovations. It provides services to clients such as Meta, Google, Bank of America, GSK, Unilever and Disney.

Problem

How to test complex LLMs and algorithms so that they surface relevant trends in minutes, within an enterprise architecture

Brandwatch owns billions of pieces of social and news data, however the core problem was surfacing relevant insights, specific to each client, at speed. Brands need to know about crises before the press do, so they can prepare for backlash and counter the news stories. They need to know about trends before their competitors do, so they can tailor products towards market tastes and market products using trending themes.

As an enterprise business with a monumental data architecture, we weren’t set up to prototype with data so we could quickly test our LLMs and algorithms. This meant our data scientists would often create for months without the ability to get feedback. As with any heavy design task, not getting feedback as you build can lead to features that don’t meet needs.

The Trends prototype running inside the live Brandwatch product with real user data.
A board of user problems distilled from generative research.
Ideation workshop board 1 of 4, with sketches and critique notes.
Ideation workshop board 2 of 4, with sketches and critique notes.
Ideation workshop board 3 of 4, with sketches and critique notes.
Ideation workshop board 4 of 4, with sketches and critique notes.
A user journey plotted from generative research.

Solutions

Creation of prototyping environments, using real client data and delivery of trend detection and alerting features

Our customers were asking for trend detection features which predicted short and long term trends in their domain and adjacent to it, broadening their awareness of both risks and opportunities. To deliver this we needed to create prototyping environments to test algorithms and then deliver detection and alerting features on the chosen solution.

Rapid prototyping with data science teams and their models

Getting feedback on analytics and research products relies on a user seeing their own data in an app. We realised that we needed to prototype with data, however achieving this in the enterprise platform was a slow process that didn’t lend to innovating. Our solution came in the form of a data science ‘sandbox’ where we could code quickly, away from the production codebase, whilst using client data. This sped the process up by months and shifted the workload away from overburdened engineers and towards our less constrained data scientists.

Trend detection, analysis and alerting features

LLM and ML features speed up users’ time-to-insight and are powerful features that our sales teams could use to entice and retain clients. Our data science team created an algorithm that identified shifts in activity in certain topics or communities and presented these to users so that they could spot opportunities to market and develop around these trends. Being first-to-market around a trend is a key advantage.

After researching to establish user needs, I worked with a designer to run ideation workshops with a tribe of 40+ people who created concepts for the algorithm. We undertook testing of the algorithm with user data to validate its accuracy and plan iterations. I then planned and ran a closed beta research study to assess how the model needed to be refined before it was rolled out to the entire client base.

The Trends prototype running inside the live Brandwatch product with real user data.
An alternative concept for the Trends feature.
Animated walkthrough of the redesigned alert configuration screens.
A prototype of sparkline features used in a team review with data scientists and engineers.

Outcomes

Faster iteration loops

Moving to a prototyping sandbox and away from our enterprise codebase, we sped the process up by months as we shifted the workload away from overburdened engineers and towards our less constrained data scientists.

Increased usage

Increase in daily visits and number of alerts triggered.

Commercial value & retention

Sales teams had original IP they could use to retain existing clients and entice new ones.

“Ben’s skillset is unique: he functions equally well as a senior UX researcher and a senior product designer, and that means he can shepherd a project from the early discovery / problem definition phases all the way through delivery and evaluation.

I worked with him in my capacity as product director for a team of about 35 engineers and data scientists building AI-oriented SaaS products, and I relied on him not only as an individual contributor for specific projects, but also as a leader who introduced and improved product development process for the entire team… He has a better understanding of the user experience side of AI products than anyone else that I’ve worked with.”

Paul SiegelProduct Director, Brandwatch

Team

Who I worked with

  • SVP & VP product, research, engineering & data science
  • PhD level data scientists & engineers
  • Product directors & owners
  • Lead researchers & UI designers
  • Customer support & sales teams
  • Solutions consultants & product marketers

Get in touch

If you’d like to discuss any roles or projects, drop me an email.

[email protected]