Different names. Same product? The challenge behind price…

Consider these two listings from our catalogue:

Retailer	Listing title
Camera House	Rode Wireless GO III - Pink Dual Channel Compact Wireless Microphone System (2.4 GHz)
Mwave	Rode Wireless GO (Gen 3) - Pink

One uses a Roman numeral. The other says “Gen 3”. One describes the wireless system in detail; the other keeps the title short.

Our manually reviewed matching data identifies them as the same product.

Now change the second listing from pink to purple. Almost every word stays the same, but the answer changes: it is a colour variant, rather than an identical product.

That distinction matters. If you're looking for the pink microphone, a cheaper purple one is useful as an alternative. It shouldn't silently become the best price for the pink one.

This is the work that happens before a price comparison can be useful.

Three answers, rather than two

It is tempting to frame product matching as a yes-or-no question. Either two listings match or they don't.

At lowerSpendings, we need three answers:

Same: two offers for the same product, with the same relevant specifications.
Variant: the same product line with a difference such as colour, storage, size or pack quantity.
Different: another model, generation or product line.

The microphone example makes the first two fairly intuitive. The third can depend on a much smaller detail.

In another reviewed example, our catalogue contains Phison Pascari X100P drives whose titles share almost all their words. Within the “Non-Opal” line, different capacities are marked as variants. Across the “Opal” and “Non-Opal” lines, the reviewed decision is different products.

A system that simply counts shared words can miss the very word that determines the answer.

And before training a model to make these decisions, we have to decide what the decisions mean. That policy is part of the engineering.

Why we can't just join everything by barcode

In an ideal catalogue, every retailer would supply a reliable identifier for every product and every variant. Matching would largely become a database lookup.

The data we receive is less tidy.

Some listings have useful manufacturer part numbers. Some have a retailer's internal stock code. Some have identifiers buried in a field intended for something else. Others give us little more than a title and description.

We use identifier rules where the evidence supports them, including barcode checks, brand and model combinations, and certain manufacturer part numbers. But an identical-looking code needs context: a retailer's internal SKU isn't automatically a universal product identifier.

The imbalance is visible in our production matching evidence. On 30 October, there were 92,239 pair labels from the shared-SKU rule, compared with 75 from the GTIN rule. Those are counts of stored pair labels, not counts of unique products or a measurement of identifier coverage. They do show how differently the rules contribute.

Identifiers give us valuable evidence. They don't resolve the whole catalogue.

Finding the pairs worth investigating

When I checked the production database for this article, it contained 606,444 retailer listings across 75 merchants. That includes stored listings, rather than only products currently in stock.

Comparing every listing with every other listing would mean more than 183 billion pairs.

Most would be pointless. A microphone and a garden chair don't need a detailed matching decision.

Instead, we first look for plausible candidates.

We use Qwen3-Embedding-0.6B to turn product text into numerical representations called embeddings. These let us search for listings with similar meaning, even when their wording differs. Milvus, our vector database, finds nearby listings across retailers and within a retailer's own catalogue.

Identifier links provide another route into the candidate set.

This stage narrows the search. A nearby listing is a candidate for investigation; similarity alone doesn't establish that it is the same product.

Two phones with different storage capacities can be extremely similar. So can two generations of the same device.

Teaching a smaller model to read the important details

To distinguish those cases, we need more than an overall similarity score. We need to extract the details that matter.

Our attribute pipeline combines retailer feed fields, rules, dictionaries and a trained language model. It looks for things such as model names, storage, RAM, colour, screen size and pack quantity, then normalises the results into comparable values.

The trained component is a DistilBERT token classifier. Its job is to mark the parts of a title that express an attribute: these words name the model, this span states the storage, that word is the colour.

The training work used 9,069 usable titles labelled with help from a larger language model. The training code checks that the proposed attribute text actually appears in the title and skips examples where it doesn't.

That gives the smaller model a deliberately bounded job. It extracts text from the listing; it doesn't need to invent a specification from its knowledge of the product. It can still pick the wrong span or misunderstand a detail, so extraction remains evidence rather than certainty.

Different methods suit different attributes. In our combined pipeline, the tagger gets preference for model names and colours, while feed fields and rules generally get preference for measurements. Condition has its own rules and merchant context.

There is also a separately trained category model: a small neural network over the product embeddings. It helps organise listings, while category agreement becomes one of the signals available to the matcher.

The practical result is a collection of models with specific jobs, working alongside ordinary code.

Combining the clues

Our trained pair matcher uses LightGBM, a gradient-boosted tree model, to weigh the evidence for “same”, “variant” and “different”.

Its inputs include semantic similarity, title overlap, model-number agreement, brand, identifiers, category information and conflicts in extracted attributes.

Those clues interact. A storage difference between listings for the same phone suggests a variant. A similar numerical difference between unrelated products tells us much less. A missing model number is different from two explicitly conflicting model numbers.

We also give the matcher compressed information about where the two embeddings agree and differ. That preserves more detail than a single similarity score.

The training process combines identifier evidence and AI-generated labels, weights the contributions, and reduces the influence of a large set of easy shared-SKU matches. Otherwise, success on one unusually convenient part of the catalogue could dominate what the model learns.

The matcher doesn't train on its own predictions. Manually reviewed director pairs are excluded from its training data and used for evaluation.

That separation matters, although a small evaluation set still has limits.

One important correction: condition belongs to the offer

An early version of our matching rules treated condition as a variant dimension. A new device and a refurbished example could therefore be labelled as variants.

We've since changed that policy.

If the model, storage, colour and other relevant specifications agree, they can share a catalogue product identity. Each retailer offer separately records its condition and, where available, its grade.

That shared identity doesn't make the offers equivalent purchases. A refurbished price needs to be understood alongside its condition, warranty and other terms.

This change reached beyond a prompt. It affected the attribute pipeline, training-label preparation, title features and automatic grouping rules.

It's a good example of why product matching requires careful definitions. A model can consistently follow a rule that later turns out to be the wrong way to represent the shopping decision.

Reviewing a whole group

Pairwise decisions introduce another problem: they have to make sense together.

If one listing matches a second, and the second matches a third, should all three be combined? A mistaken link can otherwise pull a larger group together.

Our grouping code includes conflict checks and limits on automatic chaining. Ambiguous groups can also go to a review board, where the listings are considered together.

We now use language models to propose arrangements for those boards: which offers belong to one product, which products are variants, and which listings belong elsewhere. The suggestions retain reasons and a record of the evidence. Human decisions take precedence over automatic labels.

In a saved development evaluation on 29 October, the Opus arrangement model agreed with the reviewed answers on 108 of 116 pairs, or 93.1%. Haiku agreed on 102 of 116, or 87.9%.

Those results are useful, but they aren't an accuracy figure for the website. The sample comprised 94 boards, 91 of them containing just one pair. The rules were refined using errors found in this evaluation, so it isn't an untouched final test set. It also doesn't measure matches the candidate search failed to find.

One revealing error involved treating Wi-Fi and cellular versions of a tablet as different models when our intended policy was to group them as variants. Making that rule explicit helped the evaluation results.

Sometimes the next improvement comes from clarifying the question we're asking.

Keeping the evidence, so we can improve it

The production database stores matching labels with their source, model version and confidence. Attribute evidence also records how a value was extracted.

That makes it possible to revisit a decision when a model changes, a rule improves or a person spots a mistake. We can examine disagreements rather than keeping only the final grouping and losing how we got there.

New and changed product text goes through the enrichment pipeline again. A price-only update doesn't need to repeat the semantic matching work.

There will still be missed matches and incorrect ones. Incomplete descriptions, subtle model differences and ambiguous retailer data leave plenty to improve.

But every price comparison rests on this foundation. A lower number only helps when we can explain what it is a price for.

That's why so much of the work behind lowerSpendings happens before we compare the prices at all.

Catalogue figures and listing examples were checked against production data on 30 October 2026. Matching examples reflect manually reviewed decisions. Model results come from saved development evaluations; they describe those samples, rather than catalogue-wide accuracy.