Newsletter
Stay Ahead
Occasional, engineering-led notes on applied AI — what we're building, running, and learning in production.
Occasional, engineering-led notes on applied AI — what we're building, running, and learning in production.
Use case: Deduplicate a large image library with visual embeddings
Large media libraries accumulate redundancy fast — the same product shot re-exported at a different crop, a stock photo uploaded twice under different filenames, a banner re-saved at a lower compression setting after a CMS migration. File-hash deduplication only catches byte-identical copies, so all of these near-duplicates slip through untouched. Webanto's DINOv3 endpoint gives every image in the library a 1280-dimensional, L2-normalised vector; images that are visually near-identical land close together in that vector space regardless of file format, resolution, or compression history, so a nearest-neighbour pass over the embeddings surfaces duplicate clusters that a checksum comparison would never find. The library owner still decides what happens to a flagged cluster — Webanto returns vectors, not deletions.
Surfaces near-duplicates that differ in crop, compression, resolution, or watermark — cases a byte-level or perceptual-hash comparison typically misses.
A one-time embedding pass covers the existing library; only new uploads need embedding afterwards, keeping ongoing credit spend proportional to new content, not library size.
Clustering by cosine similarity gives you a tunable precision/recall trade-off instead of a fixed yes/no duplicate flag.
Storage cleanup and licensing-cost reduction become tractable on libraries too large to review by hand.
Enumerate every asset in the library with a stable identifier — file path, DAM asset id, or CDN URL — so results can be traced back to a specific file.
Batch assets to POST /api/v1/image-embeddings, up to 60 per call, storing each returned vector against its asset id.
For a 200,000-asset library that's 200,000 divided by 20 per credit = 10,000 credits for the full one-time pass — comfortably inside a single month on the Scale plan (200,000 credits/mo, $249), or spread across roughly four monthly batches on Growth (50,000 credits/mo).
Load the vectors into pgvector or a comparable vector store and run a cosine-similarity nearest-neighbour query for every asset against the rest of the library.
Group pairs above a similarity threshold into duplicate clusters (a simple union-find over qualifying pairs works well), and treat borderline scores as 'needs review' rather than an automatic match.
Route each cluster to a human reviewer to pick the canonical asset — usually the highest resolution or most recently licensed file — before anything is archived or deleted; don't wire deletion directly to the similarity score.
On an ongoing basis, embed only newly uploaded assets and compare each new vector against the existing index, instead of re-embedding the whole library on a schedule.
A checksum only matches byte-identical files, and even perceptual hashing (pHash/dHash) is sensitive to crops, colour shifts, and heavy recompression. DINOv3 embeds the semantic visual content of the image, so it keeps matching two versions of the same photo even after a recrop, a resave at a different quality setting, or a watermark being added.
There isn't a universal number — it depends on how conservative you want deduplication to be and what the cost of a false positive is in your workflow. Start high (very close matches only), inspect a sample of flagged clusters, and lower the threshold gradually while watching for pairs that are merely similar rather than actual duplicates.
No. The API returns vectors and similarity scores; deciding which asset in a cluster is canonical and whether to archive or delete the rest is a decision your team or your DAM's own workflow should make. Automating deletion straight off a similarity score risks losing an asset with different licensing or usage rights than its near-duplicate.
Less reliably than on photographs. DINOv3 is trained primarily on natural images, so it discriminates well between photographic content but is a weaker signal for flat, high-contrast graphics where the meaningful differences are often a few pixels of line weight or colour — cases better handled by exact-match or vector-diff tooling.
Probably not. If your library is a few thousand assets or fewer, a manual review pass or your DAM's built-in duplicate finder is usually faster to set up than an embedding pipeline. The economics improve as the library grows past the point where manual review is realistic.
Shoppers who spot a product in the wild, on someone else's feed, or on a competitor's site rarely know the right keywords to find it in your catalogue. Visual search closes that gap: a shopper uploads
Reverse image search answers a narrower question than catalogue browsing: given one photo someone just handed you — a screenshot, a competitor's product shot, a customer's snapshot of a damaged item —
Invoices, purchase orders, delivery notes and scanned contracts arrive as PDFs and photos, not structured data, which means someone on your team is retyping totals, line items and dates into your acco
Start a 14-day free trial on any paid plan. No credit card required.