How is this different from checksum or perceptual-hash deduplication?

A checksum only matches byte-identical files, and even perceptual hashing (pHash/dHash) is sensitive to crops, colour shifts, and heavy recompression. DINOv3 embeds the semantic visual content of the image, so it keeps matching two versions of the same photo even after a recrop, a resave at a different quality setting, or a watermark being added.

What similarity threshold should I use to flag a duplicate?

There isn't a universal number — it depends on how conservative you want deduplication to be and what the cost of a false positive is in your workflow. Start high (very close matches only), inspect a sample of flagged clusters, and lower the threshold gradually while watching for pairs that are merely similar rather than actual duplicates.

Will this automatically delete duplicate files?

No. The API returns vectors and similarity scores; deciding which asset in a cluster is canonical and whether to archive or delete the rest is a decision your team or your DAM's own workflow should make. Automating deletion straight off a similarity score risks losing an asset with different licensing or usage rights than its near-duplicate.

Does this work well on logos, icons, and flat vector-style graphics?

Less reliably than on photographs. DINOv3 is trained primarily on natural images, so it discriminates well between photographic content but is a weaker signal for flat, high-contrast graphics where the meaningful differences are often a few pixels of line weight or colour — cases better handled by exact-match or vector-diff tooling.

Is this worth setting up for a small media library?

Probably not. If your library is a few thousand assets or fewer, a manual review pass or your DAM's built-in duplicate finder is usually faster to set up than an embedding pipeline. The economics improve as the library grows past the point where manual review is realistic.

USE CASE

Find near-duplicate images in a large media library

Use case: Deduplicate a large image library with visual embeddings

Large media libraries accumulate redundancy fast — the same product shot re-exported at a different crop, a stock photo uploaded twice under different filenames, a banner re-saved at a lower compression setting after a CMS migration. File-hash deduplication only catches byte-identical copies, so all of these near-duplicates slip through untouched. Webanto's DINOv3 endpoint gives every image in the library a 1280-dimensional, L2-normalised vector; images that are visually near-identical land close together in that vector space regardless of file format, resolution, or compression history, so a nearest-neighbour pass over the embeddings surfaces duplicate clusters that a checksum comparison would never find. The library owner still decides what happens to a flagged cluster — Webanto returns vectors, not deletions.

What to expect

Surfaces near-duplicates that differ in crop, compression, resolution, or watermark — cases a byte-level or perceptual-hash comparison typically misses.
A one-time embedding pass covers the existing library; only new uploads need embedding afterwards, keeping ongoing credit spend proportional to new content, not library size.
Clustering by cosine similarity gives you a tunable precision/recall trade-off instead of a fixed yes/no duplicate flag.
Storage cleanup and licensing-cost reduction become tractable on libraries too large to review by hand.

How to set this up

1
Enumerate every asset in the library with a stable identifier — file path, DAM asset id, or CDN URL — so results can be traced back to a specific file.
2
Batch assets to POST /api/v1/image-embeddings, up to 60 per call, storing each returned vector against its asset id.
3
For a 200,000-asset library that's 200,000 divided by 20 per credit = 10,000 credits for the full one-time pass — comfortably inside a single month on the Scale plan (200,000 credits/mo, $249), or spread across roughly four monthly batches on Growth (50,000 credits/mo).
4
Load the vectors into pgvector or a comparable vector store and run a cosine-similarity nearest-neighbour query for every asset against the rest of the library.
5
Group pairs above a similarity threshold into duplicate clusters (a simple union-find over qualifying pairs works well), and treat borderline scores as 'needs review' rather than an automatic match.
6
Route each cluster to a human reviewer to pick the canonical asset — usually the highest resolution or most recently licensed file — before anything is archived or deleted; don't wire deletion directly to the similarity score.
7
On an ongoing basis, embed only newly uploaded assets and compare each new vector against the existing index, instead of re-embedding the whole library on a schedule.

Products that power this use case

Webanto AI API

Frequently asked questions

How is this different from checksum or perceptual-hash deduplication?
A checksum only matches byte-identical files, and even perceptual hashing (pHash/dHash) is sensitive to crops, colour shifts, and heavy recompression. DINOv3 embeds the semantic visual content of the image, so it keeps matching two versions of the same photo even after a recrop, a resave at a different quality setting, or a watermark being added.
What similarity threshold should I use to flag a duplicate?
There isn't a universal number — it depends on how conservative you want deduplication to be and what the cost of a false positive is in your workflow. Start high (very close matches only), inspect a sample of flagged clusters, and lower the threshold gradually while watching for pairs that are merely similar rather than actual duplicates.
Will this automatically delete duplicate files?
No. The API returns vectors and similarity scores; deciding which asset in a cluster is canonical and whether to archive or delete the rest is a decision your team or your DAM's own workflow should make. Automating deletion straight off a similarity score risks losing an asset with different licensing or usage rights than its near-duplicate.
Does this work well on logos, icons, and flat vector-style graphics?
Less reliably than on photographs. DINOv3 is trained primarily on natural images, so it discriminates well between photographic content but is a weaker signal for flat, high-contrast graphics where the meaningful differences are often a few pixels of line weight or colour — cases better handled by exact-match or vector-diff tooling.
Is this worth setting up for a small media library?
Probably not. If your library is a few thousand assets or fewer, a manual review pass or your DAM's built-in duplicate finder is usually faster to set up than an embedding pipeline. The economics improve as the library grows past the point where manual review is realistic.

Other use cases

Ready to try it?

Start a 14-day free trial on any paid plan. No credit card required.

Start free trial