Two-stage shelf audit: YOLO finds the products, embeddings can't tell sibling SKUS apart. What should Stage 2 be? [P]

A developer is building a two-stage shelf audit tool where YOLO identifies products, and embeddings (DINOV2, SigLIP2, OpenCLIP) are used to match them against a reference gallery. The primary challenge is distinguishing between sibling SKUs, such as different sizes (e.g., 1.25 L vs. 2 L) of the same product, because the small text indicating these differences is lost when images are letterboxed to 224. The developer is seeking advice on how to improve identification, considering options like fine-tuning with hard negatives or adding OCR.

This post uniquely details a specific technical challenge in shelf auditing, where current embedding models (DINOV2, SigLIP2, OpenCLIP) fail to differentiate sibling SKUs due to image resizing, unlike general discussions on product recognition.