Hierarchical Average Precision Training for Pertinent Image Retrieval

Elias Ramzi (1, 2, 3) , Nicolas Audebert (1, 3) , Nicolas Thome (1, 4, 3) , Clément Rambour (1, 3) , Xavier Bitot (2)

1 CNAM - Conservatoire National des Arts et Métiers [CNAM]
2 Coexya
3 CEDRIC - VERTIGO - CEDRIC. Données complexes, apprentissage et représentations
4 ISIR - Institut des Systèmes Intelligents et de Robotique

Image Retrieval is commonly evaluated with Average Precision (AP) or Recall@k. Yet, those metrics, are limited to binary labels and do not take into account errors' severity. This paper introduces a new hierarchical AP training method for pertinent image retrieval (HAP-PIER). HAPPIER is based on a new H-AP metric, which leverages a concept hierarchy to refine AP by integrating errors' importance and better evaluate rankings. To train deep models with H-AP, we carefully study the problem's structure and design a smooth lower bound surrogate combined with a clustering loss that ensures consistent ordering. Extensive experiments on 6 datasets show that HAPPIER significantly outperforms state-of-the-art methods for hierarchical retrieval, while being on par with the latest approaches when evaluating fine-grained ranking performances. Finally, we show that HAPPIER leads to better organization of the embedding space, and prevents most severe failure cases of non-hierarchical methods. Our code is publicly available at: https://github.com/elias-ramzi/HAPPIER.

Mots clés: Hierarchical Image Retrieval; Hierarchical Average Precision; Ranking; Hierarchical Image Retrieval; Hierarchical Image Retrieval; Hierarchical Image Retrieval

Domaines: Intelligence artificielle [cs.AI]; Vision par ordinateur et reconnaissance de formes [cs.CV]; Apprentissage [cs.LG]; Réseau de neurones [cs.NE]; Machine Learning [stat.ML]

Elias Ramzi, Nicolas Audebert, Nicolas Thome, Clément Rambour, Xavier Bitot. Hierarchical Average Precision Training for Pertinent Image Retrieval. ECCV 2022, Oct 2022, Tel-Aviv, Israel. ⟨hal-03712933v2⟩