A Clustering Method for Zero-Shot Learning

Abstract

In certain object recognition scenarios, labeled data might not cover all categories. Zero-shot learning (ZSL) addresses this issue by leveraging auxiliary information that describes each category, aiming to develop a classifier capable of recognizing samples from categories lacking labeled instances. Transductive ZSL extends this concept by recognizing instances from unseen classes using information from both seen and unseen classes during training. In this thesis, we propose a novel approach for transductive ZSL by integrating a SOTA unsupervised clustering algorithm and modifying it to our needs. To the best of our knowledge, this is the first attempt to bridge the gap between the advances in Unsupervised Clustering literature and Zero-Shot Learning.

Initially, we employ the clustering algorithm to partition all images in the visual space into distinct sets. Subsequently, we establish a correspondence between some clusters and known classes to find a bijective mapping from the semantic space, which contains prototypes for each class, to the visual space clusters. Using this learned mapping, we project prototypes from the semantic space to the visual space and classify instances of the unseen classes based on their distance to these projected prototypes.

Through experiments on two benchmark datasets, we demonstrate the effectiveness of our approach in transductive ZSL tasks. Our method achieves performance on a par with other state-of-the-art ZSL algorithms on the AwA2 dataset, without requiring end-to-end training or fine-tuning of the ResNet101 backbone.