Language-based Interpretation of Generative Models

Abstract

Generative models have shown remarkable progress in generating realistic images and are being increasingly used in a variety of applications. However, interpreting and understanding these models remains a challenge. Two main topics have been addressed in this thesis to tackle this problem.

The first topic focuses on Glow, a flow-based generative model with exact latent-variable inference and log-likelihood. The key advantages of Glow are its invertibility and the ability to perform easy image manipulation through its latent space. This thesis proposes a novel framework for interpretable latent direction discovery in the latent space of Glow, by leveraging the text-guided image generation and manipulation capabilities of StyleCLIP. The framework is compared with existing state-of-the-art supervised and unsupervised latent direction discovery methods.

Secondly, motivated by the rapid growth of text-guided image generation and the effectiveness of diffusion models such as Stable Diffusion, this thesis proposes a systematic method to evaluate Stable Diffusion's ability to model and generate images from closely related concepts using WordNet. This study enables the detection of potential biases towards different areas of the distribution modelled by the generative model.

Overall, this thesis aims to provide a better understanding of generative models by proposing novel frameworks and evaluation methodologies for their interpretability and effectiveness. These contributions can have important implications for improving the applicability and reliability of generative models in various fields.