Training-free Style Transfer in Diffusion Models

Abstract

Diffusion models are a novel class of generative models that have shown promising results in various applications, including image synthesis, natural language processing, and audio generation. Diffusion models operate by gradually transforming a sample from a simple distribution (e.g., Gaussian noise) into a complex data distribution through a series of iterative, noise-adding, and noise-removing steps. This diploma thesis extends the application of diffusion models to the domain of style transfer, a technique pivotal to altering the output of the diffusion model and effectively guiding into producing an image that closely resembles our desired art style. It is crucial to handle semantic alignment while also preserving the texture and nuances of the desired art style. This thesis aims to achieve a fine-balance between these two components. It uses state of the art methods from papers like StyleID and InitNO to handle style transfer via attention key injection and semantic alignment via intial latent noise optimization respectively.