Please use this identifier to cite or link to this item:
http://artemis.cslab.ece.ntua.gr:8080/jspui/handle/123456789/18287
Title: | Attention-based Story Visualization |
Authors: | Τσάκας, Νικόλαος Στάμου Γιώργος |
Keywords: | GAN Story Visualization Transformer Attention |
Issue Date: | 1-Mar-2022 |
Abstract: | Story Visualization is a novel task described as the generation of an image sequence based on a short story made up of natural language sentences or other semantic information. The task borrows from Text-to-Image in its pursuit of language-image correspondence, as well as Text-to-Video in its aim for consistency across frames. Currently there are few improvements on this challenging topic as well as a scarcity of viable datasets and evaluation methods. It is the combination of recent advances in sequence transduction (Transformer) and conditional image generation (SAGAN) that motivated our approach to the task of Story Visualization, in hopes of contributing towards a model that can capture the nuances of image sequence generation and language-to-vision temporal correspondence. The main objective of this thesis is to research various improvements on the original StoryGAN and experiment with different implementations of our architectural proposals. To that end we: • Examine the effects of using a Transformer encoder in place of the original RNN. • Apply more recent architectural approaches to the image generating GAN. • Explore the effects of attention mechanisms in the model, both as presented in the SAGAN architecture and by proposing two novel attention mechanisms for image sequences. |
URI: | http://artemis.cslab.ece.ntua.gr:8080/jspui/handle/123456789/18287 |
Appears in Collections: | Διπλωματικές Εργασίες - Theses |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
tsakas_thesis.pdf | 4.44 MB | Adobe PDF | View/Open |
Items in Artemis are protected by copyright, with all rights reserved, unless otherwise indicated.