Semantic Segmentation of Coastal Images with Transformer Models

Abstract

Coastal regions play a vital role in various socio-economic activities, environmental sustainability, and biodiversity conservation. Efficient tools and methodologies are required for understanding and monitoring these dynamic and ecologically diverse ecosystems. Remote sensing, especially through aerial and satellite imagery, has emerged as a crucial technology for comprehensive coastline analysis. Furthermore, recent advancements in Transformer-based models offer promising alternatives to traditional convolutional neural networks (CNNs), particularly in capturing long-range dependencies and information. Motivated by the availability of high-resolution aerial images of the Greek coastline from the Hellenic Land Registry, this research focuses on applying state-of-the-art transformer models for semantic segmentation. Leveraging pre-existing labeled datasets from the US coastline, we adapt and train SegFormer, MaskFormer, and Mask2Former models to delineate land surface classes such as water bodies, vegetation, sediments and developed areas. Through the evaluation, Mask2Former emerges as the top-performing model, achieving 85.43% mIoU on the Greek Coastline dataset. Transfer learning proves to be vital, highlighting the value of adapting models to specific datasets. This work represents a crucial step towards leveraging computer vision techniques for remote sensing in coastal environments, paving the way for future research. Future research directions include expanding class categories, utilization of altitude information, material-level analysis, and optimization strategies for model training.