Development of an IoT data processing system using distributed technologies based on Kubernetes

Abstract

The Internet of Things (IoT) era is here, with the number of internetconnected devices expected to hit over 75.4 billion by 2025. These devices create lots of data that help us understand and control our environment from afar. Traditionally, the systems managing this data have been closedsource and outdated by the time they’re widely available. Now, cloud-native technologies and virtualization make it easier and more efficient to manage IoT systems. Kubernetes, a key tool, helps make building and maintaining IoT infrastructures simpler and less manpower-intensive. This thesis explores the design and implementation of an IoT system using distributed technologies deployed on Kubernetes, an open-source framework for automating container application management, deployment, and scaling. It discusses the need for efficient processing and management of large data volumes generated by numerous IoT devices and how implementation through Kubernetes offers a flexible and dynamically scalable solution. Additionally, it examines technologies like MQTT for efficient message transfer and Apache Kafka for real-time data processing and storage. It also analyzes the importance of using distributed systems to improve performance, scalability, and resilience compared to centralized systems, as well as the challenges encountered in managing distributed data and achieving consensus in such environments. Finally, it presents suggestions for future work, including automated scaling, monitoring metrics, continuous integration/delivery (CI/CD), and the extensive implementation of security techniques.