Development of a Framework for 3D Reconstruction and Inspection of Vineyards

Abstract

3D spatial mapping is a powerful tool that is revolutionizing the field of agriculture. By creating detailed digital representations of fields, farmers can gain insights into their land that were previously unattainable. This technology can be used to remotely monitor crop growth, identify pests and diseases, optimize irrigation and fertilizer applications, and even automate field operations. However, it is a challenge to ensure the necessary optical quality so that the farmer can easily distinguish details in the plants. The present thesis focuses on the design and implementation of an effective perception system capable of accurately reconstructing a vineyard while producing clear and comprehensive visual content for the user. The subject of the present work is not only the research and development of the necessary software to achieve the above goal, but also a review of the available hardware for that purpose and ultimately the execution of an experiment in simulation, as well as with a real robot in realistic conditions. First, the available software on 3D reconstruction and spatial mapping is reviewed. Comparing four different packages produces valuable insight regarding the benefits and drawbacks of various state-of-the-art reconstruction methods. The packages are tested on ROS noetic and CUDA hardware acceleration is enabled to speed up the process. The ZED spatial mapping tool with neural depth capabilities is chosen to complement the custom 3D reconstruction algorithm that is ultimately implemented in this work. Navigation and path planning algorithms are studied in the third chapter of the present thesis. The final design of the high-level planner which is utilized in both the simulation and the real-life experiments is based on prior studies on dual camera visual odometry conducted by the Legged Robots Team of the Control Systems Lab in NTUA, as well as on a novel obstacle avoidance algorithm which leverages polytopic decomposition and a stable trajectory tracking PID controller. Deciding which hardware components to utilize for the implementation of the perception system is next addressed in this work. A comparison of sensors and processing units, as well as an exploration of the ways in which they can be combined and placed on a quadruped robot, led to well-argued decisions about the final structure of the perception system. To justify and visualize these comparisons, a three-dimensional field of view visualization and lidar resolution analysis tool (FoVaLiRa) was developed in the Unity Development platform. The final chapter focuses on the development of the Vinymap Objective Quality Assessment, Canopy Inspection and 3D Reconstruction Algorithm. Vinymap is a novel approach to reconstructing a real vineyard while maintaining the visual features of the leaves, the grapes, and the trunk intact. It also assesses the vineyard’s canopy density and provides valuable quantitative indexes to the farmers. Vinymap was tested on a mobile robotic platform with the perception system developed in this work. It is written in python3 and utilizes open3D and openCV, two well-known and well-established open-source libraries.