Statistical Analysis of AI Use in "Software Engineering" course projects by senior ECE students

Abstract

This thesis explores how university students engage with AI tools during actual software development projects; it uses structured data collected from specific courses, namely "Software Engineering" (7th semester) and "Software as a Service Technologies" (8th semester), both in Curriculum Flow "L", at the school of Electrical and Computer Engineering, National Technical University of Athens. Rather than relying on post-project surveys, this study analyzes nearly 6,000 records submitted during the semester projects' execution, which document when, how, what, and why students used AI during the entire software development cycle. Each submission includes metadata—such as the phase, scope, action, programming language and AI model used, declared experience level on action and tool use, and perceived outcomes in time allocation, time saved, feeling and "threat" feeling level. Findings reveal that AI support is most effective in code-intensive phases like coding and testing, where students frequently describe outputs as usable with minimal revision or edit. Time savings are substantial, with many students reporting that AI completed tasks faster than they could on their own, at a ratio of at least double the time allocated. In contrast, conceptual software development phases like requirement gathering and architecture benefit more modestly—AI helps by only offering generic ideas which often lack specificity or contextual relevance resulting in need for major changes, or even characterized as unusable by the user. A subset of 51 students were registered in both courses "Software Engineering" and "Software as a Service Technologies Technologies", which enabled the analysis of the students’ experience level in the first course in comparison to the second course. The study also highlights important differences based on programming language and user experience. Python and JavaScript are associated with the highest perceived quality, while configuration or declarative languages like YAML or SQL tend to be mentioned in fewer records, indicating that the students haven’t used much AI for those languages, which partially relates to the nature of the semester projects, focusing less on SQL databases and more on architecture and services. Inexperienced users tend to view AI as a helpful partner, while experienced students approach it more critically, balancing convenience with caution. These insights suggest that AI has moved from optional add-on to a "sine qua non" tool in software development—one that requires not only technical training but also reflective guidance.