Μελέτη Σφαλμάτων σε Μεγάλα Γλωσσικά Μοντέλα

Abstract

In recent years, the emergence of Large Language Models (LLMs) has revolutionized Natural Language Processing (NLP), but has also raised critical concerns regarding the trustworthiness and accuracy of their outputs. One of the most pressing issues is hallucination, where LLMs generate information that is se- mantically inconsistent or unrelated to the input. Addressing this challenge is paramount to improving the reliability of LLMs, especially in Natural Language Generation (NLG) tasks. This thesis explores efficient methods for hallucination detection, focusing on black-box settings where the model’s internal workings are inaccessible. The research contributes to the detection of hallucinations through participation in the SemEval-2024 Task 6 (SHROOM), which involves binary classification of hallucinations across tasks such as machine translation, definition modeling, and paraphrase generation. By leveraging pre-trained models fine-tuned on hallucination detection and Natural Language Inference (NLI) datasets, this work achieves significant improvements over baseline systems, with accuracies reaching approximately 80%. Key contributions of this thesis include the development of an ensemble voting classifier combining multiple models to enhance hallucination detection, as well as an in-depth analysis of the challenges posed by different NLG tasks. The findings provide valuable insights into the nature of hallucinations in LLMs and offer a robust framework for future efforts to mitigate this inherent issue.