Open-Domain Dialogue Systems for Low-Resource Languages: The case of Greek

Abstract

Human-machine conversation has been a critical and challenging task in AI and NLP. Recent years have seen rapid progress in open-domain dialogue generation. However, because vast conversational data are only available in English, the development of generation-based chatbots for non-English languages has lagged behind. This Diploma Thesis studies dialogue generation in a low-resource language, specifically Greek, where training data and pre-trained language models are limited. We present theoretical background on machine learning (ML), deep learning (DL), and natural language processing (NLP), then study the transformer-based models used: BERT, GPT-2, T5 and XGLM. We analyze how the corresponding Greek (GREEK-BERT and GPT- 2 Greek) or multilingual (mT5) models were created. Following analysis of research on low-resource dialogue generation, we conduct experiments and discuss results. To address the lack of a Greek dialogue dataset, we used machine translation (MT) to create a Greek version of the DailyDialog dataset. We fine-tune Greek monolingual models (GREEK-BERT and GPT-2 Greek) on the translated dataset, then conduct 4 experiments with multilingual models mT5 and XGLM: 1. Native training: Fine-tuned multilingual models exclusively on the translated dataset for direct comparison with monolingual models. 2. Cross-Lingual transfer learning: Fine-tuned models using the original English DailyDialog dataset, then further fine-tuned on limited manually translated Greek examples. 3. Multitask Learning: Trained models simultaneously on both languages, utilizing the complete English dataset alongside a subset of the translated Greek dataset. 4. Prompt based Learning: Enhanced both approaches with specific prompt templates shared across languages to facilitate knowledge transfer from English to Greek dialogues. We evaluated all models using multiple metrics: Perplexity, BLEU, and BertScore for response quality, and Distinct-n for lexical diversity. Results demonstrate that native training achieved superior performance, with GPT2-Greek as the best-performing model (perplexity:12.47, BLEU B-1: 25.93, Distinct-1: 23.13%, BertScore F-1: 71.37%). Among multilingual approaches, prompt-based training significantly enhanced XGLM performance (F-1: 69.12%), while multitask learning consistently outperformed cross-lingual transfer learning. Human evaluations assessed qualitative aspects that automated metrics might not capture. These revealed that our XGLM model trained using prompt-based multitask learning achieved the best performance among our approaches, ranking second only to the much larger Meltemi model trained on substantially more Greek data. This demonstrates effective cross-linguistic knowledge transfer despite using considerably less Greek training data. This thesis opens new avenues for exploring open-domain dialogue generation for low-resource languages and proposes future extensions for further research.