LLMs and Essence: Developing a Chatbot to Support Software Engineering Practices

Recent advancements in natural language processing (NLP) have led to the development of various applications in a wide range of domains. In the field of software engineering, NLP research has extensively focused on areas like code generation while overlooking other aspects such as project management and best practice recommendations. This thesis aims to fill this gap by investigating the integration of Essence, a standard and thinking framework designed to support the management of software engineering practices, and large language models (LLMs), one of the latest NLP techniques. To achieve this, a specialised chatbot was developed as a complementary tool to assist students and professionals in learning about Essence and improving their software engineering processes. The chatbot utilises a retrieval-augmented generation system to retrieve relevant context from a curated knowledge base, augmenting the users’ queries to generate more accurate and context-aware responses. Various optimisation strategies were implemented to enhance the system’s performance, which was evaluated using metrics assessing both the relevance of retrieved context and the quality of generated responses. Comparative analysis with a general-purpose LLM demonstrated that the proposed system consistently outperforms its counterpart in domain-specific tasks. Although further testing with real users is required to fully understand the application’s potential, this work establishes a foundation for future research into improving the learning and adoption of software engineering practices.