Stepping into the NLG-Metricverse: Design, Development, and Deployment of a PyPI Library for Evaluating Artificial Text

The progress in Natural Language Generation (NLG) has resulted in the widespread use of artificial text in different areas, such as chatbots and content creation. Despite this, it is still difficult to determine the quality and coherence of generated text. This thesis tackles the issue of NLG evaluation by introducing the NLG-Metricverse— an open-source Python library that enables end-to-end evaluation of NLG. This library provides a range of evaluation metrics that allow researchers and practitioners to quantitatively analyze various aspects of generated text. These metrics can be evaluated through visualizations that aid in understanding the scores. The library’s implementation of virtual environments enhances the efficiency, modularity, and reproducibility of the NLG-Metricverse framework. It creates a controlled space to manage dependencies separately from the system-wide Python environment. The NLGMetricverse project has significantly improved its code and refined its APIs. As a result, the library has been deployed on the Python Package Index (PyPI), making it easily accessible to users in the NLG community. This achievement is expected to empower users to conduct more accurate and insightful evaluations of artificial text, thereby advancing the field of NLG. The library is accompanied by comprehensive documentation that meticulously outlines its metrics and functions, making it easy to understand how it works. Furthermore, the library structure is designed to be extensible, allowing for the incorporation of new metrics and methodologies as the field of NLG continues to evolve, hopefully following and fostering new contributions. A data app is also being developed, uploaded on Hugging Face Space, allowing future users to try the library without installing it.