Enhancing Document Parsing and Question Answering through Optimized Table Parsing

The dissertation investigates the significant impact of table parsing on enhancing the accuracy and efficiency of document parsing and question answering systems. This research is motivated by the practical challenges experienced during an internship at Bayer, where the necessity for enhanced parsing techniques became clearly evident. By integrating advanced parsing techniques with Natural Language Processing models, the research addresses the challenges of extracting and interpreting information from various types of documents, with a particular emphasis on tables. A central aspect of this work is the impact of table parsing within the document parsing and question answering processes and the evaluation of the proposed optimizations through experiments and assessments by human experts. These evaluations measure the impact of the optimizations, implemented through table parsing, on parsing quality and the question answering system. They highlight the system’s ability to accurately parse documents and generate pertinent and relevant responses to queries, underscoring the crucial role of precision in document parsing for effective question answering. The research findings demonstrate a substantial improvement in document parsing and question answering capabilities as a result of the optimized table parsing techniques. The dissertation details the advantages and limitations of different parsing methods, proposing solutions that enhance the performance of the document question answering system. Table parsing is shown to be essential for improving the system’s ability to comprehend complex queries and documents, leading to more accurate and efficient information retrieval.