Research on Hungarian Keywords and Comparing the Results of their Extraction

– Pilot Study

Abstract

Keyword and term extraction is a well-established area of research that has attracted scholarly attention for the past five decades. However, it continues to pose persistent challenges. Language models introduce a novel dimension to various facets of natural language processing, including the realm of keyword and term extraction. They offer the
capability to generate novel keywords that may be absent or only partially represented within the source text. When the authors enter keywords manually, they draw on their
own background knowledge, so these keywords are not necessarily included in the text. Manually entered keywords are therefore worth dealing with and can be considered the gold standard, a benchmark, against which to test keyword extraction applications. In our study, we conducted a comparative analysis of manually assigned keywords for 30 scientifical textual documents (from different domains) against keyword solutions provided by ChatGPT in response to various prompts. Our findings indicate that while there may not be a statistically significant difference in quantitative metrics, a qualitative examination of ChatGPT-generated solutions reveal their relevance and utility in augmenting keyword assignments. The aim of the thesis is to evaluate the outputs given by ChatGPT from the point of view of how close they are to the keywords given by the authors.