Ontology Matching with Large Language Models and Prioritized Depth-First Search

Ontology matching (OM) is crucial for data interoperability and knowledge sharing. However, it faces challenges such as the need for large training datasets and limited vocabulary processing in machine learning approaches. Recent advancements in Large Language Models (LLMs) have shown promise in OM through a retrieve-then-prompt pipeline. Despite this progress, these systems still have limitations in performance and computational overhead. To address these issues, a novel approach called MILA has been introduced. MILA embeds a retrieve-identify-prompt pipeline within a prioritized depth-first search (PDFS) strategy to efficiently identify semantic correspondences with high accuracy. This approach limits LLM requests to only the most borderline cases, reducing computational burden. The effectiveness of MILA was evaluated using the biomedical challenge from the Ontology Alignment Evaluation Initiative. Results showed that MILA achieved the highest F-Measure in four out of five unsupervised tasks, outperforming existing OM systems by up to 17%. It also performed comparably or better than leading supervised OM systems. Additionally, MILA exhibited task-agnostic performance across all tasks and settings while significantly reducing LLM requests. These findings emphasize that high-performance LLM-based OM can be achieved through a combination of programmed (PDFS), learned (embedding vectors), and prompting-based heuristics without relying on domain-specific heuristics or fine-tuning. The authors acknowledge support from the University of Santiago de Compostela and projects AF4EU and SUS-SOIL funded by the European Union's Horizon Europe program. They also express gratitude to Diego Martinez-Taboada for insightful conversations. For more information on MILA, data availability can be found at https://github.com/mariatab/MILA. Additionally, a detailed overview of state-of-the-art OM systems used for comparison in the study is provided in Table A.10 in the appendix.

Ontology matching (OM) is a crucial aspect of data interoperability and knowledge sharing. It involves identifying semantic correspondences between different ontologies, which are formal representations of knowledge in a specific domain. OM plays a vital role in various fields such as healthcare, e-commerce, and information retrieval systems. However, it faces challenges such as the need for large training datasets and limited vocabulary processing in machine learning approaches.

In recent years, there has been significant progress in using Large Language Models (LLMs) for ontology matching. LLMs are pre-trained language models that can process vast amounts of text data and generate high-quality outputs. They have shown promise in OM through a retrieve-then-prompt pipeline, where the model retrieves relevant information from existing ontologies and prompts users to provide additional input to improve accuracy.

Despite this progress, LLM-based OM systems still face limitations in performance and computational overhead. To address these issues, researchers at the University of Santiago de Compostela have introduced a novel approach called MILA (Machine Learning-based Ontology Matching with Prioritized Depth-First Search). This approach embeds a retrieve-identify-prompt pipeline within a prioritized depth-first search strategy to efficiently identify semantic correspondences with high accuracy.

The key advantage of MILA is its ability to limit LLM requests only to the most borderline cases, reducing computational burden significantly. This is achieved by combining programmed strategies (PDFS), learned embedding vectors from pre-trained LLMs, and prompting-based heuristics without relying on domain-specific heuristics or fine-tuning.

To evaluate the effectiveness of MILA, it was tested on the biomedical challenge from the Ontology Alignment Evaluation Initiative. The results showed that MILA achieved the highest F-Measure in four out of five unsupervised tasks compared to existing OM systems by up to 17%. It also performed comparably or better than leading supervised OM systems while significantly reducing LLM requests. These findings demonstrate that MILA can achieve high-performance LLM-based OM without relying on domain-specific heuristics or fine-tuning.

The authors of the research paper acknowledge support from the University of Santiago de Compostela and projects AF4EU and SUS-SOIL funded by the European Union's Horizon Europe program. They also express gratitude to Diego Martinez-Taboada for insightful conversations. The data used in this study is publicly available at https://github.com/mariatab/MILA.

For a more comprehensive understanding, Table A.10 in the appendix provides a detailed overview of state-of-the-art OM systems used for comparison in the study. This includes information such as system name, type (supervised or unsupervised), and performance metrics.

In conclusion, MILA offers a promising solution to address challenges faced by LLM-based OM systems. Its efficient retrieve-identify-prompt pipeline within a prioritized depth-first search strategy has shown superior performance compared to existing approaches while significantly reducing computational burden. With further development and refinement, MILA could potentially revolutionize ontology matching and contribute to better data interoperability and knowledge sharing across various domains.