FinReflectKG: Agentic Construction and Evaluation of Financial Knowledge Graphs
While Large Language Models (LLMs) have made it easier to generate knowledge graphs, they haven’t always guaranteed maximum quality, precision, and reliability. Existing financial Knowledge Graphs (KGs), which connect unstructured information into machine-readable, interconnected data, are often too small, overly focused on news sources, or lack rigorous evaluation — limiting their broader applicability in the industry.
Domyn’s research team introduces FinReflectKG, the first large-scale, open financial KG designed to power a new range of high-performing, transparent, and regulator-friendly AI applications. Built from over a decade of certified SEC 10-K filings (2014-2024), the team proposes a KG that transforms raw corporate disclosures into structured, machine-readable knowledge. Beyond traditional document parsing, its capabilities expand to include text and table interpretation, as well as schema detection and following, using a reflection loop to improve output quality step by step.
What makes FinReflectKG especially powerful is its potential for real-world applications, as professionals in the field can use it to ask advanced questions and get structured answers that connect multiple facts. Moreover, they can perform entity search across complex filings, detect investment signals, and even build predictive models for risk and market dynamics. Crucially, because the KG is grounded in the most authoritative data (SEC filings), it supports greater transparency, trust, and auditability, making it a reliable and strategic asset for highly regulated environments, such as financial AI.
Unlike news-based datasets that can be inconsistent, FinReflectKG grounds every data point in verified disclosures, offering unmatched quality, comparability, and credibility. This open-access resource encompasses all S&P 500 companies and brings a flexible, modular extraction framework tailored to real-world business needs. As such, they can choose to prioritize speed (single-pass), balance (multi-pass), or depth (reflection-agent).
Through its evaluation strategy, FinReflectKG proves that high-quality graphs for financial services can be a reality. Beyond traditional accuracy metrics, its holistic evaluation framework sets a new standard for KG quality by tackling rule compliance, coverage diversity, and LLM-as-Judge assessments.
This paper explores the research from Bhaskarjit Sarmah, Stefano Pasquali, Abhinav Arun, Fabrizio Dimino, Tejas Prakash – a significant benchmark for generating high-quality, trustworthy knowledge graphs, showcasing the potential of agentic workflows in LLM-driven systems.
