Selected works.

benchgap

Mind the Gap: Explicit Cross-Benchmark Calibrations for Estimating Missing LLM Evaluation Scores

Most LLMs are evaluated on only a few benchmarks: across 401 models and 146 benchmark versions, coverage is just 8.3%. benchgap estimates the missing scores without running any evaluations, calibrating each benchmark against same-capability benchmarks with monotone curves selected by leave-one-out cross-validation, and attaching an error and confidence level to every estimate. In a 15% holdout it fills 71% of masked cells at 4.6 pp MAE, 23% lower than a matrix-completion baseline. It also quantifies the harness tax, the disagreement between evaluation setups on the same models. Estimates are updated daily at benchgap.net.

Süli, P. P., Eigner, G., Hollós, R.

With Obuda University · Biomatics & Applied AI Institute · HUN-REN Centre for Agricultural Research

SchemaEvolve

Data Model Refinement for LLM-Based SQL Code Generation

Keeps the Text-to-SQL model fixed and optimises the database schema it has to read instead. An evolutionary loop proposes restructured schemas as code, each candidate must preserve the original data, and fitness is the execution accuracy of a small local translator. On 13 real databases from Spider 2.0-Lite, 20 generations raise the mean accuracy of three small local models from 27.1% to 50.6% at about $3.50 per database, and the two stronger evolved translators overtake Claude Opus 4.6 reading the original schemas. A local 27B mutator recovers most of the gains at zero API cost.

Süli, P. P., Eigner, G., Hollós, R.

With Obuda University · Biomatics & Applied AI Institute · HUN-REN Centre for Agricultural Research

Distribird

Literature-Informed Prior Distribution Design for Bayesian Model Calibration

A multi-agent LangGraph pipeline that turns scientific literature into informative Bayesian priors. Searches Semantic Scholar and OpenAlex in parallel, extracts numerical values, and fits a probability distribution via AIC model selection, improving MCMC sampling efficiency in 80% of cases across 20 parameters in ten scientific domains. Preprints are live on arXiv and Research Square; the manuscript is under review at Scientific Reports.

Süli, P. P., Eigner, G., Hollós, R.

With HUN-REN AI1Science · HUN-REN AI Center

When Graders Drift and Models Hallucinate

Auditing Human Grading with Local LLMs

A GDPR-compliant grading framework that runs four open-weight LLMs against teacher grades on six C# programming exams (159 submissions, 73 students). The best model reaches 75.3% subtask-level exact-match and ICC 0.974 with the human grader; the ensemble flags 122 likely teacher errors, 85% of which are stricter (not more lenient) than the model consensus. Presented at the 35th Annual Conference of the European Association for Education in Electrical and Information Engineering; proceedings publication forthcoming.

Süli, P. P., Balázs, E., Eigner, G.

With Obuda University · Biomatics & Applied AI Institute · Physiological Controls Research Center

Syxplain

A Neurosymbolic Framework for LLM-Guided Symbolic Regression and Automated Scientific Discovery

A neurosymbolic framework that pairs LLM-guided symbolic regression with automated scientific discovery, closing the loop from data to candidate equation to written explanation. Rather than only steering the search, it interprets the entire Pareto front and grounds every inference in real literature via deep formula research. P. P. Süli and R. Hollós contributed equally and share first authorship.

Süli, P. P., Hollós, R., Szabó, G., Eigner, G.

With HUN-REN AI1Science · HUN-REN AI Center

University Students in the Knowledge Transfer Value Chain

LLM-Mediated Knowledge Sourcing in Higher Education

Examines how LLMs may shape knowledge sourcing, the first stage of the knowledge transfer value chain in higher education. Combines a structured literature review, persona simulation (25 student personas prompted across GPT-4o, DeepSeek, Llama 3.3 and Qwen3), and a small qualitative post-test with real students at Obuda University. The simulated students use LLMs mainly for brainstorming, information acquisition and knowledge structuring, while still preferring face-to-face interaction for collaborative learning and unfamiliar material.

Buday, V., Süli, P. P., Klofsten, M., Eigner, G., Ge, J., Ding, W., Ni, Q., Wang, F.-Y.

With Obuda University · Doctoral School of Innovation and Management · Linköping University · Centre for Innovation and Entrepreneurship · CASIA · State Key Laboratory for Management and Control of Complex Systems

benchgap

Fills the empty cells of LLM leaderboards. Calibrates benchmarks against each other on the models measured on both, predicts each missing score from that model's own results, and attaches a cross-validated error and confidence level to every estimate. Data refreshed daily from BenchLM, with a public OpenAPI 3.1 JSON API. Built with Roland Hollós.

• Python
• Curve fitting
• LLM evaluation
• PHP

Distribird

Multi-agent LangGraph pipeline that turns scientific literature into informative Bayesian priors. Searches Semantic Scholar and OpenAlex in parallel, extracts reported numerical values, and fits a probability distribution via AIC model selection. Built with HUN-REN AI1Science.

• Python
• LangGraph
• LLM agents
• Bayesian inference

SpotBugs

6 merged pull requests to the widely-used Java static-analysis tool, including two new CERT Java detectors with full test cases: SER11-J (prevent overwriting of externalizable objects) and VNA04-J (ensure calls to chained methods are atomic). The atomicity detector connects to the published Acta Cybernetica paper on multithreading atomicity.

• Java
• Static analysis
• CERT Java
• AST visitors