Schreiben Sie ForschungsarbeitenLebensläufeAbschlussarbeitenFörderanträgeForschungsarbeitenmit weniger Routinearbeit.

Abstract—Compilation repair for LaTeX is easy to score badly. A candidate may compile after deleting or hiding content, while exact source equality rejects valid alternative repairs. We present TeXFix-Bench, a controlled benchmark for repairing complete, broken LaTeX sources under a pinned engine. The generator applies 12 deterministic single-fault operators to 13 hand-authored seed documents. It retains 627 engine-validated rows, comprising 300 unique hard-track inputs and 67 unique warning-only inputs after hash deduplication.

Index Terms—automated program repair, LaTeX, benchmark, compilation, large language models, reproducibility

LaTeX compilation failures are a practical target for automated repair. A missing delimiter or misspelled control sequence can prevent a complete document from building, and the first reported diagnostic may be distant from the underlying fault. LLM-assisted writing systems now operate inside LaTeX editors , but their publications do not provide a public, engine-pinned evaluation of compilation-failure repair.

Program repair benchmarks pair a defect with an executable oracle and a fixed harness . TeXFix-Bench studies a narrower contract: a broken source document in, a repaired full source out, under a specified engine.

TexOCR-Bench evaluates page-image-to-LaTeX reconstruction , and EqFix repairs errors in individual equations by synthesizing rules from examples . Overfitting to a weak oracle is a known hazard .

A system receives one broken source file and returns one repaired source file. The engine, its version, and the build command are fixed for every attempt.

Every generated row passes an engine gate before it is retained, and duplicate broken sources are removed by SHA-256 so that no input is scored twice .

Compile Recovery requires a successful Tectonic 0.17.0 run and at least 0.80 token-level LCS similarity between text extracted from the candidate and reference PDFs. Normalized exact match sits beside it, because compilation alone cannot separate a faithful repair from a deletion .

Claude Sonnet 5, GPT-5.6 Sol, and Grok 4.5 each recover 85 of 86 inputs, yet their normalized exact-match counts are 52, 66, and 59 . Compilation and edit precision must be reported together.

The contribution is an auditable task definition, an executable oracle, a fixed cohort, and a replayable candidate ledger.

[1]R. Just, D. Jalali, and M. D. Ernst, “Defects4J: A database of existing faults to enable controlled testing studies for Java programs,” in Proc. ISSTA, 2014, pp. 437–440.

[2]C. Le Goues, N. Holtschulte, E. K. Smith, Y. Brun, P. Devanbu, S. Forrest, and W. Weimer, “The ManyBugs and IntroClass benchmarks for automated repair of C programs,” IEEE Trans. Softw. Eng., vol. 41, no. 12, pp. 1236–1256, 2015.

[3]C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan, “SWE-bench: Can language models resolve real-world GitHub issues?” in Proc. ICLR, 2024.

[4]M. Chen et al., “Evaluating large language models trained on code,” arXiv:2107.03374, 2021.

[5]E. K. Smith, E. T. Barr, C. Le Goues, and Y. Brun, “Is the cure worse than the disease? Overfitting in automated program repair,” in Proc. ESEC/FSE, 2015, pp. 532–543.

[6]R. A. DeMillo, R. J. Lipton, and F. G. Sayward, “Hints on test data selection: Help for the practicing programmer,” Computer, vol. 11, no. 4, pp. 34–41, 1978.

[7]Y. Jia and M. Harman, “An analysis and survey of the development of mutation testing,” IEEE Trans. Softw. Eng., vol. 37, no. 5, pp. 649–678, 2011.

[8]R. Gupta, S. Pal, A. Kanade, and S. Shevade, “DeepFix: Fixing common C language errors by deep learning,” in Proc. AAAI, vol. 31, no. 1, pp. 1345–1351, 2017.

[9]M. Yasunaga and P. Liang, “Graph-based, self-supervised program repair from diagnostic feedback,” in Proc. ICML, PMLR, vol. 119, pp. 10799–10808, 2020.

[10]F. Zhu, Z. Liu, G. Yang, J. Li, and S. Cheung, “EqFix: Fixing LaTeX equation errors by examples,” in SETTA 2022, LNCS 13649, pp. 89–109.

[11]K. Wang et al., “TexOCR-Bench: Benchmarking page-image-to-LaTeX reconstruction,” in Proc. 64th ACL, Vol. 1, 2026.

[12]H. Wen, Z. Zhang, and Y. Li, “OverleafCopilot: Empowering academic writing in Overleaf with large language models,” arXiv:2403.09733, 2024.

[13]S. Hou et al., “PaperDebugger: Assisting scientific writing, review, and editing,” in Companion Proc. ACM Web Conf., 2026.

[14]J. Tan, K. Zhang, and X. Chen, “Inconsistencies in TeX-produced documents,” in Proc. ISSTA, 2024.

[15]L. Zheng et al., “Judging LLM-as-a-judge with MT-Bench and Chatbot Arena,” Adv. Neural Inf. Process. Syst., vol. 36, pp. 46595–46623, 2023.

[16]A. Panickssery, S. R. Bowman, and S. Feng, “LLM evaluators recognize and favor their own generations,” Adv. Neural Inf. Process. Syst., vol. 37, 2024.

[17]B. Hui et al., “Qwen2.5-Coder technical report,” arXiv:2409.12186, 2024.

[18]Meta AI, “Llama 3.2 model card,” 2024.

[19]E. Kassis et al., “Agent skills for reproducible research agents,” arXiv:2609.00065, 2026.