This week, OpenAI put COBOL in the spotlight. It announced benchmark results suggesting its latest models are getting much better at migrating COBOL programs and building software in languages with less code and documentation for models to learn from.

On OpenAI’s own COBOL migration test, GPT-6.1 Sol reached 86.6% accuracy at low reasoning effort, up from GPT-5.6 Sol’s 5.4%, while costing 39% less. At higher reasoning effort, the newer model reached 97.9%.

OpenAI has not published the details of its evaluation of COBOL (“Common Business-Oriented Language”; the programming language that AI coding tools may help maintain or migrate). It has released the MultiLangBench setup and test suite on GitHub. That benchmark tests whether models can build a SQLite-compatible database engine from scratch across 20 programming languages. 

OpenAI’s announcement calls the results “substantial improvements across both evaluations, delivering much higher accuracy at lower cost.”

Those results could matter to teams maintaining and modernizing legacy software. But without public COBOL test cases, developers cannot independently inspect that evaluation.

Per OpenAI, its new COBOL migration evaluation “tests whether a model can migrate existing programs to another language while preserving their behavior.” Specifically, it’s testing migrations from COBOL to C++, Java, Kotlin, Python, and Rust. 

To succeed, each migration must pass a series of hidden tests that require “byte-for-byte parity on observable outputs.” And with markedly greater accuracy than its predecessor, GPT-6.1 Sol is, indeed, showing substantial progress. 

The measured gains are striking, though OpenAI is not the first company to bring AI to COBOL modernization.

Source link