The transition to SAP S/4HANA, Clean Core, and ABAP Cloud presents many companies with a fundamental question:
- How do we handle accumulated custom code? Amid rising complexity, limited developer resources, and pressure to modernize, it becomes clear that traditional migration approaches often fall short.
- Given the shortage of skilled workers and cost pressures, how can we economically differentiate our SAP solutions and keep them cloud-ready?
This is precisely where the present study comes in.
For while artificial intelligence has long been seen as a beacon of hope for more efficient development and transformation, a robust foundation for objectively evaluating its actual capabilities in the ABAP environment has been lacking until now. For the first time, the study provides transparency into what today’s large language models can truly achieve—and where their limits lie. It was conducted by CONSILIO in collaboration with the Cologne University of Applied Sciences (including Prof. Dr. Hartmut Westenberger) and published in March 2026—as the first publicly available scientific study on ABAP code generation.
To this end, a systematic benchmark was developed that reflects real-world requirements from the SAP ecosystem. In a fully automated test environment, several leading AI models were challenged with a variety of practical tasks. The key point: The results are not based on subjective assessments, but on clearly measurable criteria such as compilability and functional correctness. Through iterative feedback loops, it was also possible to analyze how well the models learn from errors and improve their results.
The findings of this study provide valuable insights for IT decision-makers, architects, and developers alike. They show what role AI can play in the future of ABAP development and S/4HANA transformations—and what prerequisites must be established to make this possible.
What does this mean specifically for your company?
What potential can already be leveraged today—and where is human expertise still required?
You’ll find the answers in the full study.
Download the white paper now and gain in-depth insights into the future of ABAP development.
Our study is the first to demonstrate, on a reliable and measurable basis, just how far AI has come in the ABAP environment today and where its limits lie.