Building an AI Credit Scoring System Under the European AI Act: What We Learned from OpenScore

Artificial Intelligence has transformed credit risk assessment over the last decade. Financial institutions can now estimate default probabilities with a level of precision that was difficult to imagine only 10 to 15 years ago. Sophisticated machine learning models, deep neural networks and increasingly complex ensemble techniques have pushed predictive performance to unprecedented levels.

Yet predictive accuracy is no longer the primary factor determining whether an AI solution can be deployed in a regulated financial environment.

Under the European AI Act, the object of regulation is the complete socio-technical AI-based system responsible for producing decisions that may significantly affect individuals. For high-risk applications such as credit scoring, this changes the engineering problem entirely.

Building a model or ensemble capable of predicting financial default is relatively well understood. Building an operational AI system capable of being traceable, audited, fair, performant and updated when a risk arises from data to post-commercialization throughout its entire lifecycle is a very different discipline.

This distinction became particularly evident during Dedomena.AI's participation in the European AI Regulatory Sandbox with OpenScore, our AI system for credit scoring, solvency analysis and default risk prediction based on transactional banking data.

Rather than validating model performance, the Spanish AI Sandbox provided an opportunity to evaluate how an already mature AI solution behaves when assessed against the governance, transparency, security and lifecycle management requirements introduced by the AI Act.

The result was the first credit scoring AI system to comply with the AI Act, engineered to operate with greater robustness, transparency and regulatory readiness than its previous version, providing practical evidence of how trustworthy AI can be built under the European AI Act.

Credit scoring extends far beyond prediction

OpenScore was designed to estimate an individual's creditworthiness by analysing anonymized banking transactions and financial data through a combination of proprietary deep neural network architectures, machine learning ensembles and privacy-preserving technologies such as synthetic data generation and AI-driven anonymization.

Its analytical pipeline combines several specialized models instead of relying on a single prediction engine.

Individual services classify banking transactions, detect financial behaviour patterns, estimate future cash-flow capacity, evaluate probability of default and generate an overall solvency assessment. Explainability mechanisms accompany every prediction, allowing each result to be interpreted at both global and individual levels.

From a purely technical perspective, this architecture already represented a mature production system before entering the AI Sandbox. The solution had reached Technology Readiness Level 9 after successful validation in real operational environments and incorporated robust cybersecurity controls, technical documentation, model monitoring capabilities and advanced privacy-preserving mechanisms.

However, the AI Sandbox demonstrated (our suspicions) that technical maturity and regulatory maturity are not necessarily associated paths.

Many capabilities required by the AI Act make you re-engineer predictive performance.

Their purpose is to guarantee that the complete decision-making process remains understandable, governable and trustworthy throughout the operational life of the system.

Regulation reshapes engineering priorities

One of the most significant lessons emerging from the implementation process was the way governance requirements influence technical architecture.

Traditional AI projects naturally concentrate on model accuracy, feature engineering, computational efficiency and scalability.

The AI Act introduces additional engineering dimensions that become equally important:

• Data provenance must remain traceable across the complete analytical pipeline.

• Documentation must evolve alongside every model iteration.

• Responsibilities between technical, operational, legal and business teams require formal definition.

• Human oversight cannot depend on informal operational practices.

• Risk management becomes continuous rather than episodic.

• Evidence supporting compliance must remain available long after deployment.

• Data has to be prepared and analyzed beyond its predictive power, you need unbiased datasets.

These requirements become part of software engineering.

Throughout the AI Sandbox, activities initially perceived as documentation exercises gradually revealed themselves as architectural decisions.

Questions regarding metadata preservation affected data pipelines.

Governance requirements influenced service orchestration.

Traceability determined logging strategies.

Quality management reshaped deployment procedures.

Monitoring risks and requirements extended infrastructure design beyond conventional model performance metrics.

Compliance progressively became another engineering discipline rather than an external validation process.

This shift represents one of the most important consequences of the AI Act.

Increasingly, trustworthy AI will depend as much on operational architecture as on inference capability.

Risk management becomes an operational capability

Perhaps the deepest transformation introduced during the implementation process concerned the role of risk management.

Before participating in the AI Sandbox, OpenScore already included structured technical risk analysis covering model robustness, cybersecurity, documentation and operational reliability.

The regulatory approach required extending that perspective. Risk was no longer associated exclusively with model behaviour. It also included data governance, documentation consistency, human supervision, technical traceability, quality management, evidence preservation and post-deployment monitoring.

Rather than creating an entirely new risk methodology, the implementation focused on converting existing assessments into operational mechanisms capable of continuous observation. Each relevant risk became associated with measurable indicators. Monitoring thresholds were defined. Responsibilities for supervision were formalized. Escalation procedures became documented operational processes.

This evolution fundamentally changes the purpose of governance. Risk registers describe possible future events. Operational monitoring allows organizations to detect those events while they are beginning to emerge. The distinction may appear subtle, but it determines whether governance actively supports production systems or simply documents historical assessments.

One particularly valuable recommendation received during the AI Sandbox concerned differentiating risks associated with service operation from risks related to model degradation. Although both influence system trustworthiness, they require completely different monitoring strategies and mitigation mechanisms. This separation significantly improves both governance clarity and operational effectiveness.

Documentation becomes technical infrastructure

Software documentation has traditionally been viewed as an administrative necessity.

The AI Act elevates documentation into technical infrastructure.

During the implementation process, substantial effort was devoted to strengthening architectural documentation, model documentation, governance procedures, change management processes, audit trails, explainability of every result and long-term evidence preservation.

Roles and responsibilities were formalized. Internal review mechanisms became structured. Quality management processes were consolidated. Version control extended beyond software to include datasets, documentation and governance artefacts.

The objective was not simply producing documentation. It was ensuring that every significant technical decision remained reconstructable and auditable (especially decisions explainability) months or years after deployment.

This capability becomes particularly relevant in AI systems supporting financial decisions, where organizations may eventually need to explain not only how a prediction was generated but also under which operational conditions the system was functioning at that specific point in time.

Trust requires continuous observation

The final phase of the AI Sandbox focused on post-market monitoring, an area that fundamentally changes how AI systems evolve after deployment.

Like many mature AI solutions, OpenScore already incorporated internal monitoring procedures before entering the program. The implementation effort concentrated on formalizing those processes according to the lifecycle perspective introduced by the AI Act.

The resulting monitoring framework specifies measurable indicators for identified risks, establishes explicit governance responsibilities, documents intervention protocols and defines periodic review mechanisms capable of adapting alongside both operational experience and regulatory evolution.

The emphasis is not solely on demonstrating that a system was trustworthy when deployed. The objective is demonstrating that trustworthiness can be maintained throughout the operational lifecycle.

This perspective recognizes an unavoidable reality of modern AI systems in the last decade: models evolve, economic conditions change, data distributions shift, organisations grow. Regulations mature.

Trust therefore cannot be treated as a static property established once during development. It must become a continuously managed engineering capability.

From technical maturity to regulatory maturity

One of the most valuable outcomes of the AI Sandbox was confirming that regulatory readiness should not be understood as a final certification milestone.

It is a continuous technical and legal process that affects architecture, governance, documentation, monitoring and organizational structure simultaneously.

For OpenScore, the program reinforced an already mature technological foundation by introducing formal lifecycle management practices aligned with the European AI Act.

The result extends well beyond regulatory compliance. It produces AI systems that are more maintainable, more transparent, easier to audit and ultimately more trustworthy for organizations operating in highly regulated sectors.

As AI increasingly moves into critical business processes, this distinction will become progressively more important.

The organizations that successfully deploy high-risk AI will not necessarily be those building the most sophisticated predictive models. They will be those capable of developing and implementing complete AI systems that remain explainable, governable and operationally reliable throughout their entire lifecycle.