EU AI Act Obligations for Systemic Risk GPAI Providers

This article provides a concise breakdown of the mandatory legal obligations imposed by the European Union Artificial Intelligence Act (EU AI Act) on providers of General-Purpose AI (GPAI) models that pose systemic risks. It covers the criteria used to classify these high-impact models, the baseline rules applicable to all GPAI providers, and the stringent additional requirements—such as adversarial testing, incident reporting, and cybersecurity measures—that providers of systemic risk models must follow to maintain compliance in the EU market.

Defining Systemic Risk in GPAI Models

Under the EU AI Act, a General-Purpose AI model is classified as having "systemic risk" if it possesses high-impact capabilities. The Act establishes a specific technical threshold for this classification: any model trained using a cumulative amount of compute greater than \(10^{25}\) floating-point operations (FLOPs) is presumed to pose a systemic risk. Additionally, the European Commission can designate a model as having systemic risk based on other criteria, such as its reach, number of business users, or potential negative impact on public health, safety, and fundamental rights.

Providers must notify the European Commission within two weeks if their model meets this compute threshold.

Baseline Obligations for All GPAI Providers

Before addressing systemic risk requirements, providers must first fulfill the baseline obligations applied to all GPAI models in the EU. These include:

  • Technical Documentation: Drawing up and keeping up-to-date model documentation to provide to the AI Office and national authorities upon request.
  • Downstream Information: Providing documentation and information to downstream providers who intend to integrate the GPAI model into their own AI systems.
  • Copyright Compliance: Putting in place a policy to respect EU copyright law, particularly regarding text and data mining opt-outs.
  • Training Data Transparency: Publishing a sufficiently detailed summary of the content used for training the model, following a template provided by the AI Office.

Specific Obligations for GPAI Models with Systemic Risk

Providers of GPAI models with systemic risk must comply with all baseline rules, plus four strict additional obligations designed to mitigate large-scale harm. Unlike standard GPAI models, open-source models are not exempt from these systemic risk obligations.

1. Model Evaluation and Adversarial Testing

Providers must perform standardized model evaluations to identify and mitigate systemic risks. This explicitly requires conducting and documenting adversarial testing (often referred to as "red-teaming") both prior to the model's market placement and throughout its lifecycle. These evaluations must align with the state of the art to uncover potential vulnerabilities or dangerous capabilities.

2. Systemic Risk Assessment and Mitigation

Providers are obligated to continuously assess and mitigate possible systemic risks at the Union level. This involves tracking the sources of risks that may stem from the development, placing on the market, or use of their models. Mitigation measures must be implemented immediately when risks are identified.

3. Serious Incident Reporting

Providers must track, document, and report relevant information about serious incidents to the EU AI Office and, where appropriate, to national competent authorities. This reporting must be done without undue delay and must include details regarding any corrective measures taken to resolve the incident.

4. Adequate Cybersecurity Protection

Given the high-impact nature of these models, providers must ensure an adequate level of cybersecurity protection for the GPAI model with systemic risk, as well as for its physical infrastructure. This includes securing model weights, algorithms, and servers from unauthorized access, theft, or manipulation.

Compliance via Codes of Practice

To demonstrate compliance with these obligations, providers of GPAI models with systemic risk may rely on Codes of Practice, which are developed by industry and civil society in coordination with the EU AI Office. Once harmonized European standards are published, compliance with those standards will grant providers a formal presumption of conformity. Failure to comply with these obligations can result in fines of up to 3% of the provider's total worldwide annual turnover or €15 million, whichever is higher.