What are the effects of AI and data regulations on companies?

Who has obligations?

The legislation classifies several types of actors involved in the development, commercialization, import, or distribution of AI systems, namely: suppliers, implementers, importers, distributors, and manufacturers of devices. The main obligations they have will only materialize IF any of these actors put high-risk systems on the market. According to some estimates, 30% of AI systems produced in the EU would fall into this category, but the EU’s impact assessment suggests that it is around 15%. The situation will only become clear after these high-risk systems are registered in the European database required by the legislation.

Let’s highlight some key points from the compromise text between the Parliament and the Council.

Suppliers must:

• Conduct conformity assessments before placing high-risk systems on the market (these can be done internally or outsourced) in three areas: quality management system, technical documentation, and conformity assessment between technical documentation and the design and development process.
• Ensure that individuals providing human oversight of AI systems are informed about the risk of confirmation bias (i.e., they should not rely solely on the output of the AI system in making decisions).
• Provide specifications for the data used to train the AI system, including information about limitations and possible existing assumptions within the dataset. For example, if the data is predominantly from a specific geographic region, which could create limitations in using the systems in other regions.
• Prepare technical documentation.
• Take corrective actions to bring the system into conformity and inform the importer and implementer.

Attention! You become a supplier if you make substantial modifications to an AI system already on the market, whether it becomes high-risk or if you put your brand on an existing system.

Implementers of such systems must:

• Ensure that they put the systems into operation in accordance with the instructions for use.
• If they exercise control over the system, implement human oversight.
• Ensure that individuals overseeing the system are „competent, qualified, and adequately trained and have the resources to ensure the supervision of the AI system.”
• Monitor the implemented cybersecurity measures.
• If they exercise control over the input data, ensure that the data is relevant and representative for the intended purpose of the system.
• Inform the manufacturer, importer, distributor, and competent national authorities if the use of the system could entail risks to health, safety, or fundamental rights of individuals and suspend the use of the system if necessary.
• Maintain a log automatically generated if they have access and control over it.
• [in labour situations] Before putting a system to work, inform employee representatives and reach an agreement regarding its use. If necessary, conduct a data protection impact assessment.
• Conduct an impact assessment in the intended context of use (with a few exceptions, such as safety systems in traffic management or utility provision).

Importers must ensure that:

• The supplier has conducted a conformity assessment.
• There are technical documentation and instructions for use.
• The system bears the conformity marking.

If the system interacts with individuals, suppliers must inform users that they are interacting with an AI system. Users of deep-fake systems must also highlight that the content is falsified or artificially created.

Counterbalancing these obligations, from which I have listed only a part, is the inclusion of measures to stimulate innovation in this act. The main instrument is the „regulatory sandbox,” a tool for experimenting with products and testing innovations before they are brought to market under the supervision of competent authorities. This is not presented as an obligation of the Member States, but rather an option. Furthermore, the proposal states that startups and small providers should have priority access to such instruments.

Now, let’s look at the impact and complaints from the main actors targeted by this legislation. Let’s start with the perspective on sandboxes:

• An organization representing EU startups states that the legislation on sandboxes should be clearer because startups often do not necessarily fit into a specific domain or may change their location.
• Others are more pessimistic. A report by AI Austria shows that two-thirds of the surveyed startups believe that their activity would be slowed down due to the required adaptations. Approximately 30% of the included startups would classify their systems as high-risk, and the costs of adaptation could reach several hundred thousand euros.
• Not to mention the venture capital climate. The same report indicates that venture capital funds could redirect their investments, considering the decreased interest in high-risk systems that would become more expensive.

So, how is it? Is there too much regulation? As we can see, most of the rules apply to high-risk AI systems, and the market for such systems is limited. At least for now. According to estimates by the Commission, up to 15% of systems would fall into the high-risk category. However, the estimates from the cited reports indicate that around 30% of systems would be affected. The AI market will continue to grow, especially after the explosion of generative AI, such as Chat-GPT, which will have different rules within this legislation. But let’s not digress now.

At first glance, there doesn’t seem to be a great balance between the regulations in the legislation and the proposed regulatory sandbox. Does this compensate for the exodus of venture capital from the EU? Perhaps not, but the funds made available at the EU level through programs like the Digital Europe Programme could help. The problem is that the pitch in front of a venture capitalist is very different from the funding application required to access such funds.

Let’s not forget about the Digital Innovation Hubs (DIHs), whose role is to support companies in testing solutions before bringing them to the market. The problem here is visibility, as confirmation bias, which the AI Act aims to reduce, is present in companies. Confirmation bias of the form „it’s easier with venture capital, I don’t bother because I don’t know what I gain here, etc.”