Committee on Legal Affairs
with recommendations to the Commission on a Civil liability regime for artificial intelligence
Committee on Legal Affairs Rapporteur: Axel Voss
(Initiative – Rule 47 of the Rules of Procedure)
EN United in diversity EN
MOTION FOR A EUROPEAN PARLIAMENT RESOLUTION 3
ANNEX TO THE MOTION FOR A RESOLUTION: DETAILED RECOMMENDATIONS FOR DRAWING UP A EUROPEAN PARLIAMENT AND COUNCIL REGULATION ON LIABILITY FOR THE OPERATION OF ARTIFICIAL INTELLIGENCE-SYSTEMS 9
A. PRINCIPLES AND AIMS OF THE PROPOSAL 9
EXPLANATORY STATEMENT 26
with recommendations to the Commission on a Civil liability regime for artificial intelligence
The European Parliament,
– having regard to Article 225 of the Treaty on the Functioning of the European Union,
– having regard to Council Directive 85/374/EEC of 25 July 1985 on the approximation of the laws, regulations and administrative provisions of the Member States concerning liability for defective products,
– having regard to Council Regulation (EU) 2018/1488 of 28 September 2018 establishing the European High Performance Computing Joint Undertaking1,
– having regard to the proposal for a Regulation of the European Parliament and of the Council of 6 June 2018 establishing the Digital Europe programme for the period 2021- 2027 (COM(2018)0434),
– having regard to its resolution of 16 February 2017 with recommendations to the Commission on Civil Law Rules on Robotics2,
– having regard to its resolution of 1 June 2017 on digitizing European industry3,
– having regard to its resolution of 12 September 2018 on autonomous weapon systems4,
– having regard to its resolution of 12 February 2019 on a comprehensive European industrial policy on artificial intelligence and robotics5,
– having regard to its resolution of 12 February 2020 on automated decision-making processes: ensuring consumer protection and free movement of goods and services6,
– having regard to the Commission communication of 25 April 2018 on Artificial Intelligence for Europe (COM(2018)0237),
– having regard to the Commission communication of 7 December 2018 on a coordinated plan on artificial intelligence (COM(2018)0795),
– having regard to the Commission communication of 8 April 2019 on building trust in
1 OJ L 252, 8.10.2018, p. 1.
2 OJ C 252, 18.7.2018, p. 239.
3 OJ C 307, 30.8.2018, p. 163.
4 OJ C 433, 23.12.2019, p. 86.
5 Texts adopted, P8_TA(2019)0081.
6 Texts adopted, P9_TA(2020)0032.
human-centric artificial intelligence (COM(2019)0168),
– having regard to the Commission White Paper of 19 February 2020 on Artificial Intelligence - A European approach to excellence and trust,
– having regard to the Commission report of 19 February 2020 on safety and liability implications of Artificial Intelligence, the Internet of Things and robotics,
– having regard to the European Parliamentary Research Service STOA Policy Briefing of June 2016 on legal and ethical reflections concerning robotics,
– having regard to the Study of the Directorate General for internal policies of October 2016 for the Legal Affairs Committee entitled “European Civil Law Rules in Robotics”,
– having regard to the report of 8 April 2019 of the High-Level Expert Group on Artificial Intelligence entitled “Ethics Guidelines for trustworthy AI”,
– having regard to the report of 8 April 2019 of the High-Level Expert Group on Artificial Intelligence entitled “A definition of AI: Main Capabilities and Disciplines”,
– having regard to the report of 26 June 2019 of the High-Level Expert Group on Artificial Intelligence entitled “Policy and investment recommendations for trustworthy AI”,
– having regard to the report of 21 November 2019 of the Expert Group on Liability and New Technologies – New Technologies Formation entitled “Liability for Artificial Intelligence and other emerging digital technologies“,
– having regard to Rules 47 and 54 of its Rules of Procedure,
– having regard to the opinions of the Committee on the Internal Market and Consumer Protection and the Committee on Transport and Tourism,
– having regard to the report of the Committee on Legal Affairs (A9-0000/2020),
A. whereas the concept of ‘liability’ plays an important double role in our daily life: on the one hand, it ensures that a person who has suffered harm or damage is entitled to claim compensation from the party proven to be liable for that harm or damage, and on the other hand, it provides the economic incentives for natural and legal persons to avoid causing harm or damage in the first place;
B. whereas any future-orientated liability framework has to strike a balance between efficiently protecting potential victims of harm or damage and at the same time, providing enough leeway to make the development of new technologies, products or services possible; whereas ultimately, the goal of any liability framework should be to provide legal certainty for all parties, whether it be the producer, the deployer, the affected person or any other third party;
C. whereas the determination and attribution of liability for tortuous damage or harm generally takes place based on the principles of fault-based liability regimes; whereas
the legislator or jurisprudence of the Member States have in many cases adapted the liability regimes to different needs, for instance new emerging technologies;
D. whereas the legal system of a Member State can exclude liability for certain actors or can make it stricter for certain activities; whereas strict liability means that a party can be liable despite the absence of fault; whereas in many national tort laws, the defendant is held strictly liable if a risk materializes which that defendant has created for the public, such as in the form of cars or hazardous activities, or which he cannot control, like animals;
E. whereas Artificial Intelligence (AI)-systems present significant legal challenges for the existing liability framework and could lead to situations, in which their opacity could make it extremely expensive or even impossible to identify who was in control of the risk associated with the AI-system or which code or input has ultimately caused the harmful operation;
F. whereas this difficulty is compounded by the connectivity between an AI-system and other AI-systems and non-AI-systems, by the dependency on external data, by the vulnerability to cybersecurity breaches as well as by the increasing autonomy of AI- systems triggered by machine-learning and deep-learning capabilities;
G. whereas sound ethical standards for AI-systems combined with solid and fair compensation procedures can help to address those legal challenges; whereas fair liability procedures means that each person who suffers harm caused by AI-systems or whose property damage is caused by AI-systems should have the same level of protection compared to cases without involvement of an AI-system.
1. Considers that the challenge related to the introduction of AI-systems into society and the economy is one of the most important questions on the current political agenda; whereas technologies based on A I could improve our lives in almost every sector, from the personal sphere (e.g. personalised education, fitness programs) to global challenges (e.g. climate change, hunger and starvation);
2. Firmly believes that in order to efficiently exploit the advantages and prevent potential misuses, principle-based and future-proof legislation across the EU for all AI-systems is crucial; is of the opinion that, while sector specific regulations for the broad range of possible applications are preferable, a horizontal legal framework based on common principles seems necessary to establish equal standards across the Union and effectively protect our European values;
3. States that the Digital Single Market needs to be fully harmonized since the digital sphere is characterized by rapid cross-border dynamics and international data flows; considers that the Union will only achieve the objectives of maintaining EU’s digital sovereignty and of boosting digital innovation made in Europe with consistent and common rules;
4. Firmly believes that the new common rules for AI-systems should only take the form of a regulation; considers that the question of liability in cases of harm or damage caused
by an AI-system is one of the key aspects to address within this framework;
Liability and Artificial Intelligence
5. Believes that there is no need for a complete revision of the well-functioning liability regimes but that the complexity, connectivity, opacity, vulnerability and autonomy of AI-systems nevertheless represent a significant challenge; considers that specific adjustments are necessary to avoid a situation in which persons who suffer harm or whose property is damaged end up without compensation;
6. Notes that all physical or virtual activities, devices or processes that are driven by AI- systems may technically be the direct or indirect cause of harm or damage, yet are always the result of someone building, deploying or interfering with the systems; is of the opinion that the opacity and autonomy of AI-systems could make it in practice very difficult or even impossible to trace back specific harmful actions of the AI-systems to specific human input or to decisions in the design; recalls that, in accordance with widely-accepted liability concepts, one is nevertheless able to circumvent this obstacle by making the persons who create, maintain or control the risk associated with the AI- system, accountable;
7. Considers that the Product Liability Directive (PLD) has proven to be an effective means of getting compensation for harm triggered by a defective product; hence, notes that it should also be used with regard to civil liability claims against the producer of a defective AI-system, when the AI-system qualifies as a product under that Directive; if legislative adjustments to the PLD are necessary, they should be discussed during a review of that Directive; is of the opinion that, for the purpose of legal certainty throughout the Union, the ‘backend operator’ should fall under the same liability rules as the producer, manufacturer and developer;
8. Considers that the existing fault-based tort law of the Member States offers in most cases a sufficient level of protection for persons that suffer harm caused by an interfering third person like a hacker or whose property is damaged by such a third person, as the interference regularly constitutes a fault-based action; notes that only for cases in which the third person is untraceable or impecunious, additional liability rules seem necessary;
9. Considers it, therefore, appropriate for this report to focus on civil liability claims against the deployer of an AI-system; affirms that the deployer’s liability is justified by the fact that he or she is controlling a risk associated with the AI-system, comparable to an owner of a car or pet; considers that due to the AI-system’s complexity and connectivity, the deployer will be in many cases the first visible contact point for the affected person;
Liability of the deployer
10. Opines that liability rules involving the deployer should in principle cover all operations of AI-systems, no matter where the operation takes place and whether it happens physically or virtually; remarks that operations in public spaces that expose many third persons to a risk constitute, however, cases that require further consideration; considers that the potential victims of harm or damage are often not aware of the operation and
regularly do not have contractual liability claims against the deployer; notes that when harm or damage materialises, such third persons would then only have a fault-liability claim, and they might find it difficult to prove the fault of the deployer of the AI- system;
11. Considers it appropriate to define the deployer as the person who decides on the use of the AI-system, who exercises control over the risk and who benefits from its operation; considers that exercising control means any action of the deployer that affects the manner of the operation from start to finish or that changes specific functions or processes within the AI-system;
12. Notes that there could be situations in which there is more than one deployer; considers that in that event, all deployers should be jointly and severally liable while having the right to recourse proportionally against each other;
Different liability rules for different risks
13. Recognises that the type of AI-system the deployer is exercising control over is a determining factor; notes that an AI-system that entails a high risk potentially endangers the general public to a much higher degree; considers that, based on the legal challenges that AI-systems pose to the existing liability regimes, it seems reasonable to set up a strict liability regime for those high-risk AI-systems;
14. Believes that an AI-system presents a high risk when its autonomous operation involves a significant potential to cause harm to one or more persons, in a manner that is random and impossible to predict in advance; considers that the significance of the potential depends on the interplay between the severity of possible harm, the likelihood that the risk materializes and the manner in which the AI-system is being used;
15. Recommends that all high-risk AI-systems be listed in an Annex to the proposed Regulation; recognises that, given the rapid technological change and the required technical expertise, it should be up to the Commission to review that Annex every six months and if necessary, amend it through a delegated act; believes that the Commission should closely cooperate with a newly formed standing committee similar to the existing Standing Committee on Precursors or the Technical Committee on Motor Vehicles, which include national experts of the Member States and stakeholders; considers that the balanced membership of the ‘High-Level Expert Group on Artificial Intelligence’ could serve as an example for the formation of the group of stakeholders;
16. Believes that in line with strict liability systems of the Member States, the proposed Regulation should only cover harm to the important legally protected rights such as life, health, physical integrity and property, and should set out the amounts and extent of compensation as well as the limitation period;
17. Determines that all activities, devices or processes driven by AI-systems that cause harm or damage but are not listed in the Annex to the proposed Regulation should remain subject to fault-based liability; believes that the affected person should nevertheless benefit from a presumption of fault of the deployer;
Insurances and AI-systems
18. Considers the liability risk to be one of the key factors that defines the success of new technologies, products and services; observes that proper risk coverage is also essential for assuring the public that it can trust the new technology despite the potential for suffering harm or for facing legal claims by affected persons;
19. Is of the opinion that, based on the significant potential to cause harm and by taking Directive 2009/103/EC7 into account, all deployers of high-risk AI-systems listed in the Annex to the proposed Regulation should hold liability insurance; considers that such a mandatory insurance regime for high-risk AI-systems should cover the amounts and the extent of compensation laid down by the proposed Regulation;
20. Believes that a European compensation mechanism, funded with public money, is not the right way to fill potential insurance gaps; considers that bearing the good experience with regulatory sandboxes in the fintech sector in mind, it should be up to the insurance market to adjust existing products or create new insurance cover for the numerous sectors and various different technologies, products and services that involve AI- systems;
21. Requests the Commission to submit, on the basis of Article 225 of the Treaty on the Functioning of the European Union, a proposal for a Regulation on liability for the operation of Artificial Intelligence-systems, following the recommendations set out in the Annex hereto;
22. Considers that the requested proposal will not have financial implications;
23. Instructs its President to forward this resolution and the accompanying recommendations to the Commission and the Council.
7 OJ L 263, 7.10.2009, p. 11.
OPERATION OF ARTIFICIAL INTELLIGENCE-SYSTEMS
This Report is addressing an important aspect of digitisation, which itself is shaped by cross- border activities and global competition. The following principles should serve as guidance:
A genuine Digital Single Market requires full harmonisation by a Regulation.
New legal challenges posed by the deployment of Artificial Intelligence (AI)- systems have to be addressed by establishing maximal legal certainty for the producer, the deployer, the affected person and any other third party.
There should be no over-regulation as this would hamper European innovation in AI, especially if the technology, product or service is developed by SMEs or start- ups.
Instead of replacing the well-functioning existing liability regimes, we should make a few specific adjustments by introducing new and future-orientated ideas.
This Report and the Product Liability Directive are two pillars of a common liability framework for AI-systems and require close coordination between all political actors.
Citizens need to be entitled to the same level of protection and rights, no matter if the harm is caused by an AI-system or not, or if it takes place physically or virtually.
B. PROPOSED LEGISLATIVE TEXT
Proposal for a
REGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL
on liability for the operation of Artificial Intelligence-systems
Having regard to the Treaty on the Functioning of the European Union, and in particular Article 114 thereof,
Having regard to the proposal from the European Commission,
After transmission of the draft legislative act to the national parliaments,
Having regard to the opinion of the European Economic and Social Committee1, Acting in accordance with the ordinary legislative procedure2,
(1) The concept of ‘liability’ plays an important double role in our daily life: on the one hand, it ensures that a person who has suffered harm or damage is entitled to claim compensation from the party proven to be liable for that harm or damage, and on the other hand, it provides the economic incentives for persons to avoid causing harm or damage in the first place. Any liability framework should strive to strike a balance between efficiently protecting potential victims of damage and at the same time, providing enough leeway to make the development of new technologies, products or services possible.
(2) Especially at the beginning of the life cycle of new products and services, there is a certain degree of risk for the user as well as for third persons that something does not function properly. This process of trial-and-error is at the same time a key enabler of technical progress without which most of our technologies would not exist. So far, the accompanying risks of new products and services have been properly mitigated by strong product safety legislation and liability rules.
(3) The rise of Artificial intelligence (AI) however presents a significant challenge for the existing liability frameworks. Using AI-systems in our daily life will lead to situations in which their opacity (“black box” element) makes it extremely expensive or even impossible to identify who was in control of the risk of using the AI-system in question or which code or input has caused the harmful operation. This difficulty is even compounded by the connectivity between an AI-system and other AI-systems and non- AI-systems, by its dependency on external data, by its vulnerability to cybersecurity
1 OJ ...
2 OJ ...
breaches as well as by the increasing autonomy of AI-systems triggered by machine- learning and deep-learning capabilities. Besides these complex features and potential vulnerabilities, AI-systems could also be used to cause severe harm, such as compromising our values and freedoms by tracking individuals against their will, by introducing Social Credit Systems or by constructing lethal autonomous weapon systems.
(4) At this point, it is important to point out that the advantages of deploying AI-systems will by far outweigh the disadvantages. They will help to fight climate change more effectively, to improve medical examinations, to better integrate disabled persons into the society and to provide tailor-made education courses to all types of students. To exploit the various technological opportunities and to boost people’s trust in the use of AI-systems, while at the same time preventing harmful scenarios, sound ethical standards combined with solid and fair compensation is the best way forward.
(5) Any discussion about required changes in the existing legal framework should start with the clarification that AI-systems have neither legal personality nor human conscience, and that their sole task is to serve humanity. Many AI-systems are also not so different from other technologies, which are sometimes based on even more complex software. Ultimately, the large majority of AI-systems are used for handling trivial tasks without any risks for the society. There are however also AI-systems that are deployed in a critical manner and are based on neuronal networks and deep-learning processes. Their opacity and autonomy could make it very difficult to trace back specific actions to specific human decisions in their design or in their operation. A deployer of such an AI- system might for instance argue that the physical or virtual activity, device or process causing the harm or damage was outside of his or her control because it was caused by an autonomous operation of his or her AI-system. The mere operation of an autonomous AI-system should at the same time not be a sufficient ground for admitting the liability claim. As a result, there might be liability cases in which a person who suffers harm or damage caused by an AI-system cannot prove the fault of the producer, of an interfering third party or of the deployer and ends up without compensation.
(6) Nevertheless, it should always be clear that whoever creates, maintains, controls or interferes with the AI-system, should be accountable for the harm or damage that the activity, device or process causes. This follows from general and widely accepted liability concepts of justice according to which the person that creates a risk for the public is accountable if that risk materializes. Consequently, the rise of AI-systems does not pose a need for a complete revision of liability rules throughout the Union. Specific adjustments of the existing legislation and very few new provisions would be sufficient to accommodate the AI-related challenges.
(7) Council Directive 85/374/EEC3 (the Product Liability Directive) has proven to be an
3 Council Directive 85/374/EEC of 25 July 1985 on the approximation of the laws, regulations and administrative provisions of the Member States concerning liability for defective products, OJ L 210, 7.8.1985,
effective means of getting compensation for damage triggered by a defective product. Hence, it should also be used with regard to civil liability claims of a party who suffers harm or damage against the producer of a defective AI-system. In line with the better regulation principles of the Union, any necessary legislative adjustments should be discussed during a review of that Directive. The existing fault-based liability law of the Member States also offers in most cases a sufficient level of protection for persons that suffer harm or damages caused by an interfering third person, as that interference regularly constitutes a fault-based action. Consequently, this Regulation should focus on claims against the deployer of an AI-system.
(8) The liability of the deployer under this Regulation is based on the fact that he or she controls a risk by operating an AI-system. Comparable to an owner of a car or pet, the deployer is able to exercise a certain level of control over the risk that the item poses. Exercising control thereby should be understood as meaning any action of the deployer that affects the manner of the operation from start to finish or that changes specific functions or processes within the AI-system.
(9) If a user, namely the person that utilises the AI-system, is involved in the harmful event, he or she should only be liable under this Regulation if the user also qualifies as a deployer. This Regulation should not consider the backend operator, who is the person continuously defining the features of the relevant technology and providing essential and ongoing backend support, to be a deployer and thus, its provisions should not apply to him or her. For the purpose of legal certainty throughout the Union, the backend operator should fall under the same liability rules as the producer, manufacturer and developer.
(10) This Regulation should cover in principle all AI-systems, no matter where they are operating and whether the operations take place physically or virtually. The majority of liability claims under this Regulation should however address cases of third party liability, where an AI-system operates in a public space and exposes many third persons to a risk. In that situation, the affected persons will often not be aware of the operating AI-system and will not have any contractual or legal relationship towards the deployer. Consequently, the operation of the AI-system puts them into a situation in which, in the event of harm or damage being caused, they only have fault-based liability claims against the deployer of the AI-system, while facing severe difficulties to prove fault on the part of the deployer.
(11) The type of AI-system the deployer is exercising control over is a determining factor. An AI-system that entails a high risk potentially endangers the public to a much higher degree and in a manner that is random and impossible to predict in advance. This means that at the start of the autonomous operation of the AI-system, the majority of the potentially affected persons are unknown and not identifiable (e.g. persons on a public
square or in a neighbouring house), compared to the operation of an AI-system that involves specific persons, who have regularly consented to its deployment before (e.g. surgery in a hospital or sales demonstration in a small shop). Determining how significant the potential to cause harm or damage by a high-risk AI-system should depend on the interplay between the manner in which the AI-system is being used, the severity of the potential harm or damage and the likelihood that the risk materialises. The degree of severity should be determined based on the extent of the potential harm resulting from the operation, the number of affected persons, the total value for the potential damage as well as the harm to society as a whole. The likelihood should be determined based on the role of the algorithmic calculations in the decision-making process, the complexity of the decision and the reversibility of the effects. Ultimately, the manner of usage should depend, among other things, on the sector in which the AI- system operates, if it could have legal or factual effects on important legally protected rights of the affected person, and whether the effects can reasonably be avoided.
(12) All AI-systems with a high risk should be listed in an Annex to this Regulation. Given the rapid technical and market developments as well as the technical expertise which is required for an adequate review of AI-systems, the power to adopt delegated acts in accordance with Article 290 of the Treaty on the Functioning of the European Union should be delegated to the Commission to amend this Regulation in respect of the types of AI-systems that pose a high risk and the critical sectors where they are used. Based on the definitions and provisions laid down in this Regulation, the Commission should review the Annex every six months and, if necessary, amend it by means of delegated acts. To give businesses enough planning and investment security, changes to the critical sectors should only be made every 12 months. Developers are called upon to notify the Commission if they are currently working on a new technology, product or service that falls under one of the existing critical sectors provided for in the Annex and which later could qualify for a high risk AI-system.
(13) It is of particular importance that the Commission carry out appropriate consultations during its preparatory work, including at expert level, and that those consultations be conducted in accordance with the principles laid down in the Interinstitutional Agreement of 13 April 2016 on Better Law-Making4. A standing committee called 'Technical Committee – high-risk AI-systems' (TCRAI) should support the Commission in its review under this Regulation. That standing committee should comprise representatives of the Member States as well as a balanced selection of stakeholders, including consumer organisation, businesses representatives from different sectors and sizes, as well as researchers and scientists. In particular, to ensure equal participation in the preparation of delegated acts, the European Parliament and the Council receive all documents at the same time as Member States' experts, and their experts systematically have access to meetings of Commission expert groups as well as the standing TCRAI-
4 OJ L 123, 12.5.2016, p. 1.
committee, when dealing with the preparation of delegated acts.
(14) In line with strict liability systems of the Member States, this Regulation should cover only harm or damage to life, health, physical integrity and property. For the same reason, it should determine the amount and extent of compensation, as well as the limitation period for bringing forward liability claims. In contrast to the Product Liability Directive, this Regulation should set out a significantly lower ceiling for compensation, as it only refers to a single operation of an AI-system, while the former refers to a number of products or even a product line with the same defect.
(15) All physical or virtual activities, devices or processes driven by AI-systems that are not listed as a high-risk AI-system in the Annex to this Regulation should remain subject to fault-based liability. The national laws of the Member States, including any relevant jurisprudence, with regard to the amount and extent of compensation as well as the limitation period should continue to apply. A person who suffers harm or damage caused by an AI-system should however benefit from the presumption of fault of the deployer.
(16) The diligence which can be expected from a deployer should be commensurate with (i) the nature of the AI system, (ii) the legally protected right potentially affected, (iii) the potential harm or damage the AI-system could cause and (iv) the likelihood of such damage. Thereby, it should be taken into account that the deployer might have limited knowledge of the algorithms and data used in the AI-system. It should be presumed that the deployer has observed due care in selecting a suitable AI-system, if the deployer has selected an AI-system which has been certified under [the voluntary certification scheme envisaged on p. 24 of COM(2020) 65 final]. It should be presumed that the deployer has observed due care during the operation of the AI-system, if the deployer can prove to have actually and regularly monitored the AI-system during its operation and to have notified the manufacturer about potential irregularities during the operation. It should be presumed that the deployer has observed due care as regards maintaining the operational reliability, if the deployer installed all available updates provided by the producer of the AI-system.
(17) In order to enable the deployer to prove that he or she was not at fault, the producers should have the duty to collaborate with the deployer. European as well as non- European producers should furthermore have the obligation to designate an AI-liability- representative within the Union as a contact point for replying to all requests from deployers, taking similar provisions set out in Article 37 GDPR (data protection officers), Articles 3(41) and 13(4) of Regulation 2018/858 of the European Parliament and of the Council5 and Articles 4(2) and 5 of Regulation 2019/1020 of the European
5 Regulation (EU) 2018/858 of the European Parliament and of the Council of 30 May 2018 on the approval and market surveillance of motor vehicles and their trailers, and of systems, components and separate technical units intended for such vehicles, amending Regulations (EC) No 715/2007 and (EC) No 595/2009 and repealing Directive 2007/46/EC
Parliament and of the Council6 (manufacturer's representative) into account.
(18) The legislator has to consider the liability risks connected to AI-systems during their whole lifecycle, from development to usage to end of life. The inclusion of AI-systems in a product or service represents a financial risk for businesses and consequently will have a heavy impact on the ability and options for small and medium-sized enterprises (SME) as well as for start-ups in relation to insuring and financing their projects based on new technologies. The purpose of liability is, therefore, not only to safeguard important legally protected rights of individuals but also a factor which determines whether businesses, especially SMEs and start-ups, are able to raise capital, innovate and ultimately offer new products and services, as well as whether the customers are willing to use such products and services despite the potential risks and legal claims being brought against them.
(19) Insurance can help to ensure that victims can receive effective compensation as well as to pool the risks of all insured persons. One of the factors on which insurance companies base their offer of insurance products and services is risk assessment based on access to sufficient historical claim data. A lack of access to, or an insufficient quantity of high quality data could be a reason why creating insurance products for new and emerging technologies is difficult at the beginning. However, greater access to and optimising the use of data generated by new technologies will enhance insurers’ ability to model emerging risk and to foster the development of more innovative cover .
(20) Despite missing historical claim data, there are already insurance products that are developed area-by-area and cover-by-cover as technology develops. Many insurers specialise in certain market segments (e.g. SMEs) or in providing cover for certain product types (e.g. electrical goods), which means that there will usually be an insurance product available for the insured. If a new type of insurance is needed, the insurance market will develop and offer a fitting solution and thus, will close the insurance gap. In exceptional cases, in which the compensation significantly exceeds the maximum amounts set out in this Regulation, Member States should be encouraged to set up a special compensation fund for a limited period of time that addresses the specific needs of those cases.
(21) It is of utmost importance that any future changes to this text go hand in hand with a necessary review of the PLD. The introduction of a new liability regime for the deployer of AI-systems requires that the provisions of this Regulation and the review of the PLD should be closely coordinated in terms of substance as well as approach so that they together constitute a consistent liability framework for AI-systems, balancing the interests of producer, deployer and the affected person, as regards the liability risk.
(OJ L 151, 14.6.2018, p. 1).
6 Regulation (EU) 2019/1020 of the European Parliament and of the Council of 20 June 2019 on market surveillance and compliance of products and amending Directive 2004/42/EC and Regulations (EC) No 765/2008 and (EU) No 305/2011 (OJ L 169, 25.6.2019, p. 1).
Adapting and streamlining the definitions of AI-system, deployer, producer, developer, defect, product and service throughout all pieces of legislation is therefore necessary.
(22) Since the objectives of this Regulation, namely to create a future-orientated and unified approach at Union level, which sets common European standards for our citizens and businesses and to ensure the consistency of rights and legal certainty throughout the Union, in order to avoid fragmentation of the Digital Single Market, which would hamper the goal of maintaining digital sovereignty and of fostering digital innovation in Europe, require that the liability regimes for AI-systems are fully harmonized. Since this cannot be sufficiently achieved by the Member States due to the rapid technological change, the cross-border development as well as the usage of AI-systems and eventually, the conflicting legislative approaches across the Union, but can rather, by reason of the scale or effects of the action, be achieved at Union level. The Union may adopt measures, in accordance with the principle of subsidiarity as set out in Article 5 of the Treaty on European Union. In accordance with the principle of proportionality as set out in that Article, this Regulation does not go beyond what is necessary in order to achieve these objectives,
HAVE ADOPTED THIS REGULATION:
Chapter I General provisions
This Regulation sets out rules for the civil liability claims of natural and legal persons against the deployer of AI-systems.
1. This Regulation applies on the territory of the Union where a physical or virtual activity, device or process driven by an AI-system has caused harm or damage to the life, health, physical integrity or the property of a natural or legal person.
2. Any agreement between a deployer of an AI-system and a natural or legal person who suffers harm or damage because of the AI-system, which circumvents or limits the rights and obligations set out in this Regulation, whether concluded before or after the harm or damage has been caused, shall be deemed null and void.
3. This Regulation is without prejudice to any additional liability claims resulting from contractual relationships between the deployer and the natural or legal person who suffered harm or damage because of the AI-system.
For the purposes of this Regulation, the following definitions apply:
(a) ‘AI-system’ means a system that displays intelligent behaviour by analysing certain input and taking action, with some degree of autonomy, to achieve specific goals. AI- systems can be purely software-based, acting in the virtual world, or can be embedded in hardware devices;
(b) ‘autonomous’ means an AI-system that operates by perceiving certain input and without needing to follow a set of pre-determined instructions, despite its behaviour being constrained by the goal it was given and other relevant design choices made by its developer;
(c) ‘high risk’ means a significant potential in an autonomously operating AI-system to cause harm or damage to one or more persons in a manner that is random and impossible to predict in advance; the significance of the potential depends on the interplay between the severity of possible harm or damage, the likelihood that the risk materializes and the manner in which the AI-system is being used;
(d) ‘deployer’ means the person who decides on the use of the AI-system, exercises control over the associated risk and benefits from its operation;
(e) ‘affected person’ means any person who suffers harm or damage caused by a physical or virtual activity, device or process driven by an AI-system, and who is not its deployer;
(f) ‘harm or damage’ means an adverse impact affecting the life, health, physical integrity or property of a natural or legal person, with the exception of non-material harm;
(g) ‘producer’ means the developer or the backend operator of an AI-system, or the producer as defined in Article 3 of Council Directive 85/374/EEC7.
Chapter II High-risk AI-systems
Strict liability for high-risk AI-systems
1. The deployer of a high-risk AI-system shall be strictly liable for any harm or damage that was caused by a physical or virtual activity, device or process driven by that AI-system.
2. The high-risk AI-systems as well as the critical sectors where they are used shall be listed in the Annex to this Regulation. The Commission is empowered to adopt delegated acts in accordance with Article 13, to amend the exhaustive list in the Annex, by:
(a) including new types of high-risk AI-systems and critical sectors in which they are deployed;
(b) deleting types of AI-systems that can no longer be considered to pose a high risk; and/or
(c) changing the critical sectors for existing high-risk AI-systems.
Any delegated act amending the Annex shall come into force six months after its adoption. When determining new critical sectors and/or high-risk AI-systems to be inserted by means of delegated acts in the Annex, the Commission shall take full account of the criteria set out in this Regulation, in particular those set out in Article 3(c).
3. The deployer of a high-risk AI-system shall not be able to exonerate himself or herself by arguing that he or she acted with due diligence or that the harm or damage was caused by an autonomous activity, device or process driven by his or her AI-system. The deployer shall not be held liable if the harm or damage was caused by force majeure.
4. The deployer of a high-risk AI-system shall ensure they have liability insurance cover that is adequate in relation to the amounts and extent of compensation provided for in Article 5
7 Council Directive 85/374/EEC of 25 July 1985 on the approximation of the laws, regulations and administrative provisions of the Member States concerning liability for defective products, OJ L 210, 7.8.1985, p. 29.
and 6 of this Regulation. If compulsory insurance regimes already in force pursuant to other Union or national law are considered to cover the operation of the AI-system, the obligation to take out insurance for the AI-system pursuant to this Regulation shall be deemed fulfilled, as long as the relevant existing compulsory insurance covers the amounts and the extent of compensation provided for in Articles 5 and 6 of this Regulation.
5. This Regulation shall prevail over national liability regimes in the event of conflicting strict liability classification of AI-systems.
Amount of compensation
1. A deployer of a high-risk AI-system that has been held liable for harm or damage under this Regulation shall compensate:
(a) up to a maximum total amount of EUR ten million in the event of death or of harm caused to the health or physical integrity of one or several persons as the result of the same operation of the same high-risk AI-system;
(b) up to a maximum total amount of EUR two million in the event of damage caused to property, including when several items of property of one or several persons were damaged as a result of the same operation of the same high-risk AI-system; where the affected person also holds a contractual liability claim against the deployer, no compensation shall be paid under this Regulation if the total amount of the damage to property is of a value that falls below EUR 500.
2. Where the combined compensation to be paid to several persons who suffer harm or damage caused by the same operation of the same high-risk AI-system exceeds the maximum total amounts provided for in paragraph 1, the amounts to be paid to each person shall be reduced pro-rata so that the combined compensation does not exceed the maximum amounts set out in paragraph 1.
Extent of compensation
1. Within the amount set out in Article 5(1)(a), compensation to be paid by the deployer held liable in the event of physical harm followed by the death of the affected person, shall be calculated based on the costs of medical treatment that the affected person underwent prior to his or her death, and of the pecuniary prejudice sustained prior to death caused by the cessation or reduction of the earning capacity or the increase in his or her needs for the duration of the harm prior to death. The deployer held liable shall furthermore reimburse the funeral costs for the deceased affected person to the party who is responsible for defraying those expenses.
If at the time of the incident that caused the harm leading to his or her death, the affected
person was in a relationship with a third party and had a legal obligation to support that third party, the deployer held liable shall indemnify the third party by paying maintenance to the extent to which the affected person would have been obliged to pay, for the period corresponding to an average life expectancy for a person of his or her age and general description. The deployer shall also indemnify the third party if, at the time of the incident that caused the death, the third party had been conceived but had not yet been born.
2. Within the amount set out in Article 5(1)(b), compensation to be paid by the deployer held liable in the event of harm to the health or the physical integrity of the affected person shall include the reimbursement of the costs of the related medical treatment as well as the payment for any pecuniary prejudice sustained by the affected person, as a result of the temporary suspension, reduction or permanent cessation of his or her earning capacity or the consequent, medically certified increase in his or her needs.
1. Civil liability claims, brought in accordance with Article 4(1), concerning harm to life, health or physical integrity, shall be subject to a special limitation period of 30 years from the date on which the harm occurred.
2. Civil liability claims, brought in accordance with Article 4(1), concerning damage to property shall be subject to a special limitation period of:
(a) 10 years from the date when the property damage occurred, or
(b) 30 years from the date on which the operation of the high-risk AI-system that subsequently caused the property damage took place.
Of the periods referred to in the first subparagraph, the period that ends first shall be applicable.
3. This Article shall be without prejudice to national law regulating the suspension orinterruption of limitation periods.
Chapter III Other AI-systems Article 8
Fault-based liability for other AI-systems
1. The deployer of an AI-system that is not defined as a high-risk AI-system, in accordance to Article 3(c) and, as a result is not listed in the Annex to this Regulation, shall be subject to fault-based liability for any harm or damage that was caused by a physical or virtual activity, device or process driven by the AI-system.
2. The deployer shall not be liable if he or she can prove that the harm or damage was caused without his or her fault, relying on either of the following grounds’:
(a) the AI-system was activated without his or her knowledge while all reasonable and necessary measures to avoid such activation were taken, or
(b) due diligence was observed by selecting a suitable AI-system for the right task and skills, putting the AI-system duly into operation, monitoring the activities and maintaining the operational reliability by regularly installing all available updates.
The deployer shall not be able to escape liability by arguing that the harm or damage was caused by an autonomous activity, device or process driven by his or her AI-system. The deployer shall not be liable if the harm or damage was caused by force majeure.
3. Where the harm or damage was caused by a third party that interfered with the AI-system by modifying its functioning, the deployer shall nonetheless be liable for the payment of compensation if such third party is untraceable or impecunious.
4. At the request of the deployer, the producer of an AI-system shall have the duty of collaborating with the deployer to the extent warranted by the significance of the claim in order to allow the deployer to prove that he or she acted without fault.
National provisions on compensation and limitation period
Civil liability claims brought in accordance with Article 8(1) shall be subject, in relation to limitation periods as well as the amounts and the extent of compensation, to the laws of the Member State in which the harm or damage occurred.
Chapter IV Apportionment of liability Article 10
1. If the harm or damage is caused both by a physical or virtual activity, device or process driven by an AI-system and by the actions of an affected person or of any person for whom the affected person is responsible, the deployer’s extent of liability under this Regulation shall be reduced accordingly. The deployer shall not be liable if the affected person or the person for whom he or she is responsible is solely or predominantly accountable for the harm or damage caused.
2. A deployer held liable may use the data generated by the AI-system to prove contributory negligence on the part of the affected person.
Joint and several liability
If there is more than one deployer of an AI-system, they shall be jointly and severally liable.
If any of the deployers is also the producer of the AI-system, this Regulation shall prevail over the Product Liability Directive.
Recourse for compensation
1. The deployer shall not be entitled to pursue a recourse action unless the affected person, who is entitled to receive compensation under this Regulation, has been paid in full.
2. In the event that the deployer is held jointly and severally liable with other deployers in respect of an affected person and has fully compensated that affected person, in accordance with Article 4(1) or 8(1), that deployer may recover part of the compensation from the other deployers, in proportion to his or her liability. Deployers, that are jointly and severally liable, shall be obliged in equal proportions in relation to one another, unless otherwise determined. If the contribution attributable to a jointly and severally liable deployer cannot be obtained from him or her, the shortfall shall be borne by the other deployers. To the extent that a jointly and severally liable deployer compensates the affected person and demands adjustment of advancements from the other liable deployers, the claim of the affected person against the other deployers shall be subrogated to him or her. The subrogation of claims shall not be asserted to the disadvantage of the original claim.
3. In the event that the deployer of a defective AI-system fully indemnifies the affected person for harm or damages in accordance with Article 4(1) or 8(1), he or she may take action for redress against the producer of the defective AI-system according to Directive 85/374/EEC and to national provisions concerning liability for defective products.
4. In the event that the insurer of the deployer indemnifies the affected person for harm or damage in accordance with Article 4(1) or 8(1), any civil liability claim of the affected person against another person for the same damage shall be subrogated to the insurer of the deployer to the amount the insurer of the deployer has compensated the affected person.
Chapter V Final provisions Article 13
Exercise of the delegation
1. The power to adopt delegated acts is conferred on the Commission subject to the conditions laid down in this Article.
2. The power to adopt delegated acts referred to in Article 4(2) shall be conferred on the Commission for a period of five years from [date of application of this Regulation].
3. The delegation of power referred to in Article 4(2) may be revoked at any time by the European Parliament or by the Council. A decision to revoke shall put an end to the delegation of the power specified in that decision. It shall take effect the day following the
publication of the decision in the Official Journal of the European Union or at a later date specified therein. It shall not affect the validity of any delegated acts already in force.
4. Before adopting a delegated act, the Commission shall consult the standing Technical Committee for high-risk AI-systems (TCRAI-committee) in accordance with the principles laid down in the Interinstitutional Agreement on Better Law-Making of 13 April 2016.
5. As soon as it adopts a delegated act, the Commission shall notify it simultaneously to the European Parliament and to the Council.
6. A delegated act adopted pursuant to Article 4(2) shall enter into force only if no objection has been expressed by either the European Parliament or the Council within a period of two months of notification or if, before the expiry of that period, the European Parliament and the Council have both informed the Commission that they will not object. That period shall be extended by two months at the initiative of the European Parliament or of the Council.
By 1 January 202X [5 years after the date of application of this Regulation], and every three years thereafter, the Commission shall present to the European Parliament, the Council and the European Economic and Social Committee a detailed report reviewing this Regulation in the light of the further development of Artificial Intelligence.
When preparing the report referred to in the first subparagraph, the Commission shall request relevant information from Member States relating to case law, court settlements as well as accident statistics, such as the number of accidents, damage done, AI applications involved, compensation paid by insurance companies.
The Commission’s report shall be accompanied, where appropriate, by legislative proposals.
Entry into force
This Regulation shall enter into force on the twentieth day following that of its publication in
the Official Journal of the European Union. It shall apply from 1 January 202X.
This Regulation shall be binding in its entirety and directly applicable in the Member States.
Exhaustive list of AI-systems that pose a high risk as well as of critical sectors where the AI-systems are being deployed
AI-systems Critical sector
Unmanned aircraft within the meaning of Art 3(30) of Regulation (EU) 2018/1139
Vehicles with automation levels 4 and 5 according to SAE J3016
Autonomous Traffic Management Systems
Autonomous public places cleaning devices
This Annex should aim to replicate the level of detail that appears for instance in Annex I of Regulation 2018/858 (Approval and market surveillance of motor vehicles and their trailers, and of systems, components and separate technical units intended for such vehicle).
The concept of ‘liability’ plays an important double role in our daily life: on the one hand, it ensures that a person who has suffered harm is entitled to claim compensation from the person proven to be liable for that harm, and on the other hand, it provides the economic incentives for persons to avoid causing harm in the first place. Any future-orientated liability framework should therefore strive to strike the balance between efficiently protecting potential victims of damage and at the same time, providing enough leeway to make the development of new technologies, products or services possible.
Especially at the beginning of the life cycle of new products and services, there is a certain degree of risk for the user as well as third persons that something is not function properly. This process of trial-and-error is however also a key enabler of technical progress without whom most of our technologies would not exist today. So far, Europe’s strong product safety regulations and liability rules were more than capable to deal with the potentially higher risks of new technologies. In the eyes of many people, this certitude is now being challenged by the rise of Artificial Intelligence (AI). What makes this technology unique is its ‘opacity’ or in other words, its ‘black box’ feature. Combined with its connectivity, dependency on external data, vulnerability to cybersecurity breaches and a distinctive autonomy, the involvement of AI-systems could make it extremely expensive or even impossible to identify who was in control or which code or input has ultimately caused the harmful operation. As a result, the harmed person could face difficulties to get compensation.
Even though AI-systems are indeed posing new legal challenges to our existing liability regime, they are materially in many cases not so different to other technologies, which sometimes are based on even more sophisticated software. Modern AI-systems regularly function rather trivial and are far away from conscious robots we know from Sci-Fi movies. Any discussion about giving AI-systems legal personality is therefore obsolete. Choosing a sensible approach to address the legal challenges posed by new AI-systems means that we refrain from major changes to our liability framework. If a person suffered harm caused by a defective AI-system, the Product Liability Directive (PLD) should remain the legal means to seek compensation from the producer. If the harm was caused by an interfering third person, the existing fault-based liability system in the Member States offer (in most cases) a sufficient level of protection. In line with better regulation principles of the Union, any necessary legislative adjustments with regard to producers and interfering third persons should be discussed in these respective legal frameworks.
This report makes nonetheless one crucial exception from its faith in the existing liability regimes: it sees a legal gap when it comes to the liability of the deployers of AI-systems. Although these persons are deciding on the use of AI-systems, are the ones who are mainly exercising control over the associated risks and are benefiting from their operations, many liability claims against them would fail due to the inability of the affected persons to prove the deployer’s fault. Especially in cases, where the harm was caused by an operation of an AI- system in a public space, the potentially enormous group of affected person would regularly not hold any contractual relationship towards the deployer, leaving them with almost no chance of being compensated for their harm. The Rapporteur propose two different approaches to solve this legal gap, depending on the level of risk the AI-system entails:
(1) High-risk AI-systems: The deployer of such a system is in quite a similar position as the owner of a car or a pet. He or she exercises control over an item that significantly endangers the public, in a manner that is random and impossible to predict in advance. Consequently, the deployer - like the owner of a car or pet - should be subject to a strict liability regime and compensate the victim within a certain extent and certain amount of money for any harm to its important legally protected rights (life, health, physical integrity, property). This Report defines clear criteria on which AI-systems can qualify as high-risk and list them exhaustively in an ANNEX. Given the rapid technical and market developments and given the technical expertise that is required for an adequate review of an AI-system, it should be up to the European Commission to amend the ANNEX through delegated acts. A newly formed standing committee, involving national experts and stakeholders, should support the Commission in its review of potentially high-risk AI-systems.
(2) All other AI-systems: The person who suffered harm caused by an AI-systems that is not listed in the Annex, should nevertheless benefit from a presumption of fault towards the deployer. The national law regulating the amount and extent of compensation as well as the limitation period in case of harm caused by the AI-system remain applicable.
Any proposal for new legislation needs to analyse profoundly the existing laws to avoid duplication or conflicting provisions. Based on this principle, the Report does only cover harm to life, health, physical integrity and property. Although AI-systems can admittedly cause considerable harm to personal rights and other important legally protected interests, those infringements are much better addressed by already existing and tailor-made legal provisions in those areas (e.g. anti-discrimination law or consumer protection law). For the very same reason, the use of biometric data or of face recognition techniques by AI-systems were not incorporated by the Rapporteur; any unauthorized use in this area is already covered by specific data protection laws such as the GDPR. With regard to conflicting national liability regimes when it comes to the question if an AI-system falls under strict liability or with regard to the limiting effect of contractual agreements, this Report holds that its provisions always prevail. It moreover aims to achieve full compensation for the affected person by the deployer, before potential liability claims against the producer can be brought forward by other persons than the affected person. For the purpose of legal certainty throughout the Union, the backend operator - which is not covered by this Regulation - should fall under the same liability rules as the producer, manufacturer and developer.
As the European Union and its Member States do not require radical changes to their liability frameworks, AI-systems also should not push us away from our traditional insurances systems. Publicly funded compensation mechanisms are no adequate answer to the rise of Artificial Intelligence. Such compensation regimes would only impose an unnecessary financial burden on taxpayer. Despite the lack of access to quality historical claims data involving AI-systems, European insurers are already developing new products area-by-area and cover-by-cover as the technology develops further. If there is a need for a new cover, the insurance market will come up with an adequate solution. It would be wrong to fall for hypothetical scenarios that are being used to lobby for additional public systems. If one day a mass harm event like a large terrorist attack materializes, Member States could set up special compensation funds for a limited period of time as it already happened in the past.
Consequently, this Report solely requires deployers of high-risk AI-systems to hold an adequate liability insurance (comparable with the obligation set up by the Motor Insurance
Directive), which covers the amounts and the extent of compensation determined by this Regulation. The Rapporteur strongly believes in the insurance market to either adapt existing insurance covers or to come up with various new products that each separately cover the different types of AI-systems in different sectors.
With its narrow but clear approach on liability rules for the deployer of AI-systems, the Rapporteur is convinced to strike the balance between effectively protecting the society while allowing this exciting technology to innovate further. Way too often only the risks of Artificial Intelligence are singled out. Yes, AI-systems could be used to do bad things. But do we want to allow negative manifestations – that happen with all technologies from mobile phones to nuclear power – to restrict our general use? Do we want to pass on the help of AI- systems in our fight against climate change, to improve our health care system or to better integrate persons with disabilities? This Report strongly advises to focus on exploiting the positives effects of AI-systems, while building up strong safeguards.
Thereby, all new laws on Artificial Intelligence should be written in form of regulations. As the digital sphere is characterized by rapid cross-border dynamics, our European Digital Single Market needs to be fully harmonized to catch up with the global digital competition. It is crucial to emphasise that the political discussion on this Regulation should go hand in hand with a necessary Review of the PLD. The introduction of a new liability regime for the deployer of AI-systems requires that the negotiations on this Report and the Review of the PLD should be closely coordinated in terms of substance as well as approach so that they together constitute a consistent liability framework for AI-systems, balancing the interest of producer, deployer and the affected person, as regards the liability risk. Adapting and streamlining the definitions of AI-system, deployer, producer, developer, defect, product and service throughout all legislative initiatives seem therefore necessary.
Last but not least, the political players should realise that the technological progress does not stop during their legislative negotiations. If we are serious with our goal to keep up with digitisation, to maintain our digital sovereignty and to play a major role in the digital age, the European Institutions need to send a clear political message to our successful industry and to our bright researchers working on new AI-systems. Until the legislative response to the rise of Artificial Intelligence becomes law, industry and researchers should be able to innovate according to the current rules and should benefit from a five-year-long transition period. If we are not granting them this planning certainty, Europe will miss out on numerous new fascinating technologies, products or services.