2020-02-19T12:33:12+01:00 2020-02-19T12:33:12+01:00 engines={image=false textcolor=true vector=false text=true} tetml={filename={D:\Drim\repository\projects\drim_66_0\metadata\7\9\7988ec8056c444a2091e5c0de72cb318\7988ec8056c444a2091e5c0de72cb318.para.xml} elements={annotations=false bookmarks=false destinations=false fields=false javascripts=false} } granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} EUROPEAN E U R O P E A N COMMISSION C O M M I S S I O N Brussels, B r u s s e l s , 19.2.2020 1 9 . 2 . 2 0 2 0 COM(2020) C O M ( 2 0 2 0 ) 65 6 5 final f i n a l WHITE W H I T E PAPER P A P E R On O n Artificial A r t i f i c i a l Intelligence I n t e l l i g e n c e - - A A European E u r o p e a n approach a p p r o a c h to t o excellence e x c e l l e n c e and a n d trust t r u s t EN E N EN E N granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} White W h i t e Paper P a p e r on o n Artificial A r t i f i c i a l Intelligence I n t e l l i g e n c e A A European E u r o p e a n approach a p p r o a c h to t o excellence e x c e l l e n c e and a n d trust t r u s t Artificial A r t i f i c i a l Intelligence I n t e l l i g e n c e is i s developing d e v e l o p i n g fast. f a s t . It I t will w i l l change c h a n g e our o u r lives l i v e s by b y improving i m p r o v i n g healthcare h e a l t h c a r e (e.g. ( e . g . making m a k i n g diagnosis d i a g n o s i s more m o r e precise, p r e c i s e , enabling e n a b l i n g better b e t t e r prevention p r e v e n t i o n of o f diseases), d i s e a s e s ) , increasing i n c r e a s i n g the t h e efficiency e f f i c i e n c y of o f farming, f a r m i n g , contributing c o n t r i b u t i n g to t o climate c l i m a t e change c h a n g e mitigation m i t i g a t i o n and a n d adaptation, a d a p t a t i o n , improving i m p r o v i n g the t h e efficiency e f f i c i e n c y of o f production p r o d u c t i o n systems s y s t e m s through t h r o u g h predictive p r e d i c t i v e maintenance, m a i n t e n a n c e , increasing i n c r e a s i n g the t h e security s e c u r i t y of o f Europeans, E u r o p e a n s , and a n d in i n many m a n y other o t h e r ways w a y s that t h a t we w e can c a n only o n l y begin b e g i n to t o imagine. i m a g i n e . At A t the t h e same s a m e time, t i m e , Artificial A r t i f i c i a l Intelligence I n t e l l i g e n c e (AI) ( A I ) entails e n t a i l s a a number n u m b e r of o f potential p o t e n t i a l risks, r i s k s , such s u c h as a s opaque o p a q u e decision-making, d e c i s i o n - m a k i n g , gender-based g e n d e r - b a s e d or o r other o t h e r kinds k i n d s of o f discrimination, d i s c r i m i n a t i o n , intrusion i n t r u s i o n in i n our o u r private p r i v a t e lives l i v e s or o r being b e i n g used u s e d for f o r criminal c r i m i n a l purposes. p u r p o s e s . Against A g a i n s t a a background b a c k g r o u n d of o f fierce f i e r c e global g l o b a l competition, c o m p e t i t i o n , a a solid s o l i d European E u r o p e a n approach a p p r o a c h is i s needed, n e e d e d , building b u i l d i n g on o n the t h e European E u r o p e a n strategy s t r a t e g y for f o r AI A I presented p r e s e n t e d in i n April A p r i l 2018 2 0 1 8 1 1 . . To T o address a d d r e s s the t h e opportunities o p p o r t u n i t i e s and a n d challenges c h a l l e n g e s of o f AI, A I , the t h e EU E U must m u s t act a c t as a s one o n e and a n d define d e f i n e its i t s own o w n way, w a y , based b a s e d on o n European E u r o p e a n values, v a l u e s , to t o promote p r o m o t e the t h e development d e v e l o p m e n t and a n d deployment d e p l o y m e n t of o f AI. A I . The T h e Commission C o m m i s s i o n is i s committed c o m m i t t e d to t o enabling e n a b l i n g scientific s c i e n t i f i c breakthrough, b r e a k t h r o u g h , to t o preserving p r e s e r v i n g the t h e EU’s E U ’ s technological t e c h n o l o g i c a l leadership l e a d e r s h i p and a n d to t o ensuring e n s u r i n g that t h a t new n e w technologies t e c h n o l o g i e s are a r e at a t the t h e service s e r v i c e of o f all a l l Europeans E u r o p e a n s – – improving i m p r o v i n g their t h e i r lives l i v e s while w h i l e respecting r e s p e c t i n g their t h e i r rights. r i g h t s . Commission C o m m i s s i o n President P r e s i d e n t Ursula U r s u l a von v o n der d e r Leyen L e y e n announced a n n o u n c e d in i n her h e r political p o l i t i c a l Guidelines G u i d e l i n e s 2 2 a a coordinated c o o r d i n a t e d European E u r o p e a n approach a p p r o a c h on o n the t h e human h u m a n and a n d ethical e t h i c a l implications i m p l i c a t i o n s of o f AI A I as a s well w e l l as a s a a reflection r e f l e c t i o n on o n the t h e better b e t t e r use u s e of o f big b i g data d a t a for f o r innovation. i n n o v a t i o n . Thus, T h u s , the t h e Commission C o m m i s s i o n supports s u p p o r t s a a regulatory r e g u l a t o r y and a n d investment i n v e s t m e n t oriented o r i e n t e d approach a p p r o a c h with w i t h the t h e twin t w i n objective o b j e c t i v e of o f promoting p r o m o t i n g the t h e uptake u p t a k e of o f AI A I and a n d of o f addressing a d d r e s s i n g the t h e risks r i s k s associated a s s o c i a t e d with w i t h certain c e r t a i n uses u s e s of o f this t h i s new n e w technology. t e c h n o l o g y . The T h e purpose p u r p o s e of o f this t h i s White W h i t e Paper P a p e r is i s to t o set s e t out o u t policy p o l i c y options o p t i o n s on o n how h o w to t o achieve a c h i e v e these t h e s e objectives. o b j e c t i v e s . It I t does d o e s not n o t address a d d r e s s +the + t h e development d e v e l o p m e n t and a n d use u s e of o f AI A I for f o r military m i l i t a r y purposes.The p u r p o s e s . T h e Commission C o m m i s s i o n invites i n v i t e s Member M e m b e r States, S t a t e s , other o t h e r European E u r o p e a n institutions, i n s t i t u t i o n s , and a n d all a l l stakeholders, s t a k e h o l d e r s , including i n c l u d i n g industry, i n d u s t r y , social s o c i a l partners, p a r t n e r s , civil c i v i l society s o c i e t y organisations, o r g a n i s a t i o n s , researchers, r e s e a r c h e r s , the t h e public p u b l i c in i n general g e n e r a l and a n d any a n y interested i n t e r e s t e d party, p a r t y , to t o react r e a c t to t o the t h e options o p t i o n s below b e l o w and a n d to t o contribute c o n t r i b u t e to t o the t h e Commission’s C o m m i s s i o n ’ s future f u t u r e decision-making d e c i s i o n - m a k i n g in i n this t h i s domain. d o m a i n . 1. 1 . INTRODUCTION I N T R O D U C T I O N As A s digital d i g i t a l technology t e c h n o l o g y becomes b e c o m e s an a n ever e v e r more m o r e central c e n t r a l part p a r t of o f every e v e r y aspect a s p e c t of o f people’s p e o p l e ’ s lives, l i v e s , people p e o p l e should s h o u l d be b e able a b l e to t o trust t r u s t it. i t . Trustworthiness T r u s t w o r t h i n e s s is i s also a l s o a a prerequisite p r e r e q u i s i t e for f o r its i t s uptake. u p t a k e . This T h i s is i s a a chance c h a n c e for f o r Europe, E u r o p e , given g i v e n its i t s strong s t r o n g attachment a t t a c h m e n t to t o values v a l u e s and a n d the t h e rule r u l e of o f law l a w as a s well w e l l as a s its i t s proven p r o v e n capacity c a p a c i t y to t o build b u i l d safe, s a f e , reliable r e l i a b l e and a n d sophisticated s o p h i s t i c a t e d products p r o d u c t s and a n d services s e r v i c e s from f r o m aeronautics a e r o n a u t i c s to t o energy, e n e r g y , automotive a u t o m o t i v e and a n d medical m e d i c a l equipment. e q u i p m e n t . Europe’s E u r o p e ’ s current c u r r e n t and a n d future f u t u r e sustainable s u s t a i n a b l e economic e c o n o m i c growth g r o w t h and a n d societal s o c i e t a l wellbeing w e l l b e i n g increasingly i n c r e a s i n g l y draws d r a w s on o n value v a l u e created c r e a t e d by b y data. d a t a . AI A I is i s one o n e of o f the t h e most m o s t important i m p o r t a n t applications a p p l i c a t i o n s of o f the t h e data d a t a economy. e c o n o m y . Today T o d a y most m o s t data d a t a are a r e related r e l a t e d to t o consumers c o n s u m e r s and a n d are a r e stored s t o r e d and a n d processed p r o c e s s e d on o n central c e n t r a l cloud-based c l o u d - b a s e d infrastructure. i n f r a s t r u c t u r e . By B y contrast c o n t r a s t a a large l a r g e share s h a r e of o f tomorrow’s t o m o r r o w ’ s far f a r more m o r e abundant a b u n d a n t data d a t a will w i l l come c o m e from f r o m industry, i n d u s t r y , business b u s i n e s s and a n d the t h e public p u b l i c sector, s e c t o r , and a n d will w i l l be b e stored s t o r e d on o n a a variety v a r i e t y of o f systems, s y s t e m s , notably n o t a b l y on o n computing c o m p u t i n g devices d e v i c e s working w o r k i n g at a t the t h e edge e d g e of o f the t h e network. n e t w o r k . This T h i s opens o p e n s up u p new n e w opportunities o p p o r t u n i t i e s for f o r Europe, E u r o p e , which w h i c h has h a s a a strong s t r o n g position p o s i t i o n in i n 1 1 AI A I for f o r Europe, E u r o p e , COM/2018/237 C O M / 2 0 1 8 / 2 3 7 final f i n a l 2 2 https://ec.europa.eu/commission/sites/beta-political/files/political-guidelines-next-commission_en.pdf. h t t p s : / / e c . e u r o p a . e u / c o m m i s s i o n / s i t e s / b e t a - p o l i t i c a l / f i l e s / p o l i t i c a l - g u i d e l i n e s - n e x t - c o m m i s s i o n _ e n . p d f . 1 1 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} digitised d i g i t i s e d industry i n d u s t r y and a n d business-to-business b u s i n e s s - t o - b u s i n e s s applications, a p p l i c a t i o n s , but b u t a a relatively r e l a t i v e l y weak w e a k position p o s i t i o n in i n consumer c o n s u m e r platforms. p l a t f o r m s . Simply S i m p l y put, p u t , AI A I is i s a a collection c o l l e c t i o n of o f technologies t e c h n o l o g i e s that t h a t combine c o m b i n e data, d a t a , algorithms a l g o r i t h m s and a n d computing c o m p u t i n g power. p o w e r . Advances A d v a n c e s in i n computing c o m p u t i n g and a n d the t h e increasing i n c r e a s i n g availability a v a i l a b i l i t y of o f data d a t a are a r e therefore t h e r e f o r e key k e y drivers d r i v e r s of o f the t h e current c u r r e n t upsurge u p s u r g e of o f AI. A I . Europe E u r o p e can c a n combine c o m b i n e its i t s technological t e c h n o l o g i c a l and a n d industrial i n d u s t r i a l strengths s t r e n g t h s with w i t h a a high-quality h i g h - q u a l i t y digital d i g i t a l infrastructure i n f r a s t r u c t u r e and a n d a a regulatory r e g u l a t o r y framework f r a m e w o r k based b a s e d on o n its i t s fundamental f u n d a m e n t a l values v a l u e s to t o become b e c o m e a a global g l o b a l leader l e a d e r in i n innovation i n n o v a t i o n in i n the t h e data d a t a economy e c o n o m y and a n d its i t s applications a p p l i c a t i o n s as a s set s e t out o u t in i n the t h e European E u r o p e a n data d a t a strategy s t r a t e g y 3 3 . . On O n that t h a t basis, b a s i s , it i t can c a n develop d e v e l o p an a n AI A I ecosystem e c o s y s t e m that t h a t brings b r i n g s the t h e benefits b e n e f i t s of o f the t h e technology t e c h n o l o g y to t o the t h e whole w h o l e of o f European E u r o p e a n society s o c i e t y and a n d economy: e c o n o m y : �  for f o r citizens c i t i z e n s to t o reap r e a p new n e w benefits b e n e f i t s for f o r example e x a m p l e improved i m p r o v e d health h e a l t h care, c a r e , fewer f e w e r breakdowns b r e a k d o w n s of o f household h o u s e h o l d machinery, m a c h i n e r y , safer s a f e r and a n d cleaner c l e a n e r transport t r a n s p o r t systems, s y s t e m s , better b e t t e r public p u b l i c services; s e r v i c e s ; �  for f o r business b u s i n e s s development, d e v e l o p m e n t , for f o r example e x a m p l e a a new n e w generation g e n e r a t i o n of o f products p r o d u c t s and a n d services s e r v i c e s in i n areas a r e a s where w h e r e Europe E u r o p e is i s particularly p a r t i c u l a r l y strong s t r o n g (machinery, ( m a c h i n e r y , transport, t r a n s p o r t , cybersecurity, c y b e r s e c u r i t y , farming, f a r m i n g , the t h e green g r e e n and a n d circular c i r c u l a r economy, e c o n o m y , healthcare h e a l t h c a r e and a n d high-value h i g h - v a l u e added a d d e d sectors s e c t o r s like l i k e fashion f a s h i o n and a n d tourism); t o u r i s m ) ; and a n d �  for f o r services s e r v i c e s of o f public p u b l i c interest, i n t e r e s t , for f o r example e x a m p l e by b y reducing r e d u c i n g the t h e costs c o s t s of o f providing p r o v i d i n g services s e r v i c e s (transport, ( t r a n s p o r t , education, e d u c a t i o n , energy e n e r g y and a n d waste w a s t e management), m a n a g e m e n t ) , by b y improving i m p r o v i n g the t h e sustainability s u s t a i n a b i l i t y of o f products p r o d u c t s 4 4 and a n d by b y equipping e q u i p p i n g law l a w enforcement e n f o r c e m e n t authorities a u t h o r i t i e s with w i t h appropriate a p p r o p r i a t e tools t o o l s to t o ensure e n s u r e the t h e security s e c u r i t y of o f citizens c i t i z e n s 5 5 , , with w i t h proper p r o p e r safeguards s a f e g u a r d s to t o respect r e s p e c t their t h e i r rights r i g h t s and a n d freedoms. f r e e d o m s . Given G i v e n the t h e major m a j o r impact i m p a c t that t h a t AI A I can c a n have h a v e on o n our o u r society s o c i e t y and a n d the t h e need n e e d to t o build b u i l d trust, t r u s t , it i t is i s vital v i t a l that t h a t European E u r o p e a n AI A I is i s grounded g r o u n d e d in i n our o u r values v a l u e s and a n d fundamental f u n d a m e n t a l rights r i g h t s such s u c h as a s human h u m a n dignity d i g n i t y and a n d privacy p r i v a c y protection. p r o t e c t i o n . Furthermore, F u r t h e r m o r e , the t h e impact i m p a c t of o f AI A I systems s y s t e m s should s h o u l d be b e considered c o n s i d e r e d not n o t only o n l y from f r o m an a n individual i n d i v i d u a l perspective, p e r s p e c t i v e , but b u t also a l s o from f r o m the t h e perspective p e r s p e c t i v e of o f society s o c i e t y as a s a a whole. w h o l e . The T h e use u s e of o f AI A I systems s y s t e m s can c a n have h a v e a a significant s i g n i f i c a n t role r o l e in i n achieving a c h i e v i n g the t h e Sustainable S u s t a i n a b l e Development D e v e l o p m e n t Goals, G o a l s , and a n d in i n supporting s u p p o r t i n g the t h e democratic d e m o c r a t i c process p r o c e s s and a n d social s o c i a l rights. r i g h t s . With W i t h its i t s recent r e c e n t proposals p r o p o s a l s on o n the t h e European E u r o p e a n Green G r e e n Deal D e a l 6 6 , , Europe E u r o p e is i s leading l e a d i n g the t h e way w a y in i n tackling t a c k l i n g climate c l i m a t e and a n d environmental-related e n v i r o n m e n t a l - r e l a t e d challenges. c h a l l e n g e s . Digital D i g i t a l technologies t e c h n o l o g i e s such s u c h as a s AI A I are a r e a a critical c r i t i c a l enabler e n a b l e r for f o r attaining a t t a i n i n g the t h e goals g o a l s of o f the t h e Green G r e e n Deal. D e a l . Given G i v e n the t h e increasing i n c r e a s i n g importance i m p o r t a n c e of o f AI, A I , the t h e environmental e n v i r o n m e n t a l impact i m p a c t of o f AI A I systems s y s t e m s needs n e e d s to t o be b e duly d u l y considered c o n s i d e r e d throughout t h r o u g h o u t their t h e i r lifecycle l i f e c y c l e and a n d across a c r o s s the t h e entire e n t i r e supply s u p p l y chain, c h a i n , e.g. e . g . as a s regards r e g a r d s resource r e s o u r c e usage u s a g e for f o r the t h e training t r a i n i n g of o f algorithms a l g o r i t h m s and a n d the t h e storage s t o r a g e of o f data. d a t a . A A common c o m m o n European E u r o p e a n approach a p p r o a c h to t o AI A I is i s necessary n e c e s s a r y to t o reach r e a c h sufficient s u f f i c i e n t scale s c a l e and a n d avoid a v o i d the t h e fragmentation f r a g m e n t a t i o n of o f the t h e single s i n g l e market. m a r k e t . The T h e introduction i n t r o d u c t i o n of o f national n a t i o n a l initiatives i n i t i a t i v e s risks r i s k s to t o endanger e n d a n g e r legal l e g a l certainty, c e r t a i n t y , to t o weaken w e a k e n citizens’ c i t i z e n s ’ trust t r u s t and a n d to t o prevent p r e v e n t the t h e emergence e m e r g e n c e of o f a a dynamic d y n a m i c European E u r o p e a n industry. i n d u s t r y . This T h i s White W h i t e Paper P a p e r presents p r e s e n t s policy p o l i c y options o p t i o n s to t o enable e n a b l e a a trustworthy t r u s t w o r t h y and a n d secure s e c u r e development d e v e l o p m e n t of o f AI A I in i n Europe, E u r o p e , in i n full f u l l respect r e s p e c t of o f the t h e values v a l u e s and a n d rights r i g h t s of o f EU E U citizens. c i t i z e n s . The T h e main m a i n building b u i l d i n g blocks b l o c k s of o f this t h i s White W h i t e Paper P a p e r are: a r e : 3 3 COM(2020) C O M ( 2 0 2 0 ) 66 6 6 final. f i n a l . 4 4 AI A I and a n d digitalisation d i g i t a l i s a t i o n in i n general g e n e r a l are a r e critical c r i t i c a l enablers e n a b l e r s of o f Europe’s E u r o p e ’ s Green G r e e n deal d e a l ambitions. a m b i t i o n s . However, H o w e v e r , the t h e current c u r r e n t environmental e n v i r o n m e n t a l footprint f o o t p r i n t of o f the t h e ICT I C T sector s e c t o r is i s estimated e s t i m a t e d at a t more m o r e than t h a n 2% 2 % of o f all a l l global g l o b a l emissions. e m i s s i o n s . The T h e European E u r o p e a n digital d i g i t a l strategy s t r a t e g y accompanying a c c o m p a n y i n g this t h i s White W h i t e Paper P a p e r proposes p r o p o s e s green g r e e n transformation t r a n s f o r m a t i o n measures m e a s u r e s for f o r digital. d i g i t a l . 5 5 AI A I tools t o o l s can c a n provide p r o v i d e an a n opportunity o p p o r t u n i t y for f o r better b e t t e r protecting p r o t e c t i n g EU E U citizens c i t i z e n s from f r o m crime c r i m e and a n d acts a c t s of o f terrorism. t e r r o r i s m . Such S u c h tools t o o l s could, c o u l d , for f o r example, e x a m p l e , help h e l p identify i d e n t i f y online o n l i n e terrorist t e r r o r i s t propaganda, p r o p a g a n d a , discover d i s c o v e r suspicious s u s p i c i o u s transactions t r a n s a c t i o n s in i n the t h e sales s a l e s of o f dangerous d a n g e r o u s products, p r o d u c t s , identify i d e n t i f y dangerous d a n g e r o u s hidden h i d d e n objects o b j e c t s or o r illicit i l l i c i t substances s u b s t a n c e s or o r products, p r o d u c t s , offer o f f e r assistance a s s i s t a n c e to t o citizens c i t i z e n s in i n emergencies e m e r g e n c i e s and a n d help h e l p guide g u i d e first f i r s t responders. r e s p o n d e r s . 6 6 COM(2019) C O M ( 2 0 1 9 ) 640 6 4 0 final. f i n a l . 2 2 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} �  The T h e policy p o l i c y framework f r a m e w o r k setting s e t t i n g out o u t measures m e a s u r e s to t o align a l i g n efforts e f f o r t s at a t European, E u r o p e a n , national n a t i o n a l and a n d regional r e g i o n a l level. l e v e l . In I n partnership p a r t n e r s h i p between b e t w e e n the t h e private p r i v a t e and a n d the t h e public p u b l i c sector, s e c t o r , the t h e aim a i m of o f the t h e framework f r a m e w o r k is i s to t o mobilise m o b i l i s e resources r e s o u r c e s to t o achieve a c h i e v e an a n ‘ecosystem ‘ e c o s y s t e m of o f excellence’ e x c e l l e n c e ’ along a l o n g the t h e entire e n t i r e value v a l u e chain, c h a i n , starting s t a r t i n g in i n research r e s e a r c h and a n d innovation, i n n o v a t i o n , and a n d to t o create c r e a t e the t h e right r i g h t incentives i n c e n t i v e s to t o accelerate a c c e l e r a t e the t h e adoption a d o p t i o n of o f solutions s o l u t i o n s based b a s e d on o n AI, A I , including i n c l u d i n g by b y small s m a l l and a n d medium-sized m e d i u m - s i z e d enterprises e n t e r p r i s e s (SMEs). ( S M E s ) . �  The T h e key k e y elements e l e m e n t s of o f a a future f u t u r e regulatory r e g u l a t o r y framework f r a m e w o r k for f o r AI A I in i n Europe E u r o p e that t h a t will w i l l create c r e a t e a a unique u n i q u e ‘ecosystem ‘ e c o s y s t e m of o f trust’. t r u s t ’ . To T o do d o so, s o , it i t must m u s t ensure e n s u r e compliance c o m p l i a n c e with w i t h EU E U rules, r u l e s , including i n c l u d i n g the t h e rules r u l e s protecting p r o t e c t i n g fundamental f u n d a m e n t a l rights r i g h t s and a n d consumers’ c o n s u m e r s ’ rights, r i g h t s , in i n particular p a r t i c u l a r for f o r AI A I systems s y s t e m s operated o p e r a t e d in i n the t h e EU E U that t h a t pose p o s e a a high h i g h risk r i s k 7 7 . . Building B u i l d i n g an a n ecosystem e c o s y s t e m of o f trust t r u s t is i s a a policy p o l i c y objective o b j e c t i v e in i n itself, i t s e l f , and a n d should s h o u l d give g i v e citizens c i t i z e n s the t h e confidence c o n f i d e n c e to t o take t a k e up u p AI A I applications a p p l i c a t i o n s and a n d give g i v e companies c o m p a n i e s and a n d public p u b l i c organisations o r g a n i s a t i o n s the t h e legal l e g a l certainty c e r t a i n t y to t o innovate i n n o v a t e using u s i n g AI. A I . The T h e Commission C o m m i s s i o n strongly s t r o n g l y supports s u p p o r t s a a human-centric h u m a n - c e n t r i c approach a p p r o a c h based b a s e d on o n the t h e Communication C o m m u n i c a t i o n on o n Building B u i l d i n g Trust T r u s t in i n Human-Centric H u m a n - C e n t r i c AI A I 8 8 and a n d will w i l l also a l s o take t a k e into i n t o account a c c o u n t the t h e input i n p u t obtained o b t a i n e d during d u r i n g the t h e piloting p i l o t i n g phase p h a s e of o f the t h e Ethics E t h i c s Guidelines G u i d e l i n e s prepared p r e p a r e d by b y the t h e High-Level H i g h - L e v e l Expert E x p e r t Group G r o u p on o n AI. A I . The T h e European E u r o p e a n strategy s t r a t e g y for f o r data, d a t a , which w h i c h accompanies a c c o m p a n i e s this t h i s White W h i t e Paper, P a p e r , aims a i m s to t o enable e n a b l e Europe E u r o p e to t o become b e c o m e the t h e most m o s t attractive, a t t r a c t i v e , secure s e c u r e and a n d dynamic d y n a m i c data-agile d a t a - a g i l e economy e c o n o m y in i n the t h e world w o r l d – – empowering e m p o w e r i n g Europe E u r o p e with w i t h data d a t a to t o improve i m p r o v e decisions d e c i s i o n s and a n d better b e t t e r the t h e lives l i v e s of o f all a l l its i t s citizens. c i t i z e n s . The T h e strategy s t r a t e g y sets s e t s out o u t a a number n u m b e r of o f policy p o l i c y measures, m e a s u r e s , including i n c l u d i n g mobilising m o b i l i s i n g private p r i v a t e and a n d public p u b l i c investments, i n v e s t m e n t s , needed n e e d e d to t o achieve a c h i e v e this t h i s goal. g o a l . Finally, F i n a l l y , the t h e implications i m p l i c a t i o n s of o f AI, A I , Internet I n t e r n e t of o f Things T h i n g s and a n d other o t h e r digital d i g i t a l technologies t e c h n o l o g i e s for f o r safety s a f e t y and a n d liability l i a b i l i t y legislation l e g i s l a t i o n are a r e analysed a n a l y s e d in i n the t h e Commission C o m m i s s i o n Report R e p o r t accompanying a c c o m p a n y i n g this t h i s White W h i t e Paper. P a p e r . 2. 2 . CAPITALISING C A P I T A L I S I N G ON O N STRENGTHS S T R E N G T H S IN I N INDUSTRIAL I N D U S T R I A L AND A N D PROFESSIONAL P R O F E S S I O N A L MARKETS M A R K E T S Europe E u r o p e is i s well w e l l placed p l a c e d to t o benefit b e n e f i t from f r o m the t h e potential p o t e n t i a l of o f AI, A I , not n o t only o n l y as a s a a user u s e r but b u t also a l s o as a s a a creator c r e a t o r and a n d a a producer p r o d u c e r of o f this t h i s technology. t e c h n o l o g y . It I t has h a s excellent e x c e l l e n t research r e s e a r c h centres, c e n t r e s , innovative i n n o v a t i v e start-ups, s t a r t - u p s , a a world-leading w o r l d - l e a d i n g position p o s i t i o n in i n robotics r o b o t i c s and a n d competitive c o m p e t i t i v e manufacturing m a n u f a c t u r i n g and a n d services s e r v i c e s sectors, s e c t o r s , from f r o m automotive a u t o m o t i v e to t o healthcare, h e a l t h c a r e , energy, e n e r g y , financial f i n a n c i a l services s e r v i c e s and a n d agriculture. a g r i c u l t u r e . Europe E u r o p e has h a s developed d e v e l o p e d a a strong s t r o n g computing c o m p u t i n g infrastructure i n f r a s t r u c t u r e (e.g. ( e . g . high-performance h i g h - p e r f o r m a n c e computers), c o m p u t e r s ) , essential e s s e n t i a l to t o the t h e functioning f u n c t i o n i n g of o f AI. A I . Europe E u r o p e also a l s o holds h o l d s large l a r g e volumes v o l u m e s of o f public p u b l i c and a n d industrial i n d u s t r i a l data, d a t a , the t h e potential p o t e n t i a l of o f which w h i c h is i s currently c u r r e n t l y under-used. u n d e r - u s e d . It I t has h a s well- w e l l - recognised r e c o g n i s e d industrial i n d u s t r i a l strengths s t r e n g t h s in i n safe s a f e and a n d secure s e c u r e digital d i g i t a l systems s y s t e m s with w i t h low-power l o w - p o w e r consumption c o n s u m p t i o n that t h a t are a r e essential e s s e n t i a l for f o r the t h e further f u r t h e r development d e v e l o p m e n t of o f AI. A I . Harnessing H a r n e s s i n g the t h e capacity c a p a c i t y of o f the t h e EU E U to t o invest i n v e s t in i n next n e x t generation g e n e r a t i o n technologies t e c h n o l o g i e s and a n d infrastructures, i n f r a s t r u c t u r e s , as a s well w e l l as a s in i n digital d i g i t a l competences c o m p e t e n c e s like l i k e data d a t a literacy, l i t e r a c y , will w i l l increase i n c r e a s e Europe’s E u r o p e ’ s technological t e c h n o l o g i c a l sovereignty s o v e r e i g n t y in i n key k e y enabling e n a b l i n g technologies t e c h n o l o g i e s and a n d infrastructures i n f r a s t r u c t u r e s for f o r the t h e data d a t a economy. e c o n o m y . The T h e infrastructures i n f r a s t r u c t u r e s should s h o u l d support s u p p o r t the t h e creation c r e a t i o n of o f European E u r o p e a n data d a t a pools p o o l s enabling e n a b l i n g trustworthy t r u s t w o r t h y AI, A I , e.g. e . g . AI A I based b a s e d on o n European E u r o p e a n values v a l u e s and a n d rules. r u l e s . Europe E u r o p e should s h o u l d leverage l e v e r a g e its i t s strengths s t r e n g t h s to t o expand e x p a n d its i t s position p o s i t i o n in i n the t h e ecosystems e c o s y s t e m s and a n d along a l o n g the t h e value v a l u e chain, c h a i n , from f r o m certain c e r t a i n hardware h a r d w a r e manufacturing m a n u f a c t u r i n g sectors s e c t o r s to t o software s o f t w a r e all a l l the t h e way w a y to t o services. s e r v i c e s . This T h i s is i s already a l r e a d y happening h a p p e n i n g to t o an a n extent. e x t e n t . Europe E u r o p e produces p r o d u c e s more m o r e than t h a n a a quarter q u a r t e r of o f all a l l industrial i n d u s t r i a l and a n d professional p r o f e s s i o n a l service s e r v i c e robots r o b o t s (e.g. ( e . g . for f o r precision p r e c i s i o n farming, f a r m i n g , security, s e c u r i t y , health, h e a l t h , logistics.), l o g i s t i c s . ) , and a n d plays p l a y s an a n important i m p o r t a n t role r o l e in i n developing d e v e l o p i n g and a n d using u s i n g software s o f t w a r e applications a p p l i c a t i o n s for f o r companies c o m p a n i e s and a n d organisations o r g a n i s a t i o n s (business-to-business ( b u s i n e s s - t o - b u s i n e s s applications a p p l i c a t i o n s such s u c h as a s Enterprise E n t e r p r i s e Resource R e s o u r c e Planning, P l a n n i n g , design d e s i g n and a n d engineering e n g i n e e r i n g software) s o f t w a r e ) as a s well w e l l as a s applications a p p l i c a t i o n s to t o support s u p p o r t e-government e - g o v e r n m e n t and a n d the t h e "intelligent " i n t e l l i g e n t enterprise". e n t e r p r i s e " . 7 7 Although A l t h o u g h further f u r t h e r arrangements a r r a n g e m e n t s may m a y need n e e d to t o be b e put p u t in i n place p l a c e to t o prevent p r e v e n t and a n d counter c o u n t e r misuse m i s u s e of o f AI A I for f o r criminal c r i m i n a l purposes, p u r p o s e s , this t h i s is i s outside o u t s i d e the t h e scope s c o p e of o f this t h i s white w h i t e paper. p a p e r . 8 8 COM(2019) C O M ( 2 0 1 9 ) 168. 1 6 8 . 3 3 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} Europe E u r o p e leads l e a d s the t h e way w a y in i n deploying d e p l o y i n g AI A I in i n manufacturing. m a n u f a c t u r i n g . Over O v e r half h a l f of o f the t h e top t o p manufacturers m a n u f a c t u r e r s implement i m p l e m e n t at a t least l e a s t one o n e instance i n s t a n c e of o f AI A I in i n manufacturing m a n u f a c t u r i n g operations o p e r a t i o n s 9 9 . . One O n e reason r e a s o n for f o r Europe’s E u r o p e ’ s strong s t r o n g position p o s i t i o n in i n terms t e r m s of o f research r e s e a r c h is i s the t h e EU E U funding f u n d i n g programme p r o g r a m m e that t h a t has h a s proven p r o v e n instrumental i n s t r u m e n t a l in i n pooling p o o l i n g action, a c t i o n , avoiding a v o i d i n g duplications, d u p l i c a t i o n s , and a n d leveraging l e v e r a g i n g public p u b l i c and a n d private p r i v a t e investments i n v e s t m e n t s in i n the t h e Member M e m b e r States. S t a t e s . Over O v e r the t h e past p a s t three t h r e e years, y e a r s , EU E U funding f u n d i n g for f o r research r e s e a r c h and a n d innovation i n n o v a t i o n for f o r AI A I has h a s risen r i s e n to t o €1.5 € 1 . 5 billion, b i l l i o n , i.e. i . e . a a 70% 7 0 % increase i n c r e a s e compared c o m p a r e d to t o the t h e previous p r e v i o u s period. p e r i o d . However, H o w e v e r , investment i n v e s t m e n t in i n research r e s e a r c h and a n d innovation i n n o v a t i o n in i n Europe E u r o p e is i s still s t i l l a a fraction f r a c t i o n of o f the t h e public p u b l i c and a n d private p r i v a t e investment i n v e s t m e n t in i n other o t h e r regions r e g i o n s of o f the t h e world. w o r l d . Some S o m e €3.2 € 3 . 2 billion b i l l i o n were w e r e invested i n v e s t e d in i n AI A I in i n Europe E u r o p e in i n 2016, 2 0 1 6 , compared c o m p a r e d to t o around a r o u n d €12.1 € 1 2 . 1 billion b i l l i o n in i n North N o r t h America A m e r i c a and a n d €6.5 € 6 . 5 billion b i l l i o n in i n Asia A s i a 10 1 0 . . In I n response, r e s p o n s e , Europe E u r o p e needs n e e d s to t o increase i n c r e a s e its i t s investment i n v e s t m e n t levels l e v e l s significantly. s i g n i f i c a n t l y . The T h e Coordinated C o o r d i n a t e d plan p l a n on o n AI A I 11 1 1 developed d e v e l o p e d with w i t h Member M e m b e r States S t a t e s is i s proving p r o v i n g to t o be b e a a good g o o d starting s t a r t i n g point p o i n t in i n building b u i l d i n g closer c l o s e r cooperation c o o p e r a t i o n on o n AI A I in i n Europe E u r o p e and a n d in i n creating c r e a t i n g synergies s y n e r g i e s to t o maximise m a x i m i s e investment i n v e s t m e n t in i n the t h e AI A I value v a l u e chain. c h a i n . 3. 3 . SEIZING S E I Z I N G THE T H E OPPORTUNITIES O P P O R T U N I T I E S AHEAD: A H E A D : THE T H E NEXT N E X T DATA D A T A WAVE W A V E Although A l t h o u g h Europe E u r o p e currently c u r r e n t l y is i s in i n a a weaker w e a k e r position p o s i t i o n in i n consumer c o n s u m e r applications a p p l i c a t i o n s and a n d on o n online o n l i n e platforms, p l a t f o r m s , which w h i c h results r e s u l t s in i n a a competitive c o m p e t i t i v e disadvantage d i s a d v a n t a g e in i n data d a t a access, a c c e s s , major m a j o r shifts s h i f t s in i n the t h e value v a l u e and a n d re-use r e - u s e of o f data d a t a across a c r o s s sectors s e c t o r s are a r e underway. u n d e r w a y . The T h e volume v o l u m e of o f data d a t a produced p r o d u c e d in i n the t h e world w o r l d is i s growing g r o w i n g rapidly, r a p i d l y , from f r o m 33 3 3 zettabytes z e t t a b y t e s in i n 2018 2 0 1 8 to t o an a n expected e x p e c t e d 175 1 7 5 zettabytes z e t t a b y t e s in i n 2025 2 0 2 5 12 1 2 . . Each E a c h new n e w wave w a v e of o f data d a t a brings b r i n g s opportunities o p p o r t u n i t i e s for f o r Europe E u r o p e to t o position p o s i t i o n itself i t s e l f in i n the t h e data-agile d a t a - a g i l e economy e c o n o m y and a n d to t o become b e c o m e a a world w o r l d leader l e a d e r in i n this t h i s area. a r e a . Furthermore, F u r t h e r m o r e , the t h e way w a y in i n which w h i c h data d a t a are a r e stored s t o r e d and a n d processed p r o c e s s e d will w i l l change c h a n g e dramatically d r a m a t i c a l l y over o v e r the t h e coming c o m i n g five f i v e years. y e a r s . Today T o d a y 80% 8 0 % of o f data d a t a processing p r o c e s s i n g and a n d analysis a n a l y s i s that t h a t takes t a k e s place p l a c e in i n the t h e cloud c l o u d occurs o c c u r s in i n data d a t a centres c e n t r e s and a n d centralised c e n t r a l i s e d computing c o m p u t i n g facilities, f a c i l i t i e s , and a n d 20% 2 0 % in i n smart s m a r t connected c o n n e c t e d objects, o b j e c t s , such s u c h as a s cars, c a r s , home h o m e appliances a p p l i a n c e s or o r manufacturing m a n u f a c t u r i n g robots, r o b o t s , and a n d in i n computing c o m p u t i n g facilities f a c i l i t i e s close c l o s e to t o the t h e user u s e r (“edge ( “ e d g e computing”). c o m p u t i n g ” ) . By B y 2025 2 0 2 5 these t h e s e proportions p r o p o r t i o n s are a r e set s e t to t o change c h a n g e markedly m a r k e d l y 13 1 3 . . Europe E u r o p e is i s a a global g l o b a l leader l e a d e r in i n low-power l o w - p o w e r electronics e l e c t r o n i c s which w h i c h is i s key k e y for f o r the t h e next n e x t generation g e n e r a t i o n of o f specialised s p e c i a l i s e d processors p r o c e s s o r s for f o r AI. A I . This T h i s market m a r k e t is i s currently c u r r e n t l y dominated d o m i n a t e d by b y non-EU n o n - E U players. p l a y e r s . This T h i s could c o u l d change c h a n g e with w i t h the t h e help h e l p of o f initiatives i n i t i a t i v e s such s u c h as a s the t h e European E u r o p e a n Processor P r o c e s s o r Initiative, I n i t i a t i v e , which w h i c h focuses f o c u s e s on o n developing d e v e l o p i n g low-power l o w - p o w e r computing c o m p u t i n g systems s y s t e m s for f o r both b o t h edge e d g e and a n d next n e x t generation g e n e r a t i o n high-performance h i g h - p e r f o r m a n c e computing, c o m p u t i n g , and a n d the t h e work w o r k of o f the t h e Key K e y Digital D i g i t a l Technology T e c h n o l o g y Joint J o i n t Undertaking, U n d e r t a k i n g , proposed p r o p o s e d to t o start s t a r t in i n 2021. 2 0 2 1 . Europe E u r o p e also a l s o leads l e a d s in i n neuromorphic n e u r o m o r p h i c solutions s o l u t i o n s 14 1 4 that t h a t are a r e ideally i d e a l l y suited s u i t e d to t o automating a u t o m a t i n g industrial i n d u s t r i a l processes p r o c e s s e s (industry ( i n d u s t r y 4.0) 4 . 0 ) and a n d transport t r a n s p o r t modes. m o d e s . They T h e y can c a n improve i m p r o v e energy e n e r g y efficiency e f f i c i e n c y by b y several s e v e r a l orders o r d e r s of o f magnitude. m a g n i t u d e . Recent R e c e n t advances a d v a n c e s in i n quantum q u a n t u m computing c o m p u t i n g will w i l l generate g e n e r a t e exponential e x p o n e n t i a l increases i n c r e a s e s in i n processing p r o c e s s i n g capacity c a p a c i t y 15 1 5 . . Europe E u r o p e can c a n be b e at a t the t h e forefront f o r e f r o n t of o f this t h i s technology t e c h n o l o g y thanks t h a n k s to t o its i t s academic a c a d e m i c strengths s t r e n g t h s in i n quantum q u a n t u m computing, c o m p u t i n g , as a s well w e l l as a s European E u r o p e a n industry’s i n d u s t r y ’ s strong s t r o n g position p o s i t i o n in i n quantum q u a n t u m simulators s i m u l a t o r s and a n d programming p r o g r a m m i n g environments e n v i r o n m e n t s for f o r quantum q u a n t u m computing. c o m p u t i n g . European E u r o p e a n initiatives i n i t i a t i v e s that t h a t aim a i m to t o increase i n c r e a s e the t h e availability a v a i l a b i l i t y of o f quantum q u a n t u m testing t e s t i n g and a n d experimentation e x p e r i m e n t a t i o n facilities f a c i l i t i e s will w i l l help h e l p apply a p p l y these t h e s e new n e w quantum q u a n t u m solutions s o l u t i o n s to t o a a number n u m b e r of o f industrial i n d u s t r i a l and a n d academic a c a d e m i c sectors. s e c t o r s . 9 9 Followed F o l l o w e d by b y Japan J a p a n (30%) ( 3 0 % ) and a n d the t h e US U S (28%). ( 2 8 % ) . Source: S o u r c e : CapGemini C a p G e m i n i (2019). ( 2 0 1 9 ) . 10 1 0 10 1 0 imperatives i m p e r a t i v e s for f o r Europe E u r o p e in i n the t h e age a g e of o f AI A I and a n d automation, a u t o m a t i o n , McKinsey M c K i n s e y (2017). ( 2 0 1 7 ) . 11 1 1 COM(2018) C O M ( 2 0 1 8 ) 795. 7 9 5 . 12 1 2 IDC I D C (2019). ( 2 0 1 9 ) . 13 1 3 Gartner G a r t n e r (2017). ( 2 0 1 7 ) . 14 1 4 Neuromorphic N e u r o m o r p h i c solutions s o l u t i o n s means m e a n s any a n y very v e r y large-scale l a r g e - s c a l e system s y s t e m of o f integrated i n t e g r a t e d circuits c i r c u i t s that t h a t mimic m i m i c neuro-biological n e u r o - b i o l o g i c a l architectures a r c h i t e c t u r e s present p r e s e n t in i n the t h e nervous n e r v o u s system. s y s t e m . 15 1 5 Quantum Q u a n t u m computers c o m p u t e r s will w i l l have h a v e the t h e capacity c a p a c i t y to t o process p r o c e s s in i n less l e s s than t h a n seconds s e c o n d s many m a n y fold f o l d larger l a r g e r data d a t a sets s e t s than t h a n today’s t o d a y ’ s highest h i g h e s t performance p e r f o r m a n c e computers c o m p u t e r s allowing a l l o w i n g for f o r the t h e development d e v e l o p m e n t of o f new n e w AI A I applications a p p l i c a t i o n s across a c r o s s sectors. s e c t o r s . 4 4 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} In I n parallel, p a r a l l e l , Europe E u r o p e will w i l l continue c o n t i n u e to t o lead l e a d progress p r o g r e s s in i n the t h e algorithmic a l g o r i t h m i c foundations f o u n d a t i o n s of o f AI, A I , building b u i l d i n g on o n its i t s own o w n scientific s c i e n t i f i c excellence. e x c e l l e n c e . There T h e r e is i s a a need n e e d to t o build b u i l d bridges b r i d g e s between b e t w e e n disciplines d i s c i p l i n e s that t h a t currently c u r r e n t l y work w o r k separately, s e p a r a t e l y , such s u c h as a s machine m a c h i n e learning l e a r n i n g and a n d deep d e e p learning l e a r n i n g (characterised ( c h a r a c t e r i s e d by b y limited l i m i t e d interpretability, i n t e r p r e t a b i l i t y , the t h e need n e e d for f o r a a large l a r g e volume v o l u m e of o f data d a t a to t o train t r a i n the t h e models m o d e l s and a n d learn l e a r n through t h r o u g h correlations) c o r r e l a t i o n s ) and a n d symbolic s y m b o l i c approaches a p p r o a c h e s (where ( w h e r e rules r u l e s are a r e created c r e a t e d through t h r o u g h human h u m a n intervention). i n t e r v e n t i o n ) . Combining C o m b i n i n g symbolic s y m b o l i c reasoning r e a s o n i n g with w i t h deep d e e p neural n e u r a l networks n e t w o r k s may m a y help h e l p us u s improve i m p r o v e explainability e x p l a i n a b i l i t y of o f AI A I outcomes. o u t c o m e s . 4. 4 . AN A N ECOSYSTEM E C O S Y S T E M OF O F EXCELLENCE E X C E L L E N C E To T o build b u i l d an a n ecosystem e c o s y s t e m of o f excellence e x c e l l e n c e that t h a t can c a n support s u p p o r t the t h e development d e v e l o p m e n t and a n d uptake u p t a k e of o f AI A I across a c r o s s the t h e EU E U economy e c o n o m y and a n d public p u b l i c administration, a d m i n i s t r a t i o n , there t h e r e is i s a a need n e e d to t o step s t e p up u p action a c t i o n at a t multiple m u l t i p l e levels. l e v e l s . A. A . WORKING W O R K I N G WITH W I T H MEMBER M E M B E R STATES S T A T E S Delivering D e l i v e r i n g on o n its i t s strategy s t r a t e g y on o n AI A I adopted a d o p t e d in i n April A p r i l 2018, 2 0 1 8 , 16 1 6 in i n December D e c e m b e r 2018 2 0 1 8 the t h e Commission C o m m i s s i o n presented p r e s e n t e d a a Coordinated C o o r d i n a t e d Plan P l a n - - prepared p r e p a r e d together t o g e t h e r with w i t h the t h e Member M e m b e r States S t a t e s - - to t o foster f o s t e r the t h e development d e v e l o p m e n t and a n d use u s e of o f AI A I in i n Europe E u r o p e 17 1 7 . . This T h i s plan p l a n proposes p r o p o s e s some s o m e 70 7 0 joint j o i n t actions a c t i o n s for f o r closer c l o s e r and a n d more m o r e efficient e f f i c i e n t cooperation c o o p e r a t i o n between b e t w e e n Member M e m b e r States, S t a t e s , and a n d the t h e Commission C o m m i s s i o n in i n key k e y areas, a r e a s , such s u c h as a s research, r e s e a r c h , investment, i n v e s t m e n t , market m a r k e t uptake, u p t a k e , skills s k i l l s and a n d talent, t a l e n t , data d a t a and a n d international i n t e r n a t i o n a l cooperation. c o o p e r a t i o n . The T h e plan p l a n is i s scheduled s c h e d u l e d to t o run r u n until u n t i l 2027, 2 0 2 7 , with w i t h regular r e g u l a r monitoring m o n i t o r i n g and a n d review. r e v i e w . The T h e aim a i m is i s to t o maximise m a x i m i s e the t h e impact i m p a c t of o f investment i n v e s t m e n t in i n research, r e s e a r c h , innovation i n n o v a t i o n and a n d deployment, d e p l o y m e n t , assess a s s e s s national n a t i o n a l AI A I strategies s t r a t e g i e s and a n d build b u i l d on o n and a n d extend e x t e n d the t h e Coordinated C o o r d i n a t e d Plan P l a n on o n AI A I with w i t h Member M e m b e r States: S t a t e s : �  Action A c t i o n 1: 1 : The T h e Commission, C o m m i s s i o n , taking t a k i n g into i n t o account a c c o u n t the t h e results r e s u l t s of o f the t h e public p u b l i c consultation c o n s u l t a t i o n on o n the t h e White W h i t e Paper, P a p e r , will w i l l propose p r o p o s e to t o the t h e Member M e m b e r States S t a t e s a a revision r e v i s i o n of o f the t h e Coordinated C o o r d i n a t e d Plan P l a n to t o be b e adopted a d o p t e d by b y end e n d 2020 2 0 2 0 EU-level E U - l e v e l funding f u n d i n g in i n AI A I should s h o u l d attract a t t r a c t and a n d pool p o o l investment i n v e s t m e n t in i n areas a r e a s where w h e r e the t h e action a c t i o n required r e q u i r e d goes g o e s beyond b e y o n d what w h a t any a n y single s i n g l e Member M e m b e r State S t a t e can c a n achieve. a c h i e v e . The T h e objective o b j e c t i v e is i s to t o attract a t t r a c t over o v e r €20 € 2 0 billion b i l l i o n 18 1 8 of o f total t o t a l investment i n v e s t m e n t in i n the t h e EU E U per p e r year y e a r in i n AI A I over o v e r the t h e next n e x t decade. d e c a d e . To T o stimulate s t i m u l a t e private p r i v a t e and a n d public p u b l i c investment, i n v e s t m e n t , the t h e EU E U will w i l l make m a k e available a v a i l a b l e resources r e s o u r c e s from f r o m the t h e Digital D i g i t a l Europe E u r o p e Programme, P r o g r a m m e , Horizon H o r i z o n Europe E u r o p e as a s well w e l l as a s from f r o m the t h e European E u r o p e a n Structural S t r u c t u r a l and a n d Investment I n v e s t m e n t Funds F u n d s to t o address a d d r e s s the t h e needs n e e d s of o f less- l e s s - developed d e v e l o p e d regions r e g i o n s as a s well w e l l as a s rural r u r a l areas. a r e a s . The T h e Coordinated C o o r d i n a t e d Plan P l a n could c o u l d also a l s o address a d d r e s s societal s o c i e t a l and a n d environmental e n v i r o n m e n t a l well-being w e l l - b e i n g as a s a a key k e y principle p r i n c i p l e for f o r AI. A I . AI A I systems s y s t e m s promise p r o m i s e to t o help h e l p tackling t a c k l i n g the t h e most m o s t pressing p r e s s i n g concerns, c o n c e r n s , including i n c l u d i n g climate c l i m a t e change c h a n g e and a n d environmental e n v i r o n m e n t a l degradation. d e g r a d a t i o n . It I t is i s also a l s o important i m p o r t a n t that t h a t this t h i s happens h a p p e n s in i n an a n environmentally e n v i r o n m e n t a l l y friendly f r i e n d l y manner. m a n n e r . AI A I can c a n and a n d should s h o u l d itself i t s e l f critically c r i t i c a l l y examine e x a m i n e resource r e s o u r c e usage u s a g e and a n d energy e n e r g y consumption c o n s u m p t i o n and a n d be b e trained t r a i n e d to t o make m a k e choices c h o i c e s that t h a t are a r e positive p o s i t i v e for f o r the t h e environment. e n v i r o n m e n t . The T h e Commission C o m m i s s i o n will w i l l consider c o n s i d e r options o p t i o n s to t o encourage e n c o u r a g e and a n d promote p r o m o t e AI A I solutions s o l u t i o n s that t h a t do d o this t h i s together t o g e t h e r with w i t h the t h e Member M e m b e r States. S t a t e s . B. B . FOCUSING F O C U S I N G THE T H E EFFORTS E F F O R T S OF O F THE T H E RESEARCH R E S E A R C H AND A N D INNOVATION I N N O V A T I O N COMMUNITY C O M M U N I T Y 16 1 6 Artificial A r t i f i c i a l Intelligence I n t e l l i g e n c e for f o r Europe, E u r o p e , COM(2018) C O M ( 2 0 1 8 ) 237. 2 3 7 . 17 1 7 Coordinated C o o r d i n a t e d Plan P l a n on o n Artificial A r t i f i c i a l Intelligence, I n t e l l i g e n c e , COM(2018) C O M ( 2 0 1 8 ) 795. 7 9 5 . 18 1 8 COM(2018) C O M ( 2 0 1 8 ) 237. 2 3 7 . 5 5 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} Europe E u r o p e cannot c a n n o t afford a f f o r d to t o maintain m a i n t a i n the t h e current c u r r e n t fragmented f r a g m e n t e d landscape l a n d s c a p e of o f centres c e n t r e s of o f competence c o m p e t e n c e with w i t h none n o n e reaching r e a c h i n g the t h e scale s c a l e necessary n e c e s s a r y to t o compete c o m p e t e with w i t h the t h e leading l e a d i n g institutes i n s t i t u t e s globally. g l o b a l l y . It I t is i s imperative i m p e r a t i v e to t o create c r e a t e more m o r e synergies s y n e r g i e s and a n d networks n e t w o r k s between b e t w e e n the t h e multiple m u l t i p l e European E u r o p e a n research r e s e a r c h centres c e n t r e s on o n AI A I and a n d to t o align a l i g n their t h e i r efforts e f f o r t s to t o improve i m p r o v e excellence, e x c e l l e n c e , retain r e t a i n and a n d attract a t t r a c t the t h e best b e s t researchers r e s e a r c h e r s and a n d develop d e v e l o p the t h e best b e s t technology. t e c h n o l o g y . Europe E u r o p e needs n e e d s a a lighthouse l i g h t h o u s e centre c e n t r e of o f research, r e s e a r c h , innovation i n n o v a t i o n and a n d expertise e x p e r t i s e that t h a t would w o u l d coordinate c o o r d i n a t e these t h e s e efforts e f f o r t s and a n d be b e a a world w o r l d reference r e f e r e n c e of o f excellence e x c e l l e n c e in i n AI A I and a n d that t h a t can c a n attract a t t r a c t investments i n v e s t m e n t s and a n d the t h e best b e s t talents t a l e n t s in i n the t h e field. f i e l d . The T h e centres c e n t r e s and a n d the t h e networks n e t w o r k s should s h o u l d concentrate c o n c e n t r a t e in i n sectors s e c t o r s where w h e r e Europe E u r o p e has h a s the t h e potential p o t e n t i a l to t o become b e c o m e a a global g l o b a l champion c h a m p i o n such s u c h as a s industry, i n d u s t r y , health, h e a l t h , transport, t r a n s p o r t , finance, f i n a n c e , agrifood a g r i f o o d value v a l u e chains, c h a i n s , energy/environment, e n e r g y / e n v i r o n m e n t , forestry, f o r e s t r y , earth e a r t h observation o b s e r v a t i o n and a n d space. s p a c e . In I n all a l l these t h e s e domains, d o m a i n s , the t h e race r a c e for f o r global g l o b a l leadership l e a d e r s h i p is i s ongoing, o n g o i n g , and a n d Europe E u r o p e offers o f f e r s significant s i g n i f i c a n t potential, p o t e n t i a l , knowledge k n o w l e d g e and a n d expertise e x p e r t i s e 19 1 9 . . Equally E q u a l l y important i m p o r t a n t is i s to t o create c r e a t e testing t e s t i n g and a n d experimentation e x p e r i m e n t a t i o n sites s i t e s to t o support s u p p o r t the t h e development d e v e l o p m e n t and a n d subsequent s u b s e q u e n t deployment d e p l o y m e n t of o f novel n o v e l AI A I applications. a p p l i c a t i o n s . �  Action A c t i o n 2: 2 : the t h e Commission C o m m i s s i o n will w i l l facilitate f a c i l i t a t e the t h e creation c r e a t i o n of o f excellence e x c e l l e n c e and a n d testing t e s t i n g centres c e n t r e s that t h a t can c a n combine c o m b i n e European, E u r o p e a n , national n a t i o n a l and a n d private p r i v a t e investments, i n v e s t m e n t s , possibly p o s s i b l y including i n c l u d i n g a a new n e w legal l e g a l instrument. i n s t r u m e n t . The T h e Commission C o m m i s s i o n has h a s proposed p r o p o s e d an a n ambitious a m b i t i o u s and a n d dedicated d e d i c a t e d amount a m o u n t to t o support s u p p o r t world w o r l d reference r e f e r e n c e testing t e s t i n g centres c e n t r e s in i n Europe E u r o p e under u n d e r the t h e Digital D i g i t a l Europe E u r o p e Programme P r o g r a m m e and a n d complemented c o m p l e m e n t e d where w h e r e appropriate a p p r o p r i a t e by b y research r e s e a r c h and a n d innovation i n n o v a t i o n actions a c t i o n s of o f Horizon H o r i z o n Europe E u r o p e as a s part p a r t of o f the t h e Multiannual M u l t i a n n u a l Financial F i n a n c i a l Framework F r a m e w o r k for f o r 2021 2 0 2 1 to t o 2027. 2 0 2 7 . C. C . SKILLS S K I L L S The T h e European E u r o p e a n approach a p p r o a c h to t o AI A I will w i l l need n e e d to t o be b e underpinned u n d e r p i n n e d by b y a a strong s t r o n g focus f o c u s on o n skills s k i l l s to t o fill f i l l competence c o m p e t e n c e shortages. s h o r t a g e s . 20 2 0 The T h e Commission C o m m i s s i o n will w i l l soon s o o n present p r e s e n t a a reinforcement r e i n f o r c e m e n t of o f the t h e Skills S k i l l s Agenda, A g e n d a , which w h i c h aims a i m s to t o ensure e n s u r e that t h a t everyone e v e r y o n e in i n Europe E u r o p e can c a n benefit b e n e f i t from f r o m the t h e green g r e e n and a n d digital d i g i t a l transformations t r a n s f o r m a t i o n s of o f the t h e EU E U economy. e c o n o m y . Initiatives I n i t i a t i v e s could c o u l d also a l s o include i n c l u d e the t h e support s u p p o r t of o f sectoral s e c t o r a l regulators r e g u l a t o r s to t o enhance e n h a n c e their t h e i r AI A I skills s k i l l s in i n order o r d e r to t o effectively e f f e c t i v e l y and a n d efficiently e f f i c i e n t l y implement i m p l e m e n t relevant r e l e v a n t rules. r u l e s . The T h e updated u p d a t e d Digital D i g i t a l Education E d u c a t i o n Action A c t i o n Plan P l a n will w i l l help h e l p make m a k e better b e t t e r use u s e of o f data d a t a and a n d AI-based A I - b a s e d technologies t e c h n o l o g i e s such s u c h as a s learning l e a r n i n g and a n d predictive p r e d i c t i v e analytics a n a l y t i c s with w i t h the t h e aim a i m to t o improve i m p r o v e education e d u c a t i o n and a n d training t r a i n i n g systems s y s t e m s and a n d make m a k e them t h e m fit f i t for f o r the t h e digital d i g i t a l age. a g e . The T h e Plan P l a n will w i l l also a l s o increase i n c r e a s e awareness a w a r e n e s s of o f AI A I at a t all a l l levels l e v e l s of o f education e d u c a t i o n in i n order o r d e r to t o prepare p r e p a r e citizens c i t i z e n s for f o r informed i n f o r m e d decisions d e c i s i o n s that t h a t will w i l l be b e increasingly i n c r e a s i n g l y affected a f f e c t e d by b y AI. A I . Developing D e v e l o p i n g the t h e skills s k i l l s necessary n e c e s s a r y to t o work w o r k in i n AI A I and a n d upskilling u p s k i l l i n g the t h e workforce w o r k f o r c e to t o become b e c o m e fit f i t for f o r the t h e AI- A I - led l e d transformation t r a n s f o r m a t i o n will w i l l be b e a a priority p r i o r i t y of o f the t h e revised r e v i s e d Coordinated C o o r d i n a t e d Plan P l a n on o n AI A I to t o be b e developed d e v e l o p e d with w i t h Member M e m b e r States. S t a t e s . This T h i s could c o u l d include i n c l u d e transforming t r a n s f o r m i n g the t h e assessment a s s e s s m e n t list l i s t of o f the t h e ethical e t h i c a l guidelines g u i d e l i n e s into i n t o an a n indicative i n d i c a t i v e “curriculum” “ c u r r i c u l u m ” for f o r developers d e v e l o p e r s of o f AI A I that t h a t will w i l l be b e made m a d e available a v a i l a b l e as a s a a resource r e s o u r c e for f o r training t r a i n i n g institutions. i n s t i t u t i o n s . Particular P a r t i c u l a r efforts e f f o r t s should s h o u l d be b e undertaken u n d e r t a k e n to t o increase i n c r e a s e the t h e number n u m b e r of o f women w o m e n trained t r a i n e d and a n d employed e m p l o y e d in i n this t h i s area. a r e a . In I n addition, a d d i t i o n , a a lighthouse l i g h t h o u s e centre c e n t r e of o f research r e s e a r c h and a n d innovation i n n o v a t i o n for f o r AI A I in i n Europe E u r o p e would w o u l d attract a t t r a c t talent t a l e n t from f r o m all a l l over o v e r the t h e world w o r l d due d u e to t o the t h e possibilities p o s s i b i l i t i e s it i t could c o u l d offer. o f f e r . It I t would w o u l d also a l s o develop d e v e l o p and a n d spread s p r e a d excellence e x c e l l e n c e in i n skills s k i l l s that t h a t take t a k e root r o o t and a n d grow g r o w across a c r o s s Europe. E u r o p e . 19 1 9 The T h e future f u t u r e European E u r o p e a n Defence D e f e n c e Fund F u n d and a n d Permanent P e r m a n e n t Structured S t r u c t u r e d Cooperation C o o p e r a t i o n (PESCO) ( P E S C O ) will w i l l also a l s o provide p r o v i d e opportunities o p p o r t u n i t i e s for f o r research r e s e a r c h and a n d development d e v e l o p m e n t in i n AI. A I . These T h e s e projects p r o j e c t s should s h o u l d be b e synchronized s y n c h r o n i z e d with w i t h the t h e wider w i d e r EU E U civilian c i v i l i a n programmes p r o g r a m m e s devoted d e v o t e d to t o AI. A I . 20 2 0 https://ec.europa.eu/jrc/en/publication/academic-offer-and-demand-advanced-profiles-eu h t t p s : / / e c . e u r o p a . e u / j r c / e n / p u b l i c a t i o n / a c a d e m i c - o f f e r - a n d - d e m a n d - a d v a n c e d - p r o f i l e s - e u 6 6 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} �  Action A c t i o n 3: 3 : Establish E s t a b l i s h and a n d support s u p p o r t through t h r o u g h the t h e advanced a d v a n c e d skills s k i l l s pillar p i l l a r of o f the t h e Digital D i g i t a l Europe E u r o p e Programme P r o g r a m m e networks n e t w o r k s of o f leading l e a d i n g universities u n i v e r s i t i e s and a n d higher h i g h e r education e d u c a t i o n institutes i n s t i t u t e s to t o attract a t t r a c t the t h e best b e s t professors p r o f e s s o r s and a n d scientists s c i e n t i s t s and a n d offer o f f e r world-leading w o r l d - l e a d i n g masters m a s t e r s programmes p r o g r a m m e s in i n AI. A I . Beyond B e y o n d upskilling, u p s k i l l i n g , workers w o r k e r s and a n d employers e m p l o y e r s are a r e directly d i r e c t l y affected a f f e c t e d by b y the t h e design d e s i g n and a n d use u s e of o f AI A I systems s y s t e m s in i n the t h e workplace. w o r k p l a c e . The T h e involvement i n v o l v e m e n t of o f social s o c i a l partners p a r t n e r s will w i l l be b e a a crucial c r u c i a l factor f a c t o r in i n ensuring e n s u r i n g a a human-centred h u m a n - c e n t r e d approach a p p r o a c h to t o AI A I at a t work. w o r k . D. D . FOCUS F O C U S ON O N SMES S M E S It I t will w i l l also a l s o be b e important i m p o r t a n t to t o ensure e n s u r e that t h a t SMEs S M E s can c a n access a c c e s s and a n d use u s e AI. A I . To T o this t h i s end, e n d , the t h e Digital D i g i t a l Innovation I n n o v a t i o n Hubs H u b s 21 2 1 and a n d the t h e AI-on-demand A I - o n - d e m a n d platform p l a t f o r m 22 2 2 should s h o u l d be b e strengthened s t r e n g t h e n e d further f u r t h e r and a n d foster f o s t e r collaboration c o l l a b o r a t i o n between b e t w e e n SMEs. S M E s . The T h e Digital D i g i t a l Europe E u r o p e Programme P r o g r a m m e will w i l l be b e instrumental i n s t r u m e n t a l in i n achieving a c h i e v i n g this. t h i s . While W h i l e all a l l Digital D i g i t a l Innovation I n n o v a t i o n Hubs H u b s should s h o u l d provide p r o v i d e support s u p p o r t to t o SMEs S M E s to t o understand u n d e r s t a n d and a n d adopt a d o p t AI, A I , it i t will w i l l be b e important i m p o r t a n t that t h a t at a t least l e a s t one o n e innovation i n n o v a t i o n hub h u b per p e r Member M e m b e r State S t a t e has h a s a a high h i g h degree d e g r e e of o f specialisation s p e c i a l i s a t i o n in i n AI. A I . SMEs S M E s and a n d start-ups s t a r t - u p s will w i l l need n e e d access a c c e s s to t o finance f i n a n c e in i n order o r d e r to t o adapt a d a p t their t h e i r processes p r o c e s s e s or o r to t o innovate i n n o v a t e using u s i n g AI. A I . Building B u i l d i n g on o n the t h e forthcoming f o r t h c o m i n g pilot p i l o t investment i n v e s t m e n t fund f u n d of o f €100 € 1 0 0 million m i l l i o n in i n AI A I and a n d blockchain, b l o c k c h a i n , the t h e Commission C o m m i s s i o n plans p l a n s to t o further f u r t h e r scale s c a l e up u p access a c c e s s to t o finance f i n a n c e in i n AI A I under u n d e r InvestEU I n v e s t E U 23 2 3 . . AI A I is i s explicitly e x p l i c i t l y mentioned m e n t i o n e d among a m o n g the t h e eligible e l i g i b l e areas a r e a s for f o r the t h e use u s e of o f the t h e InvestEU I n v e s t E U guarantee. g u a r a n t e e . �  Action A c t i o n 4: 4 : the t h e Commission C o m m i s s i o n will w i l l work w o r k with w i t h Member M e m b e r States S t a t e s to t o ensure e n s u r e that t h a t at a t least l e a s t one o n e digital d i g i t a l innovation i n n o v a t i o n hub h u b per p e r Member M e m b e r State S t a t e has h a s a a high h i g h degree d e g r e e of o f specialisation s p e c i a l i s a t i o n on o n AI. A I . Digital D i g i t a l Innovation I n n o v a t i o n Hubs H u b s can c a n be b e supported s u p p o r t e d under u n d e r the t h e Digital D i g i t a l Europe E u r o p e Programme. P r o g r a m m e . �  The T h e Commission C o m m i s s i o n and a n d the t h e European E u r o p e a n Investment I n v e s t m e n t Fund F u n d will w i l l launch l a u n c h a a pilot p i l o t scheme s c h e m e of o f €100 € 1 0 0 million m i l l i o n in i n Q1 Q 1 2020 2 0 2 0 to t o provide p r o v i d e equity e q u i t y financing f i n a n c i n g for f o r innovative i n n o v a t i v e developments d e v e l o p m e n t s in i n AI. A I . Subject S u b j e c t to t o final f i n a l agreement a g r e e m e n t on o n the t h e MFF, M F F , the t h e Commission’s C o m m i s s i o n ’ s intention i n t e n t i o n is i s to t o scale s c a l e it i t up u p significantly s i g n i f i c a n t l y from f r o m 2021 2 0 2 1 through t h r o u g h InvestEU. I n v e s t E U . E. E . PARTNERSHIP P A R T N E R S H I P WITH W I T H THE T H E PRIVATE P R I V A T E SECTOR S E C T O R It I t is i s also a l s o essential e s s e n t i a l to t o make m a k e sure s u r e that t h a t the t h e private p r i v a t e sector s e c t o r is i s fully f u l l y involved i n v o l v e d in i n setting s e t t i n g the t h e research r e s e a r c h and a n d innovation i n n o v a t i o n agenda a g e n d a and a n d provides p r o v i d e s the t h e necessary n e c e s s a r y level l e v e l of o f co-investment. c o - i n v e s t m e n t . This T h i s requires r e q u i r e s setting s e t t i n g up u p a a broad- b r o a d - based b a s e d public p u b l i c private p r i v a t e partnership, p a r t n e r s h i p , and a n d securing s e c u r i n g the t h e commitment c o m m i t m e n t of o f the t h e top t o p management m a n a g e m e n t of o f companies. c o m p a n i e s . �  Action A c t i o n 5: 5 : In I n the t h e context c o n t e x t of o f Horizon H o r i z o n Europe, E u r o p e , the t h e Commission C o m m i s s i o n will w i l l set s e t up u p a a new n e w public p u b l i c private p r i v a t e partnership p a r t n e r s h i p in i n AI, A I , data d a t a and a n d robotics r o b o t i c s to t o combine c o m b i n e efforts, e f f o r t s , ensure e n s u r e coordination c o o r d i n a t i o n of o f research r e s e a r c h and a n d innovation i n n o v a t i o n in i n AI, A I , collaborate c o l l a b o r a t e with w i t h other o t h e r public-private p u b l i c - p r i v a t e partnerships p a r t n e r s h i p s in i n Horizon H o r i z o n Europe E u r o p e and a n d work w o r k together t o g e t h e r with w i t h the t h e testing t e s t i n g facilities f a c i l i t i e s and a n d the t h e Digital D i g i t a l Innovation I n n o v a t i o n Hubs H u b s mentioned m e n t i o n e d above. a b o v e . 21 2 1 ec.europe.eu/digital-single-market/en/news/digital-innovation-hubs-helping-companies-across-economy-make-most- e c . e u r o p e . e u / d i g i t a l - s i n g l e - m a r k e t / e n / n e w s / d i g i t a l - i n n o v a t i o n - h u b s - h e l p i n g - c o m p a n i e s - a c r o s s - e c o n o m y - m a k e - m o s t - digital-opportunities. d i g i t a l - o p p o r t u n i t i e s . 22 2 2 www.Ai4eu.eu. w w w . A i 4 e u . e u . 23 2 3 Europe.eu/investeu. E u r o p e . e u / i n v e s t e u . 7 7 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} F. F . PROMOTING P R O M O T I N G THE T H E ADOPTION A D O P T I O N OF O F AI A I BY B Y THE T H E PUBLIC P U B L I C SECTOR S E C T O R It I t is i s essential e s s e n t i a l that t h a t public p u b l i c administrations, a d m i n i s t r a t i o n s , hospitals, h o s p i t a l s , utility u t i l i t y and a n d transport t r a n s p o r t services, s e r v i c e s , financial f i n a n c i a l supervisors, s u p e r v i s o r s , and a n d other o t h e r areas a r e a s of o f public p u b l i c interest i n t e r e s t rapidly r a p i d l y begin b e g i n to t o deploy d e p l o y products p r o d u c t s and a n d services s e r v i c e s that t h a t rely r e l y on o n AI A I in i n their t h e i r activities. a c t i v i t i e s . A A specific s p e c i f i c focus f o c u s will w i l l be b e in i n the t h e areas a r e a s of o f healthcare h e a l t h c a r e and a n d transport t r a n s p o r t where w h e r e technology t e c h n o l o g y is i s mature m a t u r e for f o r large-scale l a r g e - s c a l e deployment. d e p l o y m e n t . �  Action A c t i o n 6: 6 : The T h e Commission C o m m i s s i o n will w i l l initiate i n i t i a t e open o p e n and a n d transparent t r a n s p a r e n t sector s e c t o r dialogues d i a l o g u e s giving g i v i n g priority p r i o r i t y to t o healthcare, h e a l t h c a r e , rural r u r a l administrations a d m i n i s t r a t i o n s and a n d public p u b l i c service s e r v i c e operators o p e r a t o r s in i n order o r d e r to t o present p r e s e n t an a n action a c t i o n plan p l a n to t o facilitate f a c i l i t a t e development, d e v e l o p m e n t , experimentation e x p e r i m e n t a t i o n and a n d adoption. a d o p t i o n . The T h e sector s e c t o r dialogues d i a l o g u e s will w i l l be b e used u s e d to t o prepare p r e p a r e a a specific s p e c i f i c ‘Adopt ‘ A d o p t AI A I programme’ p r o g r a m m e ’ that t h a t will w i l l support s u p p o r t public p u b l i c procurement p r o c u r e m e n t of o f AI A I systems, s y s t e m s , and a n d help h e l p to t o transform t r a n s f o r m public p u b l i c procurement p r o c u r e m e n t processes p r o c e s s e s themselves. t h e m s e l v e s . G. G . SECURING S E C U R I N G ACCESS A C C E S S TO T O DATA D A T A AND A N D COMPUTING C O M P U T I N G INFRASTRUCTURES I N F R A S T R U C T U R E S The T h e areas a r e a s for f o r action a c t i o n set s e t out o u t in i n this t h i s White W h i t e Paper P a p e r are a r e complementary c o m p l e m e n t a r y to t o the t h e plan p l a n presented p r e s e n t e d in i n parallel p a r a l l e l under u n d e r the t h e European E u r o p e a n data d a t a strategy. s t r a t e g y . Improving I m p r o v i n g access a c c e s s to t o and a n d the t h e management m a n a g e m e n t of o f data d a t a is i s fundamental. f u n d a m e n t a l . Without W i t h o u t data, d a t a , the t h e development d e v e l o p m e n t of o f AI A I and a n d other o t h e r digital d i g i t a l applications a p p l i c a t i o n s is i s not n o t possible. p o s s i b l e . The T h e enormous e n o r m o u s volume v o l u m e of o f new n e w data d a t a yet y e t to t o be b e generated g e n e r a t e d constitutes c o n s t i t u t e s an a n opportunity o p p o r t u n i t y for f o r Europe E u r o p e to t o position p o s i t i o n itself i t s e l f at a t the t h e forefront f o r e f r o n t of o f the t h e data d a t a and a n d AI A I transformation. t r a n s f o r m a t i o n . Promoting P r o m o t i n g responsible r e s p o n s i b l e data d a t a management m a n a g e m e n t practices p r a c t i c e s and a n d compliance c o m p l i a n c e of o f data d a t a with w i t h the t h e FAIR F A I R principles p r i n c i p l e s will w i l l contribute c o n t r i b u t e to t o build b u i l d trust t r u s t and a n d ensure e n s u r e re-usability r e - u s a b i l i t y of o f data d a t a 24 2 4 . . Equally E q u a l l y important i m p o r t a n t is i s investment i n v e s t m e n t in i n key k e y computing c o m p u t i n g technologies t e c h n o l o g i e s and a n d infrastructures. i n f r a s t r u c t u r e s . The T h e Commission C o m m i s s i o n has h a s proposed p r o p o s e d more m o r e than t h a n €4 € 4 billion b i l l i o n under u n d e r the t h e Digital D i g i t a l Europe E u r o p e Programme P r o g r a m m e to t o support s u p p o r t high-performance h i g h - p e r f o r m a n c e and a n d quantum q u a n t u m computing, c o m p u t i n g , including i n c l u d i n g edge e d g e computing c o m p u t i n g and a n d AI, A I , data d a t a and a n d cloud c l o u d infrastructure. i n f r a s t r u c t u r e . The T h e European E u r o p e a n data d a t a strategy s t r a t e g y develops d e v e l o p s these t h e s e priorities p r i o r i t i e s further. f u r t h e r . H. H . INTERNATIONAL I N T E R N A T I O N A L ASPECTS A S P E C T S Europe E u r o p e is i s well w e l l positioned p o s i t i o n e d to t o exercise e x e r c i s e global g l o b a l leadership l e a d e r s h i p in i n building b u i l d i n g alliances a l l i a n c e s around a r o u n d shared s h a r e d values v a l u e s and a n d promoting p r o m o t i n g the t h e ethical e t h i c a l use u s e of o f AI. A I . The T h e EU's E U ' s work w o r k on o n AI A I has h a s already a l r e a d y influenced i n f l u e n c e d international i n t e r n a t i o n a l discussions. d i s c u s s i o n s . When W h e n developing d e v e l o p i n g its i t s ethical e t h i c a l guidelines, g u i d e l i n e s , the t h e High-Level H i g h - L e v e l Expert E x p e r t Group G r o u p involved i n v o l v e d a a number n u m b e r of o f non-EU n o n - E U organisations o r g a n i s a t i o n s and a n d several s e v e r a l governmental g o v e r n m e n t a l observers. o b s e r v e r s . In I n parallel, p a r a l l e l , the t h e EU E U was w a s closely c l o s e l y involved i n v o l v e d in i n developing d e v e l o p i n g the t h e OECD’s O E C D ’ s ethical e t h i c a l principles p r i n c i p l e s for f o r AI A I 25 2 5 . . The T h e G20 G 2 0 subsequently s u b s e q u e n t l y endorsed e n d o r s e d these t h e s e principles p r i n c i p l e s in i n its i t s June J u n e 2019 2 0 1 9 Ministerial M i n i s t e r i a l Statement S t a t e m e n t on o n Trade T r a d e and a n d Digital D i g i t a l Economy. E c o n o m y . In I n parallel, p a r a l l e l , the t h e EU E U recognises r e c o g n i s e s that t h a t important i m p o r t a n t work w o r k on o n AI A I is i s ingoing i n g o i n g in i n other o t h e r multilateral m u l t i l a t e r a l fora, f o r a , including i n c l u d i n g the t h e Council C o u n c i l of o f Europe, E u r o p e , the t h e United U n i t e d Nations N a t i o n s Educational E d u c a t i o n a l Scientific S c i e n t i f i c and a n d Cultural C u l t u r a l Organization O r g a n i z a t i o n (UNESCO), ( U N E S C O ) , the t h e Organisation O r g a n i s a t i o n for f o r Economic E c o n o m i c Co-operation C o - o p e r a t i o n and a n d Development’s D e v e l o p m e n t ’ s (OECD), ( O E C D ) , the t h e World W o r l d Trade T r a d e Organisation O r g a n i s a t i o n and a n d the t h e International I n t e r n a t i o n a l Telecommunications T e l e c o m m u n i c a t i o n s Union U n i o n (ITU). ( I T U ) . At A t the t h e UN, U N , the t h e EU E U is i s involved i n v o l v e d in i n the t h e follow-up f o l l o w - u p of o f the t h e report r e p o r t of o f the t h e High-Level H i g h - L e v e l Panel P a n e l on o n Digital D i g i t a l Cooperation, C o o p e r a t i o n , including i n c l u d i n g its i t s recommendation r e c o m m e n d a t i o n on o n AI. A I . The T h e EU E U will w i l l continue c o n t i n u e to t o cooperate c o o p e r a t e with w i t h like-minded l i k e - m i n d e d countries, c o u n t r i e s , but b u t also a l s o with w i t h global g l o b a l players, p l a y e r s , on o n AI, A I , based b a s e d on o n an a n approach a p p r o a c h based b a s e d on o n EU E U rules r u l e s and a n d values v a l u e s (e.g. ( e . g . supporting s u p p o r t i n g upward u p w a r d regulatory r e g u l a t o r y convergence, c o n v e r g e n c e , accessing a c c e s s i n g key k e y resources r e s o u r c e s including i n c l u d i n g data, d a t a , creating c r e a t i n g a a level l e v e l playing p l a y i n g field). f i e l d ) . The T h e Commission C o m m i s s i o n will w i l l closely c l o s e l y monitor m o n i t o r the t h e policies p o l i c i e s of o f third t h i r d countries c o u n t r i e s that t h a t limit l i m i t data d a t a flows f l o w s and a n d will w i l l address a d d r e s s undue u n d u e restrictions r e s t r i c t i o n s in i n 24 2 4 Findable, F i n d a b l e , Accessible, A c c e s s i b l e , Interoperable I n t e r o p e r a b l e and a n d Reusable R e u s a b l e as a s stated s t a t e d in i n the t h e Final F i n a l Report R e p o r t and a n d Action A c t i o n Plan P l a n from f r o m the t h e Commission C o m m i s s i o n Expert E x p e r t Group G r o u p on o n FAIR F A I R data, d a t a , 2018, 2 0 1 8 , https://ec.europa.eu/info/sites/info/files/turning_fair_into_reality_1.pdf. h t t p s : / / e c . e u r o p a . e u / i n f o / s i t e s / i n f o / f i l e s / t u r n i n g _ f a i r _ i n t o _ r e a l i t y _ 1 . p d f . 25 2 5 https://www.oecd.org/going-digital/ai/principles/ h t t p s : / / w w w . o e c d . o r g / g o i n g - d i g i t a l / a i / p r i n c i p l e s / 8 8 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} bilateral b i l a t e r a l trade t r a d e negotiations n e g o t i a t i o n s and a n d through t h r o u g h action a c t i o n in i n the t h e context c o n t e x t of o f the t h e World W o r l d Trade T r a d e Organization. O r g a n i z a t i o n . The T h e Commission C o m m i s s i o n is i s convinced c o n v i n c e d that t h a t international i n t e r n a t i o n a l cooperation c o o p e r a t i o n on o n AI A I matters m a t t e r s must m u s t be b e based b a s e d on o n an a n approach a p p r o a c h that t h a t promotes p r o m o t e s the t h e respect r e s p e c t of o f fundamental f u n d a m e n t a l rights, r i g h t s , including i n c l u d i n g human h u m a n dignity, d i g n i t y , pluralism, p l u r a l i s m , inclusion, i n c l u s i o n , non- n o n - discrimination d i s c r i m i n a t i o n and a n d protection p r o t e c t i o n of o f privacy p r i v a c y and a n d personal p e r s o n a l data d a t a 26 2 6 and a n d it i t will w i l l strive s t r i v e to t o export e x p o r t its i t s values v a l u e s across a c r o s s the t h e world w o r l d 27 2 7 . . It I t is i s also a l s o clear c l e a r that t h a t the t h e responsible r e s p o n s i b l e development d e v e l o p m e n t and a n d use u s e of o f AI A I can c a n be b e a a driving d r i v i n g force f o r c e to t o achieve a c h i e v e the t h e Sustainable S u s t a i n a b l e Development D e v e l o p m e n t Goals G o a l s and a n d advance a d v a n c e the t h e 2030 2 0 3 0 Agenda. A g e n d a . 5. 5 . AN A N ECOSYSTEM E C O S Y S T E M OF O F TRUST: T R U S T : REGULATORY R E G U L A T O R Y FRAMEWORK F R A M E W O R K FOR F O R AI A I As A s with w i t h any a n y new n e w technology, t e c h n o l o g y , the t h e use u s e of o f AI A I brings b r i n g s both b o t h opportunities o p p o r t u n i t i e s and a n d risks. r i s k s . Citizens C i t i z e n s fear f e a r being b e i n g left l e f t powerless p o w e r l e s s in i n defending d e f e n d i n g their t h e i r rights r i g h t s and a n d safety s a f e t y when w h e n facing f a c i n g the t h e information i n f o r m a t i o n asymmetries a s y m m e t r i e s of o f algorithmic a l g o r i t h m i c decision-making, d e c i s i o n - m a k i n g , and a n d companies c o m p a n i e s are a r e concerned c o n c e r n e d by b y legal l e g a l uncertainty. u n c e r t a i n t y . While W h i l e AI A I can c a n help h e l p protect p r o t e c t citizens' c i t i z e n s ' security s e c u r i t y and a n d enable e n a b l e them t h e m to t o enjoy e n j o y their t h e i r fundamental f u n d a m e n t a l rights, r i g h t s , citizens c i t i z e n s also a l s o worry w o r r y that t h a t AI A I can c a n have h a v e unintended u n i n t e n d e d effects e f f e c t s or o r even e v e n be b e used u s e d for f o r malicious m a l i c i o u s purposes. p u r p o s e s . These T h e s e concerns c o n c e r n s need n e e d to t o be b e addressed. a d d r e s s e d . Moreover, M o r e o v e r , in i n addition a d d i t i o n to t o a a lack l a c k of o f investment i n v e s t m e n t and a n d skills, s k i l l s , lack l a c k of o f trust t r u s t is i s a a main m a i n factor f a c t o r holding h o l d i n g back b a c k a a broader b r o a d e r uptake u p t a k e of o f AI. A I . That T h a t is i s why w h y the t h e Commission C o m m i s s i o n set s e t out o u t an a n AI A I strategy s t r a t e g y 28 2 8 on o n 25 2 5 April A p r i l 2018 2 0 1 8 addressing a d d r e s s i n g the t h e socioeconomic s o c i o e c o n o m i c aspects a s p e c t s in i n parallel p a r a l l e l with w i t h an a n increase i n c r e a s e in i n investment i n v e s t m e n t in i n research, r e s e a r c h , innovation i n n o v a t i o n and a n d AI-capacity A I - c a p a c i t y across a c r o s s the t h e EU. E U . It I t agreed a g r e e d a a Coordinated C o o r d i n a t e d Plan P l a n 29 2 9 with w i t h the t h e Member M e m b e r States S t a t e s to t o align a l i g n strategies. s t r a t e g i e s . The T h e Commission C o m m i s s i o n also a l s o established e s t a b l i s h e d a a High-Level H i g h - L e v e l Expert E x p e r t Group G r o u p that t h a t published p u b l i s h e d Guidelines G u i d e l i n e s on o n trustworthy t r u s t w o r t h y AI A I in i n April A p r i l 2019 2 0 1 9 30 3 0 . . The T h e Commission C o m m i s s i o n published p u b l i s h e d a a Communication C o m m u n i c a t i o n 31 3 1 welcoming w e l c o m i n g the t h e seven s e v e n key k e y requirements r e q u i r e m e n t s identified i d e n t i f i e d in i n the t h e Guidelines G u i d e l i n e s of o f the t h e High-Level H i g h - L e v e l Expert E x p e r t Group: G r o u p : �  Human H u m a n agency a g e n c y and a n d oversight, o v e r s i g h t , �  Technical T e c h n i c a l robustness r o b u s t n e s s and a n d safety, s a f e t y , �  Privacy P r i v a c y and a n d data d a t a governance, g o v e r n a n c e , �  Transparency, T r a n s p a r e n c y , �  Diversity, D i v e r s i t y , non-discrimination n o n - d i s c r i m i n a t i o n and a n d fairness, f a i r n e s s , �  Societal S o c i e t a l and a n d environmental e n v i r o n m e n t a l wellbeing, w e l l b e i n g , and a n d �  Accountability. A c c o u n t a b i l i t y . In I n addition, a d d i t i o n , the t h e Guidelines G u i d e l i n e s contain c o n t a i n an a n assessment a s s e s s m e n t list l i s t for f o r practical p r a c t i c a l use u s e by b y companies. c o m p a n i e s . During D u r i n g the t h e second s e c o n d half h a l f of o f 2019, 2 0 1 9 , over o v e r 350 3 5 0 organisations o r g a n i s a t i o n s have h a v e tested t e s t e d this t h i s assessment a s s e s s m e n t list l i s t and a n d sent s e n t feedback. f e e d b a c k . The T h e High-Level H i g h - L e v e l Group G r o u p is i s in i n the t h e process p r o c e s s of o f revising r e v i s i n g its i t s guidelines g u i d e l i n e s in i n light l i g h t of o f this t h i s feedback f e e d b a c k and a n d will w i l l finalise f i n a l i s e this t h i s work w o r k by b y June J u n e 2020. 2 0 2 0 . A A key k e y result r e s u l t of o f the t h e feedback f e e d b a c k process p r o c e s s is i s that t h a t while w h i l e a a number n u m b e r of o f the t h e requirements r e q u i r e m e n t s are a r e already a l r e a d y reflected r e f l e c t e d in i n existing e x i s t i n g legal l e g a l or o r regulatory r e g u l a t o r y regimes, r e g i m e s , those t h o s e regarding r e g a r d i n g transparency, t r a n s p a r e n c y , traceability t r a c e a b i l i t y and a n d human h u m a n oversight o v e r s i g h t are a r e not n o t specifically s p e c i f i c a l l y covered c o v e r e d under u n d e r current c u r r e n t legislation l e g i s l a t i o n in i n many m a n y economic e c o n o m i c sectors. s e c t o r s . On O n top t o p of o f this t h i s set s e t of o f non-binding n o n - b i n d i n g Guidelines G u i d e l i n e s of o f the t h e High-Level H i g h - L e v e l Expert E x p e r t Group, G r o u p , and a n d in i n line l i n e with w i t h the t h e President’s P r e s i d e n t ’ s political p o l i t i c a l guidelines, g u i d e l i n e s , a a clear c l e a r European E u r o p e a n regulatory r e g u l a t o r y framework f r a m e w o r k would w o u l d build b u i l d trust t r u s t among a m o n g 26 2 6 Under U n d e r the t h e Partnership P a r t n e r s h i p Instrument, I n s t r u m e n t , the t h e Commission C o m m i s s i o n will w i l l finance f i n a n c e a a €2.5 € 2 . 5 million m i l l i o n project p r o j e c t that t h a t will w i l l facilitate f a c i l i t a t e cooperation c o o p e r a t i o n with w i t h like-minded l i k e - m i n d e d partners, p a r t n e r s , in i n order o r d e r to t o promote p r o m o t e the t h e EU E U AI A I ethical e t h i c a l guidelines g u i d e l i n e s and a n d to t o adopt a d o p t common c o m m o n principles p r i n c i p l e s and a n d operational o p e r a t i o n a l conclusions. c o n c l u s i o n s . 27 2 7 President P r e s i d e n t Von V o n der d e r Leyen, L e y e n , A A Union U n i o n that t h a t strives s t r i v e s for f o r more m o r e – – My M y agenda a g e n d a for f o r Europe, E u r o p e , page p a g e 17. 1 7 . 28 2 8 COM(2018) C O M ( 2 0 1 8 ) 237. 2 3 7 . 29 2 9 COM(2018) C O M ( 2 0 1 8 ) 795. 7 9 5 . 30 3 0 https://ec.europa.eu/futurium/en/ai-alliance-consultation/guidelines#Top h t t p s : / / e c . e u r o p a . e u / f u t u r i u m / e n / a i - a l l i a n c e - c o n s u l t a t i o n / g u i d e l i n e s # T o p 31 3 1 COM(2019) C O M ( 2 0 1 9 ) 168. 1 6 8 . 9 9 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} consumers c o n s u m e r s and a n d businesses b u s i n e s s e s in i n AI, A I , and a n d therefore t h e r e f o r e speed s p e e d up u p the t h e uptake u p t a k e of o f the t h e technology. t e c h n o l o g y . Such S u c h a a regulatory r e g u l a t o r y framework f r a m e w o r k should s h o u l d be b e consistent c o n s i s t e n t with w i t h other o t h e r actions a c t i o n s to t o promote p r o m o t e Europe’s E u r o p e ’ s innovation i n n o v a t i o n capacity c a p a c i t y and a n d competitiveness c o m p e t i t i v e n e s s in i n this t h i s field. f i e l d . In I n addition, a d d i t i o n , it i t must m u s t ensure e n s u r e socially, s o c i a l l y , environmentally e n v i r o n m e n t a l l y and a n d economically e c o n o m i c a l l y optimal o p t i m a l outcomes o u t c o m e s and a n d compliance c o m p l i a n c e with w i t h EU E U legislation, l e g i s l a t i o n , principles p r i n c i p l e s and a n d values. v a l u e s . This T h i s is i s particularly p a r t i c u l a r l y relevant r e l e v a n t in i n areas a r e a s where w h e r e citizens’ c i t i z e n s ’ rights r i g h t s may m a y be b e most m o s t directly d i r e c t l y affected, a f f e c t e d , for f o r example e x a m p l e in i n the t h e case c a s e of o f AI A I applications a p p l i c a t i o n s for f o r law l a w enforcement e n f o r c e m e n t and a n d the t h e judiciary. j u d i c i a r y . Developers D e v e l o p e r s and a n d deployers d e p l o y e r s of o f AI A I are a r e already a l r e a d y subject s u b j e c t to t o European E u r o p e a n legislation l e g i s l a t i o n on o n fundamental f u n d a m e n t a l rights r i g h t s (e.g. ( e . g . data d a t a protection, p r o t e c t i o n , privacy, p r i v a c y , non-discrimination), n o n - d i s c r i m i n a t i o n ) , consumer c o n s u m e r protection, p r o t e c t i o n , and a n d product p r o d u c t safety s a f e t y and a n d liability l i a b i l i t y rules. r u l e s . Consumers C o n s u m e r s expect e x p e c t the t h e same s a m e level l e v e l of o f safety s a f e t y and a n d respect r e s p e c t of o f their t h e i r rights r i g h t s whether w h e t h e r or o r not n o t a a product p r o d u c t or o r a a system s y s t e m relies r e l i e s on o n AI. A I . However, H o w e v e r , some s o m e specific s p e c i f i c features f e a t u r e s of o f AI A I (e.g. ( e . g . opacity) o p a c i t y ) can c a n make m a k e the t h e application a p p l i c a t i o n and a n d enforcement e n f o r c e m e n t of o f this t h i s legislation l e g i s l a t i o n more m o r e difficult. d i f f i c u l t . For F o r this t h i s reason, r e a s o n , there t h e r e is i s a a need n e e d to t o examine e x a m i n e whether w h e t h e r current c u r r e n t legislation l e g i s l a t i o n is i s able a b l e to t o address a d d r e s s the t h e risks r i s k s of o f AI A I and a n d can c a n be b e effectively e f f e c t i v e l y enforced, e n f o r c e d , whether w h e t h e r adaptations a d a p t a t i o n s of o f the t h e legislation l e g i s l a t i o n are a r e needed, n e e d e d , or o r whether w h e t h e r new n e w legislation l e g i s l a t i o n is i s needed. n e e d e d . Given G i v e n how h o w fast f a s t AI A I is i s evolving, e v o l v i n g , the t h e regulatory r e g u l a t o r y framework f r a m e w o r k must m u s t leave l e a v e room r o o m to t o cater c a t e r for f o r further f u r t h e r developments. d e v e l o p m e n t s . Any A n y changes c h a n g e s should s h o u l d be b e limited l i m i t e d to t o clearly c l e a r l y identified i d e n t i f i e d problems p r o b l e m s for f o r which w h i c h feasible f e a s i b l e solutions s o l u t i o n s exist. e x i s t . Member M e m b e r States S t a t e s are a r e pointing p o i n t i n g at a t the t h e current c u r r e n t absence a b s e n c e of o f a a common c o m m o n European E u r o p e a n framework. f r a m e w o r k . The T h e German G e r m a n Data D a t a Ethics E t h i c s Commission C o m m i s s i o n has h a s called c a l l e d for f o r a a five-level f i v e - l e v e l risk-based r i s k - b a s e d system s y s t e m of o f regulation r e g u l a t i o n that t h a t would w o u l d go g o from f r o m no n o regulation r e g u l a t i o n for f o r the t h e most m o s t innocuous i n n o c u o u s AI A I systems s y s t e m s to t o a a complete c o m p l e t e ban b a n for f o r the t h e most m o s t dangerous d a n g e r o u s ones. o n e s . Denmark D e n m a r k has h a s just j u s t launched l a u n c h e d the t h e prototype p r o t o t y p e of o f a a Data D a t a Ethics E t h i c s Seal. S e a l . Malta M a l t a has h a s introduced i n t r o d u c e d a a voluntary v o l u n t a r y certification c e r t i f i c a t i o n system s y s t e m for f o r AI. A I . If I f the t h e EU E U fails f a i l s to t o provide p r o v i d e an a n EU-wide E U - w i d e approach, a p p r o a c h , there t h e r e is i s a a real r e a l risk r i s k of o f fragmentation f r a g m e n t a t i o n in i n the t h e internal i n t e r n a l market, m a r k e t , which w h i c h would w o u l d undermine u n d e r m i n e the t h e objectives o b j e c t i v e s of o f trust, t r u s t , legal l e g a l certainty c e r t a i n t y and a n d market m a r k e t uptake. u p t a k e . A A solid s o l i d European E u r o p e a n regulatory r e g u l a t o r y framework f r a m e w o r k for f o r trustworthy t r u s t w o r t h y AI A I will w i l l protect p r o t e c t all a l l European E u r o p e a n citizens c i t i z e n s and a n d help h e l p create c r e a t e a a frictionless f r i c t i o n l e s s internal i n t e r n a l market m a r k e t for f o r the t h e further f u r t h e r development d e v e l o p m e n t and a n d uptake u p t a k e of o f AI A I as a s well w e l l as a s strengthening s t r e n g t h e n i n g Europe’s E u r o p e ’ s industrial i n d u s t r i a l basis b a s i s in i n AI. A I . A. A . PROBLEM P R O B L E M DEFINITION D E F I N I T I O N While W h i l e AI A I can c a n do d o much m u c h good, g o o d , including i n c l u d i n g by b y making m a k i n g products p r o d u c t s and a n d processes p r o c e s s e s safer, s a f e r , it i t can c a n also a l s o do d o harm. h a r m . This T h i s harm h a r m might m i g h t be b e both b o t h material m a t e r i a l (safety ( s a f e t y and a n d health h e a l t h of o f individuals, i n d i v i d u a l s , including i n c l u d i n g loss l o s s of o f life, l i f e , damage d a m a g e to t o property) p r o p e r t y ) and a n d immaterial i m m a t e r i a l (loss ( l o s s of o f privacy, p r i v a c y , limitations l i m i t a t i o n s to t o the t h e right r i g h t of o f freedom f r e e d o m of o f expression, e x p r e s s i o n , human h u m a n dignity, d i g n i t y , discrimination d i s c r i m i n a t i o n for f o r instance i n s t a n c e in i n access a c c e s s to t o employment), e m p l o y m e n t ) , and a n d can c a n relate r e l a t e to t o a a wide w i d e variety v a r i e t y of o f risks. r i s k s . A A regulatory r e g u l a t o r y framework f r a m e w o r k should s h o u l d concentrate c o n c e n t r a t e on o n how h o w to t o minimise m i n i m i s e the t h e various v a r i o u s risks r i s k s of o f potential p o t e n t i a l harm, h a r m , in i n particular p a r t i c u l a r the t h e most m o s t significant s i g n i f i c a n t ones. o n e s . The T h e main m a i n risks r i s k s related r e l a t e d to t o the t h e use u s e of o f AI A I concern c o n c e r n the t h e application a p p l i c a t i o n of o f rules r u l e s designed d e s i g n e d to t o protect p r o t e c t fundamental f u n d a m e n t a l rights r i g h t s (including ( i n c l u d i n g personal p e r s o n a l data d a t a and a n d privacy p r i v a c y protection p r o t e c t i o n and a n d non-discrimination), n o n - d i s c r i m i n a t i o n ) , as a s well w e l l as a s safety s a f e t y 32 3 2 and a n d liability-related l i a b i l i t y - r e l a t e d issues. i s s u e s . Risks R i s k s for f o r fundamental f u n d a m e n t a l rights, r i g h t s , including i n c l u d i n g personal p e r s o n a l data d a t a and a n d privacy p r i v a c y protection p r o t e c t i o n and a n d non- n o n - discrimination d i s c r i m i n a t i o n 32 3 2 This T h i s includes i n c l u d e s issues i s s u e s of o f cybersecurity, c y b e r s e c u r i t y , issues i s s u e s associated a s s o c i a t e d with w i t h AI A I applications a p p l i c a t i o n s in i n critical c r i t i c a l infrastructures, i n f r a s t r u c t u r e s , or o r malicious m a l i c i o u s use u s e of o f AI. A I . 10 1 0 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} The T h e use u s e of o f AI A I can c a n affect a f f e c t the t h e values v a l u e s on o n which w h i c h the t h e EU E U is i s founded f o u n d e d and a n d lead l e a d to t o breaches b r e a c h e s of o f fundamental f u n d a m e n t a l rights r i g h t s 33 3 3 , , including i n c l u d i n g the t h e rights r i g h t s to t o freedom f r e e d o m of o f expression, e x p r e s s i o n , freedom f r e e d o m of o f assembly, a s s e m b l y , human h u m a n dignity, d i g n i t y , non- n o n - discrimination d i s c r i m i n a t i o n based b a s e d on o n sex, s e x , racial r a c i a l or o r ethnic e t h n i c origin, o r i g i n , religion r e l i g i o n or o r belief, b e l i e f , disability, d i s a b i l i t y , age a g e or o r sexual s e x u a l orientation, o r i e n t a t i o n , as a s applicable a p p l i c a b l e in i n certain c e r t a i n domains, d o m a i n s , protection p r o t e c t i o n of o f personal p e r s o n a l data d a t a and a n d private p r i v a t e life, l i f e , 34 3 4 or o r the t h e right r i g h t to t o an a n effective e f f e c t i v e judicial j u d i c i a l remedy r e m e d y and a n d a a fair f a i r trial, t r i a l , as a s well w e l l as a s consumer c o n s u m e r protection. p r o t e c t i o n . These T h e s e risks r i s k s might m i g h t result r e s u l t from f r o m flaws f l a w s in i n the t h e overall o v e r a l l design d e s i g n of o f AI A I systems s y s t e m s (including ( i n c l u d i n g as a s regards r e g a r d s human h u m a n oversight) o v e r s i g h t ) or o r from f r o m the t h e use u s e of o f data d a t a without w i t h o u t correcting c o r r e c t i n g possible p o s s i b l e bias b i a s (e.g. ( e . g . the t h e system s y s t e m is i s trained t r a i n e d using u s i n g only o n l y or o r mainly m a i n l y data d a t a from f r o m men m e n leading l e a d i n g to t o suboptimal s u b o p t i m a l results r e s u l t s in i n relation r e l a t i o n to t o women). w o m e n ) . AI A I can c a n perform p e r f o r m many m a n y functions f u n c t i o n s that t h a t previously p r e v i o u s l y could c o u l d only o n l y be b e done d o n e by b y humans. h u m a n s . As A s a a result, r e s u l t , citizens c i t i z e n s and a n d legal l e g a l entities e n t i t i e s will w i l l increasingly i n c r e a s i n g l y be b e subject s u b j e c t to t o actions a c t i o n s and a n d decisions d e c i s i o n s taken t a k e n by b y or o r with w i t h the t h e assistance a s s i s t a n c e of o f AI A I systems, s y s t e m s , which w h i c h may m a y sometimes s o m e t i m e s be b e difficult d i f f i c u l t to t o understand u n d e r s t a n d and a n d to t o effectively e f f e c t i v e l y challenge c h a l l e n g e where w h e r e necessary. n e c e s s a r y . Moreover, M o r e o v e r , AI A I increases i n c r e a s e s the t h e possibilities p o s s i b i l i t i e s to t o track t r a c k and a n d analyse a n a l y s e the t h e daily d a i l y habits h a b i t s of o f people. p e o p l e . For F o r example, e x a m p l e , there t h e r e is i s a a potential p o t e n t i a l risk r i s k that t h a t AI A I may m a y be b e used, u s e d , in i n breach b r e a c h of o f EU E U data d a t a protection p r o t e c t i o n and a n d other o t h e r rules, r u l e s , by b y state s t a t e authorities a u t h o r i t i e s or o r other o t h e r entities e n t i t i e s for f o r mass m a s s surveillance s u r v e i l l a n c e and a n d by b y employers e m p l o y e r s to t o observe o b s e r v e how h o w their t h e i r employees e m p l o y e e s behave. b e h a v e . By B y analysing a n a l y s i n g large l a r g e amounts a m o u n t s of o f data d a t a and a n d identifying i d e n t i f y i n g links l i n k s among a m o n g them, t h e m , AI A I may m a y also a l s o be b e used u s e d to t o retrace r e t r a c e and a n d de-anonymise d e - a n o n y m i s e data d a t a about a b o u t persons, p e r s o n s , creating c r e a t i n g new n e w personal p e r s o n a l data d a t a protection p r o t e c t i o n risks r i s k s even e v e n in i n respect r e s p e c t to t o datasets d a t a s e t s that t h a t per p e r se s e do d o not n o t include i n c l u d e personal p e r s o n a l data. d a t a . AI A I is i s also a l s o used u s e d by b y online o n l i n e intermediaries i n t e r m e d i a r i e s to t o prioritise p r i o r i t i s e information i n f o r m a t i o n for f o r their t h e i r users u s e r s and a n d to t o perform p e r f o r m content c o n t e n t moderation. m o d e r a t i o n . The T h e processed p r o c e s s e d data, d a t a , the t h e way w a y applications a p p l i c a t i o n s are a r e designed d e s i g n e d and a n d the t h e scope s c o p e for f o r human h u m a n intervention i n t e r v e n t i o n can c a n affect a f f e c t the t h e rights r i g h t s to t o free f r e e expression, e x p r e s s i o n , personal p e r s o n a l data d a t a protection, p r o t e c t i o n , privacy, p r i v a c y , and a n d political p o l i t i c a l freedoms. f r e e d o m s . Certain C e r t a i n AI A I algorithms, a l g o r i t h m s , when w h e n exploited e x p l o i t e d for f o r predicting p r e d i c t i n g criminal c r i m i n a l recidivism, r e c i d i v i s m , can c a n display d i s p l a y gender g e n d e r and a n d racial r a c i a l bias, b i a s , demonstrating d e m o n s t r a t i n g different d i f f e r e n t recidivism r e c i d i v i s m prediction p r e d i c t i o n probability p r o b a b i l i t y for f o r women w o m e n vs v s men m e n or o r for f o r nationals n a t i o n a l s vs v s foreigners. f o r e i g n e r s . Source: S o u r c e : Tolan T o l a n S., S . , Miron M i r o n M., M . , Gomez G o m e z E. E . and a n d Castillo C a s t i l l o C. C . "Why " W h y Machine M a c h i n e Learning L e a r n i n g May M a y Lead L e a d to t o Unfairness: U n f a i r n e s s : Evidence E v i d e n c e from f r o m Risk R i s k Assessment A s s e s s m e n t for f o r Juvenile J u v e n i l e Justice J u s t i c e in i n Catalonia", C a t a l o n i a " , Best B e s t Paper P a p e r Award, A w a r d , International I n t e r n a t i o n a l Conference C o n f e r e n c e on o n AI A I and a n d Law, L a w , 2019 2 0 1 9 Certain C e r t a i n AI A I programmes p r o g r a m m e s for f o r facial f a c i a l analysis a n a l y s i s display d i s p l a y gender g e n d e r and a n d racial r a c i a l bias, b i a s , demonstrating d e m o n s t r a t i n g low l o w errors e r r o r s for f o r determining d e t e r m i n i n g the t h e gender g e n d e r of o f lighter-skinned l i g h t e r - s k i n n e d men m e n but b u t high h i g h errors e r r o r s in i n determining d e t e r m i n i n g gender g e n d e r for f o r darker-skinned d a r k e r - s k i n n e d women. w o m e n . Source: S o u r c e : Joy J o y Buolamwini, B u o l a m w i n i , Timnit T i m n i t Gebru; G e b r u ; Proceedings P r o c e e d i n g s of o f the t h e 1st 1 s t Conference C o n f e r e n c e on o n Fairness, F a i r n e s s , Accountability A c c o u n t a b i l i t y and a n d Transparency, T r a n s p a r e n c y , PMLR P M L R 81:77-91, 8 1 : 7 7 - 9 1 , 2018. 2 0 1 8 . Bias B i a s and a n d discrimination d i s c r i m i n a t i o n are a r e inherent i n h e r e n t risks r i s k s of o f any a n y societal s o c i e t a l or o r economic e c o n o m i c activity. a c t i v i t y . Human H u m a n decision- d e c i s i o n - making m a k i n g is i s not n o t immune i m m u n e to t o mistakes m i s t a k e s and a n d biases. b i a s e s . However, H o w e v e r , the t h e same s a m e bias b i a s when w h e n present p r e s e n t in i n AI A I could c o u l d have h a v e a a much m u c h larger l a r g e r effect, e f f e c t , affecting a f f e c t i n g and a n d discriminating d i s c r i m i n a t i n g many m a n y people p e o p l e without w i t h o u t the t h e social s o c i a l control c o n t r o l mechanisms m e c h a n i s m s that t h a t govern g o v e r n human h u m a n behaviour b e h a v i o u r 35 3 5 . . This T h i s can c a n also a l s o happen h a p p e n when w h e n the t h e AI A I system s y s t e m ‘learns’ ‘ l e a r n s ’ while w h i l e in i n operation. o p e r a t i o n . 33 3 3 Council C o u n c i l of o f Europe E u r o p e research r e s e a r c h shows s h o w s that t h a t a a large l a r g e number n u m b e r of o f fundamental f u n d a m e n t a l rights r i g h t s could c o u l d be b e impacted i m p a c t e d from f r o m the t h e use u s e of o f AI, A I , https://rm.coe.int/algorithms-and-human-rights-en-rev/16807956b5. h t t p s : / / r m . c o e . i n t / a l g o r i t h m s - a n d - h u m a n - r i g h t s - e n - r e v / 1 6 8 0 7 9 5 6 b 5 . 34 3 4 The T h e General G e n e r a l Data D a t a Protection P r o t e c t i o n Regulation R e g u l a t i o n and a n d the t h e ePrivacy e P r i v a c y Directive D i r e c t i v e (new ( n e w ePrivacy e P r i v a c y Regulation R e g u l a t i o n under u n d e r negotiation) n e g o t i a t i o n ) address a d d r e s s these t h e s e risks r i s k s but b u t there t h e r e might m i g h t be b e a a need n e e d to t o examine e x a m i n e whether w h e t h e r AI A I systems s y s t e m s pose p o s e additional a d d i t i o n a l risks. r i s k s . The T h e Commission C o m m i s s i o n will w i l l be b e monitoring m o n i t o r i n g and a n d assessing a s s e s s i n g the t h e application a p p l i c a t i o n of o f the t h e GDPR G D P R on o n a a continuous c o n t i n u o u s basis. b a s i s . 35 3 5 The T h e Commission’s C o m m i s s i o n ’ s Advisory A d v i s o r y Committee C o m m i t t e e on o n Equal E q u a l Opportunities O p p o r t u n i t i e s for f o r Women W o m e n and a n d Men M e n is i s currently c u r r e n t l y preparing p r e p a r i n g an a n “Opinion “ O p i n i o n on o n Artificial A r t i f i c i a l Intelligence” I n t e l l i g e n c e ” analysing a n a l y s i n g inter i n t e r alia a l i a the t h e impacts i m p a c t s of o f Artificial A r t i f i c i a l Intelligence I n t e l l i g e n c e on o n gender g e n d e r equality e q u a l i t y which w h i c h is i s expected e x p e c t e d to t o be b e adopted a d o p t e d by b y the t h e Committee C o m m i t t e e in i n early e a r l y 2020. 2 0 2 0 . The T h e EU E U Gender G e n d e r Equality E q u a l i t y Strategy S t r a t e g y 2020-2024 2 0 2 0 - 2 0 2 4 also a l s o addresses a d d r e s s e s the t h e link l i n k between b e t w e e n AI A I on o n gender g e n d e r equality; e q u a l i t y ; The T h e European E u r o p e a n Network N e t w o r k of o f Equality E q u a l i t y Bodies B o d i e s (Equinet) ( E q u i n e t ) will w i l l publish p u b l i s h a a report r e p o r t (by ( b y Robin R o b i n 11 1 1 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} In I n such s u c h cases, c a s e s , where w h e r e the t h e outcome o u t c o m e could c o u l d not n o t have h a v e been b e e n prevented p r e v e n t e d or o r anticipated a n t i c i p a t e d at a t the t h e design d e s i g n phase, p h a s e , the t h e risks r i s k s will w i l l not n o t stem s t e m from f r o m a a flaw f l a w in i n the t h e original o r i g i n a l design d e s i g n of o f the t h e system s y s t e m but b u t rather r a t h e r from f r o m the t h e practical p r a c t i c a l impacts i m p a c t s of o f the t h e correlations c o r r e l a t i o n s or o r patterns p a t t e r n s that t h a t the t h e system s y s t e m identifies i d e n t i f i e s in i n a a large l a r g e dataset. d a t a s e t . The T h e specific s p e c i f i c characteristics c h a r a c t e r i s t i c s of o f many m a n y AI A I technologies, t e c h n o l o g i e s , including i n c l u d i n g opacity o p a c i t y (‘black ( ‘ b l a c k box-effect’), b o x - e f f e c t ’ ) , complexity, c o m p l e x i t y , unpredictability u n p r e d i c t a b i l i t y and a n d partially p a r t i a l l y autonomous a u t o n o m o u s behaviour, b e h a v i o u r , may m a y make m a k e it i t hard h a r d to t o verify v e r i f y compliance c o m p l i a n c e with, w i t h , and a n d may m a y hamper h a m p e r the t h e effective e f f e c t i v e enforcement e n f o r c e m e n t of, o f , rules r u l e s of o f existing e x i s t i n g EU E U law l a w meant m e a n t to t o protect p r o t e c t fundamental f u n d a m e n t a l rights. r i g h t s . Enforcement E n f o r c e m e n t authorities a u t h o r i t i e s and a n d affected a f f e c t e d persons p e r s o n s might m i g h t lack l a c k the t h e means m e a n s to t o verify v e r i f y how h o w a a given g i v e n decision d e c i s i o n made m a d e with w i t h the t h e involvement i n v o l v e m e n t of o f AI A I was w a s taken t a k e n and, a n d , therefore, t h e r e f o r e , whether w h e t h e r the t h e relevant r e l e v a n t rules r u l e s were w e r e respected. r e s p e c t e d . Individuals I n d i v i d u a l s and a n d legal l e g a l entities e n t i t i e s may m a y face f a c e difficulties d i f f i c u l t i e s with w i t h effective e f f e c t i v e access a c c e s s to t o justice j u s t i c e in i n situations s i t u a t i o n s where w h e r e such s u c h decisions d e c i s i o n s may m a y negatively n e g a t i v e l y affect a f f e c t them. t h e m . Risks R i s k s for f o r safety s a f e t y and a n d the t h e effective e f f e c t i v e functioning f u n c t i o n i n g of o f the t h e liability l i a b i l i t y regime r e g i m e AI A I technologies t e c h n o l o g i e s may m a y present p r e s e n t new n e w safety s a f e t y risks r i s k s for f o r users u s e r s when w h e n they t h e y are a r e embedded e m b e d d e d in i n products p r o d u c t s and a n d services. s e r v i c e s . For F o r example, e x a m p l e , as a s result r e s u l t of o f a a flaw f l a w in i n the t h e object o b j e c t recognition r e c o g n i t i o n technology, t e c h n o l o g y , an a n autonomous a u t o n o m o u s car c a r can c a n wrongly w r o n g l y identify i d e n t i f y an a n object o b j e c t on o n the t h e road r o a d and a n d cause c a u s e an a n accident a c c i d e n t involving i n v o l v i n g injuries i n j u r i e s and a n d material m a t e r i a l damage. d a m a g e . As A s with w i t h the t h e risks r i s k s to t o fundamental f u n d a m e n t a l rights, r i g h t s , these t h e s e risks r i s k s can c a n be b e caused c a u s e d by b y flaws f l a w s in i n the t h e design d e s i g n of o f the t h e AI A I technology, t e c h n o l o g y , be b e related r e l a t e d to t o problems p r o b l e m s with w i t h the t h e availability a v a i l a b i l i t y and a n d quality q u a l i t y of o f data d a t a or o r to t o other o t h e r problems p r o b l e m s stemming s t e m m i n g from f r o m machine m a c h i n e learning. l e a r n i n g . While W h i l e some s o m e of o f these t h e s e risks r i s k s are a r e not n o t limited l i m i t e d to t o products p r o d u c t s and a n d services s e r v i c e s that t h a t rely r e l y on o n AI A I , , the t h e use u s e of o f AI A I may m a y increase i n c r e a s e or o r aggravate a g g r a v a t e the t h e risks. r i s k s . A A lack l a c k of o f clear c l e a r safety s a f e t y provisions p r o v i s i o n s tackling t a c k l i n g these t h e s e risks r i s k s may, m a y , in i n addition a d d i t i o n to t o risks r i s k s for f o r the t h e individuals i n d i v i d u a l s concerned, c o n c e r n e d , create c r e a t e legal l e g a l uncertainty u n c e r t a i n t y for f o r businesses b u s i n e s s e s that t h a t are a r e marketing m a r k e t i n g their t h e i r products p r o d u c t s involving i n v o l v i n g AI A I in i n the t h e EU. E U . Market M a r k e t surveillance s u r v e i l l a n c e and a n d enforcement e n f o r c e m e n t authorities a u t h o r i t i e s may m a y find f i n d themselves t h e m s e l v e s in i n a a situation s i t u a t i o n where w h e r e they t h e y are a r e unclear u n c l e a r as a s to t o whether w h e t h e r they t h e y can c a n intervene, i n t e r v e n e , because b e c a u s e they t h e y may m a y not n o t be b e empowered e m p o w e r e d to t o act a c t and/or a n d / o r don’t d o n ’ t have h a v e the t h e appropriate a p p r o p r i a t e technical t e c h n i c a l capabilities c a p a b i l i t i e s for f o r inspecting i n s p e c t i n g systems s y s t e m s 36 3 6 . . Legal L e g a l uncertainty u n c e r t a i n t y may m a y therefore t h e r e f o r e reduce r e d u c e overall o v e r a l l levels l e v e l s of o f safety s a f e t y and a n d undermine u n d e r m i n e the t h e competitiveness c o m p e t i t i v e n e s s of o f European E u r o p e a n companies. c o m p a n i e s . If I f the t h e safety s a f e t y risks r i s k s materialise, m a t e r i a l i s e , the t h e lack l a c k of o f clear c l e a r requirements r e q u i r e m e n t s and a n d the t h e characteristics c h a r a c t e r i s t i c s of o f AI A I technologies t e c h n o l o g i e s mentioned m e n t i o n e d above a b o v e make m a k e it i t difficult d i f f i c u l t to t o trace t r a c e back b a c k potentially p o t e n t i a l l y problematic p r o b l e m a t i c decisions d e c i s i o n s made m a d e with w i t h the t h e involvement i n v o l v e m e n t of o f AI A I systems. s y s t e m s . This T h i s in i n turn t u r n may m a y make m a k e it i t difficult d i f f i c u l t for f o r persons p e r s o n s having h a v i n g suffered s u f f e r e d harm h a r m to t o obtain o b t a i n compensation c o m p e n s a t i o n under u n d e r the t h e current c u r r e n t EU E U and a n d national n a t i o n a l liability l i a b i l i t y legislation. l e g i s l a t i o n . 37 3 7 Allen A l l e n and a n d Dee D e e Masters) M a s t e r s ) on o n “Regulating “ R e g u l a t i n g AI: A I : the t h e new n e w role r o l e for f o r Equality E q u a l i t y Bodies B o d i e s – – Meeting M e e t i n g the t h e new n e w challenges c h a l l e n g e s to t o equality e q u a l i t y and a n d non-discrimination n o n - d i s c r i m i n a t i o n from f r o m increased i n c r e a s e d digitalisation d i g i t a l i s a t i o n and a n d the t h e use u s e of o f AI”, A I ” , expected e x p e c t e d early e a r l y 2020. 2 0 2 0 . 36 3 6 An A n example e x a m p l e may m a y be b e the t h e smart s m a r t watch w a t c h for f o r children. c h i l d r e n . This T h i s product p r o d u c t may m a y cause c a u s e no n o direct d i r e c t harm h a r m to t o the t h e child c h i l d wearing w e a r i n g it, i t , but b u t lacking l a c k i n g a a minimum m i n i m u m level l e v e l of o f security, s e c u r i t y , it i t can c a n be b e easily e a s i l y used u s e d as a s a a tool t o o l to t o have h a v e access a c c e s s to t o the t h e child. c h i l d . Market M a r k e t surveillance s u r v e i l l a n c e authorities a u t h o r i t i e s may m a y find f i n d it i t difficult d i f f i c u l t to t o intervene i n t e r v e n e in i n cases c a s e s where w h e r e the t h e risk r i s k is i s not n o t linked l i n k e d to t o the t h e product p r o d u c t as a s such. s u c h . 37 3 7 The T h e implications i m p l i c a t i o n s of o f AI, A I , Internet I n t e r n e t of o f Things T h i n g s and a n d other o t h e r digital d i g i t a l technologies t e c h n o l o g i e s for f o r safety s a f e t y and a n d liability l i a b i l i t y legislation l e g i s l a t i o n are a r e analysed a n a l y s e d in i n the t h e Commission C o m m i s s i o n Report R e p o r t accompanying a c c o m p a n y i n g this t h i s White W h i t e Paper. P a p e r . 12 1 2 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} Under U n d e r the t h e Product P r o d u c t Liability L i a b i l i t y Directive, D i r e c t i v e , a a manufacturer m a n u f a c t u r e r is i s liable l i a b l e for f o r damage d a m a g e caused c a u s e d by b y a a defective d e f e c t i v e product. p r o d u c t . However, H o w e v e r , in i n the t h e case c a s e of o f an a n AI A I based b a s e d system s y s t e m such s u c h as a s autonomous a u t o n o m o u s cars, c a r s , it i t may m a y be b e difficult d i f f i c u l t to t o prove p r o v e that t h a t there t h e r e is i s a a defect d e f e c t in i n the t h e product, p r o d u c t , the t h e damage d a m a g e that t h a t has h a s occurred o c c u r r e d and a n d the t h e causal c a u s a l link l i n k between b e t w e e n the t h e two. t w o . In I n addition, a d d i t i o n , there t h e r e is i s some s o m e uncertainty u n c e r t a i n t y about a b o u t how h o w and a n d to t o what w h a t extent e x t e n t the t h e Product P r o d u c t Liability L i a b i l i t y Directive D i r e c t i v e applies a p p l i e s in i n the t h e case c a s e of o f certain c e r t a i n types t y p e s of o f defects, d e f e c t s , for f o r example e x a m p l e if i f these t h e s e result r e s u l t from f r o m weaknesses w e a k n e s s e s in i n the t h e cybersecurity c y b e r s e c u r i t y of o f the t h e product. p r o d u c t . Thus, T h u s , the t h e difficulty d i f f i c u l t y of o f tracing t r a c i n g back b a c k potentially p o t e n t i a l l y problematic p r o b l e m a t i c decisions d e c i s i o n s taken t a k e n by b y AI A I systems s y s t e m s and a n d referred r e f e r r e d to t o above a b o v e in i n relation r e l a t i o n to t o fundamental f u n d a m e n t a l rights r i g h t s applies a p p l i e s equally e q u a l l y to t o safety s a f e t y and a n d liability-related l i a b i l i t y - r e l a t e d issues. i s s u e s . Persons P e r s o n s having h a v i n g suffered s u f f e r e d harm h a r m may m a y not n o t have h a v e effective e f f e c t i v e access a c c e s s to t o the t h e evidence e v i d e n c e that t h a t is i s necessary n e c e s s a r y to t o build b u i l d a a case c a s e in i n court, c o u r t , for f o r instance, i n s t a n c e , and a n d may m a y have h a v e less l e s s effective e f f e c t i v e redress r e d r e s s possibilities p o s s i b i l i t i e s compared c o m p a r e d to t o situations s i t u a t i o n s where w h e r e the t h e damage d a m a g e is i s caused c a u s e d by b y traditional t r a d i t i o n a l technologies. t e c h n o l o g i e s . These T h e s e risks r i s k s will w i l l increase i n c r e a s e as a s the t h e use u s e of o f AI A I becomes b e c o m e s more m o r e widespread. w i d e s p r e a d . B. B . POSSIBLE P O S S I B L E ADJUSTMENTS A D J U S T M E N T S TO T O EXISTING E X I S T I N G EU E U LEGISLATIVE L E G I S L A T I V E FRAMEWORK F R A M E W O R K RELATING R E L A T I N G TO T O AI A I An A n extensive e x t e n s i v e body b o d y of o f existing e x i s t i n g EU E U product p r o d u c t safety s a f e t y and a n d liability l i a b i l i t y legislation l e g i s l a t i o n 38 3 8 , , including i n c l u d i n g sector-specific s e c t o r - s p e c i f i c rules, r u l e s , further f u r t h e r complemented c o m p l e m e n t e d by b y national n a t i o n a l legislation, l e g i s l a t i o n , is i s relevant r e l e v a n t and a n d potentially p o t e n t i a l l y applicable a p p l i c a b l e to t o a a number n u m b e r of o f emerging e m e r g i n g AI A I applications. a p p l i c a t i o n s . As A s regards r e g a r d s the t h e protection p r o t e c t i o n of o f fundamental f u n d a m e n t a l rights r i g h t s and a n d consumer c o n s u m e r rights, r i g h t s , the t h e EU E U legislative l e g i s l a t i v e framework f r a m e w o r k includes i n c l u d e s legislation l e g i s l a t i o n such s u c h as a s the t h e Race R a c e Equality E q u a l i t y Directive D i r e c t i v e 39 3 9 , , the t h e Directive D i r e c t i v e on o n equal e q u a l treatment t r e a t m e n t in i n employment e m p l o y m e n t and a n d occupation o c c u p a t i o n 40 4 0 , , the t h e Directives D i r e c t i v e s on o n equal e q u a l treatment t r e a t m e n t between b e t w e e n men m e n and a n d women w o m e n in i n relation r e l a t i o n to t o employment e m p l o y m e n t and a n d access a c c e s s to t o goods g o o d s and a n d services s e r v i c e s 41 4 1 , , a a number n u m b e r of o f consumer c o n s u m e r protection p r o t e c t i o n rules r u l e s 42 4 2 , , as a s well w e l l as a s rules r u l e s on o n personal p e r s o n a l data d a t a protection p r o t e c t i o n and a n d privacy, p r i v a c y , notably n o t a b l y the t h e General G e n e r a l Data D a t a Protection P r o t e c t i o n Regulation R e g u l a t i o n and a n d other o t h e r sectorial s e c t o r i a l legislation l e g i s l a t i o n covering c o v e r i n g personal p e r s o n a l data d a t a protection, p r o t e c t i o n , such s u c h as a s the t h e Data D a t a Protection P r o t e c t i o n Law L a w Enforcement E n f o r c e m e n t Directive D i r e c t i v e 43 4 3 . . In I n addition, a d d i t i o n , as a s from f r o m 2025, 2 0 2 5 , the t h e rules r u l e s on o n accessibility a c c e s s i b i l i t y requirements r e q u i r e m e n t s for f o r goods g o o d s and a n d services, s e r v i c e s , set s e t out o u t in i n the t h e European E u r o p e a n Accessibility A c c e s s i b i l i t y Act A c t will w i l l apply a p p l y 44 4 4 . . In I n addition, a d d i t i o n , fundamental f u n d a m e n t a l rights r i g h t s need n e e d to t o be b e respected r e s p e c t e d when w h e n implementing i m p l e m e n t i n g other o t h e r EU E U legislation, l e g i s l a t i o n , including i n c l u d i n g in i n the t h e field f i e l d of o f financial f i n a n c i a l services, s e r v i c e s , migration m i g r a t i o n or o r responsibility r e s p o n s i b i l i t y of o f online o n l i n e intermediaries. i n t e r m e d i a r i e s . While W h i l e the t h e EU E U legislation l e g i s l a t i o n remains r e m a i n s in i n principle p r i n c i p l e fully f u l l y applicable a p p l i c a b l e irrespective i r r e s p e c t i v e of o f the t h e involvement i n v o l v e m e n t of o f AI, A I , it i t is i s important i m p o r t a n t to t o assess a s s e s s whether w h e t h e r it i t can c a n be b e enforced e n f o r c e d adequately a d e q u a t e l y to t o address a d d r e s s the t h e risks r i s k s that t h a t AI A I systems s y s t e m s create, c r e a t e , or o r whether w h e t h e r adjustments a d j u s t m e n t s are a r e needed n e e d e d to t o specific s p e c i f i c legal l e g a l instruments. i n s t r u m e n t s . 38 3 8 The T h e EU E U legal l e g a l framework f r a m e w o r k for f o r product p r o d u c t safety s a f e t y consists c o n s i s t s of o f the t h e General G e n e r a l Product P r o d u c t Safety S a f e t y Directive D i r e c t i v e (Directive ( D i r e c t i v e 2001/95/EC), 2 0 0 1 / 9 5 / E C ) , as a s a a safety s a f e t y net, n e t , and a n d a a number n u m b e r of o f sector-specific s e c t o r - s p e c i f i c rules r u l e s covering c o v e r i n g different d i f f e r e n t categories c a t e g o r i e s of o f products p r o d u c t s ranging r a n g i n g from f r o m machines, m a c h i n e s , planes p l a n e s and a n d cars c a r s to t o toys t o y s and a n d medical m e d i c a l devices d e v i c e s aiming a i m i n g to t o provide p r o v i d e a a high h i g h level l e v e l of o f health h e a l t h and a n d safety. s a f e t y . Product P r o d u c t liability l i a b i l i t y law l a w is i s complemented c o m p l e m e n t e d by b y different d i f f e r e n t systems s y s t e m s of o f civil c i v i l liability l i a b i l i t y for f o r damages d a m a g e s caused c a u s e d by b y products p r o d u c t s or o r services. s e r v i c e s . 39 3 9 Directive D i r e c t i v e 2000/43/EC. 2 0 0 0 / 4 3 / E C . 40 4 0 Directive D i r e c t i v e 2000/78/EC. 2 0 0 0 / 7 8 / E C . 41 4 1 Directive D i r e c t i v e 2004/113/EC; 2 0 0 4 / 1 1 3 / E C ; Directive D i r e c t i v e 2006/54/EC. 2 0 0 6 / 5 4 / E C . 42 4 2 Such S u c h as a s the t h e Unfair U n f a i r Commercial C o m m e r c i a l Practices P r a c t i c e s Directive D i r e c t i v e (Directive ( D i r e c t i v e 2005/29/EC) 2 0 0 5 / 2 9 / E C ) and a n d the t h e Consumer C o n s u m e r Rights R i g h t s Directive D i r e c t i v e (Directive ( D i r e c t i v e 2011/83/EC). 2 0 1 1 / 8 3 / E C ) . 43 4 3 Directive D i r e c t i v e (EU) ( E U ) 2016/680 2 0 1 6 / 6 8 0 of o f the t h e European E u r o p e a n Parliament P a r l i a m e n t and a n d of o f the t h e Council C o u n c i l of o f 27 2 7 April A p r i l 2016 2 0 1 6 on o n the t h e protection p r o t e c t i o n of o f natural n a t u r a l persons p e r s o n s with w i t h regard r e g a r d to t o the t h e processing p r o c e s s i n g of o f personal p e r s o n a l data d a t a by b y competent c o m p e t e n t authorities a u t h o r i t i e s for f o r the t h e purposes p u r p o s e s of o f the t h e prevention, p r e v e n t i o n , investigation, i n v e s t i g a t i o n , detection d e t e c t i o n or o r prosecution p r o s e c u t i o n of o f criminal c r i m i n a l offences o f f e n c e s or o r the t h e execution e x e c u t i o n of o f criminal c r i m i n a l penalties, p e n a l t i e s , and a n d on o n the t h e free f r e e movement m o v e m e n t of o f such s u c h data. d a t a . 44 4 4 Directive D i r e c t i v e (EU) ( E U ) 2019/882 2 0 1 9 / 8 8 2 on o n the t h e accessibility a c c e s s i b i l i t y requirements r e q u i r e m e n t s for f o r products p r o d u c t s and a n d services. s e r v i c e s . 13 1 3 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} For F o r example, e x a m p l e , economic e c o n o m i c actors a c t o r s remain r e m a i n fully f u l l y responsible r e s p o n s i b l e for f o r the t h e compliance c o m p l i a n c e of o f AI A I to t o existing e x i s t i n g rules r u l e s that t h a t protects p r o t e c t s consumers, c o n s u m e r s , any a n y algorithmic a l g o r i t h m i c exploitation e x p l o i t a t i o n of o f consumer c o n s u m e r behaviour b e h a v i o u r in i n violation v i o l a t i o n of o f existing e x i s t i n g rules r u l e s shall s h a l l be b e not n o t permitted p e r m i t t e d and a n d violations v i o l a t i o n s shall s h a l l be b e accordingly a c c o r d i n g l y punished. p u n i s h e d . The T h e Commission C o m m i s s i o n is i s of o f the t h e opinion o p i n i o n that t h a t the t h e legislative l e g i s l a t i v e framework f r a m e w o r k could c o u l d be b e improved i m p r o v e d to t o address a d d r e s s the t h e following f o l l o w i n g risks r i s k s and a n d situations: s i t u a t i o n s : �  Effective E f f e c t i v e application a p p l i c a t i o n and a n d enforcement e n f o r c e m e n t of o f existing e x i s t i n g EU E U and a n d national n a t i o n a l legislation: l e g i s l a t i o n : the t h e key k e y characteristics c h a r a c t e r i s t i c s of o f AI A I create c r e a t e challenges c h a l l e n g e s for f o r ensuring e n s u r i n g the t h e proper p r o p e r application a p p l i c a t i o n and a n d enforcement e n f o r c e m e n t of o f EU E U and a n d national n a t i o n a l legislation. l e g i s l a t i o n . The T h e lack l a c k of o f transparency t r a n s p a r e n c y (opaqueness ( o p a q u e n e s s of o f AI) A I ) makes m a k e s it i t difficult d i f f i c u l t to t o identify i d e n t i f y and a n d prove p r o v e possible p o s s i b l e breaches b r e a c h e s of o f laws, l a w s , including i n c l u d i n g legal l e g a l provisions p r o v i s i o n s that t h a t protect p r o t e c t fundamental f u n d a m e n t a l rights, r i g h t s , attribute a t t r i b u t e liability l i a b i l i t y and a n d meet m e e t the t h e conditions c o n d i t i o n s to t o claim c l a i m compensation. c o m p e n s a t i o n . Therefore, T h e r e f o r e , in i n order o r d e r to t o ensure e n s u r e an a n effective e f f e c t i v e application a p p l i c a t i o n and a n d enforcement, e n f o r c e m e n t , it i t may m a y be b e necessary n e c e s s a r y to t o adjust a d j u s t or o r clarify c l a r i f y existing e x i s t i n g legislation l e g i s l a t i o n in i n certain c e r t a i n areas, a r e a s , for f o r example e x a m p l e on o n liability l i a b i l i t y as a s further f u r t h e r detailed d e t a i l e d in i n the t h e Report, R e p o r t , which w h i c h accompanies a c c o m p a n i e s this t h i s White W h i t e Paper. P a p e r . �  Limitations L i m i t a t i o n s of o f scope s c o p e of o f existing e x i s t i n g EU E U legislation: l e g i s l a t i o n : an a n essential e s s e n t i a l focus f o c u s of o f EU E U product p r o d u c t safety s a f e t y legislation l e g i s l a t i o n is i s on o n the t h e placing p l a c i n g of o f products p r o d u c t s on o n the t h e market. m a r k e t . While W h i l e in i n EU E U product p r o d u c t safety s a f e t y legislation l e g i s l a t i o n software, s o f t w a r e , when w h e n is i s part p a r t of o f the t h e final f i n a l product, p r o d u c t , must m u s t comply c o m p l y with w i t h the t h e relevant r e l e v a n t product p r o d u c t safety s a f e t y rules, r u l e s , it i t is i s an a n open o p e n question q u e s t i o n whether w h e t h e r stand-alone s t a n d - a l o n e software s o f t w a r e is i s covered c o v e r e d by b y EU E U product p r o d u c t safety s a f e t y legislation, l e g i s l a t i o n , outside o u t s i d e some s o m e sectors s e c t o r s with w i t h explicit e x p l i c i t rules r u l e s 45 4 5 . . General G e n e r a l EU E U safety s a f e t y legislation l e g i s l a t i o n currently c u r r e n t l y in i n force f o r c e applies a p p l i e s to t o products p r o d u c t s and a n d not n o t to t o services, s e r v i c e s , and a n d therefore t h e r e f o r e in i n principle p r i n c i p l e not n o t to t o services s e r v i c e s based b a s e d on o n AI A I technology t e c h n o l o g y either e i t h e r (e.g. ( e . g . health h e a l t h services, s e r v i c e s , financial f i n a n c i a l services, s e r v i c e s , transport t r a n s p o r t services). s e r v i c e s ) . �  Changing C h a n g i n g functionality f u n c t i o n a l i t y of o f AI A I systems: s y s t e m s : the t h e integration i n t e g r a t i o n of o f software, s o f t w a r e , including i n c l u d i n g AI, A I , into i n t o products p r o d u c t s can c a n modify m o d i f y the t h e functioning f u n c t i o n i n g of o f such s u c h products p r o d u c t s and a n d systems s y s t e m s during d u r i n g their t h e i r lifecycle. l i f e c y c l e . This T h i s is i s particularly p a r t i c u l a r l y true t r u e for f o r systems s y s t e m s that t h a t require r e q u i r e frequent f r e q u e n t software s o f t w a r e updates u p d a t e s or o r which w h i c h rely r e l y on o n machine m a c h i n e learning. l e a r n i n g . These T h e s e features f e a t u r e s can c a n give g i v e rise r i s e to t o new n e w risks r i s k s that t h a t were w e r e not n o t present p r e s e n t when w h e n the t h e system s y s t e m was w a s placed p l a c e d on o n the t h e market. m a r k e t . These T h e s e risks r i s k s are a r e not n o t adequately a d e q u a t e l y addressed a d d r e s s e d in i n the t h e existing e x i s t i n g legislation l e g i s l a t i o n which w h i c h predominantly p r e d o m i n a n t l y focuses f o c u s e s on o n safety s a f e t y risks r i s k s present p r e s e n t at a t the t h e time t i m e of o f placing p l a c i n g on o n the t h e market. m a r k e t . �  Uncertainty U n c e r t a i n t y as a s regards r e g a r d s the t h e allocation a l l o c a t i o n of o f responsibilities r e s p o n s i b i l i t i e s between b e t w e e n different d i f f e r e n t economic e c o n o m i c operators o p e r a t o r s in i n the t h e supply s u p p l y chain: c h a i n : in i n general, g e n e r a l , EU E U legislation l e g i s l a t i o n on o n product p r o d u c t safety s a f e t y allocates a l l o c a t e s the t h e responsibility r e s p o n s i b i l i t y to t o the t h e producer p r o d u c e r of o f the t h e product p r o d u c t placed p l a c e d on o n the t h e market, m a r k e t , including i n c l u d i n g all a l l components c o m p o n e n t s e.g. e . g . AI A I systems. s y s t e m s . But B u t the t h e rules r u l e s can c a n for f o r example e x a m p l e become b e c o m e unclear u n c l e a r if i f AI A I is i s added a d d e d after a f t e r the t h e product p r o d u c t is i s placed p l a c e d on o n the t h e market m a r k e t by b y a a party p a r t y that t h a t is i s not n o t the t h e producer. p r o d u c e r . In I n addition, a d d i t i o n , EU E U product p r o d u c t liability l i a b i l i t y legislation l e g i s l a t i o n provides p r o v i d e s for f o r liability l i a b i l i t y of o f producers p r o d u c e r s and a n d leaves l e a v e s national n a t i o n a l liability l i a b i l i t y rules r u l e s to t o govern g o v e r n liability l i a b i l i t y of o f others o t h e r s in i n the t h e supply s u p p l y chain. c h a i n . �  Changes C h a n g e s to t o the t h e concept c o n c e p t of o f safety: s a f e t y : the t h e use u s e of o f AI A I in i n products p r o d u c t s and a n d services s e r v i c e s can c a n give g i v e rise r i s e to t o risks r i s k s that t h a t EU E U legislation l e g i s l a t i o n currently c u r r e n t l y does d o e s not n o t explicitly e x p l i c i t l y address. a d d r e s s . These T h e s e risks r i s k s may m a y be b e linked l i n k e d to t o cyber c y b e r threats, t h r e a t s , personal p e r s o n a l security s e c u r i t y risks r i s k s (linked ( l i n k e d for f o r example e x a m p l e to t o new n e w applications a p p l i c a t i o n s of o f AI A I such s u c h as a s to t o home h o m e appliances), a p p l i a n c e s ) , risks r i s k s that t h a t result r e s u l t from f r o m loss l o s s of o f connectivity, c o n n e c t i v i t y , etc. e t c . These T h e s e risks r i s k s may m a y be b e present p r e s e n t at a t the t h e time t i m e of o f placing p l a c i n g products p r o d u c t s on o n the t h e market m a r k e t or o r arise a r i s e as a s a a result r e s u l t of o f software s o f t w a r e updates u p d a t e s or o r self-learning s e l f - l e a r n i n g when w h e n the t h e product p r o d u c t is i s being b e i n g used. u s e d . The T h e EU E U should s h o u l d make m a k e full f u l l use u s e of o f the t h e tools t o o l s at a t its i t s disposal d i s p o s a l to t o 45 4 5 For F o r instance i n s t a n c e software s o f t w a r e intended i n t e n d e d by b y the t h e manufacturer m a n u f a c t u r e r to t o be b e used u s e d for f o r medical m e d i c a l purposes p u r p o s e s is i s considered c o n s i d e r e d a a medical m e d i c a l device d e v i c e under u n d e r the t h e Medical M e d i c a l Device D e v i c e Regulation R e g u l a t i o n (Regulation ( R e g u l a t i o n (EU) ( E U ) 2017/745). 2 0 1 7 / 7 4 5 ) . 14 1 4 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} enhance e n h a n c e its i t s evidence e v i d e n c e base b a s e on o n potential p o t e n t i a l risks r i s k s linked l i n k e d to t o AI A I applications, a p p l i c a t i o n s , including i n c l u d i n g using u s i n g the t h e experience e x p e r i e n c e of o f the t h e EU E U Cybersecurity C y b e r s e c u r i t y Agency A g e n c y (ENISA) ( E N I S A ) for f o r assessing a s s e s s i n g the t h e AI A I threat t h r e a t landscape. l a n d s c a p e . As A s indicated i n d i c a t e d earlier, e a r l i e r , several s e v e r a l Member M e m b e r States S t a t e s are a r e already a l r e a d y exploring e x p l o r i n g options o p t i o n s for f o r national n a t i o n a l legislation l e g i s l a t i o n to t o address a d d r e s s the t h e challenges c h a l l e n g e s created c r e a t e d by b y AI. A I . This T h i s raises r a i s e s the t h e risk r i s k that t h a t the t h e single s i n g l e market m a r k e t may m a y be b e fragmented. f r a g m e n t e d . Divergent D i v e r g e n t national n a t i o n a l rules r u l e s are a r e likely l i k e l y to t o create c r e a t e obstacles o b s t a c l e s for f o r companies c o m p a n i e s that t h a t want w a n t to t o sell s e l l and a n d operate o p e r a t e AI A I systems s y s t e m s in i n the t h e single s i n g l e market. m a r k e t . Ensuring E n s u r i n g a a common c o m m o n approach a p p r o a c h at a t EU E U level l e v e l would w o u l d enable e n a b l e European E u r o p e a n companies c o m p a n i e s to t o benefit b e n e f i t from f r o m smooth s m o o t h access a c c e s s to t o the t h e single s i n g l e market m a r k e t and a n d support s u p p o r t their t h e i r competitiveness c o m p e t i t i v e n e s s on o n global g l o b a l markets. m a r k e t s . Report R e p o r t on o n the t h e safety s a f e t y and a n d liability l i a b i l i t y implications i m p l i c a t i o n s of o f Artificial A r t i f i c i a l Intelligence, I n t e l l i g e n c e , the t h e Internet I n t e r n e t of o f Things T h i n g s and a n d robotics r o b o t i c s The T h e Report, R e p o r t , which w h i c h accompanies a c c o m p a n i e s this t h i s White W h i t e Paper, P a p e r , analyses a n a l y s e s the t h e relevant r e l e v a n t legal l e g a l framework. f r a m e w o r k . It I t identifies i d e n t i f i e s uncertainties u n c e r t a i n t i e s as a s to t o the t h e application a p p l i c a t i o n of o f this t h i s framework f r a m e w o r k with w i t h respect r e s p e c t to t o the t h e specific s p e c i f i c risks r i s k s posed p o s e d by b y AI A I systems s y s t e m s and a n d other o t h e r digital d i g i t a l technologies. t e c h n o l o g i e s . It I t concludes c o n c l u d e s that t h a t the t h e current c u r r e n t product p r o d u c t safety s a f e t y legislation l e g i s l a t i o n already a l r e a d y supports s u p p o r t s an a n extended e x t e n d e d concept c o n c e p t of o f safety s a f e t y protecting p r o t e c t i n g against a g a i n s t all a l l kind k i n d of o f risks r i s k s arising a r i s i n g from f r o m the t h e product p r o d u c t according a c c o r d i n g to t o its i t s use. u s e . However, H o w e v e r , provisions p r o v i s i o n s explicitly e x p l i c i t l y covering c o v e r i n g new n e w risks r i s k s presented p r e s e n t e d by b y the t h e emerging e m e r g i n g digital d i g i t a l technologies t e c h n o l o g i e s could c o u l d be b e introduced i n t r o d u c e d to t o provide p r o v i d e more m o r e legal l e g a l certainty. c e r t a i n t y . �  The T h e autonomous a u t o n o m o u s behaviour b e h a v i o u r of o f certain c e r t a i n AI A I systems s y s t e m s during d u r i n g its i t s life l i f e cycle c y c l e may m a y entail e n t a i l important i m p o r t a n t product p r o d u c t changes c h a n g e s having h a v i n g an a n impact i m p a c t on o n safety, s a f e t y , which w h i c h may m a y require r e q u i r e a a new n e w risk r i s k assessment. a s s e s s m e n t . In I n addition, a d d i t i o n , human h u m a n oversight o v e r s i g h t from f r o m the t h e product p r o d u c t design d e s i g n and a n d throughout t h r o u g h o u t the t h e lifecycle l i f e c y c l e of o f the t h e AI A I products p r o d u c t s and a n d systems s y s t e m s may m a y be b e needed n e e d e d as a s a a safeguard. s a f e g u a r d . �  Explicit E x p l i c i t obligations o b l i g a t i o n s for f o r producers p r o d u c e r s could c o u l d be b e considered c o n s i d e r e d also a l s o in i n respect r e s p e c t of o f mental m e n t a l safety s a f e t y risks r i s k s of o f users u s e r s when w h e n appropriate a p p r o p r i a t e (ex. ( e x . collaboration c o l l a b o r a t i o n with w i t h humanoid h u m a n o i d robots). r o b o t s ) . �  Union U n i o n product p r o d u c t safety s a f e t y legislation l e g i s l a t i o n could c o u l d provide p r o v i d e for f o r specific s p e c i f i c requirements r e q u i r e m e n t s addressing a d d r e s s i n g the t h e risks r i s k s to t o safety s a f e t y of o f faulty f a u l t y data d a t a at a t the t h e design d e s i g n stage s t a g e as a s well w e l l as a s mechanisms m e c h a n i s m s to t o ensure e n s u r e that t h a t quality q u a l i t y of o f data d a t a is i s maintained m a i n t a i n e d throughout t h r o u g h o u t the t h e use u s e of o f the t h e AI A I products p r o d u c t s and a n d systems. s y s t e m s . �  The T h e opacity o p a c i t y of o f systems s y s t e m s based b a s e d on o n algorithms a l g o r i t h m s could c o u l d be b e addressed a d d r e s s e d through t h r o u g h transparency t r a n s p a r e n c y requirements. r e q u i r e m e n t s . �  Existing E x i s t i n g rules r u l e s may m a y need n e e d to t o be b e adapted a d a p t e d and a n d clarified c l a r i f i e d in i n the t h e case c a s e of o f a a stand-alone s t a n d - a l o n e software s o f t w a r e placed p l a c e d as a s it i t is i s on o n the t h e market m a r k e t or o r downloaded d o w n l o a d e d into i n t o a a product p r o d u c t after a f t e r its i t s placing p l a c i n g on o n the t h e market, m a r k e t , when w h e n having h a v i n g an a n impact i m p a c t on o n safety. s a f e t y . �  Given G i v e n the t h e increasing i n c r e a s i n g complexity c o m p l e x i t y of o f supply s u p p l y chains c h a i n s as a s regards r e g a r d s new n e w technologies, t e c h n o l o g i e s , provisions p r o v i s i o n s specifically s p e c i f i c a l l y requesting r e q u e s t i n g cooperation c o o p e r a t i o n between b e t w e e n the t h e economic e c o n o m i c operators o p e r a t o r s in i n the t h e supply s u p p l y chain c h a i n and a n d the t h e users u s e r s could c o u l d provide p r o v i d e legal l e g a l certainty. c e r t a i n t y . The T h e characteristics c h a r a c t e r i s t i c s of o f emerging e m e r g i n g digital d i g i t a l technologies t e c h n o l o g i e s like l i k e AI, A I , the t h e IoT I o T and a n d robotics r o b o t i c s may m a y challenge c h a l l e n g e aspects a s p e c t s of o f the t h e liability l i a b i l i t y frameworks f r a m e w o r k s and a n d could c o u l d reduce r e d u c e their t h e i r effectiveness. e f f e c t i v e n e s s . Some S o m e of o f these t h e s e characteristics c h a r a c t e r i s t i c s could c o u l d make m a k e it i t hard h a r d to t o trace t r a c e the t h e damage d a m a g e back b a c k to t o a a person, p e r s o n , which w h i c h would w o u l d be b e necessary n e c e s s a r y for f o r a a fault-based f a u l t - b a s e d claim c l a i m in i n accordance a c c o r d a n c e with w i t h most m o s t national n a t i o n a l rules. r u l e s . This T h i s could c o u l d significantly s i g n i f i c a n t l y increase i n c r e a s e the t h e costs c o s t s for f o r victims v i c t i m s and a n d means m e a n s that t h a t liability l i a b i l i t y claims c l a i m s against a g a i n s t others o t h e r s than t h a n producers p r o d u c e r s may m a y be b e difficult d i f f i c u l t to t o make m a k e or o r prove. p r o v e . �  Persons P e r s o n s having h a v i n g suffered s u f f e r e d harm h a r m caused c a u s e d with w i t h the t h e involvement i n v o l v e m e n t of o f AI A I systems s y s t e m s need n e e d to t o enjoy e n j o y the t h e same s a m e level l e v e l of o f protection p r o t e c t i o n as a s persons p e r s o n s having h a v i n g suffered s u f f e r e d harm h a r m caused c a u s e d by b y other o t h e r technologies, t e c h n o l o g i e s , whilst w h i l s t technological t e c h n o l o g i c a l innovation i n n o v a t i o n should s h o u l d be b e allowed a l l o w e d to t o continue c o n t i n u e to t o develop. d e v e l o p . �  All A l l options o p t i o n s to t o ensure e n s u r e this t h i s objective o b j e c t i v e should s h o u l d be b e carefully c a r e f u l l y assessed, a s s e s s e d , including i n c l u d i n g possible p o s s i b l e amendments a m e n d m e n t s to t o the t h e Product P r o d u c t Liability L i a b i l i t y Directive D i r e c t i v e and a n d possible p o s s i b l e further f u r t h e r targeted t a r g e t e d harmonisation h a r m o n i s a t i o n of o f national n a t i o n a l liability l i a b i l i t y rules. r u l e s . For F o r example, e x a m p l e , the t h e Commission C o m m i s s i o n is i s seeking s e e k i n g views v i e w s whether w h e t h e r and a n d to t o what w h a t extent e x t e n t it i t may m a y be b e needed n e e d e d to t o mitigate m i t i g a t e the t h e consequences c o n s e q u e n c e s of o f complexity c o m p l e x i t y by b y adapting a d a p t i n g the t h e burden b u r d e n of o f proof p r o o f required r e q u i r e d by b y national n a t i o n a l liability l i a b i l i t y rules r u l e s for f o r damage d a m a g e caused c a u s e d by b y the t h e operation o p e r a t i o n of o f AI A I applications. a p p l i c a t i o n s . 15 1 5 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} From F r o m the t h e discussion d i s c u s s i o n above, a b o v e , the t h e Commission C o m m i s s i o n concludes c o n c l u d e s that t h a t – – in i n addition a d d i t i o n to t o the t h e possible p o s s i b l e adjustments a d j u s t m e n t s to t o existing e x i s t i n g legislation l e g i s l a t i o n – – a a new n e w legislation l e g i s l a t i o n specifically s p e c i f i c a l l y on o n AI A I may m a y be b e needed n e e d e d in i n order o r d e r to t o make m a k e the t h e EU E U legal l e g a l framework f r a m e w o r k fit f i t for f o r the t h e current c u r r e n t and a n d anticipated a n t i c i p a t e d technological t e c h n o l o g i c a l and a n d commercial c o m m e r c i a l developments. d e v e l o p m e n t s . C. C . SCOPE S C O P E OF O F A A FUTURE F U T U R E EU E U REGULATORY R E G U L A T O R Y FRAMEWORK F R A M E W O R K A A key k e y issue i s s u e for f o r the t h e future f u t u r e specific s p e c i f i c regulatory r e g u l a t o r y framework f r a m e w o r k on o n AI A I intelligence i n t e l l i g e n c e is i s to t o determine d e t e r m i n e the t h e scope s c o p e of o f its i t s application. a p p l i c a t i o n . The T h e working w o r k i n g assumption a s s u m p t i o n is i s that t h a t the t h e regulatory r e g u l a t o r y framework f r a m e w o r k would w o u l d apply a p p l y to t o products p r o d u c t s and a n d services s e r v i c e s relying r e l y i n g on o n AI. A I . AI A I should s h o u l d therefore t h e r e f o r e be b e clearly c l e a r l y defined d e f i n e d for f o r the t h e purposes p u r p o s e s of o f this t h i s White W h i t e Paper, P a p e r , as a s well w e l l as a s any a n y possible p o s s i b l e future f u t u r e policy-making p o l i c y - m a k i n g initiative. i n i t i a t i v e . In I n its i t s Communication C o m m u n i c a t i o n on o n AI A I for f o r Europe E u r o p e the t h e Commission C o m m i s s i o n provided p r o v i d e d a a first f i r s t definition d e f i n i t i o n of o f AI A I 46 4 6 . . This T h i s definition d e f i n i t i o n was w a s further f u r t h e r refined r e f i n e d by b y the t h e High H i g h Level L e v e l Expert E x p e r t Group G r o u p 47 4 7 . . In I n any a n y new n e w legal l e g a l instrument, i n s t r u m e n t , the t h e definition d e f i n i t i o n of o f AI A I will w i l l need n e e d to t o be b e sufficiently s u f f i c i e n t l y flexible f l e x i b l e to t o accommodate a c c o m m o d a t e technical t e c h n i c a l progress p r o g r e s s while w h i l e being b e i n g precise p r e c i s e enough e n o u g h to t o provide p r o v i d e the t h e necessary n e c e s s a r y legal l e g a l certainty. c e r t a i n t y . For F o r the t h e purposes p u r p o s e s of o f this t h i s White W h i t e Paper, P a p e r , as a s well w e l l as a s of o f any a n y possible p o s s i b l e future f u t u r e discussions d i s c u s s i o n s on o n policy p o l i c y initiatives, i n i t i a t i v e s , it i t seems s e e m s important i m p o r t a n t to t o clarify c l a r i f y the t h e main m a i n elements e l e m e n t s that t h a t compose c o m p o s e AI, A I , which w h i c h are a r e “data” “ d a t a ” and a n d “algorithms”. “ a l g o r i t h m s ” . AI A I can c a n be b e integrated i n t e g r a t e d in i n hardware. h a r d w a r e . In I n case c a s e of o f machine m a c h i n e learning l e a r n i n g techniques, t e c h n i q u e s , which w h i c h constitute c o n s t i t u t e a a subset s u b s e t of o f AI, A I , algorithms a l g o r i t h m s are a r e trained t r a i n e d to t o infer i n f e r certain c e r t a i n patterns p a t t e r n s based b a s e d on o n a a set s e t of o f data d a t a in i n order o r d e r to t o determine d e t e r m i n e the t h e actions a c t i o n s needed n e e d e d to t o achieve a c h i e v e a a given g i v e n goal. g o a l . Algorithms A l g o r i t h m s In I n autonomous a u t o n o m o u s driving d r i v i n g for f o r example, e x a m p l e , the t h e algorithm a l g o r i t h m uses, u s e s , in i n real r e a l time, t i m e , the t h e data d a t a from f r o m the t h e car c a r (speed, ( s p e e d , engine e n g i n e consumption, c o n s u m p t i o n , shock-absorbers, s h o c k - a b s o r b e r s , etc..) e t c . . ) and a n d from f r o m the t h e sensors s e n s o r s scanning s c a n n i n g the t h e whole w h o l e environment e n v i r o n m e n t of o f the t h e car c a r (road, ( r o a d , signs, s i g n s , other o t h e r vehicles, v e h i c l e s , pedestrians p e d e s t r i a n s etc..) e t c . . ) to t o derive d e r i v e which w h i c h direction, d i r e c t i o n , acceleration a c c e l e r a t i o n and a n d speed s p e e d the t h e car c a r should s h o u l d take t a k e to t o reach r e a c h a a certain c e r t a i n destination. d e s t i n a t i o n . Based B a s e d on o n the t h e data d a t a observed, o b s e r v e d , the t h e algorithm a l g o r i t h m adapts a d a p t s to t o the t h e situation s i t u a t i o n of o f the t h e road r o a d and a n d to t o the t h e outside o u t s i d e conditions, c o n d i t i o n s , including i n c l u d i n g other o t h e r drivers’ d r i v e r s ’ behaviour, b e h a v i o u r , to t o derive d e r i v e the t h e most m o s t comfortable c o m f o r t a b l e and a n d safest s a f e s t drive. d r i v e . may m a y continue c o n t i n u e to t o learn l e a r n when w h e n in i n use. u s e . While W h i l e AI-based A I - b a s e d products p r o d u c t s can c a n act a c t autonomously a u t o n o m o u s l y by b y perceiving p e r c e i v i n g their t h e i r environment e n v i r o n m e n t and a n d without w i t h o u t following f o l l o w i n g a a pre-determined p r e - d e t e r m i n e d set s e t of o f instructions, i n s t r u c t i o n s , their t h e i r behaviour b e h a v i o u r is i s largely l a r g e l y defined d e f i n e d and a n d constrained c o n s t r a i n e d by b y its i t s developers. d e v e l o p e r s . Humans H u m a n s determine d e t e r m i n e and a n d programme p r o g r a m m e the t h e goals, g o a l s , which w h i c h an a n AI A I system s y s t e m should s h o u l d optimise o p t i m i s e for. f o r . The T h e EU E U has h a s a a strict s t r i c t legal l e g a l framework f r a m e w o r k in i n place p l a c e to t o ensure e n s u r e inter i n t e r alia a l i a consumer c o n s u m e r protection, p r o t e c t i o n , to t o address a d d r e s s unfair u n f a i r commercial c o m m e r c i a l practices p r a c t i c e s and a n d to t o protect p r o t e c t personal p e r s o n a l data d a t a and a n d privacy. p r i v a c y . In I n addition, a d d i t i o n , the t h e acquis a c q u i s contains c o n t a i n s specific s p e c i f i c rules r u l e s for f o r certain c e r t a i n sectors s e c t o r s (e.g. ( e . g . healthcare, h e a l t h c a r e , transport). t r a n s p o r t ) . These T h e s e existing e x i s t i n g provisions p r o v i s i o n s of o f EU E U law l a w will w i l l continue c o n t i n u e to t o apply a p p l y in i n relation r e l a t i o n to t o AI, A I , although a l t h o u g h certain c e r t a i n updates u p d a t e s to t o that t h a t framework f r a m e w o r k may m a y be b e necessary n e c e s s a r y to t o reflect r e f l e c t the t h e digital d i g i t a l transformation t r a n s f o r m a t i o n and a n d the t h e use u s e of o f AI A I (see ( s e e section s e c t i o n B). B ) . As A s a a consequence, c o n s e q u e n c e , those t h o s e aspects a s p e c t s that t h a t are a r e 46 4 6 COM(2018) C O M ( 2 0 1 8 ) 237 2 3 7 final, f i n a l , p. p . 1: 1 : “Artificial “ A r t i f i c i a l intelligence i n t e l l i g e n c e (AI) ( A I ) refers r e f e r s to t o systems s y s t e m s that t h a t display d i s p l a y intelligent i n t e l l i g e n t behaviour b e h a v i o u r by b y analysing a n a l y s i n g their t h e i r environment e n v i r o n m e n t and a n d taking t a k i n g actions a c t i o n s – – with w i t h some s o m e degree d e g r e e of o f autonomy a u t o n o m y – – to t o achieve a c h i e v e specific s p e c i f i c goals. g o a l s . AI-based A I - b a s e d systems s y s t e m s can c a n be b e purely p u r e l y software-based, s o f t w a r e - b a s e d , acting a c t i n g in i n the t h e virtual v i r t u a l world w o r l d (e.g. ( e . g . voice v o i c e assistants, a s s i s t a n t s , image i m a g e analysis a n a l y s i s software, s o f t w a r e , search s e a r c h engines, e n g i n e s , speech s p e e c h and a n d face f a c e recognition r e c o g n i t i o n systems) s y s t e m s ) or o r AI A I can c a n be b e embedded e m b e d d e d in i n hardware h a r d w a r e devices d e v i c e s (e.g. ( e . g . advanced a d v a n c e d robots, r o b o t s , autonomous a u t o n o m o u s cars, c a r s , drones d r o n e s or o r Internet I n t e r n e t of o f Things T h i n g s applications).” a p p l i c a t i o n s ) . ” 47 4 7 High H i g h Level L e v e l Expert E x p e r t Group, G r o u p , A A definition d e f i n i t i o n of o f AI, A I , p. p . 8: 8 : “Artificial “ A r t i f i c i a l intelligence i n t e l l i g e n c e (AI) ( A I ) systems s y s t e m s are a r e software s o f t w a r e (and ( a n d possibly p o s s i b l y also a l s o hardware) h a r d w a r e ) systems s y s t e m s designed d e s i g n e d by b y humans h u m a n s that, t h a t , given g i v e n a a complex c o m p l e x goal, g o a l , act a c t in i n the t h e physical p h y s i c a l or o r digital d i g i t a l dimension d i m e n s i o n by b y perceiving p e r c e i v i n g their t h e i r environment e n v i r o n m e n t through t h r o u g h data d a t a acquisition, a c q u i s i t i o n , interpreting i n t e r p r e t i n g the t h e collected c o l l e c t e d structured s t r u c t u r e d or o r unstructured u n s t r u c t u r e d data, d a t a , reasoning r e a s o n i n g on o n the t h e knowledge, k n o w l e d g e , or o r processing p r o c e s s i n g the t h e information, i n f o r m a t i o n , derived d e r i v e d from f r o m this t h i s data d a t a and a n d deciding d e c i d i n g the t h e best b e s t action(s) a c t i o n ( s ) to t o take t a k e to t o achieve a c h i e v e the t h e given g i v e n goal. g o a l . AI A I systems s y s t e m s can c a n either e i t h e r use u s e symbolic s y m b o l i c rules r u l e s or o r learn l e a r n a a numeric n u m e r i c model, m o d e l , and a n d they t h e y can c a n also a l s o adapt a d a p t their t h e i r behaviour b e h a v i o u r by b y analysing a n a l y s i n g how h o w the t h e environment e n v i r o n m e n t is i s affected a f f e c t e d by b y their t h e i r previous p r e v i o u s actions.” a c t i o n s . ” 16 1 6 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} already a l r e a d y addressed a d d r e s s e d by b y existing e x i s t i n g horizontal h o r i z o n t a l or o r sectoral s e c t o r a l legislation l e g i s l a t i o n (e.g. ( e . g . on o n medical m e d i c a l devices d e v i c e s 48 4 8 , , in i n transport t r a n s p o r t systems) s y s t e m s ) will w i l l continue c o n t i n u e to t o be b e governed g o v e r n e d by b y this t h i s legislation. l e g i s l a t i o n . As A s a a matter m a t t e r of o f principle, p r i n c i p l e , the t h e new n e w regulatory r e g u l a t o r y framework f r a m e w o r k for f o r AI A I should s h o u l d be b e effective e f f e c t i v e to t o achieve a c h i e v e its i t s objectives o b j e c t i v e s while w h i l e not n o t being b e i n g excessively e x c e s s i v e l y prescriptive p r e s c r i p t i v e so s o that t h a t it i t could c o u l d create c r e a t e a a disproportionate d i s p r o p o r t i o n a t e burden, b u r d e n , especially e s p e c i a l l y for f o r SMEs. S M E s . To T o strike s t r i k e this t h i s balance, b a l a n c e , the t h e Commission C o m m i s s i o n is i s of o f the t h e view v i e w that t h a t it i t should s h o u l d follow f o l l o w a a risk- r i s k - based b a s e d approach. a p p r o a c h . A A risk-based r i s k - b a s e d approach a p p r o a c h is i s important i m p o r t a n t to t o help h e l p ensure e n s u r e that t h a t the t h e regulatory r e g u l a t o r y intervention i n t e r v e n t i o n is i s proportionate. p r o p o r t i o n a t e . However, H o w e v e r , it i t requires r e q u i r e s clear c l e a r criteria c r i t e r i a to t o differentiate d i f f e r e n t i a t e between b e t w e e n the t h e different d i f f e r e n t AI A I applications, a p p l i c a t i o n s , in i n particular p a r t i c u l a r in i n relation r e l a t i o n to t o the t h e question q u e s t i o n whether w h e t h e r or o r not n o t they t h e y are a r e ‘high-risk’ ‘ h i g h - r i s k ’ 49 4 9 . . The T h e determination d e t e r m i n a t i o n of o f what w h a t is i s a a high-risk h i g h - r i s k AI A I application a p p l i c a t i o n should s h o u l d be b e clear c l e a r and a n d easily e a s i l y understandable u n d e r s t a n d a b l e and a n d applicable a p p l i c a b l e for f o r all a l l parties p a r t i e s concerned. c o n c e r n e d . Nevertheless N e v e r t h e l e s s even e v e n if i f an a n AI A I application a p p l i c a t i o n is i s not n o t qualified q u a l i f i e d as a s high-risk, h i g h - r i s k , it i t remains r e m a i n s entirely e n t i r e l y subject s u b j e c t to t o already a l r e a d y existing e x i s t i n g EU-rules. E U - r u l e s . The T h e Commission C o m m i s s i o n is i s of o f the t h e opinion o p i n i o n that t h a t a a given g i v e n AI A I application a p p l i c a t i o n should s h o u l d generally g e n e r a l l y be b e considered c o n s i d e r e d high-risk h i g h - r i s k in i n light l i g h t of o f what w h a t is i s at a t stake, s t a k e , considering c o n s i d e r i n g whether w h e t h e r both b o t h the t h e sector s e c t o r and a n d the t h e intended i n t e n d e d use u s e involve i n v o l v e significant s i g n i f i c a n t risks, r i s k s , in i n particular p a r t i c u l a r from f r o m the t h e viewpoint v i e w p o i n t of o f protection p r o t e c t i o n of o f safety, s a f e t y , consumer c o n s u m e r rights r i g h t s and a n d fundamental f u n d a m e n t a l rights. r i g h t s . More M o r e specifically, s p e c i f i c a l l y , an a n AI A I application a p p l i c a t i o n should s h o u l d be b e considered c o n s i d e r e d high-risk h i g h - r i s k where w h e r e it i t meets m e e t s the t h e following f o l l o w i n g two t w o cumulative c u m u l a t i v e criteria: c r i t e r i a : �  First, F i r s t , the t h e AI A I application a p p l i c a t i o n is i s employed e m p l o y e d in i n a a sector s e c t o r where, w h e r e , given g i v e n the t h e characteristics c h a r a c t e r i s t i c s of o f the t h e activities a c t i v i t i e s typically t y p i c a l l y undertaken, u n d e r t a k e n , significant s i g n i f i c a n t risks r i s k s can c a n be b e expected e x p e c t e d to t o occur. o c c u r . This T h i s first f i r s t criterion c r i t e r i o n ensures e n s u r e s that t h a t the t h e regulatory r e g u l a t o r y intervention i n t e r v e n t i o n is i s targeted t a r g e t e d on o n the t h e areas a r e a s where, w h e r e , generally g e n e r a l l y speaking, s p e a k i n g , risks r i s k s are a r e deemed d e e m e d most m o s t likely l i k e l y to t o occur. o c c u r . The T h e sectors s e c t o r s covered c o v e r e d should s h o u l d be b e specifically s p e c i f i c a l l y and a n d exhaustively e x h a u s t i v e l y listed l i s t e d in i n the t h e new n e w regulatory r e g u l a t o r y framework. f r a m e w o r k . For F o r instance, i n s t a n c e , healthcare; h e a l t h c a r e ; transport; t r a n s p o r t ; energy e n e r g y and a n d parts p a r t s of o f the t h e public p u b l i c sector. s e c t o r . 50 5 0 The T h e list l i s t should s h o u l d be b e periodically p e r i o d i c a l l y reviewed r e v i e w e d and a n d amended a m e n d e d where w h e r e necessary n e c e s s a r y in i n function f u n c t i o n of o f relevant r e l e v a n t developments d e v e l o p m e n t s in i n practice; p r a c t i c e ; �  Second, S e c o n d , the t h e AI A I application a p p l i c a t i o n in i n the t h e sector s e c t o r in i n question q u e s t i o n is, i s , in i n addition, a d d i t i o n , used u s e d in i n such s u c h a a manner m a n n e r that t h a t significant s i g n i f i c a n t risks r i s k s are a r e likely l i k e l y to t o arise. a r i s e . This T h i s second s e c o n d criterion c r i t e r i o n reflects r e f l e c t s the t h e acknowledgment a c k n o w l e d g m e n t that t h a t not n o t every e v e r y use u s e of o f AI A I in i n the t h e selected s e l e c t e d sectors s e c t o r s necessarily n e c e s s a r i l y involves i n v o l v e s significant s i g n i f i c a n t risks. r i s k s . For F o r example, e x a m p l e , whilst w h i l s t healthcare h e a l t h c a r e generally g e n e r a l l y may m a y well w e l l be b e a a relevant r e l e v a n t sector, s e c t o r , a a flaw f l a w in i n the t h e appointment a p p o i n t m e n t scheduling s c h e d u l i n g system s y s t e m in i n a a hospital h o s p i t a l will w i l l normally n o r m a l l y not n o t pose p o s e risks r i s k s of o f such s u c h significance s i g n i f i c a n c e as a s to t o justify j u s t i f y legislative l e g i s l a t i v e intervention. i n t e r v e n t i o n . The T h e assessment a s s e s s m e n t of o f the t h e level l e v e l of o f risk r i s k of o f a a given g i v e n use u s e could c o u l d be b e based b a s e d on o n the t h e impact i m p a c t on o n the t h e affected a f f e c t e d parties. p a r t i e s . For F o r instance, i n s t a n c e , uses u s e s of o f AI A I applications a p p l i c a t i o n s that t h a t produce p r o d u c e legal l e g a l or o r similarly s i m i l a r l y significant s i g n i f i c a n t effects e f f e c t s for f o r the t h e rights r i g h t s of o f an a n individual i n d i v i d u a l or o r a a company; c o m p a n y ; that t h a t pose p o s e risk r i s k of o f injury, i n j u r y , death d e a t h or o r significant s i g n i f i c a n t material m a t e r i a l or o r immaterial i m m a t e r i a l damage; d a m a g e ; that t h a t produce p r o d u c e effects e f f e c t s that t h a t cannot c a n n o t reasonably r e a s o n a b l y be b e avoided a v o i d e d by b y individuals i n d i v i d u a l s or o r legal l e g a l entities. e n t i t i e s . The T h e application a p p l i c a t i o n of o f the t h e two t w o cumulative c u m u l a t i v e criteria c r i t e r i a would w o u l d ensure e n s u r e that t h a t the t h e scope s c o p e of o f the t h e regulatory r e g u l a t o r y framework f r a m e w o r k is i s targeted t a r g e t e d and a n d provides p r o v i d e s legal l e g a l certainty. c e r t a i n t y . The T h e mandatory m a n d a t o r y requirements r e q u i r e m e n t s contained c o n t a i n e d in i n the t h e new n e w regulatory r e g u l a t o r y framework f r a m e w o r k on o n AI A I (see ( s e e section s e c t i o n D D below) b e l o w ) would w o u l d in i n principle p r i n c i p l e apply a p p l y only o n l y to t o those t h o s e applications a p p l i c a t i o n s identified i d e n t i f i e d as a s high-risk h i g h - r i s k in i n accordance a c c o r d a n c e with w i t h these t h e s e two t w o cumulative c u m u l a t i v e criteria. c r i t e r i a . 48 4 8 For F o r example, e x a m p l e , there t h e r e are a r e different d i f f e r e n t safety s a f e t y considerations c o n s i d e r a t i o n s and a n d legal l e g a l implications i m p l i c a t i o n s concerning c o n c e r n i n g AI A I systems s y s t e m s that t h a t provide p r o v i d e specialized s p e c i a l i z e d medical m e d i c a l information i n f o r m a t i o n to t o physicians, p h y s i c i a n s , AI A I systems s y s t e m s providing p r o v i d i n g medical m e d i c a l information i n f o r m a t i o n directly d i r e c t l y to t o the t h e patient p a t i e n t and a n d AI A I systems s y s t e m s performing p e r f o r m i n g medical m e d i c a l tasks t a s k s themselves t h e m s e l v e s directly d i r e c t l y on o n a a patient. p a t i e n t . The T h e Commission C o m m i s s i o n is i s examining e x a m i n i n g these t h e s e safety s a f e t y and a n d liability l i a b i l i t y challenges c h a l l e n g e s that t h a t are a r e distinct d i s t i n c t to t o healthcare. h e a l t h c a r e . 49 4 9 EU E U legislation l e g i s l a t i o n may m a y categorise c a t e g o r i s e “risks” “ r i s k s ” differently d i f f e r e n t l y to t o what w h a t is i s described d e s c r i b e d here, h e r e , depending d e p e n d i n g on o n the t h e area, a r e a , such s u c h as a s for f o r example, e x a m p l e , product p r o d u c t safety s a f e t y 50 5 0 The T h e public p u b l i c sector s e c t o r could c o u l d include i n c l u d e areas a r e a s like l i k e asylum, a s y l u m , migration, m i g r a t i o n , border b o r d e r controls c o n t r o l s and a n d judiciary, j u d i c i a r y , social s o c i a l security s e c u r i t y and a n d employment e m p l o y m e n t services. s e r v i c e s . 17 1 7 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} Notwithstanding N o t w i t h s t a n d i n g the t h e foregoing, f o r e g o i n g , there t h e r e may m a y also a l s o be b e exceptional e x c e p t i o n a l instances i n s t a n c e s where, w h e r e , due d u e to t o the t h e risks r i s k s at a t stake, s t a k e , the t h e use u s e of o f AI A I applications a p p l i c a t i o n s for f o r certain c e r t a i n purposes p u r p o s e s is i s to t o be b e considered c o n s i d e r e d as a s high-risk h i g h - r i s k as a s such s u c h – – that t h a t is, i s , irrespective i r r e s p e c t i v e of o f the t h e sector s e c t o r concerned c o n c e r n e d and a n d where w h e r e the t h e below b e l o w requirements r e q u i r e m e n t s would w o u l d still s t i l l apply. a p p l y . 51 5 1 As A s an a n illustration, i l l u s t r a t i o n , one o n e could c o u l d think t h i n k in i n particular p a r t i c u l a r of o f the t h e following: f o l l o w i n g : �  In I n light l i g h t of o f its i t s significance s i g n i f i c a n c e for f o r individuals i n d i v i d u a l s and a n d of o f the t h e EU E U acquis a c q u i s addressing a d d r e s s i n g employment e m p l o y m e n t equality, e q u a l i t y , the t h e use u s e of o f AI A I applications a p p l i c a t i o n s for f o r recruitment r e c r u i t m e n t processes p r o c e s s e s as a s well w e l l as a s in i n situations s i t u a t i o n s impacting i m p a c t i n g workers’ w o r k e r s ’ rights r i g h t s would w o u l d always a l w a y s be b e considered c o n s i d e r e d “high-risk” “ h i g h - r i s k ” and a n d therefore t h e r e f o r e the t h e below b e l o w requirements r e q u i r e m e n t s would w o u l d at a t all a l l times t i m e s apply. a p p l y . Further F u r t h e r specific s p e c i f i c applications a p p l i c a t i o n s affecting a f f e c t i n g consumer c o n s u m e r rights r i g h t s could c o u l d be b e considered. c o n s i d e r e d . �  the t h e use u s e of o f AI A I applications a p p l i c a t i o n s for f o r the t h e purposes p u r p o s e s of o f remote r e m o t e biometric b i o m e t r i c identification i d e n t i f i c a t i o n 52 5 2 and a n d other o t h e r intrusive i n t r u s i v e surveillance s u r v e i l l a n c e technologies, t e c h n o l o g i e s , would w o u l d always a l w a y s be b e considered c o n s i d e r e d “high-risk” “ h i g h - r i s k ” and a n d therefore t h e r e f o r e the t h e below b e l o w requirements r e q u i r e m e n t s would w o u l d at a t all a l l times t i m e s apply. a p p l y . D. D . TYPES T Y P E S OF O F REQUIREMENTS R E Q U I R E M E N T S When W h e n designing d e s i g n i n g the t h e future f u t u r e regulatory r e g u l a t o r y framework f r a m e w o r k for f o r AI, A I , it i t will w i l l be b e necessary n e c e s s a r y to t o decide d e c i d e on o n the t h e types t y p e s of o f mandatory m a n d a t o r y legal l e g a l requirements r e q u i r e m e n t s to t o be b e imposed i m p o s e d on o n the t h e relevant r e l e v a n t actors. a c t o r s . These T h e s e requirements r e q u i r e m e n t s may m a y be b e further f u r t h e r specified s p e c i f i e d through t h r o u g h standards. s t a n d a r d s . As A s noted n o t e d in i n section s e c t i o n C C above a b o v e and a n d in i n addition a d d i t i o n to t o already a l r e a d y existing e x i s t i n g legislation, l e g i s l a t i o n , those t h o s e requirements r e q u i r e m e n t s would w o u l d apply a p p l y to t o high-risk h i g h - r i s k AI A I applications a p p l i c a t i o n s only, o n l y , thus t h u s ensuring e n s u r i n g that t h a t any a n y regulatory r e g u l a t o r y intervention i n t e r v e n t i o n is i s focused f o c u s e d and a n d proportionate. p r o p o r t i o n a t e . Taking T a k i n g into i n t o account a c c o u n t the t h e guidelines g u i d e l i n e s of o f the t h e High H i g h Level L e v e l Expert E x p e r t Group G r o u p and a n d what w h a t has h a s been b e e n set s e t out o u t in i n the t h e foregoing, f o r e g o i n g , the t h e requirements r e q u i r e m e n t s for f o r high-risk h i g h - r i s k AI A I applications a p p l i c a t i o n s could c o u l d consist c o n s i s t of o f the t h e following f o l l o w i n g key k e y features, f e a t u r e s , which w h i c h are a r e discussed d i s c u s s e d in i n further f u r t h e r detail d e t a i l in i n the t h e subsections s u b s e c t i o n s below: b e l o w : �  training t r a i n i n g data; d a t a ; �  data d a t a and a n d record-keeping; r e c o r d - k e e p i n g ; �  information i n f o r m a t i o n to t o be b e provided; p r o v i d e d ; �  robustness r o b u s t n e s s and a n d accuracy; a c c u r a c y ; �  human h u m a n oversight; o v e r s i g h t ; �  specific s p e c i f i c requirements r e q u i r e m e n t s for f o r certain c e r t a i n particular p a r t i c u l a r AI A I applications, a p p l i c a t i o n s , such s u c h as a s those t h o s e used u s e d for f o r purposes p u r p o s e s of o f remote r e m o t e biometric b i o m e t r i c identification. i d e n t i f i c a t i o n . To T o ensure e n s u r e legal l e g a l certainty, c e r t a i n t y , these t h e s e requirements r e q u i r e m e n t s will w i l l be b e further f u r t h e r specified s p e c i f i e d to t o provide p r o v i d e a a clear c l e a r benchmark b e n c h m a r k for f o r all a l l the t h e actors a c t o r s who w h o need n e e d to t o comply c o m p l y with w i t h them. t h e m . a) a ) Training T r a i n i n g data d a t a It I t is i s more m o r e important i m p o r t a n t than t h a n ever e v e r to t o promote, p r o m o t e , strengthen s t r e n g t h e n and a n d defend d e f e n d the t h e EU’s E U ’ s values v a l u e s and a n d rules, r u l e s , and a n d in i n particular p a r t i c u l a r the t h e rights r i g h t s that t h a t citizens c i t i z e n s derive d e r i v e from f r o m EU E U law. l a w . These T h e s e efforts e f f o r t s undoubtedly u n d o u b t e d l y also a l s o extend e x t e n d to t o the t h e high-risk h i g h - r i s k AI A I applications a p p l i c a t i o n s marketed m a r k e t e d and a n d used u s e d in i n the t h e EU E U under u n d e r consideration c o n s i d e r a t i o n here. h e r e . 51 5 1 It I t is i s important i m p o r t a n t to t o highlight h i g h l i g h t that t h a t other o t h e r pieces p i e c e s of o f EU E U legislation l e g i s l a t i o n may m a y also a l s o apply. a p p l y . For F o r example, e x a m p l e , when w h e n incorporated i n c o r p o r a t e d into i n t o a a consumer c o n s u m e r product, p r o d u c t , the t h e General G e n e r a l Product P r o d u c t Safety S a f e t y Directive D i r e c t i v e may m a y apply a p p l y to t o the t h e safety s a f e t y of o f AI A I applications. a p p l i c a t i o n s . 52 5 2 Remote R e m o t e biometric b i o m e t r i c identification i d e n t i f i c a t i o n should s h o u l d be b e distinguished d i s t i n g u i s h e d from f r o m biometric b i o m e t r i c authentication a u t h e n t i c a t i o n (the ( t h e latter l a t t e r is i s a a security s e c u r i t y process p r o c e s s that t h a t relies r e l i e s on o n the t h e unique u n i q u e biological b i o l o g i c a l characteristics c h a r a c t e r i s t i c s of o f an a n individual i n d i v i d u a l to t o verify v e r i f y that t h a t he/she h e / s h e is i s who w h o he/she h e / s h e says s a y s he/she h e / s h e is). i s ) . Remote R e m o t e biometric b i o m e t r i c identification i d e n t i f i c a t i o n is i s when w h e n the t h e identities i d e n t i t i e s of o f multiple m u l t i p l e persons p e r s o n s are a r e established e s t a b l i s h e d with w i t h the t h e help h e l p of o f biometric b i o m e t r i c identifiers i d e n t i f i e r s (fingerprints, ( f i n g e r p r i n t s , facial f a c i a l image, i m a g e , iris, i r i s , vascular v a s c u l a r patterns, p a t t e r n s , etc.) e t c . ) at a t a a distance, d i s t a n c e , in i n a a public p u b l i c space s p a c e and a n d in i n a a continuous c o n t i n u o u s or o r ongoing o n g o i n g manner m a n n e r by b y checking c h e c k i n g them t h e m against a g a i n s t data d a t a stored s t o r e d in i n a a database. d a t a b a s e . 18 1 8 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} As A s discussed d i s c u s s e d earlier, e a r l i e r , without w i t h o u t data, d a t a , there t h e r e is i s no n o AI. A I . The T h e functioning f u n c t i o n i n g of o f many m a n y AI A I systems, s y s t e m s , and a n d the t h e actions a c t i o n s and a n d decisions d e c i s i o n s to t o which w h i c h they t h e y may m a y lead, l e a d , very v e r y much m u c h depend d e p e n d on o n the t h e data d a t a set s e t on o n which w h i c h the t h e systems s y s t e m s have h a v e been b e e n trained. t r a i n e d . The T h e necessary n e c e s s a r y measures m e a s u r e s should s h o u l d therefore t h e r e f o r e be b e taken t a k e n to t o ensure e n s u r e that, t h a t , where w h e r e it i t comes c o m e s to t o the t h e data d a t a used u s e d to t o train t r a i n AI A I systems, s y s t e m s , the t h e EU’s E U ’ s values v a l u e s and a n d rules r u l e s are a r e respected, r e s p e c t e d , specifically s p e c i f i c a l l y in i n relation r e l a t i o n to t o safety s a f e t y and a n d existing e x i s t i n g legislative l e g i s l a t i v e rules r u l e s for f o r the t h e protection p r o t e c t i o n of o f fundamental f u n d a m e n t a l rights. r i g h t s . The T h e following f o l l o w i n g requirements r e q u i r e m e n t s relating r e l a t i n g to t o the t h e data d a t a set s e t used u s e d to t o train t r a i n AI A I systems s y s t e m s could c o u l d be b e envisaged: e n v i s a g e d : �  Requirements R e q u i r e m e n t s aimed a i m e d at a t providing p r o v i d i n g reasonable r e a s o n a b l e assurances a s s u r a n c e s that t h a t the t h e subsequent s u b s e q u e n t use u s e of o f the t h e products p r o d u c t s or o r services s e r v i c e s that t h a t the t h e AI A I system s y s t e m enables e n a b l e s is i s safe, s a f e , in i n that t h a t it i t meets m e e t s the t h e standards s t a n d a r d s set s e t in i n the t h e applicable a p p l i c a b l e EU E U safety s a f e t y rules r u l e s (existing ( e x i s t i n g as a s well w e l l as a s possible p o s s i b l e complementary c o m p l e m e n t a r y ones). o n e s ) . For F o r instance, i n s t a n c e , requirements r e q u i r e m e n t s ensuring e n s u r i n g that t h a t AI A I systems s y s t e m s are a r e trained t r a i n e d on o n data d a t a sets s e t s that t h a t are a r e sufficiently s u f f i c i e n t l y broad b r o a d and a n d cover c o v e r all a l l relevant r e l e v a n t scenarios s c e n a r i o s needed n e e d e d to t o avoid a v o i d dangerous d a n g e r o u s situations. s i t u a t i o n s . �  Requirements R e q u i r e m e n t s to t o take t a k e reasonable r e a s o n a b l e measures m e a s u r e s aimed a i m e d at a t ensuring e n s u r i n g that t h a t such s u c h subsequent s u b s e q u e n t use u s e of o f AI A I systems s y s t e m s does d o e s not n o t lead l e a d to t o outcomes o u t c o m e s entailing e n t a i l i n g prohibited p r o h i b i t e d discrimination. d i s c r i m i n a t i o n . These T h e s e requirements r e q u i r e m e n t s could c o u l d entail e n t a i l in i n particular p a r t i c u l a r obligations o b l i g a t i o n s to t o use u s e data d a t a sets s e t s that t h a t are a r e sufficiently s u f f i c i e n t l y representative, r e p r e s e n t a t i v e , especially e s p e c i a l l y to t o ensure e n s u r e that t h a t all a l l relevant r e l e v a n t dimensions d i m e n s i o n s of o f gender, g e n d e r , ethnicity e t h n i c i t y and a n d other o t h e r possible p o s s i b l e grounds g r o u n d s of o f prohibited p r o h i b i t e d discrimination d i s c r i m i n a t i o n are a r e appropriately a p p r o p r i a t e l y reflected r e f l e c t e d in i n those t h o s e data d a t a sets; s e t s ; �  Requirements R e q u i r e m e n t s aimed a i m e d at a t ensuring e n s u r i n g that t h a t privacy p r i v a c y and a n d personal p e r s o n a l data d a t a are a r e adequately a d e q u a t e l y protected p r o t e c t e d during d u r i n g the t h e use u s e of o f AI-enabled A I - e n a b l e d products p r o d u c t s and a n d services. s e r v i c e s . For F o r issues i s s u e s falling f a l l i n g within w i t h i n their t h e i r respective r e s p e c t i v e scope, s c o p e , the t h e General G e n e r a l Data D a t a Protection P r o t e c t i o n Regulation R e g u l a t i o n and a n d the t h e Law L a w Enforcement E n f o r c e m e n t Directive D i r e c t i v e regulate r e g u l a t e these t h e s e matters. m a t t e r s . b) b ) Keeping K e e p i n g of o f records r e c o r d s and a n d data d a t a Taking T a k i n g into i n t o account a c c o u n t elements e l e m e n t s such s u c h as a s the t h e complexity c o m p l e x i t y and a n d opacity o p a c i t y of o f many m a n y AI A I systems s y s t e m s and a n d the t h e related r e l a t e d difficulties d i f f i c u l t i e s that t h a t may m a y exist e x i s t to t o effectively e f f e c t i v e l y verify v e r i f y compliance c o m p l i a n c e with w i t h and a n d enforce e n f o r c e the t h e applicable a p p l i c a b l e rules, r u l e s , requirements r e q u i r e m e n t s are a r e called c a l l e d for f o r regarding r e g a r d i n g the t h e keeping k e e p i n g of o f records r e c o r d s in i n relation r e l a t i o n to t o the t h e programming p r o g r a m m i n g of o f the t h e algorithm, a l g o r i t h m , the t h e data d a t a used u s e d to t o train t r a i n high-risk h i g h - r i s k AI A I systems, s y s t e m s , and, a n d , in i n certain c e r t a i n cases, c a s e s , the t h e keeping k e e p i n g of o f the t h e data d a t a themselves. t h e m s e l v e s . These T h e s e requirements r e q u i r e m e n t s essentially e s s e n t i a l l y allow a l l o w potentially p o t e n t i a l l y problematic p r o b l e m a t i c actions a c t i o n s or o r decisions d e c i s i o n s by b y AI A I systems s y s t e m s to t o be b e traced t r a c e d back b a c k and a n d verified. v e r i f i e d . This T h i s should s h o u l d not n o t only o n l y facilitate f a c i l i t a t e supervision s u p e r v i s i o n and a n d enforcement; e n f o r c e m e n t ; it i t may m a y also a l s o increase i n c r e a s e the t h e incentives i n c e n t i v e s for f o r the t h e economic e c o n o m i c operators o p e r a t o r s concerned c o n c e r n e d to t o take t a k e account a c c o u n t at a t an a n early e a r l y stage s t a g e of o f the t h e need n e e d to t o respect r e s p e c t those t h o s e rules. r u l e s . To T o this t h i s aim, a i m , the t h e regulatory r e g u l a t o r y framework f r a m e w o r k could c o u l d prescribe p r e s c r i b e that t h a t the t h e following f o l l o w i n g should s h o u l d be b e kept: k e p t : �  accurate a c c u r a t e records r e c o r d s regarding r e g a r d i n g the t h e data d a t a set s e t used u s e d to t o train t r a i n and a n d test t e s t the t h e AI A I systems, s y s t e m s , including i n c l u d i n g a a description d e s c r i p t i o n of o f the t h e main m a i n characteristics c h a r a c t e r i s t i c s and a n d how h o w the t h e data d a t a set s e t was w a s selected; s e l e c t e d ; �  in i n certain c e r t a i n justified j u s t i f i e d cases, c a s e s , the t h e data d a t a sets s e t s themselves; t h e m s e l v e s ; �  documentation d o c u m e n t a t i o n on o n the t h e programming p r o g r a m m i n g 53 5 3 and a n d training t r a i n i n g methodologies, m e t h o d o l o g i e s , processes p r o c e s s e s and a n d techniques t e c h n i q u e s used u s e d to t o build, b u i l d , test t e s t and a n d validate v a l i d a t e the t h e AI A I systems, s y s t e m s , including i n c l u d i n g where w h e r e relevant r e l e v a n t in i n respect r e s p e c t of o f safety s a f e t y and a n d avoiding a v o i d i n g bias b i a s that t h a t could c o u l d lead l e a d to t o prohibited p r o h i b i t e d discrimination. d i s c r i m i n a t i o n . The T h e records, r e c o r d s , documentation d o c u m e n t a t i o n and, a n d , where w h e r e relevant, r e l e v a n t , data d a t a sets s e t s would w o u l d need n e e d to t o be b e retained r e t a i n e d during d u r i n g a a limited, l i m i t e d , reasonable r e a s o n a b l e time t i m e period p e r i o d to t o ensure e n s u r e effective e f f e c t i v e enforcement e n f o r c e m e n t of o f the t h e relevant r e l e v a n t legislation. l e g i s l a t i o n . Measures M e a s u r e s should s h o u l d be b e 53 5 3 For F o r instance, i n s t a n c e , documentation d o c u m e n t a t i o n on o n the t h e algorithm a l g o r i t h m including i n c l u d i n g what w h a t the t h e model m o d e l shall s h a l l optimise o p t i m i s e for, f o r , which w h i c h weights w e i g h t s are a r e designed d e s i g n e d to t o certain c e r t a i n parameters p a r a m e t e r s at a t the t h e outset o u t s e t etc. e t c . 19 1 9 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} taken t a k e n to t o ensure e n s u r e that t h a t they t h e y are a r e made m a d e available a v a i l a b l e upon u p o n request, r e q u e s t , in i n particular p a r t i c u l a r for f o r testing t e s t i n g or o r inspection i n s p e c t i o n by b y competent c o m p e t e n t authorities. a u t h o r i t i e s . Where W h e r e necessary, n e c e s s a r y , arrangements a r r a n g e m e n t s should s h o u l d be b e made m a d e to t o ensure e n s u r e that t h a t confidential c o n f i d e n t i a l information, i n f o r m a t i o n , such s u c h as a s trade t r a d e secrets, s e c r e t s , is i s protected. p r o t e c t e d . c) c ) Information I n f o r m a t i o n provision p r o v i s i o n Transparency T r a n s p a r e n c y is i s required r e q u i r e d also a l s o beyond b e y o n d the t h e record-keeping r e c o r d - k e e p i n g requirements r e q u i r e m e n t s discussed d i s c u s s e d in i n point p o i n t c) c ) above. a b o v e . In I n order o r d e r to t o achieve a c h i e v e the t h e objectives o b j e c t i v e s pursued p u r s u e d – – in i n particular p a r t i c u l a r promoting p r o m o t i n g the t h e responsible r e s p o n s i b l e use u s e of o f AI, A I , building b u i l d i n g trust t r u s t and a n d facilitating f a c i l i t a t i n g redress r e d r e s s where w h e r e needed n e e d e d – – it i t is i s important i m p o r t a n t that t h a t adequate a d e q u a t e information i n f o r m a t i o n is i s provided p r o v i d e d in i n a a proactive p r o a c t i v e manner m a n n e r about a b o u t the t h e use u s e of o f high-risk h i g h - r i s k AI A I systems. s y s t e m s . Accordingly, A c c o r d i n g l y , the t h e following f o l l o w i n g requirements r e q u i r e m e n t s could c o u l d be b e considered: c o n s i d e r e d : �  Ensuring E n s u r i n g clear c l e a r information i n f o r m a t i o n to t o be b e provided p r o v i d e d as a s to t o the t h e AI A I system’s s y s t e m ’ s capabilities c a p a b i l i t i e s and a n d limitations, l i m i t a t i o n s , in i n particular p a r t i c u l a r the t h e purpose p u r p o s e for f o r which w h i c h the t h e systems s y s t e m s are a r e intended, i n t e n d e d , the t h e conditions c o n d i t i o n s under u n d e r which w h i c h they t h e y can c a n be b e expected e x p e c t e d to t o function f u n c t i o n as a s intended i n t e n d e d and a n d the t h e expected e x p e c t e d level l e v e l of o f accuracy a c c u r a c y in i n achieving a c h i e v i n g the t h e specified s p e c i f i e d purpose. p u r p o s e . This T h i s information i n f o r m a t i o n is i s important i m p o r t a n t especially e s p e c i a l l y for f o r deployers d e p l o y e r s of o f the t h e systems, s y s t e m s , but b u t it i t may m a y also a l s o be b e relevant r e l e v a n t to t o competent c o m p e t e n t authorities a u t h o r i t i e s and a n d affected a f f e c t e d parties. p a r t i e s . �  Separately, S e p a r a t e l y , citizens c i t i z e n s should s h o u l d be b e clearly c l e a r l y informed i n f o r m e d when w h e n they t h e y are a r e interacting i n t e r a c t i n g with w i t h an a n AI A I system s y s t e m and a n d not n o t a a human h u m a n being. b e i n g . Whilst W h i l s t EU E U data d a t a protection p r o t e c t i o n legislation l e g i s l a t i o n already a l r e a d y contain c o n t a i n certain c e r t a i n rules r u l e s of o f this t h i s kind k i n d 54 5 4 , , additional a d d i t i o n a l requirements r e q u i r e m e n t s may m a y be b e called c a l l e d for f o r to t o achieve a c h i e v e the t h e abovementioned a b o v e m e n t i o n e d objectives. o b j e c t i v e s . If I f so, s o , unnecessary u n n e c e s s a r y burdens b u r d e n s should s h o u l d be b e avoided. a v o i d e d . Therefore, T h e r e f o r e , no n o such s u c h information i n f o r m a t i o n needs n e e d s to t o be b e provided, p r o v i d e d , for f o r instance, i n s t a n c e , in i n situations s i t u a t i o n s where w h e r e it i t is i s immediately i m m e d i a t e l y obvious o b v i o u s to t o citizens c i t i z e n s that t h a t they t h e y are a r e interacting i n t e r a c t i n g with w i t h AI A I systems. s y s t e m s . It I t is i s furthermore f u r t h e r m o r e important i m p o r t a n t that t h a t the t h e information i n f o r m a t i o n provided p r o v i d e d is i s objective, o b j e c t i v e , concise c o n c i s e and a n d easily e a s i l y understandable. u n d e r s t a n d a b l e . The T h e manner m a n n e r in i n which w h i c h the t h e information i n f o r m a t i o n is i s to t o be b e provided p r o v i d e d should s h o u l d be b e tailored t a i l o r e d to t o the t h e particular p a r t i c u l a r context. c o n t e x t . d) d ) Robustness R o b u s t n e s s and a n d accuracy a c c u r a c y AI A I systems s y s t e m s – – and a n d certainly c e r t a i n l y high-risk h i g h - r i s k AI A I applications a p p l i c a t i o n s – – must m u s t be b e technically t e c h n i c a l l y robust r o b u s t and a n d accurate a c c u r a t e in i n order o r d e r to t o be b e trustworthy. t r u s t w o r t h y . That T h a t means m e a n s that t h a t such s u c h systems s y s t e m s need n e e d to t o be b e developed d e v e l o p e d in i n a a responsible r e s p o n s i b l e manner m a n n e r and a n d with w i t h an a n ex-ante e x - a n t e due d u e and a n d proper p r o p e r consideration c o n s i d e r a t i o n of o f the t h e risks r i s k s that t h a t they t h e y may m a y generate. g e n e r a t e . Their T h e i r development d e v e l o p m e n t and a n d functioning f u n c t i o n i n g must m u s t be b e such s u c h to t o ensure e n s u r e that t h a t AI A I systems s y s t e m s behave b e h a v e reliably r e l i a b l y as a s intended. i n t e n d e d . All A l l reasonable r e a s o n a b l e measures m e a s u r e s should s h o u l d be b e taken t a k e n to t o minimise m i n i m i s e the t h e risk r i s k of o f harm h a r m being b e i n g caused. c a u s e d . Accordingly, A c c o r d i n g l y , the t h e following f o l l o w i n g elements e l e m e n t s could c o u l d be b e considered: c o n s i d e r e d : �  Requirements R e q u i r e m e n t s ensuring e n s u r i n g that t h a t the t h e AI A I systems s y s t e m s are a r e robust r o b u s t and a n d accurate, a c c u r a t e , or o r at a t least l e a s t correctly c o r r e c t l y reflect r e f l e c t their t h e i r level l e v e l of o f accuracy, a c c u r a c y , during d u r i n g all a l l life l i f e cycle c y c l e phases; p h a s e s ; �  Requirements R e q u i r e m e n t s ensuring e n s u r i n g that t h a t outcomes o u t c o m e s are a r e reproducible; r e p r o d u c i b l e ; �  Requirements R e q u i r e m e n t s ensuring e n s u r i n g that t h a t AI A I systems s y s t e m s can c a n adequately a d e q u a t e l y deal d e a l with w i t h errors e r r o r s or o r inconsistencies i n c o n s i s t e n c i e s during d u r i n g all a l l life l i f e cycle c y c l e phases. p h a s e s . 54 5 4 In I n particular, p a r t i c u l a r , pursuant p u r s u a n t to t o Art. A r t . 13(2)(f) 1 3 ( 2 ) ( f ) GDPR, G D P R , controllers c o n t r o l l e r s must, m u s t , at a t the t h e time t i m e when w h e n the t h e personal p e r s o n a l data d a t a are a r e obtained, o b t a i n e d , provide p r o v i d e the t h e data d a t a subjects s u b j e c t s with w i t h further f u r t h e r information i n f o r m a t i o n necessary n e c e s s a r y to t o ensure e n s u r e fair f a i r and a n d transparent t r a n s p a r e n t processing p r o c e s s i n g about a b o u t the t h e existence e x i s t e n c e of o f automated a u t o m a t e d decision-making d e c i s i o n - m a k i n g and a n d certain c e r t a i n additional a d d i t i o n a l information. i n f o r m a t i o n . 20 2 0 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} �  Requirements R e q u i r e m e n t s ensuring e n s u r i n g that t h a t AI A I systems s y s t e m s are a r e resilient r e s i l i e n t against a g a i n s t both b o t h overt o v e r t attacks a t t a c k s and a n d more m o r e subtle s u b t l e attempts a t t e m p t s to t o manipulate m a n i p u l a t e data d a t a or o r algorithms a l g o r i t h m s themselves, t h e m s e l v e s , and a n d that t h a t mitigating m i t i g a t i n g measures m e a s u r e s are a r e taken t a k e n in i n such s u c h cases. c a s e s . e) e ) Human H u m a n oversight o v e r s i g h t Human H u m a n oversight o v e r s i g h t helps h e l p s ensuring e n s u r i n g that t h a t an a n AI A I system s y s t e m does d o e s not n o t undermine u n d e r m i n e human h u m a n autonomy a u t o n o m y or o r cause c a u s e other o t h e r adverse a d v e r s e effects. e f f e c t s . The T h e objective o b j e c t i v e of o f trustworthy, t r u s t w o r t h y , ethical e t h i c a l and a n d human-centric h u m a n - c e n t r i c AI A I can c a n only o n l y be b e achieved a c h i e v e d by b y ensuring e n s u r i n g an a n appropriate a p p r o p r i a t e involvement i n v o l v e m e n t by b y human h u m a n beings b e i n g s in i n relation r e l a t i o n to t o high-risk h i g h - r i s k AI A I applications. a p p l i c a t i o n s . Even E v e n though t h o u g h the t h e AI A I applications a p p l i c a t i o n s considered c o n s i d e r e d in i n this t h i s White W h i t e paper p a p e r for f o r a a specific s p e c i f i c legal l e g a l regime r e g i m e are a r e all a l l considered c o n s i d e r e d high-risk, h i g h - r i s k , the t h e appropriate a p p r o p r i a t e type t y p e and a n d degree d e g r e e of o f human h u m a n oversight o v e r s i g h t may m a y vary v a r y from f r o m one o n e case c a s e to t o another. a n o t h e r . It I t shall s h a l l depend d e p e n d in i n particular p a r t i c u l a r on o n the t h e intended i n t e n d e d use u s e of o f the t h e systems s y s t e m s and a n d the t h e effects e f f e c t s that t h a t the t h e use u s e could c o u l d have h a v e for f o r affected a f f e c t e d citizens c i t i z e n s and a n d legal l e g a l entities. e n t i t i e s . It I t shall s h a l l also a l s o be b e without w i t h o u t prejudice p r e j u d i c e to t o the t h e legal l e g a l rights r i g h t s established e s t a b l i s h e d by b y the t h e GDPR G D P R when w h e n the t h e AI A I system s y s t e m processes p r o c e s s e s personal p e r s o n a l data. d a t a . For F o r instance, i n s t a n c e , human h u m a n oversight o v e r s i g h t could c o u l d have h a v e the t h e following, f o l l o w i n g , non-exhaustive, n o n - e x h a u s t i v e , manifestations: m a n i f e s t a t i o n s : �  the t h e output o u t p u t of o f the t h e AI A I system s y s t e m does d o e s not n o t become b e c o m e effective e f f e c t i v e unless u n l e s s it i t has h a s been b e e n previously p r e v i o u s l y reviewed r e v i e w e d and a n d validated v a l i d a t e d by b y a a human h u m a n (e.g. ( e . g . the t h e rejection r e j e c t i o n of o f an a n application a p p l i c a t i o n for f o r social s o c i a l security s e c u r i t y benefits b e n e f i t s may m a y be b e taken t a k e n by b y a a human h u m a n only); o n l y ) ; �  the t h e output o u t p u t of o f the t h e AI A I system s y s t e m becomes b e c o m e s immediately i m m e d i a t e l y effective, e f f e c t i v e , but b u t human h u m a n intervention i n t e r v e n t i o n is i s ensured e n s u r e d afterwards a f t e r w a r d s (e.g. ( e . g . the t h e rejection r e j e c t i o n of o f an a n application a p p l i c a t i o n for f o r a a credit c r e d i t card c a r d may m a y be b e processed p r o c e s s e d by b y an a n AI A I system, s y s t e m , but b u t human h u m a n review r e v i e w must m u s t be b e possible p o s s i b l e afterwards); a f t e r w a r d s ) ; �  monitoring m o n i t o r i n g of o f the t h e AI A I system s y s t e m while w h i l e in i n operation o p e r a t i o n and a n d the t h e ability a b i l i t y to t o intervene i n t e r v e n e in i n real r e a l time t i m e and a n d deactivate d e a c t i v a t e (e.g. ( e . g . a a stop s t o p button b u t t o n or o r procedure p r o c e d u r e is i s available a v a i l a b l e in i n a a driverless d r i v e r l e s s car c a r when w h e n a a human h u m a n determines d e t e r m i n e s that t h a t car c a r operation o p e r a t i o n is i s not n o t safe); s a f e ) ; �  in i n the t h e design d e s i g n phase, p h a s e , by b y imposing i m p o s i n g operational o p e r a t i o n a l constraints c o n s t r a i n t s on o n the t h e AI A I system s y s t e m (e.g. ( e . g . a a driverless d r i v e r l e s s car c a r shall s h a l l stop s t o p operating o p e r a t i n g in i n certain c e r t a i n conditions c o n d i t i o n s of o f low l o w visibility v i s i b i l i t y when w h e n sensors s e n s o r s may m a y become b e c o m e less l e s s reliable r e l i a b l e or o r shall s h a l l maintain m a i n t a i n a a certain c e r t a i n distance d i s t a n c e in i n any a n y given g i v e n condition c o n d i t i o n from f r o m the t h e preceding p r e c e d i n g vehicle). v e h i c l e ) . f) f ) Specific S p e c i f i c requirements r e q u i r e m e n t s for f o r remote r e m o t e biometric b i o m e t r i c identification i d e n t i f i c a t i o n The T h e gathering g a t h e r i n g and a n d use u s e of o f biometric b i o m e t r i c data d a t a 55 5 5 for f o r remote r e m o t e identification i d e n t i f i c a t i o n 56 5 6 purposes, p u r p o s e s , for f o r instance i n s t a n c e through t h r o u g h deployment d e p l o y m e n t of o f facial f a c i a l recognition r e c o g n i t i o n in i n public p u b l i c places, p l a c e s , carries c a r r i e s specific s p e c i f i c risks r i s k s for f o r fundamental f u n d a m e n t a l rights r i g h t s 57 5 7 . . The T h e 55 5 5 Biometric B i o m e t r i c data d a t a is i s defined d e f i n e d as a s “personal “ p e r s o n a l data d a t a resulting r e s u l t i n g from f r o m specific s p e c i f i c technical t e c h n i c a l processing p r o c e s s i n g relating r e l a t i n g to t o the t h e physical, p h y s i c a l , physiological p h y s i o l o g i c a l or o r behavioural b e h a v i o u r a l characteristics c h a r a c t e r i s t i c s of o f a a natural n a t u r a l person, p e r s o n , which w h i c h allow a l l o w or o r confirm c o n f i r m the t h e unique u n i q u e authentification a u t h e n t i f i c a t i o n or o r identification i d e n t i f i c a t i o n of o f that t h a t natural n a t u r a l person, p e r s o n , such s u c h as a s facial f a c i a l images i m a g e s or o r dactyloscopic d a c t y l o s c o p i c [fingerprint] [ f i n g e r p r i n t ] data.” d a t a . ” (Law ( L a w Enforcement E n f o r c e m e n t Directive, D i r e c t i v e , Art. A r t . 3 3 (13); ( 1 3 ) ; GDPR, G D P R , Art. A r t . 4 4 (14); ( 1 4 ) ; Regulation R e g u l a t i o n (EU) ( E U ) 2018/1725, 2 0 1 8 / 1 7 2 5 , Art. A r t . 3 3 (18). ( 1 8 ) . 56 5 6 In I n connection c o n n e c t i o n to t o facial f a c i a l recognition, r e c o g n i t i o n , identification i d e n t i f i c a t i o n means m e a n s that t h a t the t h e template t e m p l a t e of o f a a person’s p e r s o n ’ s facial f a c i a l image i m a g e is i s compared c o m p a r e d to t o many m a n y other o t h e r templates t e m p l a t e s stored s t o r e d in i n a a database d a t a b a s e to t o find f i n d out o u t if i f his h i s or o r her h e r image i m a g e is i s stored s t o r e d there. t h e r e . Authentication A u t h e n t i c a t i o n (or ( o r verification) v e r i f i c a t i o n ) on o n the t h e other o t h e r hand h a n d is i s often o f t e n referred r e f e r r e d to t o as a s one-to-one o n e - t o - o n e matching. m a t c h i n g . It I t enables e n a b l e s the t h e comparison c o m p a r i s o n of o f two t w o biometric b i o m e t r i c templates, t e m p l a t e s , usually u s u a l l y assumed a s s u m e d to t o belong b e l o n g to t o the t h e same s a m e individual. i n d i v i d u a l . Two T w o biometric b i o m e t r i c templates t e m p l a t e s are a r e compared c o m p a r e d to t o determine d e t e r m i n e if i f the t h e person p e r s o n shown s h o w n on o n the t h e two t w o images i m a g e s is i s the t h e same s a m e person. p e r s o n . Such S u c h a a procedure p r o c e d u r e is, i s , for f o r example, e x a m p l e , used u s e d at a t Automated A u t o m a t e d Border B o r d e r Control C o n t r o l (ABC) ( A B C ) gates g a t e s used u s e d for f o r border b o r d e r checks c h e c k s at a t airports. a i r p o r t s . 57 5 7 For F o r example e x a m p l e on o n people’s p e o p l e ’ s dignity. d i g n i t y . Relatedly, R e l a t e d l y , the t h e rights r i g h t s to t o respect r e s p e c t for f o r private p r i v a t e life l i f e and a n d protection p r o t e c t i o n of o f personal p e r s o n a l data d a t a are a r e at a t the t h e core c o r e of o f fundamental f u n d a m e n t a l rights r i g h t s concerns c o n c e r n s when w h e n using u s i n g facial f a c i a l recognition r e c o g n i t i o n technology. t e c h n o l o g y . There T h e r e is i s also a l s o a a potential p o t e n t i a l impact i m p a c t on o n non- n o n - discrimination d i s c r i m i n a t i o n and a n d rights r i g h t s of o f special s p e c i a l groups, g r o u p s , such s u c h as a s children, c h i l d r e n , older o l d e r persons p e r s o n s and a n d persons p e r s o n s with w i t h disabilities. d i s a b i l i t i e s . Moreover, M o r e o v e r , freedom f r e e d o m of o f expression, e x p r e s s i o n , association a s s o c i a t i o n and a n d assembly a s s e m b l y must m u s t not n o t be b e undermined u n d e r m i n e d by b y the t h e use u s e of o f the t h e technology. t e c h n o l o g y . See: S e e : Facial F a c i a l recognition r e c o g n i t i o n technology: t e c h n o l o g y : fundamental f u n d a m e n t a l rights r i g h t s considerations c o n s i d e r a t i o n s in i n the t h e context c o n t e x t of o f law l a w enforcement, e n f o r c e m e n t , https://fra.europa.eu/en/publication/2019/facial-recognition. h t t p s : / / f r a . e u r o p a . e u / e n / p u b l i c a t i o n / 2 0 1 9 / f a c i a l - r e c o g n i t i o n . 21 2 1 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} fundamental f u n d a m e n t a l rights r i g h t s implications i m p l i c a t i o n s of o f using u s i n g remote r e m o t e biometric b i o m e t r i c identification i d e n t i f i c a t i o n AI A I systems s y s t e m s can c a n vary v a r y considerably c o n s i d e r a b l y depending d e p e n d i n g on o n the t h e purpose, p u r p o s e , context c o n t e x t and a n d scope s c o p e of o f the t h e use. u s e . EU E U data d a t a protection p r o t e c t i o n rules r u l e s prohibit p r o h i b i t in i n principle p r i n c i p l e the t h e processing p r o c e s s i n g of o f biometric b i o m e t r i c data d a t a for f o r the t h e purpose p u r p o s e of o f uniquely u n i q u e l y identifying i d e n t i f y i n g a a natural n a t u r a l person, p e r s o n , except e x c e p t under u n d e r specific s p e c i f i c conditions c o n d i t i o n s 58 5 8 . . Specifically, S p e c i f i c a l l y , under u n d e r the t h e GDPR, G D P R , such s u c h processing p r o c e s s i n g can c a n only o n l y take t a k e place p l a c e on o n a a limited l i m i t e d number n u m b e r of o f grounds, g r o u n d s , the t h e main m a i n one o n e being b e i n g for f o r reasons r e a s o n s of o f substantial s u b s t a n t i a l public p u b l i c interest. i n t e r e s t . In I n that t h a t case, c a s e , the t h e processing p r o c e s s i n g must m u s t take t a k e place p l a c e on o n the t h e basis b a s i s of o f EU E U or o r national n a t i o n a l law, l a w , subject s u b j e c t to t o the t h e requirements r e q u i r e m e n t s of o f proportionality, p r o p o r t i o n a l i t y , respect r e s p e c t for f o r the t h e essence e s s e n c e of o f the t h e right r i g h t to t o data d a t a protection p r o t e c t i o n and a n d appropriate a p p r o p r i a t e safeguards. s a f e g u a r d s . Under U n d e r the t h e Law L a w Enforcement E n f o r c e m e n t Directive, D i r e c t i v e , there t h e r e must m u s t be b e a a strict s t r i c t necessity n e c e s s i t y for f o r such s u c h processing, p r o c e s s i n g , in i n principle p r i n c i p l e an a n authorisation a u t h o r i s a t i o n by b y EU E U or o r national n a t i o n a l law l a w as a s well w e l l as a s appropriate a p p r o p r i a t e safeguards. s a f e g u a r d s . As A s any a n y processing p r o c e s s i n g of o f biometric b i o m e t r i c data d a t a for f o r the t h e purpose p u r p o s e of o f uniquely u n i q u e l y identifying i d e n t i f y i n g a a natural n a t u r a l person p e r s o n would w o u l d relate r e l a t e to t o an a n exception e x c e p t i o n to t o a a prohibition p r o h i b i t i o n laid l a i d down d o w n in i n EU E U law, l a w , it i t would w o u l d be b e subject s u b j e c t to t o the t h e Charter C h a r t e r of o f Fundamental F u n d a m e n t a l Rights R i g h t s of o f the t h e EU. E U . It I t follows f o l l o w s that, t h a t , in i n accordance a c c o r d a n c e with w i t h the t h e current c u r r e n t EU E U data d a t a protection p r o t e c t i o n rules r u l e s and a n d the t h e Charter C h a r t e r of o f Fundamental F u n d a m e n t a l Rights, R i g h t s , AI A I can c a n only o n l y be b e used u s e d for f o r remote r e m o t e biometric b i o m e t r i c identification i d e n t i f i c a t i o n purposes p u r p o s e s where w h e r e such s u c h use u s e is i s duly d u l y justified, j u s t i f i e d , proportionate p r o p o r t i o n a t e and a n d subject s u b j e c t to t o adequate a d e q u a t e safeguards. s a f e g u a r d s . In I n order o r d e r to t o address a d d r e s s possible p o s s i b l e societal s o c i e t a l concerns c o n c e r n s relating r e l a t i n g to t o the t h e use u s e of o f AI A I for f o r such s u c h purposes p u r p o s e s in i n public p u b l i c places, p l a c e s , and a n d to t o avoid a v o i d fragmentation f r a g m e n t a t i o n in i n the t h e internal i n t e r n a l market, m a r k e t , the t h e Commission C o m m i s s i o n will w i l l launch l a u n c h a a broad b r o a d European E u r o p e a n debate d e b a t e on o n the t h e specific s p e c i f i c circumstances, c i r c u m s t a n c e s , if i f any, a n y , which w h i c h might m i g h t justify j u s t i f y such s u c h use, u s e , and a n d on o n common c o m m o n safeguards. s a f e g u a r d s . E. E . ADDRESSEES A D D R E S S E E S In I n relation r e l a t i o n to t o the t h e addressees a d d r e s s e e s of o f the t h e legal l e g a l requirements r e q u i r e m e n t s that t h a t would w o u l d apply a p p l y in i n relation r e l a t i o n to t o the t h e high-risk h i g h - r i s k AI A I applications a p p l i c a t i o n s referred r e f e r r e d to t o above, a b o v e , there t h e r e are a r e two t w o main m a i n issues i s s u e s to t o be b e considered. c o n s i d e r e d . First, F i r s t , there t h e r e is i s the t h e question q u e s t i o n how h o w obligations o b l i g a t i o n s are a r e to t o be b e distributed d i s t r i b u t e d among a m o n g the t h e economic e c o n o m i c operators o p e r a t o r s involved. i n v o l v e d . Many M a n y actors a c t o r s are a r e involved i n v o l v e d in i n the t h e lifecycle l i f e c y c l e of o f an a n AI A I system. s y s t e m . These T h e s e include i n c l u d e the t h e developer, d e v e l o p e r , the t h e deployer d e p l o y e r (the ( t h e person p e r s o n who w h o uses u s e s an a n AI-equipped A I - e q u i p p e d product p r o d u c t or o r service) s e r v i c e ) and a n d potentially p o t e n t i a l l y others o t h e r s (producer, ( p r o d u c e r , distributor d i s t r i b u t o r or o r importer, i m p o r t e r , service s e r v i c e provider, p r o v i d e r , professional p r o f e s s i o n a l or o r private p r i v a t e user). u s e r ) . It I t is i s the t h e Commission’s C o m m i s s i o n ’ s view v i e w that, t h a t , in i n a a future f u t u r e regulatory r e g u l a t o r y framework, f r a m e w o r k , each e a c h obligation o b l i g a t i o n should s h o u l d be b e addressed a d d r e s s e d to t o the t h e actor(s) a c t o r ( s ) who w h o is i s (are) ( a r e ) best b e s t placed p l a c e d to t o address a d d r e s s any a n y potential p o t e n t i a l risks. r i s k s . For F o r example, e x a m p l e , while w h i l e the t h e developers d e v e l o p e r s of o f AI A I may m a y be b e best b e s t placed p l a c e d to t o address a d d r e s s risks r i s k s arising a r i s i n g from f r o m the t h e development d e v e l o p m e n t phase, p h a s e , their t h e i r ability a b i l i t y to t o control c o n t r o l risks r i s k s during d u r i n g the t h e use u s e phase p h a s e may m a y be b e more m o r e limited. l i m i t e d . In I n that t h a t case, c a s e , the t h e deployer d e p l o y e r should s h o u l d be b e subject s u b j e c t to t o the t h e relevant r e l e v a n t obligation. o b l i g a t i o n . This T h i s is i s without w i t h o u t prejudice p r e j u d i c e to t o the t h e question q u e s t i o n whether, w h e t h e r , for f o r the t h e purpose p u r p o s e of o f liability l i a b i l i t y to t o end-users e n d - u s e r s or o r other o t h e r parties p a r t i e s suffering s u f f e r i n g harm h a r m and a n d ensuring e n s u r i n g effective e f f e c t i v e access a c c e s s to t o justice, j u s t i c e , which w h i c h party p a r t y should s h o u l d be b e liable l i a b l e for f o r any a n y damage d a m a g e caused. c a u s e d . Under U n d e r EU E U product p r o d u c t liability l i a b i l i t y law, l a w , liability l i a b i l i t y for f o r defective d e f e c t i v e products p r o d u c t s is i s attributed a t t r i b u t e d to t o the t h e producer, p r o d u c e r , without w i t h o u t prejudice p r e j u d i c e to t o national n a t i o n a l laws l a w s which w h i c h may m a y also a l s o allow a l l o w recovery r e c o v e r y from f r o m other o t h e r parties. p a r t i e s . Second, S e c o n d , there t h e r e is i s the t h e question q u e s t i o n about a b o u t the t h e geographic g e o g r a p h i c scope s c o p e of o f the t h e legislative l e g i s l a t i v e intervention. i n t e r v e n t i o n . In I n the t h e view v i e w of o f the t h e Commission, C o m m i s s i o n , it i t is i s paramount p a r a m o u n t that t h a t the t h e requirements r e q u i r e m e n t s are a r e applicable a p p l i c a b l e to t o all a l l relevant r e l e v a n t economic e c o n o m i c operators o p e r a t o r s providing p r o v i d i n g AI-enabled A I - e n a b l e d products p r o d u c t s or o r services s e r v i c e s in i n the t h e EU, E U , regardless r e g a r d l e s s of o f whether w h e t h e r they t h e y are a r e established e s t a b l i s h e d in i n the t h e EU E U or o r not. n o t . Otherwise, O t h e r w i s e , the t h e objectives o b j e c t i v e s of o f the t h e legislative l e g i s l a t i v e intervention, i n t e r v e n t i o n , mentioned m e n t i o n e d earlier, e a r l i e r , could c o u l d not n o t fully f u l l y be b e achieved. a c h i e v e d . 58 5 8 Article A r t i c l e 9 9 GDPR, G D P R , Article A r t i c l e 10 1 0 Law L a w Enforcement E n f o r c e m e n t Directive. D i r e c t i v e . See S e e also a l s o Article A r t i c l e 10 1 0 Regulation R e g u l a t i o n (EU) ( E U ) 2018/1725 2 0 1 8 / 1 7 2 5 (applicable ( a p p l i c a b l e to t o the t h e EU E U institutions i n s t i t u t i o n s and a n d bodies). b o d i e s ) . 22 2 2 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} F. F . COMPLIANCE C O M P L I A N C E AND A N D ENFORCEMENT E N F O R C E M E N T In I n order o r d e r to t o ensure e n s u r e that t h a t AI A I is i s trustworthy, t r u s t w o r t h y , secure s e c u r e and a n d in i n respect r e s p e c t of o f European E u r o p e a n values v a l u e s and a n d rules, r u l e s , the t h e applicable a p p l i c a b l e legal l e g a l requirements r e q u i r e m e n t s need n e e d to t o be b e complied c o m p l i e d with w i t h in i n practice p r a c t i c e and a n d be b e effectively e f f e c t i v e l y enforced e n f o r c e d both b o t h by b y competent c o m p e t e n t national n a t i o n a l and a n d European E u r o p e a n authorities a u t h o r i t i e s and a n d by b y affected a f f e c t e d parties. p a r t i e s . Competent C o m p e t e n t authorities a u t h o r i t i e s should s h o u l d be b e in i n a a position p o s i t i o n to t o investigate i n v e s t i g a t e individual i n d i v i d u a l cases, c a s e s , but b u t also a l s o to t o assess a s s e s s the t h e impact i m p a c t on o n society. s o c i e t y . In I n view v i e w of o f the t h e high h i g h risk r i s k that t h a t certain c e r t a i n AI A I applications a p p l i c a t i o n s pose p o s e for f o r citizens c i t i z e n s and a n d our o u r society s o c i e t y (see ( s e e section s e c t i o n A A above), a b o v e ) , the t h e Commission C o m m i s s i o n considers c o n s i d e r s at a t this t h i s stage s t a g e that t h a t an a n objective, o b j e c t i v e , prior p r i o r conformity c o n f o r m i t y assessment a s s e s s m e n t would w o u l d be b e necessary n e c e s s a r y to t o verify v e r i f y and a n d ensure e n s u r e that t h a t certain c e r t a i n of o f the t h e above a b o v e mentioned m e n t i o n e d mandatory m a n d a t o r y requirements r e q u i r e m e n t s applicable a p p l i c a b l e to t o high-risk h i g h - r i s k applications a p p l i c a t i o n s (see ( s e e section s e c t i o n D D above) a b o v e ) are a r e complied c o m p l i e d with. w i t h . The T h e prior p r i o r conformity c o n f o r m i t y assessment a s s e s s m e n t could c o u l d include i n c l u d e procedures p r o c e d u r e s for f o r testing, t e s t i n g , inspection i n s p e c t i o n or o r certification c e r t i f i c a t i o n 59 5 9 . . It I t could c o u l d include i n c l u d e checks c h e c k s of o f the t h e algorithms a l g o r i t h m s and a n d of o f the t h e data d a t a sets s e t s used u s e d in i n the t h e development d e v e l o p m e n t phase. p h a s e . The T h e conformity c o n f o r m i t y assessments a s s e s s m e n t s for f o r high-risk h i g h - r i s k AI A I applications a p p l i c a t i o n s should s h o u l d be b e part p a r t of o f the t h e conformity c o n f o r m i t y assessment a s s e s s m e n t mechanisms m e c h a n i s m s that t h a t already a l r e a d y exist e x i s t for f o r a a large l a r g e number n u m b e r of o f products p r o d u c t s being b e i n g placed p l a c e d on o n the t h e EU’s E U ’ s internal i n t e r n a l market. m a r k e t . Where W h e r e no n o such s u c h existing e x i s t i n g mechanisms m e c h a n i s m s can c a n be b e relied r e l i e d on, o n , similar s i m i l a r mechanisms m e c h a n i s m s may m a y need n e e d to t o be b e established, e s t a b l i s h e d , drawing d r a w i n g on o n best b e s t practice p r a c t i c e and a n d possible p o s s i b l e input i n p u t of o f stakeholders s t a k e h o l d e r s and a n d European E u r o p e a n standards s t a n d a r d s organisations. o r g a n i s a t i o n s . Any A n y such s u c h new n e w mechanism m e c h a n i s m should s h o u l d be b e proportionate p r o p o r t i o n a t e and a n d non-discriminatory n o n - d i s c r i m i n a t o r y and a n d use u s e transparent t r a n s p a r e n t and a n d objective o b j e c t i v e criteria c r i t e r i a in i n compliance c o m p l i a n c e with w i t h international i n t e r n a t i o n a l obligations. o b l i g a t i o n s . When W h e n designing d e s i g n i n g and a n d implementing i m p l e m e n t i n g a a system s y s t e m relying r e l y i n g on o n prior p r i o r conformity c o n f o r m i t y assessments, a s s e s s m e n t s , particular p a r t i c u l a r account a c c o u n t should s h o u l d be b e taken t a k e n of o f the t h e following: f o l l o w i n g : �  Not N o t all a l l requirements r e q u i r e m e n t s outlined o u t l i n e d above a b o v e may m a y be b e suitable s u i t a b l e to t o be b e verified v e r i f i e d through t h r o u g h a a prior p r i o r conformity c o n f o r m i t y assessment. a s s e s s m e n t . For F o r instance, i n s t a n c e , the t h e requirement r e q u i r e m e n t about a b o u t information i n f o r m a t i o n to t o be b e provided p r o v i d e d generally g e n e r a l l y does d o e s not n o t lend l e n d itself i t s e l f well w e l l for f o r verification v e r i f i c a t i o n through t h r o u g h such s u c h an a n assessment. a s s e s s m e n t . �  Particular P a r t i c u l a r account a c c o u n t should s h o u l d be b e taken t a k e n of o f the t h e possibility p o s s i b i l i t y that t h a t certain c e r t a i n AI A I systems s y s t e m s evolve e v o l v e and a n d learn l e a r n from f r o m experience, e x p e r i e n c e , which w h i c h may m a y require r e q u i r e repeated r e p e a t e d assessments a s s e s s m e n t s over o v e r the t h e life-time l i f e - t i m e of o f the t h e AI A I systems s y s t e m s in i n question. q u e s t i o n . �  The T h e need n e e d to t o verify v e r i f y the t h e data d a t a used u s e d for f o r training t r a i n i n g and a n d the t h e relevant r e l e v a n t programming p r o g r a m m i n g and a n d training t r a i n i n g methodologies, m e t h o d o l o g i e s , processes p r o c e s s e s and a n d techniques t e c h n i q u e s used u s e d to t o build, b u i l d , test t e s t and a n d validate v a l i d a t e AI A I systems. s y s t e m s . �  In I n case c a s e the t h e conformity c o n f o r m i t y assessment a s s e s s m e n t shows s h o w s that t h a t an a n AI A I system s y s t e m does d o e s not n o t meet m e e t the t h e requirements r e q u i r e m e n t s for f o r example e x a m p l e relating r e l a t i n g to t o the t h e data d a t a used u s e d to t o train t r a i n it, i t , the t h e identified i d e n t i f i e d shortcomings s h o r t c o m i n g s will w i l l need n e e d to t o be b e remedied, r e m e d i e d , for f o r instance i n s t a n c e by b y re-training r e - t r a i n i n g the t h e system s y s t e m in i n the t h e EU E U in i n such s u c h a a way w a y as a s to t o ensure e n s u r e that t h a t all a l l applicable a p p l i c a b l e requirements r e q u i r e m e n t s are a r e met. m e t . The T h e conformity c o n f o r m i t y assessments a s s e s s m e n t s would w o u l d be b e mandatory m a n d a t o r y for f o r all a l l economic e c o n o m i c operators o p e r a t o r s addressed a d d r e s s e d by b y the t h e requirements, r e q u i r e m e n t s , regardless r e g a r d l e s s of o f their t h e i r place p l a c e of o f establishment e s t a b l i s h m e n t 60 6 0 . . In I n order o r d e r to t o limit l i m i t the t h e burden b u r d e n on o n SMEs, S M E s , some s o m e support s u p p o r t structure s t r u c t u r e might m i g h t be b e envisaged e n v i s a g e d including i n c l u d i n g through t h r o u g h the t h e Digital D i g i t a l Innovation I n n o v a t i o n Hubs. H u b s . In I n addition, a d d i t i o n , standards s t a n d a r d s as a s well w e l l as a s dedicated d e d i c a t e d online o n l i n e tools t o o l s could c o u l d facilitate f a c i l i t a t e compliance. c o m p l i a n c e . 59 5 9 The T h e system s y s t e m would w o u l d be b e based b a s e d on o n conformity c o n f o r m i t y assessment a s s e s s m e n t procedures p r o c e d u r e s in i n the t h e EU, E U , see s e e Decision D e c i s i o n 768/2008/EC 7 6 8 / 2 0 0 8 / E C or o r on o n Regulation R e g u l a t i o n (EU) ( E U ) 2019/881 2 0 1 9 / 8 8 1 (Cybersecurity ( C y b e r s e c u r i t y Act), A c t ) , taking t a k i n g into i n t o account a c c o u n t the t h e specificities s p e c i f i c i t i e s of o f AI. A I . See S e e the t h e Blue B l u e Guide G u i d e on o n the t h e Implementation I m p l e m e n t a t i o n of o f EU E U product p r o d u c t rules, r u l e s , 2014. 2 0 1 4 . 60 6 0 As A s regards r e g a r d s the t h e relevant r e l e v a n t governance g o v e r n a n c e structure, s t r u c t u r e , including i n c l u d i n g the t h e bodies b o d i e s designated d e s i g n a t e d to t o carry c a r r y out o u t the t h e conformity c o n f o r m i t y assessments, a s s e s s m e n t s , see s e e section s e c t i o n H H below. b e l o w . 23 2 3 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} Any A n y prior p r i o r conformity c o n f o r m i t y assessment a s s e s s m e n t should s h o u l d be b e without w i t h o u t prejudice p r e j u d i c e to t o monitoring m o n i t o r i n g compliance c o m p l i a n c e and a n d ex e x post p o s t enforcement e n f o r c e m e n t by b y competent c o m p e t e n t national n a t i o n a l authorities. a u t h o r i t i e s . That T h a t holds h o l d s true t r u e in i n respect r e s p e c t of o f high-risk h i g h - r i s k AI A I applications, a p p l i c a t i o n s , but b u t also a l s o in i n respect r e s p e c t of o f other o t h e r AI A I applications a p p l i c a t i o n s subject s u b j e c t to t o legal l e g a l requirements, r e q u i r e m e n t s , although a l t h o u g h the t h e high-risk h i g h - r i s k nature n a t u r e of o f the t h e applications a p p l i c a t i o n s at a t issue i s s u e may m a y be b e reason r e a s o n for f o r the t h e competent c o m p e t e n t national n a t i o n a l authorities a u t h o r i t i e s to t o give g i v e particular p a r t i c u l a r attention a t t e n t i o n to t o the t h e former. f o r m e r . Ex-post E x - p o s t controls c o n t r o l s should s h o u l d be b e enabled e n a b l e d by b y adequate a d e q u a t e documentation d o c u m e n t a t i o n of o f the t h e relevant r e l e v a n t AI A I application a p p l i c a t i o n (see ( s e e section s e c t i o n E E above) a b o v e ) and, a n d , where w h e r e appropriate, a p p r o p r i a t e , a a possibility p o s s i b i l i t y for f o r third t h i r d parties p a r t i e s such s u c h as a s competent c o m p e t e n t authorities a u t h o r i t i e s to t o test t e s t such s u c h applications. a p p l i c a t i o n s . This T h i s may m a y be b e especially e s p e c i a l l y important i m p o r t a n t where w h e r e risks r i s k s to t o fundamental f u n d a m e n t a l rights r i g h t s arise, a r i s e , which w h i c h are a r e context c o n t e x t dependent. d e p e n d e n t . Such S u c h monitoring m o n i t o r i n g of o f compliance c o m p l i a n c e should s h o u l d be b e part p a r t of o f a a continuous c o n t i n u o u s market m a r k e t surveillance s u r v e i l l a n c e scheme. s c h e m e . Governance-related G o v e r n a n c e - r e l a t e d aspects a s p e c t s are a r e further f u r t h e r discussed d i s c u s s e d in i n section s e c t i o n H H below. b e l o w . Moreover, M o r e o v e r , both b o t h for f o r high- h i g h - risk r i s k AI A I applications a p p l i c a t i o n s and a n d for f o r other o t h e r AI A I applications, a p p l i c a t i o n s , effective e f f e c t i v e judicial j u d i c i a l redress r e d r e s s for f o r parties p a r t i e s negatively n e g a t i v e l y affected a f f e c t e d by b y AI A I systems s y s t e m s should s h o u l d be b e ensured. e n s u r e d . Issues I s s u e s related r e l a t e d to t o liability l i a b i l i t y are a r e further f u r t h e r discussed d i s c u s s e d in i n the t h e Report R e p o r t on o n the t h e safety s a f e t y and a n d liability l i a b i l i t y framework f r a m e w o r k accompanying a c c o m p a n y i n g this t h i s White W h i t e Paper. P a p e r . G. G . VOLUNTARY V O L U N T A R Y LABELLING L A B E L L I N G FOR F O R NO-HIGH N O - H I G H RISK R I S K AI A I APPLICATIONS A P P L I C A T I O N S For F o r AI A I applications a p p l i c a t i o n s that t h a t do d o not n o t qualify q u a l i f y as a s ‘high-risk’ ‘ h i g h - r i s k ’ (see ( s e e section s e c t i o n C C above) a b o v e ) and a n d that t h a t are a r e therefore t h e r e f o r e not n o t subject s u b j e c t to t o the t h e mandatory m a n d a t o r y requirements r e q u i r e m e n t s discussed d i s c u s s e d above a b o v e (see ( s e e sections s e c t i o n s D, D , E E and a n d F F above), a b o v e ) , an a n option o p t i o n would w o u l d be, b e , in i n addition a d d i t i o n to t o applicable a p p l i c a b l e legislation, l e g i s l a t i o n , to t o establish e s t a b l i s h a a voluntary v o l u n t a r y labelling l a b e l l i n g scheme. s c h e m e . Under U n d e r the t h e scheme, s c h e m e , interested i n t e r e s t e d economic e c o n o m i c operators o p e r a t o r s that t h a t are a r e not n o t covered c o v e r e d by b y the t h e mandatory m a n d a t o r y requirements r e q u i r e m e n t s could c o u l d decide d e c i d e to t o make m a k e themselves t h e m s e l v e s subject, s u b j e c t , on o n a a voluntary v o l u n t a r y basis, b a s i s , either e i t h e r to t o those t h o s e requirements r e q u i r e m e n t s or o r to t o a a specific s p e c i f i c set s e t of o f similar s i m i l a r requirements r e q u i r e m e n t s especially e s p e c i a l l y established e s t a b l i s h e d for f o r the t h e purposes p u r p o s e s of o f the t h e voluntary v o l u n t a r y scheme. s c h e m e . The T h e economic e c o n o m i c operators o p e r a t o r s concerned c o n c e r n e d would w o u l d then t h e n be b e awarded a w a r d e d a a quality q u a l i t y label l a b e l for f o r their t h e i r AI A I applications. a p p l i c a t i o n s . The T h e voluntary v o l u n t a r y label l a b e l would w o u l d allow a l l o w the t h e economic e c o n o m i c operators o p e r a t o r s concerned c o n c e r n e d to t o signal s i g n a l that t h a t their t h e i r AI-enabled A I - e n a b l e d products p r o d u c t s and a n d services s e r v i c e s are a r e trustworthy. t r u s t w o r t h y . It I t would w o u l d allow a l l o w users u s e r s to t o easily e a s i l y recognise r e c o g n i s e that t h a t the t h e products p r o d u c t s and a n d services s e r v i c e s in i n question q u e s t i o n are a r e in i n compliance c o m p l i a n c e with w i t h certain c e r t a i n objective o b j e c t i v e and a n d standardised s t a n d a r d i s e d EU-wide E U - w i d e benchmarks, b e n c h m a r k s , going g o i n g beyond b e y o n d the t h e normally n o r m a l l y applicable a p p l i c a b l e legal l e g a l obligations. o b l i g a t i o n s . This T h i s would w o u l d help h e l p enhance e n h a n c e the t h e trust t r u s t of o f users u s e r s in i n AI A I systems s y s t e m s and a n d promote p r o m o t e the t h e overall o v e r a l l uptake u p t a k e of o f the t h e technology. t e c h n o l o g y . This T h i s option o p t i o n would w o u l d entail e n t a i l the t h e creation c r e a t i o n of o f a a new n e w legal l e g a l instrument i n s t r u m e n t that t h a t sets s e t s out o u t the t h e voluntary v o l u n t a r y labelling l a b e l l i n g framework f r a m e w o r k for f o r developers d e v e l o p e r s and/or a n d / o r deployers d e p l o y e r s of o f AI A I systems s y s t e m s that t h a t are a r e not n o t be b e considered c o n s i d e r e d as a s high-risk. h i g h - r i s k . While W h i l e participation p a r t i c i p a t i o n in i n the t h e labelling l a b e l l i n g scheme s c h e m e would w o u l d be b e voluntary, v o l u n t a r y , once o n c e the t h e developer d e v e l o p e r or o r the t h e deployer d e p l o y e r opted o p t e d to t o use u s e the t h e label, l a b e l , the t h e requirements r e q u i r e m e n t s would w o u l d be b e binding. b i n d i n g . The T h e combination c o m b i n a t i o n of o f ex e x ante a n t e and a n d ex e x post p o s t enforcement e n f o r c e m e n t would w o u l d need n e e d to t o ensure e n s u r e that t h a t all a l l requirements r e q u i r e m e n t s are a r e complied c o m p l i e d with. w i t h . H. H . GOVERNANCE G O V E R N A N C E A A European E u r o p e a n governance g o v e r n a n c e structure s t r u c t u r e on o n AI A I in i n the t h e form f o r m of o f a a framework f r a m e w o r k for f o r cooperation c o o p e r a t i o n of o f national n a t i o n a l competent c o m p e t e n t authorities a u t h o r i t i e s is i s necessary n e c e s s a r y to t o avoid a v o i d fragmentation f r a g m e n t a t i o n of o f responsibilities, r e s p o n s i b i l i t i e s , increase i n c r e a s e capacity c a p a c i t y in i n Member M e m b e r States, S t a t e s , and a n d make m a k e sure s u r e that t h a t Europe E u r o p e equips e q u i p s itself i t s e l f progressively p r o g r e s s i v e l y with w i t h the t h e capacity c a p a c i t y needed n e e d e d for f o r testing t e s t i n g and a n d certification c e r t i f i c a t i o n of o f AI-enabled A I - e n a b l e d products p r o d u c t s and a n d services. s e r v i c e s . In I n this t h i s context, c o n t e x t , it i t would w o u l d be b e beneficial b e n e f i c i a l to t o support s u p p o r t competent c o m p e t e n t national n a t i o n a l authorities a u t h o r i t i e s to t o enable e n a b l e them t h e m to t o fulfil f u l f i l their t h e i r mandate m a n d a t e where w h e r e AI A I is i s used. u s e d . A A European E u r o p e a n governance g o v e r n a n c e structure s t r u c t u r e could c o u l d have h a v e a a variety v a r i e t y of o f tasks, t a s k s , as a s a a forum f o r u m for f o r a a regular r e g u l a r exchange e x c h a n g e of o f information i n f o r m a t i o n and a n d best b e s t practice, p r a c t i c e , identifying i d e n t i f y i n g emerging e m e r g i n g trends, t r e n d s , advising a d v i s i n g on o n standardisation s t a n d a r d i s a t i o n activity a c t i v i t y as a s well w e l l as a s on o n certification. c e r t i f i c a t i o n . It I t should s h o u l d also a l s o play p l a y a a key k e y role r o l e in i n facilitating f a c i l i t a t i n g the t h e implementation i m p l e m e n t a t i o n of o f the t h e legal l e g a l framework, f r a m e w o r k , such s u c h as a s through t h r o u g h issuing i s s u i n g guidance, g u i d a n c e , opinions o p i n i o n s and a n d expertise. e x p e r t i s e . To T o that t h a t effect, e f f e c t , it i t should s h o u l d rely r e l y on o n a a network n e t w o r k of o f national n a t i o n a l authorities, a u t h o r i t i e s , as a s well w e l l as a s sectorial s e c t o r i a l networks n e t w o r k s and a n d regulatory r e g u l a t o r y authorities, a u t h o r i t i e s , at a t national n a t i o n a l and a n d EU E U level. l e v e l . Moreover, M o r e o v e r , a a committee c o m m i t t e e of o f experts e x p e r t s could c o u l d provide p r o v i d e assistance a s s i s t a n c e to t o the t h e Commission. C o m m i s s i o n . 24 2 4 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} The T h e governance g o v e r n a n c e structure s t r u c t u r e should s h o u l d guarantee g u a r a n t e e maximum m a x i m u m stakeholders s t a k e h o l d e r s participation. p a r t i c i p a t i o n . Stakeholders S t a k e h o l d e r s – – consumer c o n s u m e r organisation o r g a n i s a t i o n and a n d social s o c i a l partners, p a r t n e r s , businesses, b u s i n e s s e s , researchers, r e s e a r c h e r s , and a n d civil c i v i l society s o c i e t y organisations o r g a n i s a t i o n s – – should s h o u l d be b e consulted c o n s u l t e d on o n the t h e implementation i m p l e m e n t a t i o n and a n d the t h e further f u r t h e r development d e v e l o p m e n t of o f the t h e framework. f r a m e w o r k . Given G i v e n already a l r e a d y existing e x i s t i n g structures s t r u c t u r e s such s u c h as a s in i n finance, f i n a n c e , pharmaceuticals, p h a r m a c e u t i c a l s , aviation, a v i a t i o n , medical m e d i c a l devices, d e v i c e s , consumer c o n s u m e r protection, p r o t e c t i o n , data d a t a protection, p r o t e c t i o n , the t h e proposed p r o p o s e d governance g o v e r n a n c e structure s t r u c t u r e should s h o u l d not n o t duplicate d u p l i c a t e existing e x i s t i n g functions. f u n c t i o n s . It I t should s h o u l d instead i n s t e a d establish e s t a b l i s h close c l o s e links l i n k s with w i t h other o t h e r EU E U and a n d national n a t i o n a l competent c o m p e t e n t authorities a u t h o r i t i e s in i n the t h e various v a r i o u s sectors s e c t o r s to t o complement c o m p l e m e n t existing e x i s t i n g expertise e x p e r t i s e and a n d help h e l p existing e x i s t i n g authorities a u t h o r i t i e s in i n monitoring m o n i t o r i n g and a n d the t h e oversight o v e r s i g h t of o f the t h e activities a c t i v i t i e s of o f economic e c o n o m i c operators o p e r a t o r s involving i n v o l v i n g AI A I systems s y s t e m s and a n d AI-enabled A I - e n a b l e d products p r o d u c t s and a n d services. s e r v i c e s . Finally, F i n a l l y , if i f this t h i s option o p t i o n is i s pursued, p u r s u e d , the t h e carrying c a r r y i n g out o u t of o f conformity c o n f o r m i t y assessments a s s e s s m e n t s could c o u l d be b e entrusted e n t r u s t e d to t o notified n o t i f i e d bodies b o d i e s designated d e s i g n a t e d by b y Member M e m b e r States. S t a t e s . Testing T e s t i n g centres c e n t r e s should s h o u l d enable e n a b l e the t h e independent i n d e p e n d e n t audit a u d i t and a n d assessment a s s e s s m e n t of o f AI-systems A I - s y s t e m s in i n accordance a c c o r d a n c e with w i t h the t h e requirements r e q u i r e m e n t s outlined o u t l i n e d above. a b o v e . Independent I n d e p e n d e n t assessment a s s e s s m e n t will w i l l increase i n c r e a s e trust t r u s t and a n d ensures e n s u r e s objectivity. o b j e c t i v i t y . It I t could c o u l d also a l s o facilitate f a c i l i t a t e the t h e work w o r k of o f relevant r e l e v a n t competent c o m p e t e n t authorities. a u t h o r i t i e s . The T h e EU E U enjoys e n j o y s excellent e x c e l l e n t testing t e s t i n g and a n d assessment a s s e s s m e n t centres c e n t r e s and a n d should s h o u l d develop d e v e l o p its i t s capacity c a p a c i t y also a l s o in i n the t h e area a r e a of o f AI. A I . Economic E c o n o m i c operators o p e r a t o r s established e s t a b l i s h e d in i n third t h i r d countries c o u n t r i e s wanting w a n t i n g to t o enter e n t e r the t h e internal i n t e r n a l market m a r k e t could c o u l d either e i t h e r make m a k e use u s e of o f designated d e s i g n a t e d bodies b o d i e s established e s t a b l i s h e d in i n the t h e EU E U or, o r , subject s u b j e c t to t o mutual m u t u a l recognition r e c o g n i t i o n agreements a g r e e m e n t s with w i t h third t h i r d countries, c o u n t r i e s , have h a v e recourse r e c o u r s e to t o third-country t h i r d - c o u n t r y bodies b o d i e s designated d e s i g n a t e d to t o carry c a r r y out o u t such s u c h assessment. a s s e s s m e n t . The T h e governance g o v e r n a n c e structure s t r u c t u r e relating r e l a t i n g to t o AI A I and a n d the t h e possible p o s s i b l e conformity c o n f o r m i t y assessments a s s e s s m e n t s at a t issue i s s u e here h e r e would w o u l d leave l e a v e the t h e powers p o w e r s and a n d responsibilities r e s p o n s i b i l i t i e s under u n d e r existing e x i s t i n g EU E U law l a w of o f the t h e relevant r e l e v a n t competent c o m p e t e n t authorities a u t h o r i t i e s in i n specific s p e c i f i c sectors s e c t o r s or o r on o n specific s p e c i f i c issues i s s u e s (finance, ( f i n a n c e , pharmaceuticals, p h a r m a c e u t i c a l s , aviation, a v i a t i o n , medical m e d i c a l devices, d e v i c e s , consumer c o n s u m e r protection, p r o t e c t i o n , data d a t a protection, p r o t e c t i o n , etc.) e t c . ) unaffected. u n a f f e c t e d . 6. 6 . CONCLUSION C O N C L U S I O N AI A I is i s a a strategic s t r a t e g i c technology t e c h n o l o g y that t h a t offers o f f e r s many m a n y benefits b e n e f i t s for f o r citizens, c i t i z e n s , companies c o m p a n i e s and a n d society s o c i e t y as a s a a whole, w h o l e , provided p r o v i d e d it i t is i s human-centric, h u m a n - c e n t r i c , ethical, e t h i c a l , sustainable s u s t a i n a b l e and a n d respects r e s p e c t s fundamental f u n d a m e n t a l rights r i g h t s and a n d values. v a l u e s . AI A I offers o f f e r s important i m p o r t a n t efficiency e f f i c i e n c y and a n d productivity p r o d u c t i v i t y gains g a i n s that t h a t can c a n strengthen s t r e n g t h e n the t h e competitiveness c o m p e t i t i v e n e s s of o f European E u r o p e a n industry i n d u s t r y and a n d improve i m p r o v e the t h e wellbeing w e l l b e i n g of o f citizens. c i t i z e n s . It I t can c a n also a l s o contribute c o n t r i b u t e to t o finding f i n d i n g solutions s o l u t i o n s to t o some s o m e of o f the t h e most m o s t pressing p r e s s i n g societal s o c i e t a l challenges, c h a l l e n g e s , including i n c l u d i n g the t h e fight f i g h t against a g a i n s t climate c l i m a t e change c h a n g e and a n d environmental e n v i r o n m e n t a l degradation, d e g r a d a t i o n , the t h e challenges c h a l l e n g e s linked l i n k e d to t o sustainability s u s t a i n a b i l i t y and a n d demographic d e m o g r a p h i c changes, c h a n g e s , and a n d the t h e protection p r o t e c t i o n of o f our o u r democracies d e m o c r a c i e s and, a n d , where w h e r e necessary n e c e s s a r y and a n d proportionate, p r o p o r t i o n a t e , the t h e fight f i g h t against a g a i n s t crime. c r i m e . For F o r Europe E u r o p e to t o seize s e i z e fully f u l l y the t h e opportunities o p p o r t u n i t i e s that t h a t AI A I offers, o f f e r s , it i t must m u s t develop d e v e l o p and a n d reinforce r e i n f o r c e the t h e necessary n e c e s s a r y industrial i n d u s t r i a l and a n d technological t e c h n o l o g i c a l capacities. c a p a c i t i e s . As A s set s e t out o u t in i n the t h e accompanying a c c o m p a n y i n g European E u r o p e a n strategy s t r a t e g y for f o r data, d a t a , this t h i s also a l s o requires r e q u i r e s measures m e a s u r e s that t h a t will w i l l enable e n a b l e the t h e EU E U to t o become b e c o m e a a global g l o b a l hub h u b for f o r data. d a t a . The T h e European E u r o p e a n approach a p p r o a c h for f o r AI A I aims a i m s to t o promote p r o m o t e Europe’s E u r o p e ’ s innovation i n n o v a t i o n capacity c a p a c i t y in i n the t h e area a r e a of o f AI A I while w h i l e supporting s u p p o r t i n g the t h e development d e v e l o p m e n t and a n d uptake u p t a k e of o f ethical e t h i c a l and a n d trustworthy t r u s t w o r t h y AI A I across a c r o s s the t h e EU E U economy. e c o n o m y . AI A I should s h o u l d work w o r k for f o r people p e o p l e and a n d be b e a a force f o r c e for f o r good g o o d in i n society. s o c i e t y . With W i t h this t h i s White W h i t e Paper P a p e r and a n d the t h e accompanying a c c o m p a n y i n g Report R e p o r t on o n the t h e safety s a f e t y and a n d liability l i a b i l i t y framework, f r a m e w o r k , the t h e Commission C o m m i s s i o n launches l a u n c h e s a a broad b r o a d consultation c o n s u l t a t i o n of o f Member M e m b e r States S t a t e s civil c i v i l society, s o c i e t y , industry i n d u s t r y and a n d academics, a c a d e m i c s , of o f concrete c o n c r e t e proposals p r o p o s a l s for f o r a a European E u r o p e a n approach a p p r o a c h to t o AI. A I . These T h e s e include i n c l u d e both b o t h policy p o l i c y means m e a n s to t o boost b o o s t investments i n v e s t m e n t s in i n research r e s e a r c h and a n d innovation, i n n o v a t i o n , enhance e n h a n c e the t h e development d e v e l o p m e n t of o f skills s k i l l s and a n d support s u p p o r t the t h e uptake u p t a k e of o f AI A I by b y SMEs, S M E s , and a n d proposals p r o p o s a l s for f o r key k e y elements e l e m e n t s of o f a a future f u t u r e regulatory r e g u l a t o r y framework. f r a m e w o r k . This T h i s consultation c o n s u l t a t i o n will w i l l 25 2 5 granularity=word tetml={elements={line} glyphdetails={all}} contentanalysis={punctuationbreaks=false dehyphenate=false shadowdetect=false} layoutanalysis={tabledetect=0 layoutdetect=3} structureanalysis={list=true} allow a l l o w a a comprehensive c o m p r e h e n s i v e dialogue d i a l o g u e with w i t h all a l l concerned c o n c e r n e d parties p a r t i e s that t h a t will w i l l inform i n f o r m the t h e next n e x t steps s t e p s of o f the t h e Commission. C o m m i s s i o n . The T h e Commission C o m m i s s i o n invites i n v i t e s for f o r comments c o m m e n t s on o n the t h e proposals p r o p o s a l s set s e t out o u t in i n the t h e White W h i t e Paper P a p e r through t h r o u g h an a n open o p e n public p u b l i c consultation c o n s u l t a t i o n available a v a i l a b l e at a t https://ec.europa.eu/info/consultations_en. h t t p s : / / e c . e u r o p a . e u / i n f o / c o n s u l t a t i o n s _ e n . The T h e consultation c o n s u l t a t i o n is i s open o p e n for f o r comments c o m m e n t s until u n t i l 19 1 9 May M a y 2020. 2 0 2 0 . It I t is i s standard s t a n d a r d practice p r a c t i c e for f o r the t h e Commission C o m m i s s i o n to t o publish p u b l i s h submissions s u b m i s s i o n s received r e c e i v e d in i n response r e s p o n s e to t o a a public p u b l i c consultation. c o n s u l t a t i o n . However, H o w e v e r , it i t is i s possible p o s s i b l e to t o request r e q u e s t that t h a t submissions, s u b m i s s i o n s , or o r parts p a r t s thereof, t h e r e o f , remain r e m a i n confidential. c o n f i d e n t i a l . Should S h o u l d this t h i s be b e the t h e case, c a s e , please p l e a s e indicate i n d i c a t e clearly c l e a r l y on o n the t h e front f r o n t page p a g e of o f your y o u r submission s u b m i s s i o n that t h a t it i t should s h o u l d not n o t be b e made m a d e public p u b l i c and a n d also a l s o send s e n d a a non-confidential n o n - c o n f i d e n t i a l version v e r s i o n of o f your y o u r submission s u b m i s s i o n to t o the t h e Commission C o m m i s s i o n for f o r publication. p u b l i c a t i o n . 26 2 6 0.00 0.13 0.00 0.00 1.00