With the EU's AI Act coming into force imminently,
those designing, developing, and/or deploying AI will need to start
getting to grips with the myriad of new obligations, including new
cyber security requirements.
Among other cyber security requirements, the AI Act introduces a
requirement for High Risk systems to be 'resilient against
attempts by unauthorised third parties to alter their use, outputs
or performance by exploiting system vulnerabilities'. There is
some element of proportionality in the obligation, which recognises
that technical solutions aiming to ensure cyber security of the
systems must be appropriate to the risks and circumstances, but it
also highlights a series of vulnerabilities the technical solutions
should address (where appropriate). These include data or model
poisoning, model evasion, confidentiality attacks, and model
flaws.
