Bradu Neagu & Associates | View firm profile
Authored by Noemi Cădariu, Managing Associate
Sure, ChatGPT writes our emails, drafts our LinkedIn posts, translates documents, and generates content that substantially reduces the time spent on a single task. The widespread adoption and normalization of artificial intelligence in our daily lives—both at work and at home—creates even more room for error, as we place blind trust in content generated without citing sources, in a format that’s easy to take “as is,” instantly translated into any language, created by software capable of sensing or reading its audience (in the term “sentient,” which is difficult to translate).
Furthermore, we wish to highlight the ease with which the end user of a piece of material, document, result, or response can be misled—how do we distinguish between an AI-generated response and one provided by a competent person who knows their field? How do we distinguish between an author’s manuscript and a text generated by AI that has learned their writing style? How do we distinguish a lawyer’s legal opinion from one written by AI with access to legislation? After all, both come with a disclaimer.
How do we manage, at the end of the day, to actually recognize the human factor?
We are aware of the obvious downside to the daily benefits that artificial intelligence brings in automating and streamlining tasks: the laziness of the human mind, both from an active perspective—by shifting active components of a project onto the AI —as well as from a passive perspective—by easily and quickly obtaining information perceived as truth but never verified.
But what are we doing about the neglected phenomenon of deepfakes? We smile when artificial intelligence generates content favorable to us, losing sight of the fact that there are always two sides to every coin, and that AI content—being in constant development—can at any time generate content that is completely unfavorable to humans (individually or as a group).
- What, exactly, is the deepfake phenomenon?
A deepfake is an advanced form of digital manipulation of media content, involving the use of machine learning (i.e., artificial intelligence) and image generation technologies to create fake audio, photo, or video material that appears authentic and credible. The term “deepfake” combines the concepts of “deep learning” and “fake,” reflecting the very complexity of its sophisticated nature.
The process of creating deepfake visual content involves a deep learning algorithm (deep AI) trained on a significant amount of data (photos, video, or audio recordings) related to the target person of the desired content. The algorithm is trained to learn the target person’s facial and vocal characteristics in order to recreate and manipulate any kind of media content—ultimately with the goal of making that person appear to be in a certain place, saying certain words, or performing certain activities that, of course, did not actually happen and do not reflect reality.
There are various techniques used in the process of creating deepfakes, among which we note the generation of images using generative adversarial networks—that is, a pair of neural networks with diametrically opposed objectives, one tasked with generating fake content and the other with detecting fakes. This system thus creates an iterative process through which the quality of the fakes improves via simultaneous positive and negative reinforcement.
Although it may seem like a form of creative entertainment when its purpose is harmless, the deepfake phenomenon is overshadowed by a dark cloud of malicious intent, ranging from political manipulation and propaganda to pornographic content (with an alarming percentage of child pornography). According to an analysis by Deeptrace Labs, over 96% of the deepfake content identified online in 2019 was non-consensual pornographic content—where the victims include both public figures and ordinary people, both adults and minors, predominantly women, thereby exacerbating the phenomenon of revenge porn.
In the political sphere, the dissemination of visual and audio deepfake content is capable of creating hysteria, inciting violence, and spreading propaganda more easily than through any other communication channel, particularly on social media platforms—especially given that the tendency to verify the sources of content consumed on these platforms is steadily declining.
For example, deepfake images and videos were widely circulated during two highly publicized armed conflicts of our time, namely the Russia-Ukraine and Israel-Hamas conflicts, generating false political statements attributed to the leaders of these countries (including (i) Volodymyr Zelensky’s deepfake speech in which he ordered Ukrainians to surrender to the Russian army[1] , (ii) deepfake content about victims of the Israel-Hamas conflict[2] , (iii) Iranian hackers disrupting broadcasts in the UAE with deepfake news[3] , and (iv) the deepfake interview with President Vladimir Putin[4] ).
Although efforts are currently underway to improve countermeasures against the negative effects of deepfake content—through the development of technologies to detect this type of content and by attempting to educate the general public in the critical evaluation of falsified media content—such a result is unrealistic at this time.
Given the way information is consumed—particularly the speed at which we actually consume the content we scroll through every day— on social media platforms designed to deliver dopamine through short, concentrated, and fast-paced content—discerning and filtering the information we consume becomes a utopian concept, especially for Gen Z and subsequent generations, who have not been instilled with the principle of “check first, then believe.”
Furthermore, with the delayed identification of deepfake content, this type of content has enough time to linger online, spreading false and misleading information, inciting violence, hatred, and racial segregation, as well as destroying the reputations of women who are victims of the wave of deepfake porn. By the time such an image or recording is identified as machine-generated, the damage has already been done, reputations have been tarnished, and the public has been influenced.
- The European Approach to the Deepfake Phenomenon and National Regulation
- The European stance
At the end of 2022, the European Commission proposed a new directive to combat violence and domestic violence against women in various forms. Currently, the European Council and the European Parliament support the proposal to criminalize, among other things, the non-consensual distribution of AI-generated pornographic content, with the aim of curbing the phenomenon of revenge porn.
The EU’s response comes in the wake of the most recent deepfake porn scandal, in which Taylor Swift herself was the victim earlier this year, with the video garnering over 45 million views online. Since early February, the European Union has proposed criminalizing the distribution of such content within the EU (including revenge porn and online harassment)—a measure expected to take effect in mid-2027.
European Commission Vice President Věra Jourová told Politico[5] that “the latest repugnant way to humiliate women involves distributing intimate images generated by artificial intelligence in less than a few minutes. Such images can cause serious harm, not just to pop stars, but to any woman who will then be forced to prove at work or at home that those images are the result of deepfakes.”
At the EU level, the European Parliament approved on Wednesday, March 13, 2024, the regulation on artificial intelligence, the EU AI Act, considered to be the world’s first official regulation of artificial intelligence. The European Parliament’s priority is to ensure that AI systems used in the EU are safe, transparent, traceable, non-discriminatory, and environmentally friendly. In particular, the aim is for these systems to be supervised by humans—rather than through automated processes themselves—to prevent harmful outcomes resulting from their use. Within the scope of these regulations and future sanctions, the phenomenon of deepfakes is also addressed as a component of generative artificial intelligence systems.
We emphasize that the adoption of such a regulation represents a global first, positioning the EU as a leader in establishing a legal framework for artificial intelligence systems, a regulation that will certainly serve as a source of inspiration for non-member countries, particularly the U.S.—where the phenomenon is far more widespread than within the EU.
Final approval will be granted by the Council of the European Union—expected by the end of 2024—and the regulation will enter into force 20 days after its publication in the Official Journal, while the prohibitions and restrictions will take effect 6 months from that date. The application of certain restrictions and bans may be deferred for up to 12 months after the directive enters into force; however, all other obligations set forth in this directive will be enforced, without exception, no later than 36 months after its entry into force, i.e., by the end of 2027 (under current conditions).
The Act regulates different risk categories regarding various artificial intelligence systems, ranging from unacceptable risk—considered a threat to people and therefore prohibited (i.e., the biometric identification and classification of individuals)—to high risk—which has a negative impact on people’s safety and fundamental rights and must be assessed before being placed on the market (i.e., software for managing migration, asylum, and border control; software that assists in the interpretation and application of the law), and limited risk—a category that includes general and generative artificial intelligence (i.e., ChatGPT; systems that generate and manipulate images, audio, and video content, such as deepfakes)[6] .
However, it is important to note that these rules and classifications were created and drafted three years ago, at a time when applications such as ChatGPT and OpenAI did not have the popularity and prevalence they have today—nor did they possess such learning capabilities and such an extensive database. Given their prevalence, ease of access, and the potential factual and legal implications of this type of output, the classification of generative content systems as “low-risk” may be “upgraded” to a higher risk level over time.
- National Legislative Framework
At the national level, however, as early as April 2023, Bill No. 471/2023 on the responsible use of technology in the context of the deepfake phenomenon was introduced; it is still under review—sent for further report to two committees, with a deadline of March 1, 2024, a deadline that has already passed.
In the explanatory memorandum, the Romanian legislature takes into account the rapid development of technology, artificial intelligence, and techniques for creating artificial virtual reality, as well as the impressive advances in machine learning tools, deeming it necessary to intervene to limit the malicious effects of the false content generated by these tools. The legislature also points out that experts in the field consider deepfakes to be far more dangerous and to have a much greater impact than the fake news disseminated in the media to which we are already accustomed, since deepfakes can generate both visual and audio content intrinsically linked to the victim being impersonated.
We therefore welcome the initiative of Representative Eugen Bejinariu and Senator Robert-Marius Cazanciuc, as the sponsors of this bill, and concur with the view expressed regarding the deepfake phenomenon—namely, that the creation and distribution of such materials (referred to as “severe fakes”) constitute—each, in its own material form—a deliberate act, carried out with discernment and direct intent, aimed at harming the victim’s image, reputation, or dignity through the dissemination of misleading and falsified materials that create the appearance of originating directly from the victim or place the victim in scenarios that do not reflect factual, objective reality.
We include, by way of example, the use of deepfake technology to portray the President of Romania in a derogatory video, the accuracy of which comes close to mimicking a real video and speech[7] . We can easily dismiss such content when we assume it is fake and when its very title makes it clear that it was generated by AI software; however, without these assumptions and given the continuous development and refinement of such software, how much longer will we truly be able to distinguish it?
In the form adopted by the Senate, the law provides four articles, including the definition of its scope of application, the definition of the deepfake phenomenon, the conditions under which AI-generated material must disclose its nature, and who is responsible for verifying compliance with these legal requirements.
The law defines the “deepfake” phenomenon as “any falsified image, audio, and/or video content created, as a rule, using artificial intelligence, virtual reality, augmented reality, or other means, such that it creates the appearance that a person has said or done things to which they have not given their consent, and which in reality were not said or done by that person.”
The distribution and broadcast of such deepfake materials in the media are prohibited unless they are accompanied by a warning displayed on at least 10% of the screen area and throughout the entire duration of the broadcast of the visual content, or by an audio message at the beginning and end of the audio content: “This material contains fictional scenes.”
- Undermining the credibility of previously indisputable evidence
What disservice are we truly doing to ourselves by perfecting generative artificial intelligence systems? In terms of evidence, at least until now, imagery (photos or videos) as well as audio and audio-video recordings have been—in both criminal and civil trials—indisputable evidence, evidence whose authenticity was rarely contested, evidence on which we could rely without question.
However, in this surreal context where:
- we can create an omniscient person—who appears in multiple places simultaneously in surveillance footage or photographs;
- we can portray an individual in a place, circumstance, or context in which they have never been;
- we can generate false testimonies, confessions, and statements—attributed to a person who never made them;
- we can manipulate or generate/fabricate evidence—e., alter the content of surveillance footage or images from searches;
- we can create events through photos or videos that simulate an event that never happened—which is important for linking persons of interest to an alleged event;
- We can defame and discredit any individual by spreading deepfake content that depicts them engaging in illegal, scandalous, defamatory, and—most likely—false behavior, ultimately damaging their reputation;
- We can discredit witnesses, either by creating content that contradicts their statements or by placing them in other locations in time and space that would make it impossible for them to have been present as witnesses to the events they are testifying about;
- we can easily clone and steal identities for the purpose of committing fraud (cyber, banking, financial) by impersonating a real person whom people trust and to whom they will provide personal data;
- we can easily violate people’s privacy and harass them online by creating fake profiles with deepfake content that portrays the person in question in unrealistic situations capable of damaging their reputation;
- we can forge documents by cloning logos and stamps, artificially imitating handwriting styles, or simply making credible alterations to only certain parts of documents, which can ultimately have financial or even criminal implications.
Although such content could be generated in the past, it was far more rudimentary than content developed by artificial intelligence—which is constantly refining its work to perfectly mimic the human element—and the programs were not readily accessible to the general public. Now, with a single Google search—accessible to anyone (including minors)—we can find dozens of software programs available for free or at a minimal cost, where even the most basic and accessible software can generate credible content.
The proliferation of deepfake content is constantly on the rise; the software already available is becoming increasingly sophisticated and user-friendly, making it easy for anyone to generate, in under 30 minutes, content capable of causing serious reputational damage to the chosen victim. Furthermore, unlike news platforms or traditional media, social media platforms do not filter their content through human verification, but rather through bots that are, unfortunately, far too easy to fool.
Social media algorithms can quickly spread (sometimes in a matter of a few hours) deepfake material to tens or hundreds of thousands of users, and the support provided by these platforms is slow, with minimal chance of reaching a human representative beyond the support bots. Thus, although the content may be removed after numerous appeals and reports—a process that can take days or even weeks—the deepfake has time to circulate, linger, cause outrage, and quietly poison public opinion, without the pressure of a ticking clock.
Finally, in the context of generative artificial intelligence, which floats freely online and is accessible to anyone who wants it, in the complete absence of a legal framework governing AI tools that would impose limitations or clarify who is actually liable for the harm caused by such content (especially when the author cannot be identified), data protection is expanding its scope to include databases that can access your personal images (such as Facebook, Instagram, TikTok, Twitter, WhatsApp, etc.).
[1] Deepfakes from the Gaza War Fuel Fears About AI’s Ability to Mislead | AP News
[2] TikTok Struggles to Remove Deepfake Videos of Hamas Victims (bloomberglaw.com)
“They follow a similar pattern: a tragic death appears in the news, and within a few days or even hours, users post videos featuring a lookalike of that person, recounting how they died. The format of this trend typically includes an introduction from that person’s perspective and a doctored image of them on screen, telling the story of how they died.”
[3] Iranian Hackers Interrupt UAE Broadcasts With Deepfake News (voanews.com)
[4] OECD AI Policy Observatory Portal
[5] Taylor Swift deepfakes nudge the EU to get real about AI – POLITICO
[6] EU AI Law: The First Regulation on Artificial Intelligence | Topics | European Parliament (europa.eu)
[7] Iohannis: Do You Want to Be a Billionaire? [deepfake] (youtube.com)