The Deepfake Dilemma Protecting Democracy In The AI Era

The Rising Tide of Misinformation

According to most surveys, about 1 in 5 Americans dismiss or deny the impacts of global climate change. Towards the end of the global COVID pandemic, 1 in 5 Americans believed that Bill Gates planned to use COVID-19 to implement a mandatory vaccine program with microchips for tracking people. Furthermore, after Joe Biden won the 2020 presidential election, more than half of Republican voters believed Donald Trump was the rightful winner.

Basic facts concerning our planet's health, public health, and the very foundations of our democracy are being denied by a significant portion of citizens. This widespread alternate reality is not confined to the United States; this plague of lies, conspiracies, misinformation, and disinformation is a global phenomenon.

Much of the belief in these and other baseless claims is spread through traditional channels and social media, amplified by well-known personalities, influencers, politicians, and some billionaires. What will happen when the environment that has allowed these dangerous conspiracies to flourish is super-charged by generative AI?

Understanding Deepfakes The Technology Behind Deception

Before the more neutral term "generative AI" became common, AI-generated content was known as "deepfakes." This term originated from the username of a Reddit user in 2017 who used this emerging technology to create non-consensual intimate imagery (NCII), often mislabeled as "revenge porn."

Today, generative AI can create hyper-realistic images, voices, and videos of people saying or doing almost anything. These technologies promise to revolutionize many industries while also dangerously accelerating the spread and belief in harmful lies and conspiracies.

Text-to-image AI models are trained on billions of images with descriptive captions. They learn by progressively corrupting each training image until only visual noise remains, and then learn to reverse this process to denoise the image. Once trained, these models can generate an image consistent with any text prompt, such as "Please generate an image of the great Egyptian Sphinx and pyramids during a snowstorm." (A side note from AI researcher Sarah Barrington: always be polite to AI models, just in case they take over the world.)

Figure 1: An AI-generated image of the Sphinx and pyramids during a snowstorm.

Video deepfakes generally fall into two categories: text-to-video and impersonation. Text-to-video deepfakes are an extension of text-to-image models, trained to generate videos from text prompts. These have become much more convincing in the last year. A year ago, prompts like "Will Smith eating spaghetti" produced obviously fake videos. You can see an example of this older technology in this video.

Today's videos, while not perfect, are stunningly realistic and temporally consistent, quickly becoming hard to distinguish from reality. The updated version of Will Smith enjoying spaghetti demonstrates this progress, viewable in this newer video.

Impersonation deepfakes include popular types like lip-syncs and face-swaps. Given a source video of someone speaking and a new audio track (AI-generated or impersonated), a lip-sync deepfake creates a new video where the person's mouth movements match the new audio. Since it is relatively easy to clone a person's voice from just 30 seconds of audio, lip-sync deepfakes are commonly used to hijack identities of celebrities or politicians for scams and disinformation.

A face-swap deepfake replaces one person's identity (from eyebrows to chin) with another in a video. This is most common in creating non-consensual intimate imagery. Face-swaps can also be done in real-time, meaning you might soon not know if the person in a video call is real.

The trend over the past few years shows all forms of image, audio, and video deepfakes continuing to improve rapidly in realism, ease of use, accessibility, and weaponization.

Deepfakes in Action The 2024 US Election Case Study

It is hard to quantify how much deepfakes impacted the 2024 U.S. presidential elections. However, there is no doubt that deepfakes were present in many forms. Regardless of their impact, their use in this election is a warning for future elections worldwide.

Deepfakes in the election ranged from direct attempts at voter suppression to disinformation campaigns meant to confuse voters or cast doubt on the election's outcome.

For example, in January 2024, tens of thousands of Democratic voters received a robocall with an AI-generated voice of President Biden telling them not to vote in the New Hampshire primaries. The perpetrators were a political consultant, a magician paid $150 to create the fake audio using a platform called ElevenLabs (which offers voice cloning for as little as $5 a month), and a telecommunications company.

Throughout the campaign, viral AI-generated images of Black people supporting Donald Trump garnered millions of views on social media. Cliff Albright of Black Voters Matter stated these images pushed a "strategic narrative" to show Trump's popularity in the Black community, noting, "There have been documented attempts to target disinformation to black communities again, especially younger black voters," as reported by the BBC.

To cast doubt on election fairness, numerous fake videos, traced back to Russia, circulated, supposedly showing an election official destroying ballots for Trump. An endless stream of viral AI-generated images and videos polluted social media, from fake images promoting a socialist/communist narrative about Kamala Harris to a fake image of Taylor Swift endorsing Donald Trump.

The Dangerous Power of the Liar's Dividend

While deepfakes are already a threat, a more insidious outcome is that when anything we see or hear could be fake, then nothing has to be real. In the deepfake era, a liar can both spread lies and, by invoking the specter of deepfakes, cast doubt on inconvenient truths - this is the so-called liar's dividend.

Trump, for instance, publicly accused the Harris-Walz campaign of posting AI-generated images of large rally crowds. This claim was baseless. Denying crowd size might seem petty, but it could be part of a more nefarious strategy. Trump had stated he would deny election results if he lost, so disputing crowd sizes pre-election could give him ammunition to claim voter fraud later. The January 6 insurrection following the 2020 election showed the high stakes for democracy. As deepfakes improve, wielding the liar's dividend will become easier.

Figure 2: An authentic photo of a Harris-Walz rally that, during the 2024 U.S. national election, Donald Trump claimed was fake.

Strategies to Protect Democracy from Deepfakes

If the past two decades of technological revolution (and its disastrous outcomes like privacy invasions and toxic social media) have taught us anything, it is that things will not end well if we ignore or downplay the malicious uses of generative AI and deepfakes.

Reasonable and proportional interventions, from creation through distribution, and across academia, government, and the private sector, are necessary and beneficial for everyone in the long term. Here are some practical interventions that can keep us safe while allowing innovation to flourish.

At the Point of Creation

Content online has three main phases: creation, distribution, and consumption. The Coalition for Content Provenance and Authentication (C2PA) is a multi-stakeholder, open-source initiative to establish trust in digital media. C2PA has created standards to ensure authenticity and provenance at the point of recording or creation. This includes adding metadata, embedding an imperceptible watermark, and extracting a distinct digital signature for identification even if credentials are stripped. All AI services should be required to implement this standard to make identifying AI-generated content easier.

Managing Distribution Channels

Social media platforms need to take more responsibility for their role in sharing content, from unlawful to lawful-but-awful items amplified by their recommendation algorithms. However, they are not solely culpable. Social media operates within a larger online ecosystem powered by advertisers, financial services, and hosting/network services. These often hidden institutions must also take responsibility for how their services enable online harms.

Empowering Consumption Through Education

When discussing deepfakes, the most common question is: "What can the average consumer do to distinguish real from fake?" The answer is always: "Very little." Artifacts in today's deepfakes - like seven fingers or incoherent text - will be gone tomorrow, and instructions on spotting them provide a false sense of security. Generative AI is moving too fast, and forensic examination is too complex for the average consumer to be an armchair detective. Instead, we need a massive investment in primary and secondary education to empower consumers with skills to understand how and where to get reliable news and information.

Authenticating Digital Content

Identifying manipulated content by qualified experts is divided into active and reactive approaches. Active approaches include C2PA content credentials. Reactive techniques operate without such credentials. There are many techniques for detecting manipulated or AI-generated content. These can be effective, but a major limitation is that by the time malicious content is uploaded, flagged, analyzed, and fact-checked, it may have already received millions of views. This makes such authentication suitable for post-mortems but not for the billions of daily uploads.

The Role of Legislation

So far, only a few nations and some U.S. states have moved to mitigate deepfake harms. While individual state efforts are commendable, internet regulation needs more than a patchwork of local laws. A coordinated national and international effort is required. The European Union's Digital Safety Act, the United Kingdom's Online Safety Act, and Australia's Online Safety Act offer roadmaps for the United States. Global regulation will not be easy, but common ground can surely be found among the U.S. and its allies, serving as a template for other nations.

Academic Responsibility

In the 1993 movie Jurassic Park, Jeff Goldblum's character, Dr. Ian Malcolm, criticized reckless technological advancement without ethical consideration: "Your scientists were so preoccupied with whether they could, they didn't stop to think if they should." This sentiment is one all scientists should absorb.

Many current generative-AI systems used for harmful content derive from academic research. For example, UC Berkeley researchers developed pix2pix, which transforms image appearances. Shortly after its release, this open-source software was used to create DeepNude, software that transforms an image of a clothed woman into an unclothed one. The pix2pix creators could and should have foreseen this weaponization and deployed their software more carefully. This was not the first such case, nor will it be the last. Researchers need to consider how to develop technologies safely, and sometimes, whether they should be created at all.

The Path Forward Navigating the Deepfake Challenge

There is much to be excited about in this latest technology wave. But past technology waves have taught us that, left unchecked, technology will work against us and our democracy. We need not repeat past mistakes. We are nearing a fork in the road regarding technology's role in our future. If we maintain the status quo, technology will continue to be weaponized against individuals, societies, and democracies. However, changes in corporate accountability, regulation, liability, and education can yield a world where technology and AI work with and for us.

Jordan Peele's 2018 public service announcement on fake news and deepfakes offers relevant advice. The PSA ends with a Peele-controlled President Obama stating: "How we move forward in the age of information is gonna be the difference between whether we survive or whether we become some kind of f***ed up dystopia." I couldn't agree more.