VASA-1: Lifelike audio-driven talking faces generated in real time

On the other hand, if deepfaking becomes common enough that everyone stops trusting everything they read / see on the internet, it would be a net good against the spread of disinformation compared to today.

Because the text for this is only a slight variation of the tech for a broad range of legitimate applications?

Because even this precise tech has legitimate use cases?

The only purpose of this technology I can think of is getting spies to abuse others.

Can you really not think of any other use cases?

I get the feeling it's "someone's going to do this, so it might as well be us."

It's fascinating how research can take on a life of its own and will be pushed, by someone, to its own conclusion. Even for immensely destructive technologies (e.g., atomic weapons, viruses), the impact of a technology is its own attractor (could you say that's risk-seeking behavior?)

Am I going to have to do AuthN and AuthZ on every phone call and zoom now?

"Alexa, I need an alibi for yesterday at noon."

Newscasters and other talking heads will be out of business. Just pipe the script into some AI and get video.

We all know why this is really happening. Clippy 2.0.

Why is this research being done?

I think it's mostly "because it can be done". These types of impressive demos have become relatively low hanging fruit in terms of how modern machine learning can be applied.

One could imagine commercial applications (VR, virtual "try before you buy", etc), but things like this can also be a flex by the AI labs, or a PhD student wanting to write a paper.

Advertising. Now you and your friends star in the streaming commercials and digital billboards near you! (whether you want to or not)

Video games, entertainment, and avatars seems like the big ones.

State disinformation and propaganda campaigns.

Imagine being the CEO and you just grab your salary and options, go home, sit in the hot tub while one of the interns carefully prompts GPT and VASE how you are giving a speech online about strategic directions. /s

Apple Vision Pro personas competition

The purpose is to give remote workers the ability to clone themselves and automate their many jobs. /s

(but actually, because laziness is the driver of all innovation, I wouldn't be surprised if this happens).

In this case, replacing humans in service jobs. From the paper:

"Such technology holds the promise of enriching digital communication, increasing accessibility for those with communicative impairments, transforming education methods with interactive AI tutoring, and providing therapeutic support and social interaction in healthcare."

A convincing simulacrum of empathy could plausibly be the most profitable product since oil.

Entertainment maybe? I know that's not necessarily an ethical reason but some have made hilarious AI-generated songs already.

If beautiful people have an advantage in the job market, maybe people will use deepfake technology when doing zoom interviews? Maybe they will use it to alter their accent?

Propaganda, political manipulation, narrative nudging, regular scams and advertising.

Even though most of those things are illegal you could just have foreign cat's paw firms do it. Maybe you fire them for "going to far" after the damage is done, assuming some even manages to connect the dots.

One the surface, it's a simple, understandable demo for the masses. While at the same time, it hints at deeper commercial usage.

Disney has been using digital likeness to maintain characters who's actors/actresses have died. Princess Leia is the most prominent example. Arguably, there is significant realistic value in being able to generate a human-like character that doesn't have to be recast. That character can be any age, at any time, and look exactly like the actor/actress.

For actors/actresses, I suspect many of them will start licensing their image/likeness as they look to wind down their careers. It gives them on-going income with very little effort.

We already rely on chains of trust going back to the original source, and will still. I find these alarmist posts a bit mystifying – before photography, anyone could fake a quote of anyone, and human civilisation got quite far. We had a bit over a hundred years where phographic-quality images were possible and very hard to fake (which did and still does vary with technology), but clearly now we’re past that. We’ll manage!

People in my circles have been saying this for a few years now, and we've yet to see it happen.

I've got my popcorn ready.

But you can rest easy. Everyone just votes for the candidate their party picked, anyway.

Today a big ML model can do this

Not that big:

https://github.com/Zejun-Yang/AniPortrait

https://huggingface.co/ZJYang/AniPortrait/tree/main

How will we vote for national candidates if nobody knows what they think or say?

i’m going to burst your bubble here, but most voters have no idea about policies or candidates. most voters vote based on inertia or minimal cues, not on policies or candidates.

i suggest you look up “The American Voter”, “The Democratic Dilemma: Can Citizens Learn What They Need to Know?” and “American National Election Studies”.

Hyper targeted placement of generated content designed to entice you to donate to political campaigns and to vote. Perhaps leading to a point where entire video clips are generated for a single viewer. Politicians and political commentators will lease their likeness and voice out for targeted messaging to be generated using their likeness. Less reputable platforms will allow disinformation campaigns to spread.

People already believe any quote you slap on a JPEG.

Anyone have any good ideas for how we're going to do politics now?

If a business is showing a demo of this you can be assured that the Government already has this tech and has for a period of time.

How will we vote for national candidates if nobody knows what they think or say?

You don't know what they think or say now - hopefully this disabuses people of this notion.

DNS? Might be that we need a radical (for some) change of viewpoint.

Just as there's no privacy on the internet, how about 'theres very little trust on the internet'. Assume everything not securely signed by a trusted party is false.

No, we are merely returning to the pre-photography state of things where a mere printed image is not sufficient evidence for anything.

I expect this type of system to be implemented in my lifetime. It will allow whistleblowers and investigative sources to be discredited or tracked down and persecuted.

I think it's practically impossible for such a system to be globally trustworthy due to the practical inevitability of "but improper storage of private keys lead to vulnerabilities" scenarios.

People will expect or require that chain of custody only if all or at least the vast majority of the content they want would have that chain of custody.

Photo/video content will have that chain of custody only if all or almost all of devices recording that content will support it - including all the cheapest mass-produced devices in reasonably widespread use anywhere in the world.

And that chain of custody provides the benefit only if literally 100% of these manufacturers have their private keys secure 100% of the time, which is simply not happening; at least one such key will leak, if not unintentionally then intentionally for some intelligence agency who wants to fake content.

And what do you do once you see a leak of the private keys used for signing the certificates for the private keys securely embedded in (for example) all of 2029 Huawei smartphones, which could be like 200 million phones? The users won't replace their phones just because of that, and you'll have all these users making content - so everyone will have to choose to either auto-block and discard everything from all those 200 million users, or permit content with a potentially fake chain of custody; and I'm totally certain that most people will prefer the latter.

Is every single executive and manager at my credit card company completely unaware that right now anyone can clone anyone else's voice by obtaining a short sample audio clip taken from any social network?

Your mistake is assuming the company cares. The "company" is a hundred different disjointed departments that only care about not getting caught Equifax-style (or filing for bankruptcy if caught). If the marketing director sees a shiny new thing that might boost some random KPI they may not really care about security.

However in the rare chance that your bank is actually half decent, I'd suggest contacting their IT/Security teams about your concerns. Maybe you'll save some folks from getting scammed?

Any time you add a "new" security gate to your product, it should be in addition to and not instead of the existing gates. Biometrics should not replace username/password, they should be in addition to. Security Questions like "What was your first pet's name" should not be able to get you in the backdoor. SMS verification alone should not allow you to reset your password. Same with this voice authentication stuff. It should be another layer, not a replacement of your actual credentials.

If you treat it as OR instead of AND, then your security is only as good as the worst link in the chain.

I mean, what do you want them to do? If we think their security officers are freaking out and holding meetings right now about what to do, or if they're asleep at the wheel, we'd be seeing the same thing from the outside, no?

That scene from Sneakers would be so different nowadays. [1]

"My voice is my passport. Verify me." [2]

1. https://youtu.be/WdcIqFOc2UE?si=Df3DtSakatp9eD0L

2. https://youtu.be/-zVgWpVXb64?si=yT2GZpb7E2yZoEYl

Yes, they are and they also know it isn't foolproof so that isn't the only information being compared against. Some services compare the calling number is compared against live activity on the PSTN (ie, subscriber's phone is not in an active call, but their number is being presented as as the caller ID is one such metric). Many of these deep fake generators with public access have watermarks in the audio. The audio stream comparison continues, it needs to speak like you, word and phrase choices. There are other fingerprints of generated audio that you can't hear, but are still obvious at the moment. With security, it always cat and mouse with fraudsters on one hand and the effort/frustration with customers on the other.

Asking customers questions that they don't remember and that fraudsters have in front of them isn't working and the time it takes for agents to authenticate is very expensive.

While there is no doubt that companies will screw up with security, you are making wild accusations without reference to any evidence.

until we are certain that the technology will be used responsibly ...

That's basically "never" then, so we'll see how long they hold out.

Scammers are already using the existing voice/image/video generation apparently fairly successfully. :(

money will change that

/s it doesn’t have the phrase LLM in the title

Translation: "We're attempting to preserve our moat, and this is the correct PR blurb. We'll release an API once we're far enough ahead and extracted enough money."

Like somebody on Ars noted "anybody notice it's an election year?" You don't need to release an API, all online videos are now suspicious authenticity. Somebody make a video of Trump or Biden's eyes following the mouse cursor around. Real videos turned into fake videos.

I wonder how video evidence in court is going to be affected by this. Both from a defense and prosecution perspective.

Technically videos could've been faked before but it would require a ton of effort and skill that no average person would have.

I still find the faces themselves to be really obviously wrong. The sound is just off, close enough to tell who is being imitated but not particularly good.

counterpoint: we don't need any more legislation

Legislation only impairs the good guys.

In which jurisdiction?

This is good but nowhere as good as EMO https://humanaigc.github.io/emote-portrait-alive/ (https://news.ycombinator.com/item?id=39533326)

This one has too much movement and looks eerie/robotic/uncanny valley. While EMO looks just perfect.

Such an exciting startup idea! I'm thrilled!

As if this technology was needed.

Yeah, even if they just use LLMs to do all the work, or are a LLM themselves, as long as they can do the work I guess.

Weird implications for various regulations though.

It would be interesting that a remote candidate could easily identify as whatever ethnicity, age or even gender they consider most beneficial for hiring to avoid discrimination or fit certain diversity incentives.

Tech like this has the potential to bring us back to the days of "on the Internet, nobody know's you're a dog" https://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_...

yeah, teeth, tongue movement and lack of tongue shape and the "stretching" of the skin around the cheeks in the images pushed the videos right into the uncanny valley for me.

The teeth resizing dynamically is incredibly distracting, or more positively, a nice way to identify fakes. For now.

Another research project with 0 model release

the busted two scrolling sections on mobile really doesnt help

It really is something. 40 FPS on a 4090, damn.

Now I can join the meeting "in a suit" while being out paddleboarding!

I find the hairs to be the least realistic, they look elastic, which is unsurprising: highly detailed things like hairs are hard to simulate with good fidelity.