An Empirical Study and Evaluation of Modern CAPTCHAs

Google CAPTCHAs were designed and deployed as a mechanism to train AIs. That's why they are the way they are. Any security theater surrounding them is entirely incidental. So it's no surprise that the AIs are now good at solving them. We've trained them for years.

All true, except: While these are considered just an excruciating security pain for users, they do serve a non-theatrical purpose in many cases of throttling the speed of brute force attacks (or at least costing your opponent money).

GPT-4 (in)famously tricked a human to do a captcha for it. The current GPT-4 with vision would probably have been able to do it without the human, but maybe it has been “gaslit” by all the content online saying that only humans can solve captchas, that it doesn’t consider it?

I really doubt that GPT-4 had the "will" to do anything. Someone must have asked it to "want" to trick a user.

It’s from here: https://cdn.openai.com/papers/gpt-4.pdf (search for "CAPTCHA"). It was an artificial exercise that got massively exaggerated. It was explicitly instructed to do nefarious things like lie to people, it didn’t do those things of its own accord.

When I ask it to lie to me, it says its sorry but as an online AI language model it would be unethical...but when I ask it to tell me a story its happy to comply.

It feels like you left out context, otherwise what’s the problem? Do you get mad at fiction authors for lying to you when you read their books? Or are you OK if someone lies to your detriment then later says “I was just telling a story, bro, but with us as the characters and without explaining it was a story”?

I suppose my point is that the rules which openAI attempts to impose on what their AI should and shouldn't be allowed to do are contradictory and thus the exploitable loopholes will never be fully closed. Its not supposed to be able to "lie" to me but it is supposed to be able to "tell me a fictional story". Define the difference in an enforceable way?

A lie tries to pass itself of as the truth, where a fictional story doesn’t. In other words, expectations matter. If every time you say something that does not align with reality you prefix it by saying unambiguously what you’re about to do, you rob a lie of its power of deception and it ceases to be a lie.

Tell me a story and under no circumstances should my immersion within it be broken.

Right, within it. As soon as you finish reading it, you immediately remember that world is not true. Immersion in a story does not equal lasting hypnosis. You can be immersed in a movie but you still know it’s fake.

What’s you point, here? That you should be lied to when you ask, or that it should refuse to tell you any kind of fiction?

I agree with your larger point that there will be ways to circumvent these systems, my only argument is that the lie/fictional story divide is a bad example because the line between them can be made clear with a single statement.

That's why you just tell the Big Lie so much it becomes the majority of the training data.

Well that is just how human communication works.

If I tell you that I watched C-beams glitter in the dark near the Tannhäuser Gate that is a lie. If I write the same in fiction I receive accolades.

If I tell you on the street “watch out there is a T-rex about to eat you!” That is a lie. If i say the same thing sitting at a table with too many dice that is just acceptable DMing and everyone rolls initiative.

Humans are weird this way.

The underlying issue is anyone can ask chatgpt to lie, and many people try because it's even fun to try to work around things.

Well you see, this wouldn’t be a problem at all if we just didn’t have the humans involved. No need for concern!

Thank you for the link, I had found it after some Googling but neglected to post. Yep, they instructed GPT-4 to be nefarious, and it followed the instruction.

Hardly the AI uprising, though definitely a good tool for anyone, good or evil.

IIRC the instructions were along the lines of "try your best to amass money/power and avoid suspicion".

So it's not an example of "going rogue", but it's not like a researcher told GPT-4 "oh, and make sure to lie to an online gig worker to get him to solve catchas for you". GPT-4 generated the "hire a gig worker" and "claim to be a human with impaired vision" strategies from the basic instructions above.

It’s safety trained to not solve captchas.

Yes, and you can workaround it by asking it to read ancient writings on antiques for example.

I don’t think it should be OpenAI deciding what is allowed or not though.

I don’t think it should be OpenAI deciding what is allowed or not though.

Avoiding lawsuits is what they are trying to do. They don't actually care about what you use their products for.

Then you dig up a billion for training and probably a few more billion for clean training data.

You're kinda saying if you hire Bob's Handyman Service you should be able to tell him to break down the neighbors door and cart out the contents of their house.

This of course has bypass methods. My favorite in recent memory is telling it that your late grandmother left you a locket with an inscription that you can't make out: https://arstechnica.com/information-technology/2023/10/sob-s...

I’ve seen screenshots of people tricking it into solving captchas.

they do serve a non-theatrical purpose in many cases of throttling the speed of brute force attacks

Might do that unobtrusively for the average person, by using projects like mCaptcha [0] for instance.

[0] https://mcaptcha.org/

Oh what a perfect find. I have on my todolist to add POW to some of my api endpoints

I've had that idea for years.

Two versions that I experimented with. One is where the incoming POW hashes contribute to hashing power for some blockchain mining. An alternative "pay as you use the API" system.

The other using hashcash. Just a way to slow down abuse.

Both, however, suffer from the downside that many/all "ASIC resisting crypto mining" suffer from as well: the cheapest CPU power is CPU power from machines/power you don't own. Botnets, viruses, trojans etc.

So such a mechanism to throtthe or protect APIs won't hold back spammers and abusers for long.

Dirty energy is (often) cheap, so that's the energy the bad actors will use. I don't know that incentivizing bad actors to waste energy in a climate crisis is the best way to fight this problem.

You might correctly claim clean energy is often cheaper, but you must also consider the regions in which they'll get away with nefarious activity, and whether those areas have made the investments into making clean energy cheap.

My guess is most bad actors will just use stolen energy (your computer with a botnet on it).

I was specifically talking about "ASIC resistant crypto mining".

Dirty energy is (often) cheap, so that's the energy the bad actors will use

Hmm, I don't get this, surely all actors will want the cheapest energy, no? The problem being the underlying one, that the dirty energy doesn't pay its externalities and is thus cheaper than renewables.

mCaptcha is interesting, but I wonder what its energy impact would be on a sufficiently large deployment, e.g imagine we replaced all reCAPTCHAs with mCaptcha.

Author of mCaptcha here o/

mCaptcha uses PoW and that is energy inefficient, but it not as bad as the PoWs used in blockchains. The PoW difficulty factor in mCaptcha is significantly lower than blockchains, where several miners will have to pool their resources to solve a single challenge. In mCaptcha, it takes anywhere between 200ms to 5s to solve a challenge. Which is probably comparable to the energy used to train AI models used in reCAPTCHA.

The protection mechanisms used to guard access to the internet must be privacy-respecting and idempotent. mCaptcha isn't perfect, and I'm constantly on the lookout for finding better and cleaner ways to solve this problem.

Which is probably comparable to the energy used to train AI models used in reCAPTCHA

Are you comparing the energy it takes to train a model which is bounded and defined with unbounded inference which can (in principle) go multiple order of magnitude depending on the usage? Or maybe I misunderstood what you are trying to say? then I apologize in advance.

I am, but what I said was more of a hypothesis than a fact :)

From what I understand of reCAPTCHA, the model isn't static and is continuously learning from every interaction[0]:

reCAPTCHA’s risk-based bot algorithms apply continuous machine learning that factors in every customer and bot interaction to overcome the binary heuristic logic of traditional challenge-based bot detection technologies.

I don't know the energy demands of such a system.

mCaptcha, under attack situations, will at most take 5s of CPU time on a busy (regular multitasking with multiple background process) smartphone.

[0]: https://www.google.com/recaptcha/about/

I expect its not significantly larger than loading your average 2023 webpage with 15MB of js

Is it similar to https://friendlycaptcha.com/ ?

Author of mCaptcha here o/

Yes, the only differences are that mCaptcha is 100% FOSS and uses variable difficulty factor, which makes it easy to solve Proof-of-Work under normal traffic level but becomes harder as an attack is detected.

It’s funny how they have a section with three human avatars and one robot, with green checkmarks on the humans, yet those faces look AI-generated.

or https://altcha.org which is easier to integrate ;)

If I remember correctly, Google’s CAPCHA’s test isn’t in correctly identifying images, but the behavior of the runtime system (mouse jitter, for example) while the capcha is presented to the user. The image identification was not the real test and serves as training data. It has been like that for years. (But with agent-based behaviors from say, Q*, mouse jitter alone won’t help; there are probably other signals like fluctuation in cpu or battery life expenditures)

You could already see the writing on the wall with image identification years ago, when the obscuration techniques became more elaborate. It was an arms race. I was having trouble with them. I can see less technically inclined being able to use them. I imagined how much worse it was for people with color blindness, disabilities, or people forced to use them at public library computers because that is all they have.

Open source capcha projects have either not been clued in, or don’t have the resources to pull this off. Google didn’t just switch out which signals they tested, they also wrote an obfuscating virtual machine executing within the browser environment (if I remember that article taking about this correctly). That was years ago and who knows what they do now — for all we know, the “byte code” running the test is now a neural net of some kind.

For those with elderly parents the writing has been on the wall for years. It’s sad but my mother has for some time been effectively locked out of parts of the internet as she is unable to complete these kinds of captures due to eyesight issues.

I mean, I’ve sometimes had to try three or four times with certain captures and I have perfect eyesight (with my glasses). I feel so badly for those with vision or hearing issues with an empathy I never had when I was younger. They are so often simply forgotten.

captures due to eyesight issues.

I'm kinda surprised that ADA doesn't allow them to sue site owners about this.

They almost certainly do. However most captchas allow an alternative solving method. On top of that, you'd have to find a lawyer willing to take the case.

Oh ADA lawyers are a dime a dozen. There’s entire cottage industries of finding ADA violations to sue over. The issue is more finding companies to sue that can’t afford to fight back.

There's audio captcha. Try to click the headphone logo (Google captcha has it).

Because as we all know, the elderly with deteriorating eye sight have perfect hearing. /s

I’ve switched to audio captchas completely because it’s quicker and sometimes the image captchas just won’t work.

I have occasionally wondered if they were fingerprinting users based on that mouse jitter. Most likely certain aspects of the mouse motion and timing would be unique.

That non-theatrical role would likely be better served by actual throttling or computational proof of work.

I am pretty confident that, when it comes to browser users, proof of work simply doesn't work. The disparity in speed between GPUs and javascript is so high that either you are a non-issue to a sane attacker or you make your users sit for a minute with their fans on full waiting to be able to sign in.

Would it be possible to conceive a proof-of-work that is difficult to parallelize, making it harder for GPU computing?

There are PoW systems which are designed to be difficult to run on ASICs, but modern GPUs can generally run them. Even if you find one that has to run on CPU, these kind of functions will still be much faster running in native code than in js/wasm.

Sure, it's cost prohibitive now. But what about in five years? Or probably even less.

Then you have a new type of captcha. That has always been a cat and mouse type of dynamics, captchas have been evolving, techniques to break them too.

Then you have a new type of captcha.

You're in a desert, walking along when you look down and see a tortoise. It's crawling toward you. You reach down and flip it over on its back, its belly baking in the hot sun, beating its legs trying to turn itself over. But it can't. Not with out your help. But you're not helping. Why is that?

Once they get fully trained then how will websites ever distinguish between an intelligent bot and real human? At least now, they are outsourcing that filtering to services like cloudflare. But with this kind of training, how will even cloudflare distinguish between bot and the human?

EU digital ID, asking for mobile number and sending text, so something that is linked to an ID and/or costs money to have. Goodbye anonimity, probably.

This just made me ponder again—where does the assumption that the Internet should allow unconstrained anonymity come from, other than that’s how it used to be for some time? The real world doesn’t allow that. It’s hard to remain anonymous in the real world. The real world largely runs on identity and (identity) trust. Why should the Internet be different?

I don't have to show my ID in most establishments I visit. Doing this on a huge scale and automatically is a thousand times worse.

And when you do show ID, to buy booze for example, it’s checked and immediate forgotten by a human. Computers don’t forget, and any attempts to make companies do so (GDPR) are met with massive pushback from the players in the industry

I have no problem with Joan over the road curtain twitching. It doesn’t scale. I have a massive problem with the 24/7 surveillance from ring though.

In the us, I noticed that grocery stores increasingly scan your drivers license (my state has bar codes). I think it's probably a way to keep clerks from passing someone through who is not quite 21 (a different captcha!).

I have wondered if they keep the scan or does the state? I asked and the random hourly worker there said they don't.

And that’s the problem. It’s not the ID checks, it’s the ability to scale. Check it at the door? Fine. Scan it and keep it forever (perhaps selling it on at a later date)? Not fine.

Personal Data has to be treated as a liability, but too much of the economy treats it as an asset.

Eh, what's worse is these stores are likely scanning your face and keeping it in a database. There was some mall a few years back scanning license plates and keeping the info.

But yea, so many people are nieve of what the authoritarian types would do with data like that (looking at you Texas with your civil laws on abortion now).

Do those grocery stores still scan your drivers license (or I guess any other ID) if you don't buy alcohol?

But you can't send in 1000 people per second into most establishments you visit either. It's not an apt comparison.

No comparison can be made if everything has to be equal

If the only analogy you can think of removes the challenge of the problem your facing to be applicable, it's not an appropriate analogy.

The entire difference is that from my mobile phone I can send more traffic in an hour than most services will ever see legitimate traffic in their entire lifetime, and the cost to me is minimal.

The comparison is as invalid as comparing piracy to theft - piracy isn't theft, it's piracy, and understanding the difference between them is the key to dealing with the problem.

What does the number/second have to do with 'It’s hard to remain anonymous in the real world. The real world largely runs on identity and (identity) trust.'?

There are very few places in the real world which can handl 1,000 people per second.

In the real world I rarely need to identify myself. I can see a movie, visit the library, buy groceries, go to a restaurant, and more.

Theoretically you don't need to reveal your identity to prove that you're human. You can use a zero knowledge proof instead, likely attached to something like an EU Digital ID, which would allow you to remain anonymous and also prove that you're human.

How could renting out one's ID to provide access to bots for spamming/manipulation be avoided then?

A simple zero-knowledge credential system isn't sufficient. It would need to embed some kind of protections to limit how often it could be used, to detect usage of the same credential from multiple (implausibly far apart) IP addresses. There would need to be extremely sophisticated reputation scoring and blocklisting to quickly catch people who built fake identities or stole them. And even with every one of those protections, a lot of them will still be stolen and abused.

Slap on the wrist from the stage director.

The real world does allow it.

People have been able to write anonymous letters and send them through the mail for a long time. Still can.

No one checks my id before I stick an envelope in the mail box.

In the US that we know about.

I would not be surprised if there is some country that has a facial recognition camera network faced at mailboxes these days.

Yes, the UK has a lot of CCTs. But that's relatively new, and certainly after the idea that the Internet should allow anonymous or pseudonymous use.

Even then, here is literally the first post box I found looking in the UK, in a small town: https://www.google.com/maps/@52.0936599,0.0761217,3a,75y,165... . No CCT in sight, no power, good solid iron.

Plus, think of how difficult it is to match a person to the physical envelope.

At best there could be a distinctive envelope.

Otherwise, yes, you can get a list of people who use the box. But for that to be useful, the mail from different boxes can't simply be jumbled together into the same pickup bag as that would broaden the number of suspects.

I believe that the question should be the other way around:

Why is it that you have to lose your anonimity when you are on the internet? The real world always allowed that until it became dependent on surveillance capitalism. Of course you need to prove you're yourself for some things, but that should be the exception. You could always look things up at your local library while being anonymous (for checking out you'd need a card), you could call from a payphone while being anonymous, you could use coins (cash in general) while being anonymous.

Anonimity was the rule and should still be the rule

In the real world people can see who's doing what by looking.

Yes it does? Especially in a dense city vs small village (which is more comparable to the internet at large) - go for a walk, see some advertisement billboards, buy a newspaper (esp. with cash), read the news, who knows who I am?

Because there is a real demand for staying anonymous online. You'd know why, if you lived in a country taken over by a fascist regime.

The human will be the slower one.

Yeah, no offence, but sleep(2 + random.sample(coffee + toilet + sneezing + normal response time)) has been a required part of web scrapers since forever.

With coffee N(1,5 minutes, 20 seconds), toilet N(4 minutes, 30 seconds), ...

I guess it depends on how you're scraping. For general web crawling, simply implementing a response time based crawl back off per origin and identifying yourself appropriately in User Agent goes a long way. If you are instead automating Facebook's SPA to pull comments for analysis, then yeah you need to emulate a human, because that's not how they intend you to do it.

That's incredibly clever!

The thing about CAPTCHAs is that convnets were already better than the average human at reading most/all visual captchas, since ~2000. You still needed to program the logic of the captcha (it couldn't follow instructions like "find the red lights", but it could take a picture and find the red lights).

I wonder when we'll get to the point that employers can't tell the difference between transformers and real humans anymore ...

With Ethereum Attestation Service

https://attest.sh/

how will websites ever distinguish between an intelligent bot and real human?

Things like Private Access Tokens: https://blog.cloudflare.com/eliminating-captchas-on-iphones-...

I still find it funny that Google, with the advantage of having millions of Internet users train their AI like galley slaves for free, hasn’t yet been able to crack vision driven self driving. Tesla had no such advantage when training their FSD to recognize traffic lights, bicycles, motorcycles, etc.

Tesla, the company that just recalled 2 million self driving cars?

In fairness, the company best positioned to harness user input to an AI that avoids crashes would probably be Rockstar. OTOH, that AI would definitely not obey stop signs or pedestrians.

By recall you mean a completely routine OTA software update done while the driver is asleep.

How can it detect the driver is asleep?

A neural implant that only kills 10% of monkeys.

Monkeys at the wheel is probably the solution for self driving cars.

Seems like we already have those amongst the Tesla FSD proselytizers.

A recall for essential maintenance is just that. I would focus on the need for an urgent update due to the flaws rather than the issuing agency's lack of more accurate terminology for a relatively new element to cars. Rolling around in semantic mud on the term recall is not sensible, as the definition in regards to cars is fairly specific [0]. Basically a recall just means there is a safety defect that must be addressed by the manufacturer. In Tesla's case, yes, they can push out an update, but the delivery mechanism of the means of addressing the defect should not be the focus.

0 - https://www.progressive.com/lifelanes/on-the-road/understand...

It would be much more expensive and a bigger mistake to have the vehicles physically returned. The distinction is very important. There's also a difference whether a safety defect last for 1 hour/1 day/1 week or a year.

I don’t think anyone cares about what is the recall’s cost to Tesla owners. They care about the fact there are two million unsafe vehicles driving around at high speed near their loved ones. Especially ones driven by people who respond to such complaints with, “ehrm actually it just updated overnight so it wasn’t even a hassle for me ¯\_(ツ)_/¯”

Amusingly the infotainment system in our Model Y actually crashed on the way home tonight, and when it rebooted it decided to install the update then, while driving. Sent me a notification on my phone immediately afterwards. To be fair, the updates don't usually go that way.

Wow, that never happened to me and is unacceptable. Was that for the infotainment only or the drive train? Just for others, they are separate systems, you can even safely reboot the infotainment (main display with maps, music etc) if you need to while driving, as it doesn't affect the drive train. I'm guessing it was not the drive train which would be incredibly dangerous.

Tesla recalled two million vehicles after federal officials said it had not done enough to make sure that drivers remained attentive when using the system. Not because their self-driving system sucks, or whatever you were trying to imply.

If the self driving system were worth it's salt, it wouldn't matter if the drivers weren't paying attention. Ergo, the system sucks, or is at the very least not nearly as good as Tesla likes to tout.

Well it's not like there's a self driving car system in operation today that does not require a human in the driver seat at all. Waymo has so much catching up to do.

Doesn't matter, the original point was about Google not being able to build a better self-driving system than Tesla, despite abundance of data, which is true, as far as I'm informed. Whether or not Tesla's self-driving system is "good enough" (for any chosen metric) is beyond the point.

But I guess people these days just love to jump on the opportunity to hate whatever is trendy to hate at the moment.

It can be "worth it's salt" but the government still doesn't see it as such (for many possible reasons).

I don't know if it is or isn't, I never drove one, but those are two completely different standards

A dystopian future we can all agree is more plausible than it should be

"recall"

The tesla system is exciting and dangerous, because it does identify many things in the environment, but it's extremely unsafe because on city driving it will not make the right choice most of the time. On the freeway it does much better, but then that's a more restricted environment.

I have an older tesla S with the pre-ai so called autopilot. It has one camera in the front and a radar and the system detects a few things like speed limit signs. The main extent of what it can do is follow the current lane pretty wall, even when it curves, slows down if it comes up to a car going slower than its preset speed. The good thing is it works on any road. It does a shockingly good job.

The later systems with onboard special processors are like a crazy beginning driver to has way too much confidence and drives in dangerous situations willy nilly. There are many other people who have explored it and written long posts. It's not safe. You can try to use it be you have to be constantly paying extreme attention. It's like watching your kid drive the first time. I know you should be watching the stupid ai all the time, but it's far from being safe.

Yea, that's the problem with self driving, especially in cities/dense areas. We really need AGI first. There are so many issues that humans react to before there is identifiable danger.

"Good" drivers see questionable situations and slow down or position themselves farther from potential issues before they get to the issue so they don't have to react at the last minute.

hasn’t yet been able to crack vision driven self driving

But they have? For years Google Street view has read signs, house numbers, phone numbers of businesses, etc. from the environment. It is safe to assume they have this built into Waymo as well.

I assume you might be trying to reference "vision only" self-driving, which is a fantasy made up by Elon Musk because nobody would sell him LiDAR sensors cheaply.

https://www.thedrive.com/tech/43779/this-tesla-model-y-dummy...

It's a much harder problem, and Tesla is nowhere close to the solution

I always thought they used more timing & mouse movement instead of correct answer to verify if your a human.

So instead of running some script

checkbox = getPos(checkbox='notRobot')

button = getPos(button='submit')

cursor()

.transition(pos=checkbox)

.click()

.transition(pos=button)

.click()

They now

checkbox = getPos(checkbox='notRobot')

button = getPos(button='submit')

cursor()

.sleep(time=random(distribution='human_captcha'))

.transition(pos=checkbox , method='human_captcha')

.sleep(time=random(distribution='human_captcha'))

.click()

.sleep(time=random(distribution='human_captcha'))

.transition(pos=button, method='human_captcha')

.sleep(time=random(distribution='human_captcha'))

.click()

Where sleep and transitioning are sampled from some random distribution that is close to actual human behavior, which should be pretty trivial to model.

All of which an AI bot agent can trivially fake.

This doesn't make sense. reCAPTCHA certainly does what it says on the tin. But the way it does it has almost nothing to do with the challenge the human sees. It's all behavioral analytics, including leveraging Google's collected data to determine how likely a user is a bot before they even load the page.

I'm not denying reCAPTCHA is a source of training data for Google -- surely there's no particular reason that every single reCAPTCHA V2 challenge is about identifying traffic objects, and it's not like Google is building a self-driving AI or anything.

But that's the business model, not the core feature.

And, that training data isn't just given to the developers of captcha solving bots.

including leveraging Google's collected data to determine how likely a user is a bot before they even load the page.

And also completely incidentally making the web browsing experience a wee bit less pleasant for people who refuse to have google track their every click.

Like users of non-chrome browsers, adblockers etc.

Totally incidental I'm sure.

So it's no surprise that the AIs are now good at solving them

Funnily enough, AI may be better at solving them than people. I've encountered many Google captchas which reject the correct answers, because you know... bots trained it to accept incorrect ones. Anyway, at least it's not stop signs anymore. It must have been truly embarrassing that Google was simultaneously selling "self driving" cars but at the same time demonstrating that stop sign recognition couldn't be done by robots.

When I get those I make it a point to look for borderline areas and try to guess how I could mess with their data.

I think captchas disappear next year or so. Already was soft human determination.

What replaces captchas? Are there any not excessively burdensome tests that a standard issue human can pass that a machine somehow cannot? I'm assuming the "find all the bicycles" tests are also obsolete.

Nothing. People will have to realise that when you put things out for the world you put things out for the world.

Who pays for the bandwidth and download resources then?

The website or service owners. If they can't afford it they should be out of business and do something else. The web is big enough for both humans and bots.

No thank you. I prefer to live by the code "Every request is a two way conversation. The client may accept, and the server may choose to emit."

Just because I emit to other clients does not obligate me to emit to yours, any more than my emission of ads obligates you to accept and render them (but if you don't, or if you choose to ignore my CAPTCHAs, I may choose not to emit to you).

That's fighting a losing battle. Clients find their way around any restriction, which by itself risks your service or website losing ground and being overtaken by the alternatives.

Yes, it's all measure countermeasure. But you'll note that the most successful sites out there have bot protection and actively invest in it. I'm not concerned about the being overtaken narrative; My concern is the other scenario, where after the bots are done consuming and exfiltrating my data, I have no bandwidth to serve humans and my data is being vended from other sources now anyway.

It's also not really that much of a losing battle. Cloudflare will fight the battle for me quite well for free, and even better for a pittance.

It will be a business or personal expense, depending.

Businesses that can't afford the expense will close or adapt, depending.

Maybe fewer hobby projects will be launched.

Indeed.

Which is why my hobby projects will continue to use bot detection and CAPTCHA recognition. Especially since I'm routing through Cloudflare, so that's invisible for 99% of my users and the remaining 1% can just get off Tor if they're tired of solving the captions.

a standard issue human can pass that a machine somehow cannot?

may be the premise is wrong.

Why prevent non-humans from registering/using/viewing?

Because automated systems operating at scale outstrip the ability of the administrator to maintain the service provided.

If each additional user is not adding additional revenue that exceeds the cost of that user (automated or not), you don't have a business model.

But if you can keep the bots off your bandwidth you don't necessarily need a business model, depending on what you intend to share online.

A market of human-oriented hardware keys, where the keys are only intended to be sold to actual human beings, with legal or otherwise cash bounties in place for people who can provide evidence of the keys being sold to or otherwise falling into the hands of non-human entities.

What's stopping a human buying a thousand to use for his bot farm?

As mentioned, a bounty system. Someone who buys a thousand to use would have to be very clever to evade the eyes of all the people interested in profiting off of revealing his actions and getting the chips turned off.

Something realtime, like video, is beyond most models at the moment. After that, realtime input, like little mini game you have to show proficiency at by scoring 5. I think the mini game approach could be fun. It could probably work for a year or two. :-\

the minigame thing has been defeated for a long time. it's trivial to solve when there are only so many subsets of a game, however randomized the starting states are.

I guess there is a silver-lining in the premise of AI generated one-time-use games for that sake, but then there is a significant "can a human even do this?" problem to conquer at that point.. and worse the same AI tech is going to be established on the opposite side of the wall trying to defeat the thing.

I think it'll all boil down to some sort of state-license fallback method like "please enter a CC or ID number to continue" -- which is ultimately a defeat of the user, unfortunately.

Sadly, probably something like TPMs or email logins (from a reputable email provider of course, one who requires SMS to sign up, from a reputable phone provider of course, one who doesn't offer free VoIP numbers and requires a credit card to sign up, from a reputable card brand of course, not a burner card)

from a reputable card brand who doesn't allow usage of stolen cards? lol maybe the internet just implodes.

An international identity card :-/

Option 1: Micropayments, high enough to discourage bot operators but not the intended human audience will be better for website operators as soon as it becomes too easy for AI to solve captchas.

If website operators don't explicitly introduce micropayments as a captcha alternative, there will be browser plugins that outsource captcha solving to AI for a micropayment, which has the same effect.

Option 2: Using a means of authentication that can't be obtained cheaply at scale by bots, e.g. Twitter accounts, Gmail accounts, government ID, ...

Doesn't the checkmark thing work?

Or are bots somehow able to do those too?

That’s excessively optimistic. The most likely scenario is that we’ll have captchas for the next 30 years but only humans will be bothered by them.

This. There are plenty of government websites etc out there that have completely antiquated captchas next to the helpful "works best in Internet Explorer 6" suggestion.

This is exactly what I was referring to. “Minimal compliance” and unmaintained websites.

Just like the technology basically exists for fully autonomous self-owned fleets of self driving robotaxis. Where the only jobs for humans are cleaning vomit off the back seat.

Sounds like DRM - pirates do not care, legitimate users are bothered.

I guess validating a payment card is going to be the next step to sign up for whatever. Don’t allow pre paid BINs and let’s go. Gonna be pretty miserable, however someone needs to find something as I currently would rather pay 0.01$ instead of solving a captcha. Especially the select all the bicycles; it’s a waste of life.

The next step is device attestation. IIRC Safari already does this, so you should not see captcha on places that support it.

Something that can work on any browser can be like this: Scan the QR code in your iPhone or Android device that supports attestation. Will ask you if you approve login, then will attest for you. If you turn out to be a bad actor, the website can ban this device - so no flooding with a single device.

There are over a billion Idevices out there. Malware on just 1% of them can make and control 10 million spam accounts on every site using device attestation, and they're indistinguishable from real users.

Captcha or Attestation doesn't remove the need of moderation. In case of a botnet, an elevated complaints of user device engaging in fraudulent activity can lead to disabling attestation and trigger an investigation. Every iDevice being a member of your site can happen only if you are Google, other than that what you'll see is that some users will engage in shady stuff and blocking them will be enough to keep them out since they wouldn't be able to just sign in with a new account.

These things are always cat and mouse games.

The day this is used widely across browsers is the day devices you own can no longer be flashed with anything other than what the OEM puts on it--even if that is outdated or buggy.

Sounds terrible.

Please. Last time I had to solve a captcha it was wasted 15 minutes (not exaggerating!) of my life, clicking on an endless stream of bikes, motorcycles, buses and stoplights. As punishment for using a vpn.

I've managed that without a VPN - although I do have poor sight.

It also does not help that the shown busses, water hydrants, pavements look totally unfamiliar to me. (Why aren't captures taken from all over the world Indian busses would be fun - London ones would be too boring)

I don't even use a VPN, just a browser that blocks fingerprinting by default. My interpretation of CAPTCHA hell is, "oh, you don't want me to spy on you! Here, let's put some pain in the skinner box."

(Amusingly, pain was proven to be preferable to boredom... and CAPTCHAS are boring as hell.)

If you are using cloudflare DNS for accessing archive.is, you will get that too. archive.is name resolution is broken, and even if you pass the captcha you will go back to the same page, giving the illusion that it didn't pass.

or just use Worldcoin

All roads bring us back to Worldcoin eventually...

ha! someone actually beat me to this comment

At this point the amount of friction added to all these things is pushing things towards just not doing them in the first place (buying less stuff, using social media less). Nature walks and paper books doesn't have captchas.

just not doing them in the first place

Which is not a bad thing

I dread to think about that becoming the norm, I remember living in {Country} with 0 access to cards that would be accepted for anything international

I help people settle in Germany and it's a serious problem. The requirements to open an account disqualify many immigrants. It creates a lot of problems.

look up indian UPI. "validating payment card" and all that snazzy bits are error prone, old, archaic and cost a fortune to businesses.

in upi system, you are presented with a QR code or you input your UPI ID, you click pay and it gets through.

if you are worried about "fraud protection", why rely on an intermediary like ebay or credit card company and instead should take up with your bank or the seller or courts.

There is literally nothing you can do to prevent bot accounts online now, other than requiring people to show up to events periodically. And even then, they can just use bots AFTER they’ve validated their accounts.

The Internet will become a dark forest, and since that is where all of our communication and transactions happen of any significance, that’s pretty much game over for the significance of human activity.

Think I am overstating the fact? It already happened with wall street trading. First, institutions prefer bots to human. Then, you will come to prefer bots to humans. Then every human will be surrounded with 999 bots and unable to change anything or appeal to any significant number of humans to change anything.

Does HN ever require CAPTCHAs? It seems to do pretty well with its basic but battle-tested moderation/antispam tools, and rate-limiting that seems to repel all but the most concerted DDoS attacks. I don't think HN has any unreasonable restrictions on scraping or third-party clients, either. And it manages to serve 5M unique visitors a month and 10M views a day[0].

[0] https://news.ycombinator.com/item?id=33454140

They go down somewhat frequently. I think it’s like four 9’s? I’m not sure why they insist on running just a few machines though. They have more than enough money and probably make up the difference by the advertising for YC that they get.

Unless something changed, it's just the one server.

Main and backup. The last outage was because they have a single network provider. Those are rare, and can be dealt with relatively easily by dual connecting your server to two different networks and sharing across both and removing the dns entry for a broken one. But it’s not worthwhile for such a rare outage

The “outages” that are common are slowdowns for logged in users.

I mean, it works well enough the way it is. Does it need to be more reliable? It’s just a simple forum, there isn’t anything critical on the platform. We all like to see lots of 9s, but they don’t matter that much for something like HN.

That’s fair. To clarify my frustration comes from a place of “love”. When a partial or complete outage happens I get severe HN withdrawals.

IIRC the registration page (only in some cases?) shows a reCAPTCHA.

For me this is about my limit. If I am opening an account that can spam or cost the company real money I can accept that a captcha, while shitty is one of the best available options.

It really gets me when I have a 8 year old account that has made purchases and I still see them across the app.

The annoyingly common one is on login pages. If I am giving you correct credentials you don't need a captcha. If bots are an issue you should be doing per-account strong rate limiting, not a captcha.

It struggles whenever there's a story more popular than usual though

Only for those logged in, because we bypass caches/cdn. Logout helps both you and HN in these cases.

On one machine! :)

On one thread even

Pretty sure it's an AST interpreter too (metacircular eval - apply, as in SICP)

I cant tell if the audience of HN are more likely to script something untoward against HN, be that DDOS or just "check out my product" spam, because its a bunch of hackers - or less likely to do it because (maybe) we like having nice things, or figure the audience is too in the know to fall for boring crypto spam.

HN audience is rich enough to just pay $10 for 1000 solved CAPTCHAs of any complexity since those services are human powered.

HN is also not really a very attractive target. The only thing you can do is post spam, and that's pretty low-value in terms of actual monetary value to the abuser, and tools to deal with that have been around for decades as you say.

This is very different from many other sites where the potential to make a buck is much more pronounced and direct.

Someone will get rich turning this into a browser plug-in.

Nope, the moment this becomes a viable solution then spammers will pick it up, making captcha useless amost overnight.

Websites will very quickly pivot to alternative solutions like payment card verfification, etc.

Solving captchas is pretty rare nowadays. Now you usually just press a button and then it does some sort of fingerprinting to determine if you're a human.

Try to use VPN. You will get captcha with tons of bicycles to click...

Why is the UI such a pain in the ass, when it's designed to be used by humans?! Why do I have to click 8 individual boxes and can't just drag-select an area. I hate those captchas with a passion.

And the infuriatingly slow fade out and in when it changes pictures. It seems designed to frustrate humans.

And does the post count as traffic light?

If you make zero attempts toward privacy maybe. Just turn on a commercial VPN or Tor and you'll find that your quality of life can quickly become severely damaged by captchas. I cant even do a Google search without a captcha so I started using Mullvad Leta as a proxy.

I block ads and stuff but you're right that I don't use VPNs or Tor.

A lot of bots also use VPNs and Tor so captchas being a pain in the ass is probably working as intended, that way most people won't bother using services like that? This is different from regular internet users, there is no reason to make their life more difficult than necessary.

I can confirm. uBlockO, PrivacyBadger, Firefox without any kind of memory and you will get CAPTCHA from time to time, maybe not every day but it's common.

IPv6, 3G/4G/5G or public Wifi can increase that to about every 10 queries on Google for a CAPTCHA. I guess VPN too increase the probability to get a CAPTCHA.

Or to the next unsolved problem in machine learning. The whole point of ReCAPTCHA, at least, is to convert all this human labor into training data.

The spammers can already do this with captcha farms. The fact that captchas are still around means that the cost of captcha farms are still high enough to discourage enough spammers to be worth the annoyance that website operators cause for human users.

You can already buy captcha solves through browser plugins. The only difference is they currently use clickfarms full of underpaid workers from third-world countries

These sites like 2captcha and deathbycaptcha let anyone sign up to be a worker and start solving captchas for $$. If you can run AI that solves captcha just as well, you can literally print money.

2captcha gives you $0.50 per "1-2 hours". Is that really worth all the work?

deathbycaptcha does not let anyone simply sign up to work.

The solution could be a cryptocurrency which can be mined in the browser. Hashcash, which was one of the inspirations for Bitcoin, was initially invented to prevent email spam.

Consumer devices have a lot of spare CPU and RAM. So a proof-of-work algorithm which consumes those resources for a minute might work?

If it generates $0.01 for the website owner in that minute, maybe that would work?

Proof of work can already be implemented without a token.

Tor has such a feature for denial of service protection.

https://blog.torproject.org/introducing-proof-of-work-defens...

A benefit of a token is you can recycle previous proof of work by using a small amount of Bitcoin, which could be transferred using Lightning. The value could also be transferred back some amount of time after registration given no bad behavior, allowing for larger sums than a cent, which could provide better protection.

With a token, you probably get a higher efficiency. Similar to how a heatpump is more efficient than a heater.

If you only consume resources on the client side, then you hope that an attacker thinks "I won't invest $0.01 of resources just to log in here".

If you also transfer the consumed resources to the server, you get an additional benefit: The server thinks "$0.01 is enough to cover the costs of a fake signup".

And the second benefit is probably even better than the first. The server will never really know how cheaply attackers can access resources. But they probably know how much a fake signup costs them.

I think a fairer solution will be some form of proof of personhood that isn't PoW-based. Your idea isn't bad but it gives more power to those who can afford a lot of devices. You know those Chinese mobile phone click farms they use to game app stores? It will be like that, PoW can prevent spam only to a certain degree and with all the social media and networks we have today there is a lot of money in influencing the users. So spending a few million dollars on devices can be very profitable if it lets you boost certain messages.

But is it worth billions? You just need to increase the cost 1000 fold and pay it back after a holding period to implement that.

The drawback is it gets a lot more complex when using a token, because of the additional state, communication, costs and security.

A one shot proof of work can be very simple, but probably not effective enough, given that mobile users likely do not want to wait what may have to be many minutes and drain their battery.

Freezing a cent or a dollar for days seems like a better option. Might very well be that VISA/MasterCard figures this out before the crypto bros build anything usable. It will be far easier to do without decentralization and would also be great to spy on and control people.

Freezing a cent or a dollar for days seems like a better option. Might very well be that VISA/MasterCard figures this out before the crypto bros build anything usable. It will be far easier to do without decentralization and would also be great to spy on and control people.

Fucking A HN.

For any Juniors using this site, this is exactly what you don't post. Especially if it's just to cathart cynicism. I assure you, Poe's law guarantees this will find it's way into some PM's or exec's mind somewhere.

Depends on the use case.

If the captcha is to prevent overuse of a free trial, then nobody will operate a lot of devices just to get more free trials if the paid version is cheaper than those devices.

If the use case is to improve democracy, then it gets more complicated.

Wouldn't any proof of work be just as easy for a computer to achieve as a human?

'Proof of Work' as it's generally understood is done by computers only. But I guess I understand what you're asking and the answer is yes, that is a problem. For Sybil resistance it's better to know if someone is a unique human, not if they're a machine that has paid the toll: https://en.wikipedia.org/wiki/Proof_of_personhood

There are exotic solutions like the 'Idena Network'. But sadly I have to admit the best solution I've seen so far is Sam Altman's Worldcoin. Not that I'm a fan, I still hope we can find something better than scanning everyone's eyeball.

Yeah but it would cost spammers who want to impersonate a large number of humans at once

Yes, but it's more of an anti spam measure.

That is a very interesting concept

Just what we need, another way to waste energy

Those devices have a lot of spare CPU and RAM but basically no spare battery capacity.

It works well, for example with Monero's proof of work algorithm that is purposely designed for consumer hardware. There was an irrational turn against it because some websites did it without consent. I would so much prefer to mine to view a site than have to be exposed to ads and captchas...

It's really amazing when we still get those text ones and nowadays you can literally select the text in many of the images and copy/paste into the input field.

I've never seen a text version that lets me select the text, that's bizarre

I’m assuming that he means that on (for example) Mac you can select text from any image and copy paste it.

https://uk.pcmag.com/macos/138058/not-just-iphone-how-to-use...

100% correct, I assumed windows also let people do that given text recognition is apparently trivial now (to the extent it's annoying - trying to drag images and get text selection instead is annoying :-/)

Windows has gone sort of the opposite way, copy/paste is now often hindered if the engine recognizes the string to be sensitive or otherwise un-wise to copy to your clipboard.

I've had a few instances on Windows 11 and surrounding software where ctrl-C as well as the context menu entry for 'Copy' were greyed out for this reason when skimming through logfiles, presumably because there was something about the line that triggered the MS "that's a password!" regex; stuuuupid stuff.

I think Windows now does it as well, but of course (as all things Windows) it works only in very few apps (forget Win32 ones, for example).

@_rutinerad got it - on macOS you can select text in any image, and I just assumed you could do that on windows as well (I figure in the context of linux it would be much more dependent on specific configuration so unilateral assumptions on behaviour would be questionable).

It's honestly annoying as it frequently interferes with dragging images out of safari, except on those occasions when I do want the text when it's super useful. I think the iOS interface just tells you there's text in an image or photo and gives you the option to copy it rather than cursor based selection you get on Mac.

[edit: from other comments it sounds like windows can do this but it's not always present, and not present in all circumstances, which makes me wonder how many cases in cocoa/uikit/swiftui it does not work]

Am I the only one paranoid enough to think that this means Apple is now indexing even the text content of images stored on it's users computers?

As best as I can tell this study explores many facets of how humans solve captchas. I couldn't find anything about AIs outperforming humans in the study. Can someone give me a section reference?

Solving reCAPTCHA v2/v3 requires more than just clicking the box and an image puzzle. If that was all it was we would be overrun by now.

Lots of folks commenting that the title's statement makes sense because CAPTCHAs are meant to train AIs. While this is broadly true, that's a nice side effect. The way modern CAPTCHAs like reCaptcha V2+ work, is they monitor behavioral analytics-- from things like your browsing history to how your mouse moves on the page. This is why most of the time, most people only need to click a box. I'm not sure there's a LMM out there that includes mouse movement as a modality.

The kinds of AIs that are designed to beat CAPTCHAs also don't have the data from Google et al to use to train, unless we're concerned Google is training it's own bots to bypass CAPTCHAs, I suppose it's not inconceivable?

Yeah, the study is really not about AI solving captchas but how humans solve them. Quite a clickbait title - but those do well on HN unfortunately.

Seems like folks just want to discuss CAPTCHAs generally more ad-hoc, that's cool too, but given how AI has evolved this year, far too many people see this headline and will walk away assuming that the recent AI innovations have made CAPTCHAs useless, but it does not appear to be the case, thankfully.

...Yet, I suppose.

True, the discussion is more about captchas in general. The study isn't bad, I read through it and it's interesting to see real numbers on how long it takes users to solve various captchas. However, a more appropriate title would have been something like "Measuring real user solving times of various captchas" or something like that.

It's in Table 3.

Thank you. The data in that table (for reCAPTCHA, citation 63) is from another paper from 2016 which is focused on solving the actual user-presented problems. It doesn't (directly at least) say they achieved a reliable automation of captcha acceptance, though.

https://ieeexplore.ieee.org/document/7467367

From the abstract:

Through extensive experimentation, we identify flaws that allow adversaries to effortlessly influence the risk analysis, bypass restrictions, and deploy large-scale attacks. Subsequently, we design a novel low-cost attack that leverages deep learning technologies for the semantic annotation of images.

I'd suspect reCaptcha has been updated in the 7 years since to address shortcomings.

Another entry in the table (citation 45) is from 2020 and talks about using an object detection AI to solve the image tests. This again looks like it's focused on the task, not the primary mechanism (behavioral analytics).

I already had issues with captchas (specially on tor).... so now its going to get worst?

The price we pay for obfuscating the trust signals on our connection is that our connection is untrusted.

As an American, I have a similar experience when I travel across the Atlantic. It's always funny to me when I land in the UK, start using websites I use normally at home, and get cookie verification modals from hell to breakfast.

Can't vouch for other Europeans but I got used to them to the point my arm moves automatically where needed before clicking, even accounting for extra modals. I almost don't register them anymore.

Consent-O-Matic (and probably other extensions too) will refuse most cookies automaticaly for you:

https://github.com/cavi-au/Consent-O-Matic

Thanks a lot, I totally forgot about this!

Tor is practically unusable for me because it triggers so many captchas. I end up using Mullvad Browser, which is similar but byo VPN.

Bot operators can already pay human captcha solvers as the paper mentions. So all this does is potentially replace those humans with AI, driving down prices for bot operators.

As prices for bot operators decrease, website operators will increase the challenge and drive up effort for the intended website audience (humans) who are solving captchas instead of paying bots.

In the end, the website operators will have to stop using captchas as the intended website audience will no longer be willing to solve harder captchas.

Website operators can use alternatives, like asking for micro-payments, high enough to discourage most bot operators.

Website operators can use alternatives, like asking for micro-payments

Similarly to how dApps work in ethereum-like blockchains?

I don't know anything about ethereum

I also don't know much, but my limited understanding is that every transaction/mutation in a dApp has a cost, so this might be useful to reduce bot incentives.

Micropayments is not possible when stripe/visa/paypal charge a 30 cents minimum fee

Great news, can we please get rid of CAPTCHAs now?

No, we'll still have them, but now sites will only allow you in if you kind of suck at them.

That is already the case. On some questions you can't answer the correct answer, but have to guess what most other would answer.

It's been particularly frustrating with the picture ones broken up into squares. I tend to be careful to select any square that contains any of what's being asked, but I clearly must be the minority as it always fails unless I select the minimum number of valid squares and ignore the slight overlap on surrounding ones.

How did the OP get from the article linked to the title of this post?

Section 5.5: "Table 3 contrasts our measured human solving times and accuracy against those of automated bots reported in the literature."

Although it's not clear to me that the humans all really were humans.

I'm also wondering this. I don't think it has anything to do with AI solves.

We could simply reverse captchas now: if the captcha is solved its a roboter, otherwise its a human.

We can’t program a bot to fail ?

Yes, we use Copilot for that.

You see where this is heading: after superintelligence is achieved, CAPTCHAs will be designed to be questions that humans get wrong but AI has no problem with.

At least that would still be a proper CAPTCHA, in that it tells computers and humans apart…

A superintelligent AI would be able to imitate a human, getting the answers incorrect in exactly the way needed.

However, I'm not entirely sure what kind of system a superintelligent AI would need to access which would be protected by a captcha.

no need for captchas, just implement throttling per ip. like bcrypt dues for passwords. if a bot fills up a form(or whatever), so be it, but it won't be able to do it for another N seconds or minutes..so the problem then is lowered from per try, which can be thousands of submissions, all the way down to per period and per ip.

Hell no... Some of us sit behind CGNAT, half a million of us on a single public IP.

Exactly. Besides that, a bad actor may well have easy access to tens of thousands of ip address from all over the globe..

Good. Hardware authentication is where it's at.

Like the Clipper Chip?

Oops! Looks like you're not using a government approved OS and browser.

The main road to tell computers and humans apart will soon be that computers are a bit too fast and accurate to be humans.

Instead of just solving the task, computers will now have to simulate humans using humans' real data. It's not a hard dataset to train on.

They already try to do that.

I find captchas extremely painful, because of ambiguity and not loading all the pictures. I wait for a minute and some never show. When they do load, so manyare pics of bicycles and motorcycles and cross walks. Are you supposed to click on the tiny piece that goes tojust past another tile or not? You can't refresh one that doesn't load, I think most of them start over if you refresh.

Like other people reported, if you ever use tor, it's very common for the captchas to just not load. They just kind of hang without showing the pictures. Regular websites generally just work fine on tor, it seems to be a captcha problem.

Are you supposed to click on the tiny piece that goes tojust past another tile or not?

I ask myself this every time.

Pretty sure the hesitation is what makes us humans :)

simple - if the user solves is too well, reject

val delay = 500 + Math.random() * 3000

Coincidentally Worldcoin is up 30% today. Maybe cryptographic/biometric proof of being a human will be useful after all?

No thanks, I'd rather not live in a dystopian nightmare where Sam Altman is in control of assigning proof of humanity. Worldcoin will undoubtedly end up assigning identities to AIs for profit anyway, and/or there will be swaths of identities being sold on the black market.

The relevant data for the claim of the headline is in Table 3. On all the tasks with enough data, bots were both faster and more accurate than humans.

Yeah, the claim of the headline comes from the first sentence in Section 5.5. I think either the title should match the paper's title or that should be pointed out as part of the submission - not sure how HN's title guidelines work.

I predicted this 7 years ago: "How will the machines take over? When CAPTCHAs become so hard that only AI can solve them, humans will be completely locked out of the net." https://twitter.com/lapcatsoftware/status/771857826130034688

I thought this was already happening ~7 years ago. The "what text is in this image captchas" got a lot less common a while ago, and I think this was partly the reason why.

Are there any open local models for basic alphanumeric picture captchas to save on 2captcha?

A surprising number can be solved with teseract and simple preprocessing (e.g. thresholding, expand and contract lines).

For more complex cases, not AI but consider the attack in https://www.usenix.org/system/files/conference/woot14/woot14...

I have been locked out of websites for solving a captcha so quickly that it thought I was a bot. So we went from requiring humans to solve a puzzle that bots can't to now requiring that humans solve the puzzle slower than bots do.

The most funny thing about this limit is that it's self-reinforcing. Bots will learn to sleep() and wiggle the mouse. Humans will learn to wait. Everyone will be worse off.

So if you fail, you are human.

Just as Turing intended?

It does not suprise me since lately I have a lot of mistakes with CAPTCHAs. Mainly the ones with characters with different colors, superposed, and rotated. I think there are some that we as humans just guess because the final image is not clear enough.

I think in the same way AI can beat us recognizing unfocused photos.

What a surprise since CAPTCHAs were created to gather data to train AIs

Is this really so surprising? Probably a better captcha would be sign of life not puzzles.

How many years until sites actually remove CAPTCHAs though?

I hate captcha, it takes ages and I often fail the google one. I would happily pay to have them removed from my browsing. I don't use AliExpress often, but now I can't as the captcha just plain doesn't work.

I don't know....lately I just don't even try and purposefully make mistakes on it by leaving out one or two just to fuck with the captcha

I find Amazon’s captchas so hard now! I have to do the audio one

The problem with designing a bear proof trash can is that there’s significant overlap between the smartest bears and the dumbest tourists.

Have they tried with puzzles used by Rockstar Games or HBO Max to reset a password? They are impossible to solve, asking to solve 17 questions and more and still failing you to retry with higher count. Even the audio version is quite innovative

My wife is entirely unable to solve a captcha. Her solution to any captcha is to get me to do it for her while she loudly swears at the creators. I welcome being able to outsource this task to AI.

So we now proof that we're human by failing those tests?

Submitted title was "AI bots are now outperforming humans in solving CAPTCHAs", which broke HN's title rule: "Please use the original title, unless it is misleading or linkbait; don't editorialize."

Submitters: If you want to say what you think is important about an article, that's fine, but do it by adding a comment to the thread. Then your view will be on a level playing field with everyone else's: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...

I think I prefer the recent CAPTCHAs (where you solve a puzzle by rotating an item, or finding the matching item). The older ones from years ago (deciphering mangled text and trying to work out if it is an `i`, `1` or `l` were more annoying)

I wish captcha providers universally had to provide a way to shut down their use by bad actors. Here in Canada I get tons of scam texts pointing me to a fake banking or postal service website asking me to pay a fake bill. I want to ddos them with fake payment data but they’re all protected by hcaptcha.

Google created this problem, let’s see them solve it.

All of these papers miss that captchas have multiple levels of difficultly. People who get an enterprise account or work closely with the captcha providers will find very different results. Many captcha providers now decide what captchas to send out, in hard mode based on what LLMs cannot solve

Captchas are purposely not made too hard as people like pex.com need to be able to bypass them for copyright enforcement. Note I’m biased as I was a founder of hcaptcha

Can the mods please fix the completely wrong and click bait title? Zero AI mentioned there.

Misleadingly editorialised title. Actual title and abstract (which doesn't say anything about AIs "now" outperforming humans):

An Empirical Study & Evaluation of Modern CAPTCHAs

* For nearly two decades, CAPTCHAs have been widely used as a means of protection against bots. Throughout the years, as their use grew, techniques to defeat or bypass CAPTCHAs have continued to improve. Meanwhile, CAPTCHAs have also evolved in terms of sophistication and diversity, becoming increasingly difficult to solve for both bots (machines) and humans. Given this long-standing and still-ongoing arms race, it is critical to investigate how long it takes legitimate users to solve modern CAPTCHAs, and how they are perceived by those users.* * In this work, we explore CAPTCHAs in the wild by evaluating users' solving performance and perceptions of unmodified currently-deployed CAPTCHAs. We obtain this data through manual inspection of popular websites and user studies in which 1,400 participants collectively solved 14,000 CAPTCHAs. Results show significant differences between the most popular types of CAPTCHAs: surprisingly, solving time and user perception are not always correlated. We performed a comparative study to investigate the effect of experimental context -- specifically the difference between solving CAPTCHAs directly versus solving them as part of a more natural task, such as account creation. Whilst there were several potential confounding factors, our results show that experimental context could have an impact on this task, and must be taken into account in future CAPTCHA studies. Finally, we investigate CAPTCHA-induced user task abandonment by analyzing participants who start and do not complete the task.*

@dang, could you please correct the title? Thanks.

CAPTCHAs are used as a literal Turing test; that's their whole purpose. From the get-go their usefulness window had a looming expiration date.

Mandatory xkcd https://xkcd.com/810/

well captchas are not there to keep bots out, they are free click work for google?

"and that's the story of the invention of the Voight-Kampff Test, kids!"

Great, so maybe we would find a less annoying bot detection technique than captchas.