Most of the time a web page reports an error or seems to be down, or it gives you another captcha no matter how many times you correctly answer it, it's just denying you access, but it's also trying to hide that from you. Whether there's captchas or an outright denial, those pages are guarded by bots that only allow other bots to proceed to the page. We've known for several years that captchas only deter humans (https://www.newscientist.com/article/2384228-bots-are-better...) so clearly anyone currently using them doesn't want humans visiting the page.
Most of the time, the Gemini summary that Google includes in their search results isn't particularly useful, but extracting data from those bot-only pages is one thing it excels at. So, whenever a search result links to a page guarded by a bot, and it senses that I am a human and denies access, I go back to the Google search results page, and ask Gemini to scrape the page and provide the data I want from it.
Look, I want to see WCAG adopt "Any ad that covers content, a cookie banner, or Anubis or Cloudflare harassment is an immediate F grade" I'm a little more hopeful than that than to get a requirement like "you should actually have a disabled person test it"
What the people who like Anubis don't get is that the cost for honest people is very real, the cost for crawlers is basically $0 because: most of the people who run crawlers are just idiots and aren't paying attention to what's happening and get crap data and don't even know they are getting bounced and probably don't even look at the crap data they collect, the worst of the crawlers are running botnets using stolen CPU, the real AI companies don't have any financial discipline and wouldn't care it if anubis wasted a CPU*hour per page.
Or rather, those "click on the bus" images raise your blood pressure but are close to trivial for a bot to automate... It's all performative... We're good open source people who prove we are good open source people because we put Anubis on our page to shame those bad AI people.
I did not use chatbot for search until Anubis, Cloudflare and recaptcha are so prevalent that chatbot literally parses through the pages faster than I can open any one of them, using my daily browser, with my own hands. It seems not only my choice does not matter, but most websites also do not care about human readers. At this point I just use chatbot as search engine out of vengeance.
Also, optimized Anubis & other proof of work challenge solvers are much faster than a stock browser.
Yes, I feel like there has been a sharp uptick in this the past couple months.
Whatever magic is built into the CloudFlare checkbox and my motion to click on it, I really wish was just part of the browser and my motion to click on everything else.
It's even worse when you are traveling cause then you run into country-blocks on websites and videos on top of all of that, and you didn't even mention the forced logging-in or registration walls to continue browsing/shopping/etc!
Feels like the whole internet is hitting that hockey stick growth... towards becoming shit.
Once you get past that, it's on to the auth walls where you have to login every day or week. We have perfect the nagging modals to suit all anti-consumer patterns.
This is the internet in 2026 and it sure as hell won't get any better.
Kinda disappointing that this doesn't get daily attention on HN of all places.
I have two Firefox extensions that solve captchas for me. They are just eye exams and most unwelcome.
Buster: Captcha Solver for Humans [0]
NopeCHA: CAPTCHA Solver [1]
OF COURSE, these are band-aids on an internet under full assault.
P.S. Paywall busters are a nuisance to run through. Often, a paywalled article is syndicated (MSN does this a lot) or available at arxiv, or an author's institution, so "Use the web search, Luke."
I call those bot-only web pages.
Most of the time a web page reports an error or seems to be down, or it gives you another captcha no matter how many times you correctly answer it, it's just denying you access, but it's also trying to hide that from you. Whether there's captchas or an outright denial, those pages are guarded by bots that only allow other bots to proceed to the page. We've known for several years that captchas only deter humans (https://www.newscientist.com/article/2384228-bots-are-better...) so clearly anyone currently using them doesn't want humans visiting the page.
Most of the time, the Gemini summary that Google includes in their search results isn't particularly useful, but extracting data from those bot-only pages is one thing it excels at. So, whenever a search result links to a page guarded by a bot, and it senses that I am a human and denies access, I go back to the Google search results page, and ask Gemini to scrape the page and provide the data I want from it.
Look, I want to see WCAG adopt "Any ad that covers content, a cookie banner, or Anubis or Cloudflare harassment is an immediate F grade" I'm a little more hopeful than that than to get a requirement like "you should actually have a disabled person test it"
What the people who like Anubis don't get is that the cost for honest people is very real, the cost for crawlers is basically $0 because: most of the people who run crawlers are just idiots and aren't paying attention to what's happening and get crap data and don't even know they are getting bounced and probably don't even look at the crap data they collect, the worst of the crawlers are running botnets using stolen CPU, the real AI companies don't have any financial discipline and wouldn't care it if anubis wasted a CPU*hour per page.
Or rather, those "click on the bus" images raise your blood pressure but are close to trivial for a bot to automate... It's all performative... We're good open source people who prove we are good open source people because we put Anubis on our page to shame those bad AI people.
I did not use chatbot for search until Anubis, Cloudflare and recaptcha are so prevalent that chatbot literally parses through the pages faster than I can open any one of them, using my daily browser, with my own hands. It seems not only my choice does not matter, but most websites also do not care about human readers. At this point I just use chatbot as search engine out of vengeance.
Also, optimized Anubis & other proof of work challenge solvers are much faster than a stock browser.
I think thought user behaviour telemetry was a big part of the captcha recipe. Not so much the clicking of the right squares per se
Yes, I feel like there has been a sharp uptick in this the past couple months.
Whatever magic is built into the CloudFlare checkbox and my motion to click on it, I really wish was just part of the browser and my motion to click on everything else.
It's even worse when you are traveling cause then you run into country-blocks on websites and videos on top of all of that, and you didn't even mention the forced logging-in or registration walls to continue browsing/shopping/etc!
Feels like the whole internet is hitting that hockey stick growth... towards becoming shit.
How long do you spend on the page?
Once you get past that, it's on to the auth walls where you have to login every day or week. We have perfect the nagging modals to suit all anti-consumer patterns.
This is the internet in 2026 and it sure as hell won't get any better.
Kinda disappointing that this doesn't get daily attention on HN of all places.
I have two Firefox extensions that solve captchas for me. They are just eye exams and most unwelcome.
Buster: Captcha Solver for Humans [0]
NopeCHA: CAPTCHA Solver [1]
OF COURSE, these are band-aids on an internet under full assault.
P.S. Paywall busters are a nuisance to run through. Often, a paywalled article is syndicated (MSN does this a lot) or available at arxiv, or an author's institution, so "Use the web search, Luke."
[0] https://github.com/dessant/buster
[1] https://nopecha.com/
[dead]