Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> I don’t see how proof of work could possibly be a sustainable strategy to keep bots out without ruining human user experience.

Lots of work has been done to do just that by monero’s proof of work algorithm. They achieved ASIC and GPU resistance by making you generate entire programs (with quite a bit of RAM usage) and finding one that produces a desired output. Then you submit the seed used for the program (a series of programs actually but let’s not get into the details)



Monero is fine with making people wait, that's literally the whole basis of mining blocks in "crypto currency". It's tuned to the time required.

Anubis wants "slow for bots but fast for humans" which is not possible if somebody has a farm of devices that costs them basically zero.


One possible way to do this: rather than simply burning hashes for nothing, Anubis can burn hashes to mine crypto. This way, the bots pay for the infrastructure they consume, at least in part.

Combine with heuristics like in the article, so you can crank up the difficulty only for likely bot traffic, and you've got a bot supported site.


CPU crypto mining stopped being viable over a decade ago. If you want a client to pay for the infra cost of their request, they're probably going to need to work for hours to do so.


Disappointed at the lack of replies here, I've had a similar thought before and I'd be curious to hear about the technical viability of this


This was done in the past[1] and was widely regarded as a "cryptojacking" scheme serving "malicious" content to users.

[1] https://krebsonsecurity.com/2018/03/who-and-what-is-coinhive...


If it was for profit. Here it's in response to abuse by AI companies.


> Anubis wants "slow for bots but fast for humans"

That's not true. The goal is to raise the cost for bulk scrapers to the extent possible while minimizing any inconvenience to regular users. It's not about being slow for bots but rather reducing the asymmetry of the problem (from the perspective of the server) by imposing a (minor) cost on anyone requesting a page.


That's a better way to put the goal. But does it actually raise the cost or reduce asymmetry in a meaningful way? If it makes crawling 0-5% slower then it's really not worth the annoyance.

I know having the Anubis javascript at all gets in the way of some bots. But is the calculation helping enough?


> does it actually raise the cost or reduce asymmetry in a meaningful way?

It's complicated. I think anubis currently only collects low hanging fruit so to speak but in practice that's the bulk of what was causing problems so it's good enough.

PoW in general absolutely yes - I think monero clearly demonstrates this. Last I checked it remained viable to mine monero on at least somewhat low end and at least somewhat outdated consumer CPUs. It seems to me that fact implies that a well constructed PoW scheme can in practice squash the impact of any performance differences to within at most a couple orders of magnitude.


But a couple orders of magnitude is still a ton of asymmetry.

And even if you can guarantee the same CPU time on bots and legitimate requests, you still have a bunch of asymmetry in favor of the bots.


Sure. Even if PoW in general is sound, when applied to scraping it's always going to be a matter of what the client is willing to "pay". It's entirely within the realm of possibility that a scraper could be willing to "outbid" the user of a 5 year old tablet.

But it's also true that charging anything is going to be expected to reduce server load, that the cheaper something is the more carelessly people will tend to make use of it, and that at least in general bot operators are expected to be more cost sensitive than regular visitors on account of scale.

To that last point consider that a regular visitor will generally have an idle CPU and multiple gigabytes of free RAM. Meanwhile a competent scraper will generally be optimizing request parallelism per cheap VPS while an incompetent one will just point and shoot (and proceed to blindly visit every cgit url). The latter is by far the most troublesome and conveniently also the most likely to be impacted by even a poorly implemented PoW scheme.

I keep seeing comments along the lines of anubis only working because it bounces bots running lightweight setups without js but stop and ask yourself why are they running such lightweight setups to begin with?


> why are they running such lightweight setups to begin with?

It's easier and still gives access to almost everything. I don't think CPU limitations are the main issue there. As far as computer resources, I'd bet RAM is the bottleneck to running the script, not cycles. That suggests a RAM-heavy hash might do better than SHA256, but still agrees with the idea that the PoW in Anubis is not the important part.

> To that last point consider that a regular visitor will generally have an idle CPU and multiple gigabytes of free RAM. Meanwhile a competent scraper will generally be optimizing request parallelism per cheap VPS while an incompetent one will just point and shoot (and proceed to blindly visit every cgit url). The latter is by far the most troublesome and conveniently also the most likely to be impacted by even a poorly implemented PoW scheme.

I considered that. But while a visitor is idle they need the result quickly and they face the full brunt of the challenge all at once. A scraper can be working on a thousand sites at once so the per-request impact of the challenges is a lot lower, and patiently working through challenges in the background usually reduces the real cost, potentially to almost zero.

And you'd better not need multiple gigabytes of RAM to visit your site, that'll freeze my computer.


> [...] which is not possible if somebody has a farm of devices that costs them basically zero.

Opportunity costs are real.


Monero is fascinating. I downloaded the spec and tried to digest it, but the mathematics and cryptography are quite dense, at least for a casual read.

I might try to tackle it again when I can give it some more attention.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: