Proof of Work Invisible Challenges as a Deterrent for Botting
Organizations of all sizes — whether startups or large corporations — have always had a perpetual problem of having to deal with bots on their websites. Bot traffic has always been a concern for everything from skewing engagement data for web marketing to actually attacking endpoints of key services on websites through DDoS attacks as well as all types of attacks in between including phishing and spam content.
While many larger corporations usually apply rate limiting to reduce the impact of these attack vectors, many businesses also use CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart) challenges — those challenges that ask you which images contain traffic lights or to type the letters/numbers or to even simply check a box to provide some information to the site provider — on login and signup pages.
What are the problems with existing methods?
The biggest problem with existing methods for dealing with bots is that they are too intrusive for real users to complete and they are not as effective on bots unfortunately. Rate limiting, or putting a cap on network traffic, doesn’t necessarily distinguish bots from real users, but rather puts a cap to prevent DoS attacks and limit web scraping. This is a shotgun solution to a problem that requires surgical precision.
CAPTCHA is also incredibly intrusive and adds a ton of friction to websites for real users for whom it is triggered. Forbes published an article highlighting how friction from CAPTCHAs can cause 8–29% of real users to drop off depending on the type of CAPTCHA being shown. Even when you apply a bot detection model for displaying CAPTCHAs, there are always a significant number of false positives which are also caught in these models, and while sites that used CAPTCHA saw a 88% decline in bots, they also saw a 3.2% drop in sales conversion. Moreover, CAPTCHAs do not do a lot of good for more sophisticated botters that will usually bypass the endpoint CAPTCHA is shown on, rendering the challenge useless.
This is where invisible Proof of Work (PoW) challenges that are based on real-time browser fingerprints can really solve for many of these fundamental problems that exist in Web2 user experience and security limitations.
What is Proof of Work (PoW) and what are PoW Invisible Challenges?
Investopedia provides a great high level definition for PoW. In effect, PoW is a decentralized consensus mechanism that requires members of a network to expend effort solving an arbitrary mathematical puzzle to prevent anybody from gaming the system. It is often used for validating cryptocurrency transactions and mining new tokens. Bitcoin is the classic example used for a PoW cryptocurrency.
Instead of using CAPTCHA, PoW invisible challenges are exactly what they sound like: challenges that utilize PoW consensus without displaying anything to the user. Effectively, validating a user’s identity is done in the background, so it is much harder to bypass an endpoint if you are unaware it exists. The difficulty of the challenge can be amended based on whether a user is detected as a bot or not. There have been a few similar implementations using HashCash, but these are arguably destructive implementations that basically mine crypto using a user’s CPU while they are visiting a website, which is a bad practice (in my opinion) since this is as intrusive as malware in terms of damage to the end user. There is also an implementation done by FriendlyCaptcha which utilizes PoW, but this falls into the same trap where if bots are aware of a specific endpoint containing proof of work, they will try to bypass it.
What is Browser Fingerprinting?
Pixel Privacy provides a good summary on what browser fingerprinting is. Many businesses will often use browser fingerprinting solutions to track browser type, user agent, IP, etc, typically both for security and targeted advertising based on where you click on a page. Many companies will use browser fingerprints to augment existing data that they have about a consumer to derive insights into their browsing behavior and also for bot detection. A common fingerprinting solution for developers is FingerprintingJS (which is also the backbone for the invisible challenge demo that I created — see a couple sections down).
How can Browser Fingerprinting be used to Augment Invisible Challenges?
Using real-time telemetry from a user’s browser fingerprint, we can compare every event triggered from an action taken on a webpage to see if a user is trying to mask certain properties of their fingerprint such as IP, browser ID, user agent, etc., which is what many bot accounts do. If they do such things or spam a specific event multiple times consecutively, you can trigger a harder PoW challenge that takes much longer to solve. This translates to longer load times which would discourage botters from continuing their action/attack on the specific web endpoint. On the flip side, if users are detected as being real users, then they will receive less difficult invisible challenges so they can expect normal load times for a page. Below is a quick flow diagram explaining the feedback loop for this user experience.
It’s important to note that because the invisible challenge is not exposed directly on an endpoint, it would be much harder for a botter to bypass the endpoint without recreating all past nodes for blockchain events on the site. It is also important to note that even if they are aware of the invisible challenge, because of the computational intensity of trying to recreate the entire blockchain for validation, they will likely be discouraged from continuing with their present action.
How would this type of solution look in practicality?
I built a demo solution that can be viewed at this link: https://pow-browser-fingerprinting-demo.com/
Any time you load/reload the page, type in one of the text/password fields, or click the example button, an event is logged in the developer console which can be reached by right clicking on the page, clicking “Inspect”, and clicking the “Console” tab at the top. See example images of the output below.
As you can see, multiple events including the user agent, hardware concurrency, session storage, local storage, etc. are stored. As a nefarious user or botter, if you were to try to mask your fingerprint on every new event by changing your user agent, using VPN, etc. those events would be logged and for the next event, you will be served a harder invisible challenge. For the purposes of the demo, everything — including the completed workers for each challenge and the PoW ID and hash are outputted to the console — but in practice, these will all be hidden from the user in a cloud database and server.
Interested in learning more?
Whether you are a business owner, part of a business interested in improving your website security, or other interested party, feel free to reach out to me at roshan.srin@gmail.com or fill out this survey which is also linked in the demo above and I will make sure to get back to you! I’m always looking for feedback on how to improve this product as I continue building so I am happy to discuss any feedback or specific use cases you may have.
