Skip to content
DEFLECTO
Documentation / Proof of work

The challenge, and the callers that cannot answer one.

Proof of work is how a suspicious client is charged for the privilege of asking, rather than guessed about. Most of this page is about the one thing that goes wrong with it, which is your own automation being asked to do arithmetic it has no engine for.

Before any challenge: the ruleset

Proof of work answers the question of whether a client is worth serving. It says nothing about what the request is asking for, so it runs alongside a signature ruleset that inspects every request whatever the client looks like. A visitor who solved a challenge an hour ago is still refused if the next thing they send is an exploit.

The ruleset ships enabled on every plan, is maintained by us, and needs no configuration from you. Requests are scored rather than matched once, so a single odd-looking parameter is not treated like an obvious attack. These are the families it covers, with the kind of thing each one actually stops:

Scanners
Known tooling identified by its client string, requests with no User-Agent at all, and the header artefacts scanners inject. This is the largest category by volume and almost none of it is aimed at you specifically.
Path traversal
../ sequences in the path or in any parameter, including the double-encoded and partially-encoded spellings, and direct requests for sensitive files such as /etc/hosts, /etc/passwd, /proc/self/environ and ~/.ssh.
Application probes
The endpoints that get hit on every site on the internet regardless of what software it runs: xmlrpc.php, wp-login.php, wp-admin/, attempts to read wp-config.php, and PHP webshell and administration scripts.
Remote code execution
Shell metacharacters followed by a command, command substitution and backticks, output piped into an interpreter, reverse shell invocations, Shellshock function definitions, and dangerous PHP calls carried in a parameter.
Injection
SQL injection across roughly a dozen shapes, from UNION SELECT and boolean tautologies to time-based blind probes and hex obfuscation. Also NoSQL operators, XXE, Java deserialization gadgets, prototype pollution and server-side template injection.
Log4Shell and SSRF
JNDI lookup expressions including the nested evasions, cloud instance metadata endpoints, and loopback or private addresses passed to a parameter that fetches a URL.
File disclosure
Requests for backups, archives, database dumps, editor and merge artefacts, framework key files, and dotfiles holding credentials or version-control state such as .git/ and .env.
Protocol abuse
NUL bytes and CR/LF in the request target, invalid or overlong UTF-8, double-encoded metacharacters, absurd repetition, and routing-override headers the origin might interpret differently from us.

You can add your own rules on top through the panel. You cannot subtract ours, which is deliberate: an exemption that could switch off the ruleset would be the first thing an attacker looked for.

Why a challenge rather than a block

Every filter eventually meets a request it cannot classify. It looks automated, it is arriving faster than a person browses, and it might still be a real customer on a corporate VPN with an unusual browser. The three available answers are not equally good.

  • A block is binary, and it is wrong about real people. Shared mobile carrier addresses, office egress, VPNs, hosting ranges used by perfectly legitimate integrations: refuse those and the cost lands on a customer who did nothing, silently, in a way neither of you can see.
  • A CAPTCHA outsources the cost to the human. It charges a real visitor several seconds of attention and an accessibility problem, to slow down an attacker who is increasingly able to solve it more cheaply than the visitor can.
  • Proof of work charges the client’s CPU. The cost falls on the machine making the request rather than on the person behind it. A browser loading one page pays it once and does not notice. A flood pays it on every single request, which is the part of a flood that used to be free.

That asymmetry is the whole mechanism. It does not need to decide whether a client is hostile, which is a question with no reliable answer. It only needs the answer to be expensive at volume and free at human scale.

What the visitor experiences

An interstitial page appears in place of the one that was requested. There is nothing to click, nothing to read and nothing to identify. The browser computes a solution, posts it back, and the original request continues.

  • It is fast. On a current browser the work finishes in well under a second even at the hardest setting the node will use, and it is spread across workers so the page does not freeze while it happens.
  • The page fetches nothing from anywhere else. No script from a third party, no font, no image request, so it still works on a connection that is already struggling, which is generally the situation it is being served in.
  • Most visitors never see one at all. A client is challenged because something about it already looked automated, not as a matter of routine.

The clearance that follows

A solved challenge earns a clearance. It is carried in a cookie, and its only job is to stop the same client being asked again for the rest of its visit. Every subsequent request presents it and passes straight through.

  • It expires. A clearance is a grant with a lifetime, on the order of an hour by default, not a permanent pass. A client that goes quiet and comes back much later can be asked again.
  • It is bound to the client that earned it. A clearance lifted from one machine and replayed from another is not a free pass for a botnet, which is the obvious attack on any scheme like this one and the reason the cookie is not simply a bearer token.
  • It is not a login. It says one client did some arithmetic once. It carries no identity, and your application should neither read it nor care that it exists.

A challenged request is never metered

The commercial half of the mechanism

A request that was challenged did not reach your origin, so it costs you nothing. Your quota counts wards, one ward is one request delivered to your origin, and challenged, blocked and dropped traffic spends none of them. This holds however long an attack runs and however large it gets.

This is worth stating plainly because the alternative is what most per-request billing does: it charges you precisely on the day you were attacked, which is to say on the day the product worked. Charging for somebody else’s botnet is not a business we want to be in, so the meter only moves when your application actually had to serve something.

Difficulty follows the load

The puzzle is not one fixed size. Difficulty tracks what the node is currently dealing with, so a mildly suspicious client on a quiet afternoon gets a trivial one, and the same client during a flood gets a harder one.

The effect is that the cost of a flood rises with the size of the flood, while a real browser barely notices the difference: the work scales into a range a laptop still finishes in a moment, and there is a ceiling it does not go past, because a puzzle that locks up a phone has stopped defending anything.

What breaks: callers with no browser

This is the single most common support issue, and it is worth understanding before it happens rather than at the moment a payment goes missing.

A payment provider posting a notification to your site has no JavaScript engine. It follows no redirect, keeps no cookie jar, and gives up after a handful of attempts. Challenging one does not inconvenience an attacker in the slightest, it loses you a payment. The same is true of monitoring probes, uptime checks, health checks, feed readers, mobile clients talking to your API, and any server to server integration you have.

The fix is to exempt them, and there are two ways to do it. They are not interchangeable.

By path, which is what almost every provider needs

You name the paths on your own site that must never be challenged. Providers rotate their egress ranges without notice, so for them this is the only exemption that keeps working.

  • Matching is exact, or a trailing /* for a prefix. The query string is ignored, so a provider appending ?id=1234 cannot quietly stop matching a path that worked yesterday.
  • Give the narrowest pattern that solves the problem. /webhooks/oxapay rather than /webhooks/*, and never a pattern that covers the whole site.
  • A pattern with a * anywhere but the end is refused rather than accepted and quietly reinterpreted, because a rule that does not mean what it looks like is worse than one that fails loudly.
  • A tenant may hold up to 16 of them. The ceiling is deliberate: a list long enough that nobody remembers what is on it is a list nobody audits.
An exemption removes the puzzle, not the filtering

Only the challenge is lowered. Every rule still inspects the request, blocks and drops are untouched, and rate limits, bans and the concurrency ceiling all still apply. An exempt path is not a trusted caller and it is not an injection endpoint.

Because the match is on the path alone, anyone who guesses the path also skips the puzzle. Your handler must verify the provider’s signature. The network cannot tell a real notification from a forged one that is shaped like it, only the signature can, and signature verification rather than address filtering is how webhooks are actually authenticated.

Exempted traffic is not invisible. It is recorded as a logged delivery rather than disappearing from the counters, so a hole you opened stays something you can see and measure. It is also a delivery, which means it spends a ward like any other request that reached your origin. The exemption takes away the arithmetic, not the fact that your application served the call.

By address, for callers whose addresses you own

The other option is an allow list of networks in CIDR form. It is a much bigger statement than a path exemption, because it says trust this source rather than do not ask this path for arithmetic, so keep it for things you actually control: your own monitoring host, a deployment runner, an office range. Do not try to list a payment provider’s ranges. They will change them and you will find out from a missing payment.

Testing that a path is exempt

Test it the way the provider will: from outside your network, with a client that runs no JavaScript and keeps no cookies. That is precisely what a webhook sender is.

Ask as an automated caller would
curl -sS -X POST -o - -w '\n%{http_code}\n' \
  -H 'content-type: application/json' \
  --data '{}' \
  https://example.com/webhooks/oxapay
  • What you want to see is your own handler’s answer, whatever status it chooses for an unsigned body. A rejection from your application is a pass for this test.
  • What says it is not working is an HTML page with a puzzle in it. That is the interstitial, and it means the pattern is not matching the path that actually arrived.
  • Compare the raw path, character for character. /webhooks/oxa%70ay is not exempt even though it decodes to a path that is. The origin routes on the raw path, and honouring the decoded form would let a caller choose which of the two forms the filter and your application each see.
  • Confirm it from the counters, not only from the response. The calls should appear as logged deliveries rather than as challenges. A test run on a quiet afternoon has mostly confirmed that nothing was being challenged anyway, which is not the condition you are protecting against.

If a provider is still failing after all of that, send us the exact URL it posts to, including the query string, and the timestamps of two failed attempts. Those three things are enough to find the request in the history and say what the node decided about it.