Question

Lately my site has been getting hit by some people who have bots which spam HTTP requests, everything is basically random gibberish which can’t really be filtered in any meaningful way (without some very serious work), but one inconsistency I found was in their referer string.

[22/Sep/2012:03:27:50 +0000] "GET /vb/ HTTP/1.0" 403 345 "1mc9py1amv.net" "Mozilla/3.0 (compatible; NetPositive/2.2)"

as you can see, their ref has no protocol. Is there any way to block refs with no protocol while still allowing blank (none) referer things to get into the site?

Something like, if it contains “http” or “https” or is blank, it will allow, but everything else is denied?

I tried something like

$HTTP["referer"] !~ "(http://.*|-)" {
    url.access-deny =  ( "" )
}

but that banned blank ref and didn’t allow https.

Answer 1

    $HTTP["referer"] !~ "^($|https://.*|http://.*)" {
            url.access-deny =  ( "" )
    }

This did the trick. It’s quite useful if you’ve encountered something similar.