Question

I have just stumbled upon Bad Behavior – a plugin for PHP that promises to detect spam and malicious crawlers by preventing them from accessing the site at all.

Does something similar exist for ASP.NET and ASP.NET MVC?

I am interested in blocking access to the site altogether, not in detecting spam after it was posted.

EDIT: I am interested specifically in solutions that will detect access patterns to the site – these would prevent screen scraping the site as a whole, or at least make it a very slow process for the offender because the bot would have to act as a regular user in frequency of his clicks and in patterns found in the clicks (100 links visited in depth-first succession are a clear indication that someone is crawling the site).

By malicious crawler I mean primarily bots that are building duplicate copies of other sites but also spambots that are trying to locate some submittable forms on the site and submit their crap.

Answer 1

The NoBot control in the ASP.NET AJAX Toolkit might help, if I understand you correctly. From the description page:

NoBot is a control that attempts to provide CAPTCHA-like bot/spam prevention without requiring any user interaction. This approach is easier to bypass than an implementation that requires actual human intervention, but NoBot has the benefit of being completely invisible.

I haven’t used this one myself, so I can’t speak to its effectiveness against sophisticated bots, but it sounds like it’s meant to be exactly what you’re talking about. It has the benefit of being very simple; essentially a drop-in you can add to a master page.