Question

I have a site in development that is password protected using the the apache htpasswd.

How can I exclude request coming from FB share such that it is not password protected?

Meaning, I still want the site to be password protected when a user try to access through a browser, but if the request comes from FB share ping, it should allow it.

I see that FB share ping has this user agent, facebookexternalhit/1.1, but I’m not sure how to implement that.

Thank you,
Tee

Answer 1

This answer assumes you’re using Apache 2.1 or later:

With the mod_authz_host module, you can do access control by hostname in .htaccess or in the <Location> or <Directory> section of your httpd.conf file, for example:

<Location />
  Allow from apache.org
  Allow from .net example.edu 
</Location>

Note that you can use partial domain names here, so http://www.apache.org would be allowed access in this example.

To ensure that you have mod_authz_host installed, check for a line similar to the following in your httpd.conf:

LoadModule authz_host_module modules/mod_authz_host.so

One downside with this is that it will do a reverse lookup for each access, which may or may not be a performance issue for you.

Another option is to restrict by User-Agent. However, this is an unreliable technique, because the User-Agent can be very easily spoofed. But it may be OK for your purposes depending on your paranoia level 😀

To restrict by User-Agent, you need to make sure you’re loading the mod_setenvif module in addition to the mod_authz_host module. Check for a line similar to the following in your httpd.conf:

LoadModule setenvif_module modules/mod_setenvif.so

Then, you can configure the access control in <Location>, <Directory>, or .htaccess as follows:

SetEnvIf User-Agent ^facebookexternalhit/1\.1 let_me_in
<Location />
  Order Deny,Allow
  Deny from all
  Allow from env=let_me_in
</Location>