I need dev and beta sites hosted on the same server as the production environment (let’s let that fly for practical reasons).
To keep things simple, I can accept the same protections in place on both dev and beta — basically don’t let it get spidered, and put something short of user names and passwords in place to prevent everyone and their brother from gaining access (again, there’s a need to be practical). I realize that many people would want different permissions on dev than on beta, but that’s not part of the requirements here.
Using robots.txt file is a given, but then the question: should the additional host(s) (aka “subdomain(s)”) be submitted to the Google Webmaster tools as an added preventive measure against inadvertent spidering? It should go without saying, but there will be no linking into the dev/beta sites directly, so you’d have to type in the address perfectly (with no augmentation by URL Rewrite or other assistance).
How could access be restricted to just our team? IP addresses won’t work because of the various methods of internet access (meetings at lunch spots with wifi, etc.).
Perhaps having dev/beta and production INCLUDE a small file (or call a component) that looks for URL variable to be set (on the dev/beta sites) or does not look for the URL variable (on the production site). This way you could leave a different INCLUDE or component (named the same) on the respective sites, and the source would otherwise not require a change when it’s moved from development to production.
I really want to avoid full-on user authentication at any level (app level or web server), and I realize that leaves things pretty open, but the goal is really just to prevent inadvertent browsing of pre-production sites.
Usually I see web server based authentication with a single shared username and password for all users, this should be easy to set up. An interesting trick might be to check for a cookie instead, and then just have a better hidden page to set that cookie. You can remove that page when everyone’s visited it, or implement authentication just for that file, or allow access to it just from the office and require people working from home to use VPN or visit the office if they clear their cookies.