Home/ Questions/Q 3609676
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-18T21:39:28+00:00

I have this Rails app, with domain name abc.com . I am also having

  • 0

I have this Rails app, with domain name abc.com. I am also having a separate subdomain for Piwik stats, in this subdomain stats.abc.com.

Googlebot somehow listed some of the links with my subdomain too.

http://abc.com/login
http://stats.abc.com/login
http://abc.com/signup
http://stats.abc.com/signup

The ones with stats will reference to the same page in the app, but are treated entirely different website.

I have put in robots.txt in stats after this matter, but wondering if there is any appropriate way to block this because I may have new subdomains in future. Here’s my content in robots.txt

User-agent: *
Disallow: /

Thanks.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Robots.txt will block it just fine. It’s just important to remember BEFORE you publish a site – Google is pretty fast. Some search engines ignore robots.txt. Best thing to do is not have subdomains that don’t really fit you situation. I recommend reading the true purpose of subdomains. You should not be serving the same site on different domains. You should use a 301 direct or have different contents on different (sub)domains… Unless stats.abc.com contains different material, it shouldn’t be a subdomain. What exactly do you need so many subdomains for?

    You could detect the user-agent, and if it’s a bot, return a 404 too

    • 0