Question

This is my robot:

User-agent: *

Disallow: /feed/
Disallow: /trackback/
Disallow: /wp-admin/
Disallow: /wp-content/
Disallow: /wp-includes/
Disallow: /xmlrpc.php
Disallow: /wp-

Sitemap: http://chineselearnonline.com/sitemap.xml

Strange, I took that from a blog. But I noticed it made Google stopp indexing my WordPress site.

What is there problem there?

Answer 1

It sounds like, when you moved your site to another server, you may have inadvertently changed the base url that was used to access your site. For instance, you mentioned that your site used to be accessed at http://chineselearnonline.com/v4, and now it is located at http://chineselearnonline.com/, without the v4.

Considering the site topic, it’s possible that others have linked to you, either through blogging or other forms of sharing. If the search engines try to traverse your links and find that they’re broken, your ranking in the search engines will decrease.

Note that this isn’t the same as changing the location of the code on the actual server, just the url that points to that folder.

To resolve this, you may be able to edit your htaccess file to add a rule to 301 redirect all of the /v4 links to the new base url, so that any users, or search engine entities, would automatically redirect to the valid urls. This also tells the search engines that your content has permanently moved to the new location.

If your links disappeared in Google, establishing the 301 redirect may fix any broken links on the Internet and prevent everyone from needing to update the link physically in their blogs and websites. This should help build your SEO back up.