Question

I want to prevent the robots from accessing URLs that end with /new. I am modifying my robots.txt file as follows:

Disallow: /*/new

Is this the correct pattern to use to disallow access to all urls terminating in /new?

Answer 1

Yes seems google supports this

Try https://www.google.com/webmasters “Site Configuration > Crawler access” to test your robots.txt file for given URLs

Here is the result of this tool

http://www.***.com/new/asdasd
Allowed by line 6: Allow: /

http://www.***.com/asdasd/new/
Blocked by line 8: Disallow: /*/new
Detected as a directory; specific files may have different restrictions

http://www.***.com/asdasd/new   
Blocked by line 8: Disallow: /*/new