We archive documentation for every release, but the top google result is often several years old instead of from the latest release. If we use robots.txt to disallow indexing the old stuff entirely, then a search including that version number would no longer be helpful. The old docs would still be accessible by navigating our site, but not in the google result.
There are several unofficial mirrors that are typically not updated at all, so if we disallow the archives, I’m concerned that searches for deprecated APIs will land on those mirrors instead of more appropriate places like our changelog/deprecation notice/man page.
Is there a way to say that a specific version is preferred without entirely de-indexing the old version? The SEO Starter Guide doesn’t seem to have anything to say about this scenario.
Your best bet is to create a google xml sitemap and set the priority of your content pages higher than the archives. This is no guarantee of anything, but it’s probably the only thing you can really do to attempt to control it. I wouldn’t worry about it too much, just put a notice about the deprecated APIs and a link back to the home page.