Question

I have multiple web sites for my clients and each client has a directory labelled articles. I just inherited this system and I until I can fix the issue I found, I am looking for stopgap solution, one that will eliminate the 404 errors after a file has been deleted.

All these directories have static pages for the articles, as well as an index page that lists all the articles.

Based on the logs it generates many errors from over the years. I can just imagine it is causing havoc the search engines as well.
With little knowledge of mod rewrite that I have, I managed to put this together which I plan to put into place within the Apache configuration. Before I do, is this good solution or is there something else I should do.

<Directory "/home/www/public_html/clients">
    Options +FollowSymlinks
    RewriteEngine On
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{HTTP_HOST} ^(.+)$ [NC]
    RewriteRule ^(.*)$ http://%1/articles/index.html [R=301,L]
</Directory>

RewriteCond %{REQUEST_FILENAME} !-f is looking to see if the file exists and if does display it while ignoring the rest of rewrite.

RewriteCond %{REQUEST_FILENAME} !-d is looking to see if the directory exists and if does display it while ignoring the rest of rewrite.

RewriteCond %{HTTP_HOST} ^(.+)$ [NC] grabs the domain to passes it to the last rewrite as 301 redirect.

I have it working locally and like a few opinions before making live.

Answer 1

1. No need for RewriteCond %{HTTP_HOST} ^(.+)$ [NC] line — just replace your %1 in RewriteRule by %{HTTP_HOST}

2. From rewriting point of view the solution is OK

3. From SEO point of view — not so sure — better have it 404 or 410 instead (since the article is no longer there). I think it will be better to display custom page to the customer while sending 404 or 410 to the browser:

   • Browser/Search Engine will see error code
   • User will see explanation instead while offered to see related to the requested URL pages/short index.

4. From User point of view — not good: I would like to know that URL/article is no longer available straight away (see #3) and browse your site around if I find it useful, rather than seeing some irrelevant (at first) index page and telling myself — I do not remember clicking this link, and go back to search engine/referral and click again. If I will see the same index page again — I understand (most likely) that something wrong with that page and just turn away (unless I really interested in that page’s content).

---

UPDATE:

I would do it this way:

Options +FollowSymlinks
RewriteEngine On

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} ^/articles/ [NC]
RewriteRule .* /articles/notfound.php?url=%{REQUEST_URI} [L]

1. Redirect non-existing URLs to notfound.php (or whatever else name it may have) ONLY if requested URL has anything to do with articles (URL starts with /articles/)

2. On that page (has to be dynamic (PHP or similar) and not static HTML) respond with 410 Gone Error Code (for browser/spider) and display a page explaining that this URL is no longer here but you can look at these links (and some useful links — could be mini-index/recent articles etc) — this is for user.