404 Not Found and
301 Moved Permanently

(Basic programming for beginners)

You have updated your website and are happy to have changed the names of your pages so that they are easier to navigate and their names reflect what they contain. Finally, “about_fr”, “quote_fr” pages from your french side of your site are removed. Welcome to the names that say something, “devis” and “apropos” pages. About those changes, have you thought about the old indexing from Google? Do you have links that come from other websites that you are not responsible of? If you do not deal with that, will it affect your site?

Yes, it can in part have a negative effect. Maybe even on the ranking of your site. Dead links mostly result in a 404 error (not found, topic lightly discussed later in the text or read this ), which is against the ideology of easy surfing, especially if it is not personalized. According to my deductions, the search giant would say that it is better to do a “redirection” for the pages permanently moved, but not just any way. Simply redirecting to the other page using a “.html” document does not tell Google’s crawlers that your page has moved and to keep the “Rank” of it. In addition, the “Googlebot” could perhaps see this as an pitfall for the user who does not accept simple redirect, such as those of advertisers. The way Google proposed is a permanent redirect (code 301).

A 301 redirect can be done different ways. If you can access to your server’s .htaccess document, it’s easy to set up the necessary redirects. Important note, the .htaccess document starts with a “.”, which makes it normally an invisible document on your computer, so, before downloading it, change its name. Modify it temporarily, for example, by adding 000 in front of the “.” (000.htacess). At the end of the changes, after you have uploaded it to your server, at that moment, you can remove all three zeros to give it its original name. The small modifications to do are to add the lines for the address changes, one line per redirection:

In my case, I customized the 404’s error document. Doing that, it became at the same time the 301 redirection page (redirect 301 are removed from .htacess document). As you can see in the image above, my server will use the 404.php document for URL errors. So when a user enters a URL that is not recognized as part of the site, the PHP document checks if the requested page is in those that have moved.

Sample script document 404.php

«$_SERVER[‘REQUEST_URI’]»

This code will fetch a part of the URL address, the one found after the .com, .net, .org or .ca, and will also captured the variables (ex.: www.MyWebSite.ca/YoBozo.php?variable=here). When the capture of the URL is equal to one of the old addresses, it will be indicated as “301 Moved Permanently” for the Robot, and then redirected to the correct address. In case the capture of the URL is not found in the list, the “else” option will be used and will include the 404b.php error page. At this moment the user receives in his browser the custom error page. Here, as the page to be included is also PHP script, we can take advantage of the variable $_URLvar and include it in the error page to display to the user the exact address that was asked to the server by his browser.

Sample script of the 404b.php document that will be included in 404.php if the page is not a page to redirect.

You can download the documents here and edit them for your use:
.htaccess for redirect and error pages in HTML (download)
.htaccess for redirect and error pages in HTML and PHP + 404.php and 404b.php (download)

Do not forget to create or modify your personal error pages, here is a main list:

  • 301: document moved permanently
  • 302: document moved temporarily
  • 401: unauthenticated user (if you have protected folders with a password)
  • 403: denied access (if you have protected folders with a password)
  • 404: file not found
  • 418: the HTCPCP server is a teapot.
  • 500 et 503: server error

You can find the full list on Wikipedia.
To make some tests to see what “Googlebot” may see, you can use this sites.