HTTP 404 - File not found is a browser error message that we've all grown to know and hate. On a number of search engines, it gets to a stage where we feel that all their links will return this result? How is this happening, who is to blame and how do we fix it?
On reviewing my server logs, I had noticed around 1% of all requests from my site will return this HTTP error code. One of reasons for it was a stupid mistake I made around 6 months ago. I wasn't happy with naming of a couple of my files, so I renamed them without considering consequences:
- The files had been on my site for a few days
- During that time a couple of search engine bots had crawled through pages. A search engine bot is a software program that scours web sites for content and returns results to a search engine database. The search engine interface feeds off this to return links and descriptions to surfers when they have entered their search criteria
- Since I changed names of files after bot went through, pages in their original state no longer "existed".
- The search engine query results reflect database entries, pointing to wrong filename, visitor clicks on result - 404.... aaaaaaaaargh!
It would appear that some search engines only update their results once or twice a year, such is hugeness of task of spidering web these days. Last month, percentage of requests to my site that resulted in this HTTP error code was 2%, previous month to that - slightly higher. While I am happy that this is reducing, I am kicking myself that I didn't think things through six months ago - we live and we learn. 404 errors may also be caused through a malformed browser request (user error - wrong URL typed into address bar)
Save yourself shame and embarassment (and homicidal thoughts directed at you from frustrated potential visitors) - plan your site carefully before you promote to search engines.
If you do or have found yourself in same situation as me, there is something that you can do about it (dependant upon your hosting service set up). Instead of a visitor being directed to those rather horrible "file not found" pages, you can create custom error pages. Here is an example:
http://www.tamingthebeast.net/aaaargh The above link is incomplete which triggers a 404 response on my server.
By implementing custom error pages, you have a good chance of retaining visitor, especially if you include standard navigation buttons. It also acts as a means of apologising to visitor for inconvenience. It isn't just 404 error messages that you can apply this to. There are a number of error code returns that you could customise, all with goal of alleviating visitor stress and encouraging them to further explore your site. A listing of http error codes can be viewed here: