Comments on: 404 Search Page…

By: Exordium

Exordium — Tue, 30 Nov 1999 00:00:00 +0000

404 Error Search Redux
I noticed an entry over on Antipixel about Erik’s code for redirecting “404 File Not Found” errors to a search page. Well, I looked at the code and decided that I wanted to play around with it to make it

By: Ian Gregory Online

Ian Gregory Online — Tue, 30 Nov 1999 00:00:00 +0000

Seemed like a good idea at the time
ScriptyGoddess posted a link to tip from NSLog(); suggesting a cool way of automatically searching from 404 pages, which I implemented a couple of days ago. Unfortunately I didn’t take Googlebot into account, and when it visited today it generated

By: Ian Gregory Online

Ian Gregory Online — Tue, 30 Nov 1999 00:00:00 +0000

By: tarun

tarun — Tue, 30 Nov 1999 00:00:00 +0000

Why not just have it check if it is a robot searching the site and spit out a standard page versus a user searching a site, or just limit number of queries per unit time…

By: The Long Letter

The Long Letter — Tue, 30 Nov 1999 00:00:00 +0000

No more 404’s
I just implemented a very nice little PHP script for my website that ties into my site search function — the end result being that my site no longer has a “404 File Not Found” error page!

By: Rob

Rob — Tue, 30 Nov 1999 00:00:00 +0000

To help the googlebot problem I set my script so that you can search using a directory in my weblog (such as /blog/searchterm) and not just /searchterm. If you just go to /searchterm it redirects to the standard 404 page. Im sure its possible to get length of the request_uri and see if its longer than say, 20 characters, and if so just send them automatically to the 404.

By: Kristian

Kristian — Tue, 30 Nov 1999 00:00:00 +0000

A thought: Perhaps a check before it executes any other code (like those big database queries) it checks the useragent string to see if GoogleBot is listed in it and simply dump out a standard 404 response? Simple regex, right?

By: Clint

Clint — Tue, 30 Nov 1999 00:00:00 +0000

You *are not* making it easier. If someone is going to DDoS your site they can do it *just as easy using mt-search.cgi*. Neither way is “easier” or “harder”, if someone is going to do something, they’ll do it. If you’re worried about the mt-search.cgi then remove it, it is the source of your problem

By: Ian

Ian — Tue, 30 Nov 1999 00:00:00 +0000

Just been chatting to Brian about this - he’s suggested adding a check to see what the server load average is before carrying out the search, and if it is over a present value simply return a standard 404 page. I might try making this modification when I get a second, but I don’t think there is an easy way to make it cross-platform.

By: Jennifer

Jennifer — Tue, 30 Nov 1999 00:00:00 +0000

This is in the trackback pings, but I think I need to update this post based on the problem he encountered… in the off chance people implement this without reading the “warnings”. I think this IS a great work in progress and it’s great idea… but it still needs some work.