[Date Prev][Date Next][Thread Prev] [Thread Next][Date Index] [Thread Index]

Lycos out of Control


  • From: Astachoth  
  • Date: Wed, 17 Sep 2003 03:54:21 EDT

Dear Gunnar & Ringlink,
     I just noticed that the Lycos search engine is "stuck in a loop". I like having my list pages indexed in the search engines, but Lycos is following the next links at the bottom of the list pages in my rings. It now has over a hundred links to my same list pages, because of the "offset=" extensions and different site ids, which are in other members codes.
     Is there a way to prevent this from happening without adding "robots noindex" meta tags to the list pages? I don't want my site to appear to be "spamming" the search engine. The other search engines don't appear to be having the same problem. They usually list a "list page" only a couple times and then stop.
     Also, wouldn't it be a good idea to cause a "404 not found" page to appear when someone tries to access a deleted WebRing within a WebRing system. Currently, if someone previously ran a WebRing, (which is indexed in a search engine), on a particular system, it will remain in the search engine even after deletion, because of the error page. The search engine doesn't realise that the WebRing has been deleted, because a valid HTML page still displays. Eventually, the search engines will become filled with links to deleted rings resulting in error pages.
     I have noticed that webring.com has the same problem. Deleted rings remain in the search listings.

Sincerely,
Tzabaoth


Follow-Ups from:
Gunnar Hjalmarsson
Gunnar Hjalmarsson
Gunnar Hjalmarsson
Gunnar Hjalmarsson

[Date Prev][Date Next][Thread Prev] [Thread Next][Date Index] [Thread Index]