SQL Query Help

[fluiduk]

Hi All

I am experimenting with Sphider as a search engine for a site I am working on. However I cant stop the spider from indexing some pages that I dont want in there. So to get rid of them I will need to run an sql query.

I need the query to do the following.

DELETE all from Links where '**' does not contain product_details.php

Have been on the goog for ages and not found anything. Any ideas/

Thanks
Aaron

 

[fluiduk]

The problem with robots.txt is that it will stop the spider from spidering the pages, I want it to spider but not to index.

Basically I only want it to index pages that have product_details.asp in the URL

Thanks
Aaron

 

[fluiduk]

Is that not the whole point of spidering web pages - to index them?

TrUz

No in this case. We are only using the spider to make an index of all of the product pages on our site. To make the on site search better

Its not for google etc