Semi-representing webmasters everywhere (and in frustration due to the large
number of robots that don't adhere to it), please make sure that the
robot/spider that you choose adheres to the robot exclusion standard so that
it doesn't index or search any pages that the webmaster has deemed off-limits:
http://info.webcrawler.com/mak/projects/robots/exclusion.html
If it doesn't, there are a lot of webmasters (us included) that just tend to
block access for the entire offending network; that's probably NOT in your
client's best interest.
/jason
>Hello,
>
>I have a client that wants perform a daily search over the internet or at
>least over the main search engines of each country with a main office.
>
>I need that the search show me only the last modifications or aditions,
>something like a news clipping of the company.
>
>Can someone help me?
>
>Carlo