Author: Brian Cluff Date: To: listserv Subject: Re: [Az-Geocaching] Shout out to Snaptek
Brian Casteel wrote: > A solution would be that those who want the stats updated contribute to an
> account that could get a series of 5 PQs daily to keep at least part of the
> stats system operational/correct. Otherwise, the IP azgc.com runs on, may
> very well be permanently banned if updates continue.
I believe that gc.com actually likes what we are doing... and or they
consider us grandfathered in. Either way, they have been very nice in
letting us continue to scrape their site on a daily basis.
We only had to write some speed throttling code into the crawler so that
it wouldn't activate their auto blocker... We've been told as long as we
don't exceed an certain rate that we should be fine.
(text/plain)