[Az-Geocaching] Another Download Format - XML

[Brian Cluff]

> Do you mind telling us about the resources SnapTek has dedicated to
> azgeocaching.com?

Currently we have 2 machines, one that serves the web pages and one that
server the database.  The database server also crawles the officials sites
web pages and parses out the information that we need.  That how we have a
local caches copy of the pages.

> How do you interface with the official site?  Do you crawl the pages or is
> there an api of some sort?

Every day at 2 the server gets a list of archived pages off a list of
known archived pages.  Then at 3 it pulls a list of all the arizona caches
and crawles them one by one to make them all current pages.  After that is
runs them all though the same parsing engine, wipes the old database clean
and replaces the data with the new information.

This is definatly not the best way to do it, and I want to switch, in the
near future, to just pulling the latest cache activity page sevral times a
day and updating just the pages that have actually changed with a complete
crawl a couple times a month to make sure we haven't missed anything.
That way we are more up to date since we can check for new stuff once an
hour or so and we aren't pulling down 570 cache pages and climbing every
day.
The upside could be that this new type of crawling could allow us to keep
everything up to date, meaning cache stats for the whole world.  The
biggest reason that we DONT wan't to do that is the amount of traffic that
would bring to our little non-profit web site.  It's very manageable now
and has plenty of room to grow, but with the whole world going to it, it
would easily get out of hand and we would have to start figureing how to
get money to support it, and thats just too much for a little hobbie web
site.  What we have been thinking of doing is going ahead and doing that,
but then letting someone step in from another state and kinda giving them
a franchise type of thing so they could be the official snaptek stats page
for their particual state and we would then give them a download of the
data they needed for their state and everyone is happy.  That would also
releave a ton of stress from the official site and everyone should be
happy.

Oh man, I'm rambling again..... probably badly typed as usual, and as
usual, I'm not going to bother to reread what I have typed, so you will
get the usual often times sureal reading adventure from me :)

Brian Cluff
Team Snaptek

BTW the code that I use for the themes is an open source project that I
released to the public some time ago at phpthemes.sourceforge.net.  If you
are interested in it, it's written in PHP and licensed under the LGPL.
It's a much older version with little to no documentation due to my
putting some fairly heavey development on lately.  If you are serious
about wanting to use it, lemme know and I can give you a snapshot of my
current code.