NosferatusCoffin, those kinds of URLs will be problematic since you can't remove the session ID using normal methods. Even if the GSiteCrawler...
Can you PM me your URL so that I can take a quick look? How do the search engines index the site?
Yep, session-ids is a problem... but not just with the GSiteCrawler - it will kill just about any crawler, including those from the search...
The simple solution would be to do something like the old Experts Exchange site: keep the original posting on top, add a *large* block of "please...
Actually, the Adsense bot is the Googlebot :) -- they use a crawl caching proxy which means that it will try to crawl the content once and re-use...
Actually, it's fairly easy to prove / disprove: send yourself a link to a unique URL, track the accesses. My bet is that you'll never see any...
BBC = Bad, Bad Cpammer (english spelling)...
Most default forums are a killer for crawlers - they generate upwards from 10 URLs for the same content. This is not a problem with Sitemap...
Don't worry about it :-) It's just the stylesheet which the browser is looking for. Automated readers (like Google) don't use the stylesheet,...
They all had a chance to pre-register brand names, company names, trademarks, etc. I have a list of 3-letter .eu domain names that are still...
See + post here: http://groups.google.com/group/google-sitemaps/browse_thread/thread/f205fbfdafd68c56/a060130ccbba4c4b That's going to be an...
Have you tried phpSitemapsNG? It can run on a schedule, re-creating your sitemap whenever you need it. Good stuff!
Separate names with a comma.
