Look how old this is! I post at SearchCommander.com now, and this post was published 18 years 10 months 16 days ago. This industry changes FAST, so blindly following the advice here *may not* be a good idea! If you're at all unsure, feel free to hit me up on Twitter and ask.
Did you ever wonder what pages of your website get “indexed” or “spidered” by a search engine? Improper design can make some of your pages invisible to the search engines.
John Alexander, of Search Engine Workshops brought this Search Engine Spider simulator to my attention in his “SEO tip of the day”.
Just type in your URL, and you’ll get a list of every page the spider can get to. Are some of your pages completely missing? Find out here.
