BeauScott.com

David Wallace @ SearchEngineWatch.com has written an interesting article on the effect of new-age web technologies like AJAX and Web2.0 on search engine crawlability. While I wasn’t aware of the impact of CSS on search engine behavior, and the impact of Web2.0 applications like Wikis and other community-driven technologies is somewhat obvious, he makes a very good point of the potentially negative impact of AJAX (or excessive AJAX) on search engines’ abilities to crawl and index your site. Wallace points out that because of the nature of AJAX, and the inability of search engines to interact with JavaScript, AJAX enabled websites will experience some hindrance of crawlability. While this is true, this can be minimized by following some basic guidelines when implementing AJAX on your web application:

• Do not make your application UI completely AJAX driven unless it is in an environment that you do not wish to have indexed. For example: Google’s GMail is a very well-done AJAX application that effectively uses no static HTML. Every component is rendered via JavaScript in some fashion. However, since this UI is only displayed to the user after they have signed in, a search engine would never see the content anyway.
  An prime example of AJAX gone wrong is Gucci.com, who’s entire public site — as attractive as it is — is unreachable without JavaScript support. Having said that…
• Try viewing the page with JavaScript disabled in your browser. This will help you to see what the search engine will see. A good tool to quickly toggle JavaScript support in FireFox is Chris Pederick’s WebDeveloper Plugin. Simply right-click the window, go to Web Developer -> Disable -> JavaScript.
  If you try this on Gucci.com, you’ll notice that the only thing you get is a blank screen. They may be trying some clever hack like User-Agent detection to show search engines a different set of content, but judging from a simple Google search on their domain name, I severely doubt that is the case (notice that the first few search results are blank except the domain name itself).
• AJAX widgets add a lot of pizazz to a user’s experience on your website, but they should be used to deliver minimal content to the page, if any. Utilities like the HelpBalloons I offer here do bring content to a page dynamically, but are not intended to do heavy lifting. They are meant to add context to
• Use the <noscript> tag to add static content to areas that would regularly have valuable content populated via AJAX.

If used properly AJAX can significantly improve a user’s experience by adding standalone application feel to an otherwise boring web site. If used improperly, AJAX can severely hurt your site’s potential by cutting off search engine indexing.

Please feel free to suggest more best-practice guidelines to append to this list.