I've made the mistake of generating almost all of my content as it's requested (from XML files), via Javascript. Now I've learned that this is horrible for SEO. I've also since learned about progressive enhancement and am sad that I missed that boat as well.

I've been researching my remaining SEO options and came across Google's AJAX-crawlable suggestions, but subsequently read a lot of well-founded opinions on why hashbangs are bad.

Whatever option I use, deeplinking is also important to me. I'm currently using JQuery-BBQ, and it works well for deeplinking, but offers nothing to search engines. I'm thinking about switching to an option that better supports HTML5 history state, like History.js. This will help to clean up URLs and avoid hashes, but I'm not sure it will actually make a difference for SEO, since all of my content is generated dynamically from XML files.

What options do I have remaining to achieve decent SEO? White-hat only please, no cloaking.

Note that my concerns are primarily about SEO, not accessibility -- this is a personal portfolio site and I'm okay with my content being invisible to some humans, but I'm not okay with it being invisible to spiders. Here's the site (in progress, but mostly there architecturally).

Edit: yes, I know there are a number of SO questions similar to this one, but most of them have answers that would require significant restructuring. I'm not looking for an answer that I would use if I was starting from scratch, I'm looking for something to patch over my previous ignorance as gracefully as possible...


One is you have to interpret the request coming to your site and if it is from a google bot you can return html version of your site and if it is normal user then redirect to normal site.for returning html version of your site without much effort of changing the code you can do screen scraping of your site and return the rendered html response if its a google bot. you can do this in couple of ways either use a headless browser like phantomjs with Nodejs server or use python beautiful soup or java htmlkit.

  • yeah, the google-ajax-crawlable page i mentioned does suggest screen scraping to generate html. i'll look into that, thanks. – ericsoco Jul 10 '12 at 16:51

Start over and make the site in such a way that it will work without JavaScript. Then (optionally) use JavaScript to enhance the experience at will.

  • i'm not going to start over. this is not helpful. – ericsoco Jul 10 '12 at 16:49

Despite the downvotes and suggestions to start over, I found an answer. I generate a stripped-down plain html version of my content using php, which has the additional benefit of providing a usable (if not ideal) experience for anyone with javascript disabled.

I also use a media query to display the javascripted DOM elements if viewport size is large enough to accommodate them. Since spiders don't do much with CSS, they see only the php-generated plain html, and have good content to suck down for their search engines.

See transmote.com for an example of this in action; reduce your browser width < 640px or turn off CSS with a tool like Pendule to see the php-generated plain html content.

Your Answer

