A session at SES San Francisco last week got me thinking about about site migrations. They’re very easy to screw up, and here’s a few ways you can do it.
1. Forget the Robots.txt File
In a world of distributed computing and automation, many development and IT departments use automated scripts to deploy code to one or more servers. It’s far too easy to forget that the robots.txt file from the staging or development site shouldn’t go to the live version. Nothing like blocking an entire domain to search bots completely on accident.
You can dynamically generate a robots.txt file from within the CMS or platform – leaving the options in the database can help ensure that a wayward robots.txt file doesn’t get pushed. There’s even a WordPress plugin for this (by yours truly).
You should also be sure to avoid putting your robots.txt file under version control. This will prevent any deployment system that uses version control like git or svn from pushing a bad robots file.
2. Leave a Bad Meta Robots Tag
Keeping on the crawler access theme, it’s far to easy to forget settings or templating details that insert a meta robots tag. Crawl the staging site before deployment and the product site after. Look for wayward meta tags.
3. Take a Step Down in SEO Quality
Launch a new platform/site whose pages don’t match up with the previous site’s standards. By that, I mean having a new site without the same on page SEO features: titles, descriptions, headers, semantic markup. This is a big task, one that requires SEO input from the development stages.
Get the SEO department or agency involved from the beginning development and spec phases. Make sure SEO is baked into the new site, and be sure to migrate all the SEO data from the old site to the new.
4. Don’t Do a Content Audit
Search engines love quality content. As a website ages, much of its content becomes outdated or irrelevant. Enter the content audit – something that should be done quarterly. Evaluate all content on the site: what needs to move to the new site? What pages should be removed?
Pages that are going to be moved to the new site/platform should have the same URLs or be 301 redirected. Pages that don’t make the cut should throw an appropriate status code. If there’s an equivalent new page on the site, be sure to 301 redirect the pages that get dropped – otherwise a 410 Gone status would be appropriate.
5. Believe Nothing Will Change in Your Numbers
Site migrations are a huge thing. Moving from one platform to another often means changes URLs and changing content. Moving domains means 100% of the URLs change. Even with the best plan, you should expect the migration to impact traffic and other metrics.
When done right, these effects are minimal and the site should quickly recover.
Post image by USFWS Mountain Prairie.