Testing Cmas vs Xmas in Google Adwords

I remember a teacher pointing out when I was just a kid that using "xmas" to stand for Christmas pretty much removed the whole point of Christmas. It removed "Christ." So when I shorten it, I've always had the habit of using "Cmas" instead. Makes a lot more sense to me.

So when I tried running some Google Adwords ads related to Christmas products, I wanted to use Cmas ... but knew that most people would probably better recognize Xmas. So I decided to try both and see which one got a better click through rate, or CTR.

I ran two ads that used Xmas in the headline, and two identical ads that used Cmas in the headline. Of course you cannot perfectly control all other variables in an outside setting like Google Adwords. The position of the ad is one such factor; the other ads or organic listings running nearby could also alter someone's actions; etc.

So to some degree, I have to consider this lack of control in the results, but I'll point out that the best performing ad in terms of CTR was only in the second best Average Position (4.8), and was nearly in the same position as two other ads (5.1). The worst performing ad Averaged a Position of 4.3. On that factor alone, you would expect it to have more visibility and therefore more clicks.

In fact I found what you might have predicted: Xmas performed 92% better than Cmas in terms of CTR (if you combine the two ads for Xmas and the two for Cmas). And this was verified as statistically significant through this handy calculator, which I recommend you use if you are running your own tests like this, lest you assume valid results too soon.

Now this isn't to say that Cmas wouldn't outperform Xmas in another setting, for a different audience or whatever. But this is just a simple Christmastime example of why it's important to run different ads in your campaigns and see where you get better results ... as long as you have enough search volume to legitimately test them.

One caution: remember that CTR isn't always your test subject. If I offer a "Free Science Fiction Book" in my ad, I might get a lot more clicks than if I tell people the book costs $12.95. But if the book does in fact cost $12.95, all the "free book" clicks (a high CTR) would be useless and a wasted investment; those clicking on the $12.95 ad (a low CTR) would be much more targeted to actually coughing up money for that book. In short, not a bad idea to weed out the wrong audience with your ad; save money on those clicks for a better audience!