seotworank: Google Hides Search Referral Data with New SSL Implementation

Posted by Aaron Wheeler

On Tuesday, Google announced that signed-in users will, by default, be routed to the SSL version of Google (https://www.google.com). Before Tuesday, most users used non-SSL Google for their searches. Now, according to Google, "...a web site accessed through organic search results on http://www.google.com (non-SSL) can see both that the user came from google.com and their search query... However, for organic search results on SSL search, a web site will only know that the user came from google.com." The effects were obvious immediately. Here's a screenshot of our GA account showing the quantity of "(not provided)" keywords going up from Sunday to today:

Google Analytics (not provided) visitors

Clearly, the inbound marketing community isn't thrilled. Take Ian Lurie of Portent, for example: he declared war with Google outright. Having a bunch of "(not provided)" referral keywords in Google Analytics is definitely not pretty. Fortunately, as Avinash Kaushik explains in this Google+ post, there's something you can do to at least gauge the effects on your analytics, and as Rand will explain, the effects aren't as devastating for most users as they could be. Yet.

In this emergency Whiteboard Friday, Rand will go over the changes Google has made, why it happened (and why it really might have happened), and what you can do to stay calm and fight back. Let us know how this change has affected your sites in the comments below!

Video Transcription

Howdy SEOmoz fans. Welcome to a special emergency edition of Whiteboard Day Agnostic We'll Interrupt Any Day to Do This. Unfortunately, Google has made a big change to the way that they are serving keyword referral data from their search results, and this is going to have an unfortunate impact on all of us who do white hat SEO, who do web analytics, and who try to learn from this practice.

I want to try in this Whiteboard video to explain why this has happened, what Google is doing, why they claim they're doing it, and then also explore some of the reasons that they might actually be doing it, and try to provide some actual information about what folks in the web analytics and SEO spheres can do since this data may become less available.

So let's start by explaining what happens when you do a Google search today. For example, I have done a Google search here for "learn SEO." I click the Search button and some results pop up, and here's this nice learn SEO, SEOmoz, www.seomoz.org, learn SEO, and then there's an ad over here, "Learn SEO from PayMeBucks.com." Click on my ad. Dude, I need your visits bad. That probably would not get approved by the AdWords people, but you can get the idea.

Now previously, if I were to click this result or this result, the web analytics tool, whatever it is - your Webtrends, your Omniture, your Google Analytics - at the other end would get some referral data, so with your log file, get some referral data about what sent that visit, which keyword sent that visit. So in this case, it would be "learn SEO" sent a visit from Google.com search over to my website. It would track whether it's a paid or an organic ad.

This is changing. It is changing only for folks who are logged in. If you are searching from Google and you are logged in, this will be changing so that the logged in behavior, the keyword that referred the visit will be shown as (Not provided). This will show in your web analytics. That's what Google will say. They will use these parenthesis. That's how you can see it in the Google Analytics dashboard currently. However, if you click this paid search ad, they will still be providing the keyword "learn SEO." So logged out behavior in purple here. Logged out behavior always gets keyword "learn SEO" as the referrer. Logged in behavior gets keyword (Not Provided) if you click on organic results. But if you're paying Google, you will still be able to see the referral information.

Now Google claims they're doing this to protect user privacy so that users who are logged in will by default not be showing their searches to the websites that they visit. Unfortunately, I think that there are a lot of people in the search world and folks who observe this who have rightfully stated, well, if Google were trying to protect privacy, they've already to some extent done that by providing a secure search - https search, which is what's doing this as well, the SSL search - for those people who would not like to provide that information. Some very small portion of people do use that form of Google search, the sort of protected search.

So it's already available. The reason they're doing this by default I think that many people suspect . . . I'll link to a great article by Ian Lurie of Portent Interactive, who I think prognosticates or posits the actual reason for this is that ad networks today are being very successful using search referral data from visitors, and they're able to leverage that data across multiple websites. So Google is hoping to remove that ability and be the only ad network that can be aware of your search behavior, thus sort of blocking out other providers using their near monopoly in search to exclude other people from being able to use this data,

That's frustrating. It's sad. It's upsetting. It certainly doesn't fit with what we know about Google. But I think the unfortunate thing here is that those of us in the web analytics/SEO sphere are going to have a tough battle to fight from a PR angle because Google can play the "no this is to protect your privacy" card and use that as their excuse. Of course, if that were the case, it seems very odd that you can pay them and still get the data. But I'm going to reserve judgment on that, and I'll let folks make their own decisions. I do think it's very important that we not just get upset about this, but we also think about what we can do actionably. Anytime a major player in the search world or social world or inbound world makes a big change, we need to figure out what is it, how is that we can best respond, how can we use data, how can we continue to be great marketers.

There are a couple of things that I would recommend. First off, you should be measuring the quantity and percent of the lost keyword data. That is a very important metric that you're going to want to track over time. To do this, you simply go to your web analytics tool, you grab the number of (Not Provided) keywords or referrals, visits that came to, divide that by your total visits from Google organic, and you will get the percent of search referrals affected by this. You want to track this over time because you want to know if that's going up, if more people who are logged into Google are searching and finding your site, what percent of data you're losing, whether this is going to be a big problem as Google rolls it out more broadly, and you can see some data from SEOmoz.

So let's take a look at our own data. This is from Sunday to Thursday of this week, so ending yesterday. We're filming this on Friday for release tomorrow, Saturday. You can see (Not Provided) was 1,062 or 1.2% of the visits over these 5 days. However, the number is going up. So as of Sunday, we had zero visits that did not contain any keyword data. Monday had 90. Tuesday had 111. Wednesday had 381. Thursday had 421. That is 2.2%. So you can see that we've lost keyword information on a little over 2% of our visits and climbing. So this is frustrating. Google has said that they expect this will be less than 10% for most websites. So we hope to continue to get 90% of the data.

That leads me to number two. You can continue to leverage data from sources like the existing Google data, which should be hopefully around 90% of what you have today, Bing and Yahoo data, of course, which are responsible for around anywhere between 10% and 20% of your search referrals depending on your industry and niche, and of course, your internal search query data. This data is invaluable not only for doing keyword research and targeting, but also figuring out conversion rates, trying to optimize for those visitors, make their user experience better. It's really only for white hat types of activities. So it's frustrating that Google pulled this, rather than maybe tackling something more black hat focused. But we have what we have.

Number three, if you do feel strongly about this issue, there are lots of opportunities - I don't want to say complain - but lots of opportunities to let Google know how you feel. This is a change that they are making, and they are currently planning on making and rolling on and have been rolling out. But that doesn't mean that they might not backtrack if user feedback is overwhelmingly negative, and certainly that would be nice for those of us in the analytics sphere who like to use this data.

So you can obviously blog about it, write about it. You could even write to your congressional rep. There are several forums. The Google blog post announcing this accepts comments. The Google Webmaster Tools forum certainly accepts comments. You can also contact your AdWords representatives and let them know that you're not totally thrilled by this move either. Remember AdWords data is still passing the refer. It's organic search that is affected.

So hopefully this won't affect too big a percentage of search queries and thus will still continue to have some good data, but given Google's efforts to try and make more people be logged into Google Plus, to Gmail, to Google hosted apps, I don't know. There is a lot of, I think, fear and uncertainty right now in the analytics world.

But with that said, you have some actionable things you can do. You should definitely start tracking this data, and hopefully we will see you again next week for another edition of Whiteboard Friday, rather than an emergency, interrupting version. We hope we don't have too many of these. Take care everyone.