Tag Archives: Google Webmaster Blog

Webmaster Tools Search Queries data is now available in Google Analytics

Posted by Christina Chen, Product Manager – Tuesday, October 04, 2011
Webmaster level: All

Earlier this year we announced a limited pilot for Search Engine Optimization reports in Google Analytics, based on Search queries data from Webmaster Tools. Thanks to valuable feedback from our pilot users, we’ve made several improvements and are pleased to announce that the following reports are now publicly available in the Traffic Sources section of Google Analytics.

  • Queries: impressions, clicks, position, and CTR info for the top 1,000 daily queries
  • Landing Pages: impressions, clicks, position, and CTR info for the top 1,000 daily landing pages
  • Geographical Summary: impressions, clicks, and CTR by country

All of these Search Engine Optimization reports offer Google Analytics’ advanced filtering and visualization capabilities for deeper data analysis. With the secondary dimensions, you can view your site’s data in ways that aren’t available in Webmaster Tools.


To enable these Search Engine Optimization reports for a web property, you must be both a Webmaster Tools verified site owner and a Google Analytics administrator of that Property. Once enabled, administrators can choose which profiles can see these reports.

If you have feedback or suggestions, please let us know in the Webmaster Help Forum.

Making the most of improvements to the +1 button

Posted by Daniel Dulitz, Group Product Manager

Wednesday, August 24, 2011

Webmaster level: All

For the past few months, you might have used +1 buttons to help visitors recommend your content on Google Search and on their Google Profiles. We’ve just announced a few changes that make +1 even more useful.

First, the +1 button now lets visitors share links to your pages on Google+. If someone wants to start a conversation about your content, it’s easy for them to do so. Second, you can use +Snippets to customize the name, image and description that appear when your content is shared. Finally, new inline annotations help increase engagement after users see a friend’s recommendation right on your page.

Here are a couple of tips to help you take full advantage of these improvements:

+Snippets
The +1 button opens up your site to a valuable new source of traffic with sharing on Google+. +Snippets let you put your best face forward by customizing exactly what appears when your content is shared.

For example, if you’re running a movie review site, you might want visitors to share posts containing the title, movie poster, and a brief synopsis:

You may already be using this markup to build rich annotations for your pages on Google Search. If not, marking up your pages is simple. Just add the correct schema.org attributes to the data already present on your pages. You’ll set a name, image, and description in your code:

body itemscope itemtype="http://schema.org/Article"
h1 itemprop="name"This is the article name/h1
img itemprop="image" src="thumbnail.jpg" /
p itemprop="description"This is the description of the article./p
/body

For more details on alternate markup types, please see our technical documentation.

Inline annotations
Now, when a person visits a page that someone they know has +1’d, they can see a name and face reminding them to pay special attention to your content. Here’s how it looks:


Inline annotations let people see which of their friends +1’d your content

To add inline annotations, you need to update your +1 button code. Visit the configuration tool, select ‘inline’ from the ‘Annotation’ menu, and grab a new snippet of code.

Both sharing from +1 and inline annotations are rolling out fully over the next few days. To test these improvements right now, join our Platform Preview group.

Update later the same day, August 24, 2011: If you have any thoughts or feedback you’d like to share, continue the conversation on Google+.

PDFs in Google search results

Posted by Gary Illyes, Webmaster Trends Analyst

Thursday, September 01, 2011
Webmaster level: All

Our mission is to organize the world’s information and make it universally accessible and useful. During this ambitious quest, we sometimes encounter non-HTML files such as PDFs, spreadsheets, and presentations. Our algorithms don’t let different file types slow them down; we work hard to extract the relevant content and to index it appropriately for our search results. But how do we actually index these filetypes, and—since they often differ so much from standard HTML—what guidelines apply to these files? What if a webmaster doesn’t want us to index them?

Google first started indexing PDF files in 2001 and currently has hundreds of millions of PDF files indexed. We’ve collected the most often-asked questions about PDF indexing; here are the answers:

Q: Can Google index any type of PDF file?
A: Generally we can index textual content (written in any language) from PDF files that use various kinds of character encodings, provided they’re not password protected or encrypted. If the text is embedded as images, we may process the images with OCR algorithms to extract the text. The general rule of the thumb is that if you can copy and paste the text from a PDF document into a standard text document, we should be able to index that text.

Q: What happens with the images in PDF files?
A: Currently the images are not indexed. In order for us to index your images, you should create HTML pages for them. To increase the likelihood of us returning your images in our search results, please read the tips in our Help Center.

Q: How are links treated in PDF documents?
A: Generally links in PDF files are treated similarly to links in HTML: they can pass PageRank and other indexing signals, and we may follow them after we have crawled the PDF file. It’s currently not possible to “nofollow” links within a PDF document.

Q: How can I prevent my PDF files from appearing in search results; or if they already do, how can I remove them?
A: The simplest way to prevent PDF documents from appearing in search results is to add an X-Robots-Tag: noindex in the HTTP header used to serve the file. If they’re already indexed, they’ll drop out over time if you use the X-Robot-Tag with the noindex directive. For faster removals, you can use the URL removal tool in Google Webmaster Tools.

Q: Can PDF files rank highly in the search results?
A: Sure! They’ll generally rank similarly to other webpages. For example, at the time of this post, [mortgage market review], [irs form 2011] or [paracetamol expert report] all return PDF documents that manage to rank highly in our search results, thanks to their content and the way they’re embedded and linked from other webpages.

Q: Is it considered duplicate content if I have a copy of my pages in both HTML and PDF?
A: Whenever possible, we recommend serving a single copy of your content. If this isn’t possible, make sure you indicate your preferred version by, for example, including the preferred URL in your Sitemap or by specifying the canonical version in the HTML or in the HTTP headers of the PDF resource. For more tips, read our Help Center article about canonicalization.

Q: How can I influence the title shown in search results for my PDF document?
A: We use two main elements to determine the title shown: the title metadata within the file, and the anchor text of links pointing to the PDF file. To give our algorithms a strong signal about the proper title to use, we recommend updating both.

If you want to learn more, watch Matt Cutt’s video about PDF files’ optimization for search, and visit our Help Center for information about the content types we’re able to index. If you have feedback or suggestions, please let us know in the Webmaster Help Forum.

Posted by

The +1 Button: Now Faster

Posted by David Byttow, Software Engineer
Tuesday, July 26, 2011
Webmaster level: All

One of the 10 things we hold to be true here at Google is that fast is better than slow. We keep speed in mind in all things that we do, and the +1 button is no exception. Since the button’s launch, we have been hard at work improving its load time. Today, we’re proud to announce two updates that will make both the +1 button and the page loading it, faster.

First, we’ve begun to roll out out a set of changes that will make the button render up to 3x faster on your site. No action is required on your part, so just sit back, relax, and watch as the button loads more quickly than before.

In addition to the improvements made to the button, we’re also introducing a new asynchronous snippet, allowing you to make the +1 experience even faster. The async snippet allows your web page to continue loading while your browser downloads the +1 JavaScript. By loading these elements in parallel, we’re ensuring the HTTP request to get the +1 button JavaScript doesn’t lead to an increase in your page load time. For those of you who have already implemented the button, you’ll need to update the code to the new async snippet, and then you should see an overall improvement in your page load time.

To generate the new async snippet, use our +1 Configuration Tool. Below, you’ll find an example of the code, which should be included below the last g:plusone tag on your page for best performance.


If you haven’t already implemented the +1 button on your site, we’re excited for your first experience to be a fast one. This is a great opportunity to allow your users to recommend your site to their friends, potentially bringing in more qualified traffic from Google search. To those that already have the button, we hope that you enjoy the improvements in speed. Our team will continue to work hard to enhance the +1 button experience as we know that “fast is better than slow” is as true today as it’s ever been.

If you have any questions, please join us in the Webmaster forum. To receive updates about the +1 button, please subscribe to the Google Publisher Buttons Announce Group. For advanced tips and tricks, check our Google Code site.

Page Speed Service

Posted by Ram Ramani, Engineering Manager
July 28, 2011 – Webmaster level: Advanced

Two years ago we released the Page Speed browser extension and earlier this year the Page Speed Online API to provide developers with specific suggestions to make their web pages faster. Last year we released mod_pagespeed, an Apache module, to automatically rewrite web pages. To further simplify the life of webmasters and to avoid the hassles of installation, today we are releasing the latest addition to the Page Speed family: Page Speed Service.

Page Speed Service is an online service that automatically speeds up loading of your web pages. To use the service, you need to sign up and point your site’s DNS entry to Google. Page Speed Service fetches content from your servers, rewrites your pages by applying web performance best practices, and serves them to end users via Google’s servers across the globe. Your users will continue to access your site just as they did before, only with faster load times. Now you don’t have to worry about concatenating CSS, compressing images, caching, gzipping resources or other web performance best practices.

In our testing we have seen speed improvements of 25% to 60% on several sites. But we know you care most about the numbers for your site, so check out how much Page Speed Service can speed up your site. If you’re encouraged by the results, please sign up. If not, be sure to check back later. We are diligently working on adding more improvements to the service.

At this time, Page Speed Service is being offered to a limited set of webmasters free of charge. Pricing will be competitive and details will be made available later. You can request access to the service by filling out this web form.

Introducing new and improved sitelinks

Written by Harvey Jones, Software Engineer, Raj Krishnan, Product Manager, Sitelinks team
Tuesday, August 16, 2011 Webmaster level: All

This week we launched an update to sitelinks to improve the organization and quality of our search results. Sitelinks are the two columns of links that appear under some search results and ads that help users easily navigate deeper into the site. Sitelinks haven’t changed fundamentally: they’re still generated and ranked algorithmically based on the link structure of your site, and they’ll only appear if useful for a particular query.


Sitelinks before today’s changes
Here’s how we’ve improved sitelinks with today’s launch:

  • Visibility. The links have been boosted to full-sized text, and augmented with a green URL and one line of text snippet, much like regular search results. This increases the prominence of both the individual sitelinks and the top site overall, making them easier to find.

 

  • Flexibility. Until now, each site had a fixed list of sitelinks that would either all appear or not appear; there was no query-specific ranking of the links. With today’s launch, sitelink selection and ranking can change from query to query, allowing more optimized results. In addition, the maximum number of sitelinks that can appear for a site has been raised from eight to 12, and the number shown also varies by query.

 

 

  • Clarity. Previously, pages from your site could either appear in the sitelinks, in the regular results, or both. Now we’re making the separation between the top domain and other domains a bit clearer. If sitelinks appear for the top result, then the rest of the results below them will be from other domains. One exception to this is if the top result for a query is a subpart of a domain. For instance, the query [the met exhibitions] has www.metmuseum.org/special/ as the top result, and its sitelinks are all from within the www.metmuseum.org/special section of the site. However, the rest of the results may be from other parts of the metmuseum.org domain, like store.metmuseum.org or blog.metmuseum.org/alexandermcqueen/about.

 

 

  • Quality. These user-visible changes are accompanied by quality improvements behind the scenes. The core improvement is that we’ve combined the signals we use for sitelinks generation and ranking — like the link structure of your site — with our more traditional ranking system, creating a better, unified algorithm. From a ranking perspective, there’s really no separation between “regular” results and sitelinks anymore.

 


Sitelinks after today’s changes
These changes are also reflected in Webmaster Tools, where you can manage the sitelinks that appear for your site. You can now suggest a demotion to a sitelink if it’s inappropriate or incorrect, and the algorithms will take these demotions into account when showing and ranking the links (although removal is not guaranteed). Since sitelinks can vary over time and by query, it no longer makes sense to select from a set list of links — now, you can suggest a demotion of any URL for any parent page. Up to 100 demotions will be allowed per site. Finally, all current sitelink blocks in Webmaster Tools will automatically be converted to the demotions system. More information can be found in our Webmaster Tools Help Center.

It’s also worth mentioning a few things that haven’t changed. One-line sitelinks, where sitelinks can appear as a row of links on multiple results, and sitelinks on ads aren’t affected. Existing best practices for the link structure of your site are still relevant today, both for generating good quality sitelinks and to make it easier for your visitors. And, as always, you can raise any questions or comments in our Webmaster Help Forum.

Add +1 to help your site stand out

Written by Evan Gilbert, Software Engineer, +1 Button

When we introduced the +1 button in March, Google search took a small step in an important direction. Search results can be more helpful, and more personal, when recommendations from the people you trust are there to guide your way.

The +1 button can help publishers, too. As potential visitors see recommendations from their friends and contacts beneath your Google search results, you could see more, and better qualified, traffic coming from Google.

Since we announced +1, we’ve gotten lots of requests from Google search users and webmasters alike for +1 buttons in more places than just search results. That’s why today we’re making the +1 button available to sites across the web. Sometimes you want to recommend a web page after you’ve visited it. After all, how do you know you want to suggest that great article on Spanish tapas if you haven’t read it yet?

We’ve partnered with a few sites where you’ll see +1 buttons over the coming days:

 

Partner Logos
You’ll also start to see +1 buttons on other Google properties such as Android Market, Blogger, Product Search and YouTube.

Adding +1 buttons to your pages is a great way to help your content stand out in Google search. By giving your visitors more chances to +1 your pages, your search results and search ads could show up with +1 annotations more often, helping users see when your pages are most likely to be useful.

 


To get started, visit the +1 button tool on Google Webmaster Central. You’ll be able to configure a small snippet of JavaScript and add it to the pages where you want +1 buttons to appear. You can pick from a few different button sizes and styles, so choose the +1 button that best matches your site’s layout.


In the common case, a press of the button +1’s the URL of the page it’s on. We recommend some easy ways to ensure this maps as often as possible to the pages appearing in Google search results.

If your site primarily caters to users outside of the US and Canada, you can install the +1 button code now; the +1 button is already supported in 44 languages. However, keep in mind that +1 annotations currently only appear for English search results on Google.com. We’re working on releasing +1 to searchers worldwide in the future.

If you have users who love your content (and we bet you do), encourage them to spread the word! Add the +1 button to help your site stand out with a personal recommendation right at the moment of decision, on Google search.

To stay current on updates to the +1 button large and small, please subscribe to the Google Publisher Buttons Announce Group. For advanced tips and tricks, check our Google Code site. Finally, if you have any questions about using the +1 button on your websites, feel free to drop by the Webmaster Help Forum.

Troubleshooting Instant Previews in Webmaster Tools

Posted by Michael Darweesh, Software Engineer

Webmaster level: All

In November, we launched Instant Previews to help users better understand if a particular result was relevant for a their search query. Since launch, our Instant Previews team has been keeping an eye on common complaints and problems related to how pages are rendered for Instant Previews.

When we see issues with preview images, they are frequently due to:

  • Blocked resources due to a robots.txt entry
  • Cloaking: Erroneous content being served to the Googlebot user-agent
  • Poor alternative content when Flash is unavailable

To help webmasters diagnose these problems, we have a new Instant Preview tool in the Labs section of Webmaster Tools (in English only for now).

Here, you can input the URL of any page on your site. We will then fetch the page from your site and try to render it both as it would display in Chrome and through our Instant Preview renderer. Please keep in mind that both of these renders are done using a recent build of Webkit which does not include plugins such as Flash or Silverlight, so it’s important to consider the value of providing alternative content for these situations. Alternative content can be helpful to search engines, and visitors to your site without the plugin would benefit as well.

Below the renders, you’ll also see automated feedback on problems our system can detect such as missing or roboted resources. And, in the future, we plan to add more informative and timely feedback to help improve your Instant Previews!

Please direct your questions and feedback to the Webmaster Forum.

Easier URL removals for site owners

Written by Jonathan Simon, Webmaster Trends Analyst

Webmaster Level: All

We recently made a change to the Remove URL tool in Webmaster Tools to eliminate the requirement that the webpage’s URL must first be blocked by a site owner before the page can be removed from Google’s search results. Because you’ve already verified ownership of the site, we can eliminate this requirement to make it easier for you, as the site owner, to remove unwanted pages (e.g. pages accidentally made public) from Google’s search results.

Removals persist for at least 90 days
When a page’s URL is requested for removal, the request is temporary and persists for at least 90 days. We may continue to crawl the page during the 90-day period but we will not display it in the search results. You can still revoke the removal request at any time during those 90 days. After the 90-day period, the page can reappear in our search results, assuming you haven’t made any other changes that could impact the page’s availability.

Permanent removal
In order to permanently remove a URL, you must ensure that one of the following page blocking methods is implemented for the URL of the page that you want removed:

This will ensure that the page is permanently removed from Google’s search results for as long as the page is blocked. If at any time in the future you remove the previously implemented page blocking method, we may potentially re-crawl and index the page. For immediate and permanent removal, you can request that a page be removed using the Remove URL tool and then permanently block the page’s URL before the 90-day expiration of the removal request.

For more information about URL removals, see our “URL removal explained” blog series covering this topic. If you still have questions about this change or about URL removal requests in general, please post in our Webmaster Help Forum.

Do 404s hurt my site?

Posted by Susan Moskwa, Webmaster Trends Analyst

Webmaster level: Beginner/Intermediate

So there you are, minding your own business, using Webmaster Tools to check out how awesome your site is… but, wait! The Crawl errors page is full of 404 (Not found) errors! Is disaster imminent??

 


Fear not, my young padawan. Let’s take a look at 404s and how they do (or do not) affect your site:

Q: Do the 404 errors reported in Webmaster Tools affect my site’s ranking?
A:
404s are a perfectly normal part of the web; the Internet is always changing, new content is born, old content dies, and when it dies it (ideally) returns a 404 HTTP response code. Search engines are aware of this; we have 404 errors on our own sites, as you can see above, and we find them all over the web. In fact, we actually prefer that, when you get rid of a page on your site, you make sure that it returns a proper 404 or 410 response code (rather than a “soft 404”). Keep in mind that in order for our crawler to see the HTTP response code of a URL, it has to be able to crawl that URL—if the URL is blocked by your robots.txt file we won’t be able to crawl it and see its response code. The fact that some URLs on your site no longer exist / return 404s does not affect how your site’s other URLs (the ones that return 200 (Successful)) perform in our search results.

Q: So 404s don’t hurt my website at all?
A:
If some URLs on your site 404, this fact alone does not hurt you or count against you in Google’s search results. However, there may be other reasons that you’d want to address certain types of 404s. For example, if some of the pages that 404 are pages you actually care about, you should look into why we’re seeing 404s when we crawl them! If you see a misspelling of a legitimate URL (www.example.com/awsome instead of www.example.com/awesome), it’s likely that someone intended to link to you and simply made a typo. Instead of returning a 404, you could 301 redirect the misspelled URL to the correct URL and capture the intended traffic from that link. You can also make sure that, when users do land on a 404 page on your site, you help them find what they were looking for rather than just saying “404 Not found.”

Q: Tell me more about “soft 404s.”
A:
A soft 404 is when a web server returns a response code other than 404 (or 410) for a URL that doesn’t exist. A common example is when a site owner wants to return a pretty 404 page with helpful information for his users, and thinks that in order to serve content to users he has to return a 200 response code. Not so! You can return a 404 response code while serving whatever content you want. Another example is when a site redirects any unknown URLs to their homepage instead of returning 404s. Both of these cases can have negative effects on our understanding and indexing of your site, so we recommend making sure your server returns the proper response codes for nonexistent content. Keep in mind that just because a page says “404 Not Found,” doesn’t mean it’s actually returning a 404 HTTP response code—use the Fetch as Googlebot feature in Webmaster Tools to double-check. If you don’t know how to configure your server to return the right response codes, check out your web host’s help documentation.

Q: How do I know whether a URL should 404, or 301, or 410?
A:
When you remove a page from your site, think about whether that content is moving somewhere else, or whether you no longer plan to have that type of content on your site. If you’re moving that content to a new URL, you should 301 redirect the old URL to the new URL—that way when users come to the old URL looking for that content, they’ll be automatically redirected to something relevant to what they were looking for. If you’re getting rid of that content entirely and don’t have anything on your site that would fill the same user need, then the old URL should return a 404 or 410. Currently Google treats 410s (Gone) the same as 404s (Not found), so it’s immaterial to us whether you return one or the other.

Q: Most of my 404s are for bizarro URLs that never existed on my site. What’s up with that? Where did they come from?
A:
If Google finds a link somewhere on the web that points to a URL on your domain, it may try to crawl that link, whether any content actually exists there or not; and when it does, your server should return a 404 if there’s nothing there to find. These links could be caused by someone making a typo when linking to you, some type of misconfiguration (if the links are automatically generated, e.g. by a CMS), or by Google’s increased efforts to recognize and crawl links embedded in JavaScript or other embedded content; or they may be part of a quick check from our side to see how your server handles unknown URLs, to name just a few. If you see 404s reported in Webmaster Tools for URLs that don’t exist on your site, you can safely ignore them. We don’t know which URLs are important to you vs. which are supposed to 404, so we show you all the 404s we found on your site and let you decide which, if any, require your attention.

Q: Someone has scraped my site and caused a bunch of 404s in the process. They’re all “real” URLs with other code tacked on, like http://www.example.com/images/kittens.jpg” width=”100″ height=”300″ alt=”kittens”//a… Will this hurt my site?
A:
Generally you don’t need to worry about “broken links” like this hurting your site. We understand that site owners have little to no control over people who scrape their site, or who link to them in strange ways. If you’re a whiz with the regex, you could consider redirecting these URLs as described here, but generally it’s not worth worrying about. Remember that you can also file a takedown request when you believe someone is stealing original content from your website.

Q: Last week I fixed all the 404s that Webmaster Tools reported, but they’re still listed in my account. Does this mean I didn’t fix them correctly? How long will it take for them to disappear?
A:
Take a look at the ‘Detected’ column on the Crawl errors page—this is the most recent date on which we detected each error. If the date(s) in that column are from before the time you fixed the errors, that means we haven’t encountered these errors since that date. If the dates are more recent, it means we’re continuing to see these 404s when we crawl.

After implementing a fix, you can check whether our crawler is seeing the new response code by using Fetch as Googlebot. Test a few URLs and, if they look good, these errors should soon start to disappear from your list of Crawl errors.

Q: Can I use Google’s URL removal tool to make 404 errors disappear from my account faster?
A:
No; the URL removal tool removes URLs from Google’s search results, not from your Webmaster Tools account. It’s designed for urgent removal requests only, and using it isn’t necessary when a URL already returns a 404, as such a URL will drop out of our search results naturally over time. See the bottom half of this blog post for more details on what the URL removal tool can and can’t do for you.

Still want to know more about 404s? Check out 404 week from our blog, or drop by our Webmaster Help Forum.