In June 2005, Google announced a new service called Google Sitemaps. This program allows webmaster to submit an index of URLs which they want to have included in Googles web search. It’s free to use and helps Google to get a more complete overview of your pages.
Overview
What is or are Google Sitemaps?
Basically, a Google Sitemap is a file which contains URLs and some additional information for all public pages or documents of your website. Google can read this file and add the defined pages to their index. The Google Sitemaps program is part of the «Google webmaster tools«.
How can I create Google Sitemap for my website?
If you are using the blogging software WordPress, you can use the Google Sitemap Generator for WordPress plugin. Otherwise, you can check the list of third party programs at code.google.com.
Do I need a Google Account to use Google Sitemaps?
There are two ways to notify Google about your sitemap.
- You can register to the Google Sitemaps program and submit your sitemap. This will also allow you to see some interesting statistics about your site like the most used keywords and spidering problems.
- If you don’t have a Google account or don’t want to create one, you can notify Google about your sitemap by «pinging» their sitemap server. All you need to is to point your browser to http://www.google.com/webmasters/sitemaps/ping?sitemap=http://www.name.com/sitemap.xml
Google will check your sitemap for updates regularly, so you don’t need to do this more than once.
Will Google Sitemaps affect my ranking?
There are no evidences that a Google Sitemap will directly affect your ranking. However, it can help Google to index and crawl your page better which may result in a more complete index of your page.
Where can I submit my German (or international) sitemap?
There are no different sitemap programs for languages or countries. Just register to the international Google Sitemaps Program, which is localized in many languages.
Technical questions
What is the maximum size of a Google Sitemap?
According to the Google Sitemap FAQ, your sitemap can contain up to 50,000 URLs or reach a file size of 10MB (uncompressed!). However, I would recommend you to split such large sitemaps into various smaller ones which allows Google to retrieve only the latest ones regularly. This will save you a lot of traffic.
Can I use RSS as my Google Sitemap?
Yes, Google Sitemaps supports RSS 2.0 and Atom 0.3 feeds. However, a Google Sitemap should contain as much pages of your site as possible but a RSS feed contains only the latest ones normally.
Do I have to resubmit my sitemap every time I change it?
Google will check your sitemap for updates regularly, so you don’t need to inform them. However, you can resubmit your sitemap by clicking the «Resubmit» button on the Google Sitemaps Site Overview (Google Account required) or by pointing your browser to http://www.google.com/webmasters/sitemaps/ping?sitemap=http://www.name.com/sitemap.xml
If you are using a program to generate your sitemap it’s likely that there is an option to notify Google about changes automatically.
Can I hide my sitemap from other people?
I don’t know any reason why you should hide your sitemap from other people. They can find most of your URLs by searching with the «site:www.exampe.com» operator in Google. Of course, you don’t need to call it sitemap.xml but my-completely-hidden-sitemap.xml for example.
Does my Google Sitemap have to end with .xml?
No, you can name it whatever you like, just make sure you are sending the correct mime type (text/xml for xml data). You can configure your Apache server to send «text/xml» for your favourite extension by adding «AddType text/xml .yourext» to your .htaccess file or httpd.conf.
Can you give me an example of a Google Sitemap?
Basically, a Google Sitemap looks like the following sample. A «real» sitemap can be found here. (Note that the linked example is styled via XSLT to make it more readable. Use the «Show Source»e; function of your browser to see the actual XML code.) All fields except of «loc» are optional.
-
<?xml version=»1.0″ encoding=»UTF-8″?>
-
<urlset>
-
<url>
-
<loc>http://www.arnebrachhold.de/</loc>
-
<lastmod>2006-05-22T12:31:11+00:00</lastmod>
-
<changefreq>weekly</changefreq>
-
<priority>1</priority>
-
</url>
-
<url>
-
<loc>http://www.arnebrachhold.de/imprint/</loc>
-
<lastmod>2006-05-22T13:31:11+00:00</lastmod>
-
</url>
-
<url>
-
<loc>http://www.arnebrachhold.de/foo/</loc>
-
</url>
-
<!– This is a comment –>
-
</urlset>
How do I know if my sitemap.xml is correct?
You can just submit your sitemap to the Sitemaps program and wait until it’s downloaded the first time. Google will tell you if there are any errors. If you want to validate your sitemap before submitting it, you can use a XML validator to validate the structure of your sitemap file.
Help with error messages
What means the error message «This url is not allowed for a sitemap at this location»?
This error means that your sitemap contains URLs which are not alllowed in your sitemap. Your sitemap can only include URLs which point to files on same domain and the same or deeper directories. Let’s take a small example to illustrate it:
Your sitemap file is saved at http://www.example.com/herbert/sitemap.xml
Your sitemap is allowed to contain URLs like
- http://www.example.com/herbert/
- http://www.example.com/herbert/home.html
- http://www.example.com/herbert/test/index.html
- http://www.example.com/herbert/info/about/guestbook
but you can’t include URLs like
- http://www.example.com/ (Higher directory level than your sitemap)
- http://www.example.com/herbert (Higher directory level, missing trailing slash so «herbert» is treated like a file)
- http://www.example.com/lusie/ (Higher directory level than your sitemap)
- http://www.herbert.com (Not the same domain)
You can include the 3 first denied URLs by moving your sitemap one directory higher to http://www.example.com/sitemap.xml
Now you can include all URLs which point to http://www.example.com/ like
- http://www.example.com/herbert
- http://www.example.com/
- http://www.example.com/lusie/
- http://www.example.com/lusie/all-about-google.html
What means the error message «Invalid date»?
This error means that your sitemap contains an entry which has an invalid last modified date. Google Sitemaps requires the ISO-8601 encoding which has two variations:
– 2005-02-21
This one just contains the year (4 digit), the month (2 digit) and the day (2 digit)
– 2005-02-21T18:00:15+00:00
This one is more complex and includes the year (4 digit), the month (2 digit) and the day (2 digit) followed by the character «T», the hour (2 digit), colon, minute (2 digit), colon, second (2 digit) and finally the time zone with the character «+» OR «-«, the timezone offset in hours (2 digit) and minutes (2 digit).
It’s important to include ALL parts of the chosen date format and pay attention to the correct order. See the ISO-8601 specification for more examples.
What means the error message «We couldn’t find your verification file.»?
Double check that you named your verification file correctly (like Google told you) and that there are no spaces in front or after the file name. If it still doesn’t work, make sure your server returns correct status headers for existing documents (200) and not existing documents (404). You can check this by using a sniffer or HTTP Request Tool.
Enter the URL to your verification file and check that your server returns «200 OK» as the status code. Then try a not existing page and verify that your server returns «404 Not found». If you got a «200 OK» again, review your server configuration about error documents.
Statistics and verification
May I delete my verification file after I verified my site?
You can delete the verification file but Google will check the existence of the file regularly so you will have to create it again soon.
Why does Google check the existence of my verification file regularly or why should I not delete it?
This will allow Google to ensure that you are the current owner of the domain or have the permission to use it. If you buy a domain from another person he will lose the access to your Google Sitemaps statistics after you deleted his verification file.
Can other people view my statistics?
Only people who have write access to your webserver, via FTP for example, can view your statistics after they successfully verified their Google account with putting a file with a specified name on your web space. The file name is given by Google and depends on the Google account name. As long as your webserver is secure, nobody beside you can view your Google Sitemaps statistics.
Do I need a Google Sitemap to view statistics about my website?
No, just sign up to Google Webmaster Tools and add your website. As soon as you verified your submission, you are able to view statistics like crawling errors, top keywords and so on. You don’t need to submit a sitemap for that.
Sitemap Generator for WordPress Plugin FAQ
WordPress needs to long to create my sitemap or I get a timeout error / blank page
Try to increase the memory and time limits which are located under Advanced optionsat the sitemap configuration page. See also this question.
I have no comments (or disabled them) and all my postings have a priority of zero!
Disable automatic priority calculation and define a static priority for posts! There is also an option to define a minimum post priority.
Do i need a Google Account for this plugin?
Maybe. If the «Auto-Ping Google Sitemaps» Feature works, you don’t need a Google Account. If your host disabled the required PHP functions for this feature, you need a Google Account and submit your sitemap once. Google will check the sitemap file periodically for changes.
Do I always have to click on «Rebuild Sitemap» if I modified a post?
No need to do that. If you edit/publish/delete a post, your sitemap gets automatically regenerated!
So much configuration options… Do I need to change them?
No, only if you want. Default values should be ok!
Does it work with all WordPress versions?
This plugin works with WordPress 1.5.1.1 or higher only. Please upgrade if your are using an older version.
I get an fopen error and / or permission denied
If you get permission errors make sure that the script has writing rights in your blog directory. Try to create the sitemap.xml resp. sitemap.xml.gz manually, upload them with a ftp program and set the rights to 777 with CHMOD. Then restart sitemap generation on the administration page. A good tutorial for changing file permissions can be found on the WordPress Codex.
Which MySQL versions are supported?
This plugin works with all MySQL 4, 5 and newer MySQL 3 builds.
Do I need cronjobs to run this plugin?
No, you don’t need any cronjobs. The sitemap gets rebuilt if you edit a post.
The sitemap files could not be written!
Make sure that the files «sitemap.xml» and / or «sitemap.xml.gz» are writable. You have two options to ensure this:
- Make your blog root writable
You can make your whole blog root folder writable and the plugin will create the files for you. You can do this by using a FTP program and setting CHMOD to 755 or 777 to your web-root folder. This folder is often named «htdocs», «html», «public» or «httpdocs».
- Create the sitemap files and make them writable
If you can’t make your blog root folder writable or don’t want to do so, you can create two new files, name them «sitemap.xml» and «sitemap.xml.gz», upload them to your blog root and use a FTP program to apply CHMOD 755 or 777 to them. To create these files, simply open Notepad, click on «File > Save As», choose «Filetype: All File» and enter «sitemap.xml» as the name. Repeat this step for «sitemap.xml.gz».
- More information about changing file permissions
Please look at the WordPress Codex or this toturial to get step by step advices for your FTP program or check the manual.
The last run didn’t finish or I just get a white screen
It could be that your server is not configured to run memory-heavy scripts like a sitemap generator.
- Try to increase the memory limit on the sitemap options page (start by «4M» and raise the value if it doesn’t help).
- Try to increase the time limit on the sitemap options page (start by «20» and raise the value if it doesn’t help).
- If it’s still not working you may not have the permission to change these settings so you need to ask your hoster to raise the limit
What’s the difference between the «sitemap.xml» and «sitemap.xml.gz» files?
The «sitemap.xml.gz» is a compressed version of the «sitemap.xml» file. It has the same content, but is significantly smaller than the other one. This helps you and the search engines to save a lot of traffic. Since all search engines support compressed sitemaps, you actually don’t need the «sitemap.xml», but maybe you or your visitors want to view them from time to time so keeping it doesn’t hurt.
What are the different building modes?
You can choose when your sitemap gets regenerated:
- Rebuild sitemap if you change the content of your blog
Your sitemap gets automatically refreshed when you publish or delete a post. If you have really much post, the process may take some time and you have to wait on the posting screen until it’s finished.
- Enable manual sitemap building via GET Request
This option allows you to refresh your sitemap using a special URL which is displayed when you click on the «[?]» sign. This url can be used with a cron job for example which refreshed the sitemap every day or every hour. This mode is prefered if you have thousands of post and the automatic building needs to long
What is the update notification?
This plugin can automatically notify Google and YAHOO when the content of your blog changes. This service is free to use, YAHOO just requires an API key which can be freely obtained here. After the search engines recieved the «ping» they may come and crawl your site again. Since the sitemap files contain the last change of every post or page, the spiders should just retrieve the changed ones and save your traffic. The plugin stops the time the notification of every search engines needs and might recommend you to disable this service if it slows down the building process significantly.
About the advanced options
- Limit the number of posts
If you have problems with the maximum execution time or memory limit you can limit the number of posts which will be included in the sitemap. Newer posts are included first so your sitemap will stay up-to-date.
- Increase the memory limit
Building the sitemap needs a lot of memory. If the memory size is limited via configuration and the script can’t finish the sitemap, you can try to increase this limit by entering a higher value. The values are in megabytes so you can start with «2M» for smaller sites and raise the number until it works. However, it could be that you don’t have the permission to change this value so if it still doesn’t work and you already tried a very high value like 16M, you will need to contact your webhoster and ask him to raise it for you.
- Increase the maximum execution time
Like the memory, the maximum execution time can also be limited. If the script doesn’t finish, try to set the time limit to «0» which means unlimited or a high value like «30» seconds.
- Include a XSLT stylesheet
Since version 3.05b, the plugin ships a default XSLT stylesheet which makes your XML sitemap human readable. You can specify you own by entering a full or relative URL. Please note that the XSLT stylesheet must be on the same server for security reasons.
- Enable MySQL standard mode
Per default, the plugin uses a separate MySQL connection to query the post data in a very effective, memory-saving way. If this doesn’t work on your hosting configuration you can enable the MySQL Standard mode which uses much more memory but should always work.
- Build the sitemap in a background process
If your blog contains a large number of posts you may experience a delay after editing or saving a new post or page since it needs some time to generate the sitemap. If you activate this option, your sitemap will be built in background using wp-cron which will avoid the delay. Your sitemap will be generated a few seconds after you’ve hit the save button so the sitemap status at the administration panel won’t show the changes immediately.
- Exclude posts or pages
Here you can enter the IDs of posts or pages which will not be included in your sitemap. You can see the IDs of the post or pages in the corresponding management pages. Separate multiple IDs by comma.
Google Sitemaps and robots.txt
You can use the robots.txt file to inform search engines about your sitemap. If you activate this option at the administration panel, the plugin will try to create the file in your blog root. The «File permissions» status below the checkbox will give you a hint if this is possible or not. If the robots.txt file cannot be generated due to insufficient file permissions, please create the robots.txt file by yourself and make it writable via CHMOD. A good tutorial for changing file permissions can be found on the WordPress Codex. The plugin will NOT delete your existing robots.txt file but append the new values at the end.
About this FAQ
I didn’t find an answer to my problem, are there any additional resources?
You can look at the Official Google Sitemaps FAQ, join the Google Sitemaps Group or leave a comment on this post.
What can I do if I have suggestions or corrections for this FAQ?
Suggestions and corrections are always welcome, please write me a mail if you have some and I will update this page .
