Believe it or not, Google actually has a page dedicated to how to submit your content. However, this page does not include all the different ways you can get your content indexed in the Google search engine.

For example, adding your URL is just the beginning. There are things you can do to help the search engine spiders crawl your website with ease.

An XML sitemap is placed on your Web server to tell search engine crawlers what pages are present within your website. There are several XML site map generator tools available on the Web that generate the code based on your URL. After creating an XML site map, upload it to your server and submit via your Google Webmaster Account. Google will ask you to verify that you are the webmaster by uploading an HTML file to your server with a specified file name.

The robot.txt file is used to give crawling instructions to Web robots. Use a plain text editor (such as Notepad) to create your robots file. Include the following information:

User-agent: *

Sitemap: http://www.yourdomain.com/sitemap.xml

Disallow:

“User-agent: *” indicates that the instructions apply to all robots.

“Site Map” explains where your XML site map is located.

“Disallow:” explains that no pages are off limits. If you did want to block a page from being indexed, you would include after “Disallow:”

Broken links can hinder a search engine spider when it is crawling your site. You can use online link checker tools to review your site and identify broken links.