How to Get Google to Index a Site and Page

Recently, I launched my NY Local SEO page on my WordPress website to start ranking for local keywords. After a few days, I noticed the page was not collecting impression data in Google Search Console. Not seeing an impression data was surprising since I was confident it would rank well. After reviewing the page using Google Search Console, I saw that it was never indexed. Here are the steps marketers need to take to get Google to index any page on their site.

 

What Does Indexing in Search Engines Mean?

Indexing is the search bots’ process of crawling a web page and then putting that page in their index. Indexing does not mean that your page will rank in a search engine’s algorithm, but it just means that crawlers, like Googlebot and Bingbot, were able to put that page in their memory. The first step in winning in SEO is to get your content crawled and indexed by search engines.

 

Steps to Take to Force Google to Crawl Any Page

If you want to make sure your target page is in Google and Bing, you need to search for it. One tip is to put the exact URL in both search engines to see if it appears. If the URL is not in the search engines, you need to follow the steps outlined below.

 

Robots.txt File

By default, Google and Bing respect the robots.txt file DISALLOW rules on where to crawl. The robots.txt file tells Google and Bing where they should and should not look by a folder path on your website. If you want to confirm that your new content has a chance for crawling, you can use the robots.txt tester in both Google and Bing. The primary thing you want to see is if both bots are allowed to crawl the path.

 

Check for a NOINDEX Tag

A NOINDEX tag tells Google and Bing that they can crawl a page, but they can’t show it in their index. A reason for this meta tag is if you don’t want your Thank You page to be in Google’s index for tracking reasons. To remove that page from entering Google and Bing’s index, you would administer a NOINDEX tag. One way to check if the NOINDEX tag is on the page is to run a test (URL Inspection Tool) in Google Search Console and Microsoft Bing Webmaster Tools. In addition to that, you can right-click on the page in your browser and search for NOINDEX.

 

Check for a NOFOLLOW Tag

A NOFOLLOW tag is different from a NOINDEX tag as it tells Google and Bing to stop following the links on a site. For example, you may have a link from one blog post to another, but you have a NOFOLLOW tag on that previous blog post. Having that NOFOLLOW tag tells Google and Bing not to follow any links to that new blog post, which can hurt your organic traffic potential if that was the only way for search engines to get to that new page.

 

Review the Canonical Tag

A canonical tag tells search engines what version of a page they (search engines) should showcase in the SERP. A Canonical Tag becomes an issue if the canonical tag is not the same as the page’s actual URL. For example, you may have a canonical tag for the homepage, but the existing page is different. Therefore, Google and Bing would say that they already show the homepage in their results so that they won’t display this new page with a different URL.

 

Is This Duplicate Content?

Site owners may not realize this, but they might have duplicate content on their site, leading to significant technical errors. Duplicate content wastes Google crawl’s budget, which means they won’t get to important pages to crawl and index. To preserve the crawl budget of search engines, you need to have a 301 redirect set up for every variation of your site, so Google/Bing only goes to one preferred version of the site. Another thing to remember is a new site won’t have as much indexation as older sites. That means that new websites need to make sure that there are no duplicate content issues on their site. Since Google and Bing won’t crawl the new domain that often because it may seem like low-quality, you need to give both search engines a clear path to crawl only essential pages.

 

Make Sure that the Page has an Internal Crawl Path to Follow

The only way that Google and Bing will crawl new pages is if there is a link path for them to follow. Some tools that I use to confirm a link path include Screaming Frog, Sitebulb, and SEMRush. A link path can range from a page already indexed, an HTML sitemap, an XML sitemap, or the level above that folder path. If you can confirm an internal link that can bring someone and a bot to that page, you should be all set.

 

Confirm if the Page Above is in Google’s Index

After running through this  process above, I found a NOINDEX issue on the page level above my local services. While there was no error with the Local search page, I NOINDEXED the main search page because I thought the content was not high-quality content. After removing the NOINDEX tag from that page’s header, I set a recrawl in Google and Bing to add the service page back to the index.

 

Request Indexing on the Target Page

After waiting a day for my service page to appear in Google, I then requested indexing on my local page. Once one day went by, I saw Google and Bing’s message that this new page was now in the index. Getting this page in the index meant that I would have a chance to start ranking for non-brand keywords.

 

Conclusion

Having a successful SEO Strategy means allowing Google and Bing to crawl and index your content. Without getting your content in the SERP, you won’t drive organic traffic to your site. Whether you are trying to get a specific page in the index or need your whole site indexed, you can follow these steps above.

 

As you can see in the photo above, I’m now on page 1 of Google for a target keyword.

 

About TM Blast LLC

TM Blast is a New York SEO Company helping grow the traffic to clients’ websites. If you are looking for a Free SEO Analysis, you can fill out the contact form. Getting a Free SEO Analysis of your site is a great way to optimize important pages on your website. In addition to the audit, I also provide the Free SEO Workshop for anyone to take and learn SEO. Finally, you can also become an affiliate of TM Blast by heading over to the affiliate page.