Robots Meta Tag Mistakes: Why Product Pages Are Hidden from Search in Azerbaijan

The Robots Meta Tag Problem in Azerbaijan

A common situation in Azerbaijani e-commerce sites: the site owner doesn't intend to hide certain pages from Google search, but the misconfiguration of the robots meta tag effectively erases them. Within weeks, sales drop significantly, but the cause isn't immediately obvious.

What is the Robots Meta Tag?

The robots meta tag is a directive written in the <head> section of your web page's HTML code. This tag tells Google, Bing, and other search engines whether your page should be indexed and whether its links should be followed. For example, <meta name='robots' content='index, follow'> tells search engines to index the page and follow its links, while <meta name='robots' content='noindex'> hides the page from search results.

The Most Common Mistakes in Azerbaijan

1. The 'noindex' Tag Left on Product Pages

The most frequent case: a product page wasn't supposed to be indexed during development, but the <meta name='robots' content='noindex'> tag was never removed. As a result, Google never indexes the page and it never appears in search results. Sales drop immediately.

2. Too Strict Rules in Robots.txt

Some Azerbaijani companies, especially small e-commerce sites, write 'Disallow: /' in robots.txt during testing and forget to change it for production. This is the most severe error, hiding your entire site from Google.

3. Wrong Global Settings in WordPress

If the 'Discourage Search Engines from Indexing This Site' option in WordPress Settings is unchecked, your entire site becomes hidden. Many small business owners deactivate this during testing but forget to reactivate it for production.

4. Empty Meta Tags on Dynamic Pages

Product pages with filters (for example, 'red sneakers') may not inherit the correct robots tag from their template, resulting in duplicate indexing or no indexing at all.

How to Identify the Problem

  • Google Search Console: The 'Coverage' report shows how many pages aren't being indexed.
  • Site Search: Type 'site:yoursite.az' in Google. If your product pages don't appear, there's a problem.
  • HTML Source Code: Check the <head> section of your page's HTML for the robots tag.
  • Robots Meta Tag Checker Tools: Use online SEO tools to verify your robots configuration.

How to Fix It

  • For WordPress: Ensure the 'Discourage Search Engines from Indexing This Site' option is unchecked in Settings. Verify your SEO plugins (Yoast, Rankmath) are configured correctly.
  • Fix Robots.txt: Review your robots.txt file to ensure product and category pages aren't blocked.
  • Check Every Page's Meta Tag: All product pages should have <meta name='robots' content='index, follow'>.
  • Request Reindexing in Search Console: After fixing the issue, use the 'URL Inspection' tool in Google Search Console to request reindexing.

Conclusion

Correct use of the robots meta tag is fundamental for the success of Azerbaijani e-commerce sites. A small tagging error can cause significant sales losses within weeks. Check your site immediately, audit your robots settings, and implement the necessary fixes without delay.

Chat on WhatsApp