Overcoming SEO limitations on Shopify – Part 1: Unalterable Robots.txt and locked sitemaps

This series will discuss the SEO limitations marketers may face when using the e-commerce platform, Shopify, and share insight on how they can combat these issues.

[Update as of 7/7/2021]

As of June 18, 2021, Shopify allows its customers to edit their robots.txt files. You can now control and edit how search engine bots crawl your site through the robots.txt liquid template

This is a major enhancement for the CMS because, as we described in detail below, an uneditable robots.txt file makes things more difficult for SEO.


Many brick and mortar stores are now closed due to COVID-19, so retail businesses have to adapt. As such, e-commerce has become a vital channel for brands to drive conversions. E-commerce website platforms are great content management systems (CMS) to sell products and services online. They are also great alternatives if you are just getting started in retail because of how simple they are to use. In this blog series, we will be examining Shopify, an out-of-the-box platform, popular because of its low start up cost and easy-to-use interface.

Limitations of Shopify

With Black Friday and Cyber Monday rapidly approaching, this time period is critical for brands to promote their products to consumers. Many retail businesses will be aiming to optimize their marketing strategies to reach their target market. Unfortunately, many of these smaller online retailers using Shopify will come to see that running an e-commerce store that works exactly like thousands of others is ultimately not ideal. 

Izabela WIsniewska, Managing Director and Head of Search at Creatos Media, recently spoke at BrightonSEO 2020 about the Shopify platform’s Search Engine Optimization (SEO) limitations and the best ways to overcome them. She shared that companies using Shopify have more difficulty ranking highly for profitable, hyper-competitive search keywords, especially when competing against more established brands. There are a number of reasons for this, including the higher likelihood for more established brands to have greater control over their CMS – this allows them to better optimize for SEO because of it. That doesn’t mean Shopify cannot still compete well for these high value search terms – but it is harder.

If you want to rank highly on search engines like Google and still use Shopify, keep in mind that Shopify poses some limitations that can be problematic for your search optimization. Some common problems are:   

  • Unalterable robots.txt
  • Locked sitemap
  • Duplicate content
  • Unalterable tag pages
  • Forced URL structure

In this post, we will be discussing two problems in particular and how to navigate around them: unalterable Robots.txt and locked sitemap.

Unalterable Robots.txt

Robots.txt is a file that sends signals to search engines on how to crawl and index your site. Shopify automatically creates one for you, but you cannot alter the file. Below is an example of Shopify’s robots.txt file:

So what does this mean? This means that Google may not see important pages on your site that are optimized to drive traffic and conversions, rendering them ineffective. So what can you do? You can leave the robots.txt file as is, or find a workaround like utilizing Cloudflare workers and Edge SEO. Cloudflare workers is a serverless technology that executes Javascript on the edge without impacting the underlying infrastructure. This serverless application allows you to modify the underlying codebase as it passes through its content delivery network. Edge SEO uses edge computing technologies to create new SEO implementation, testing, and research processes outside of the current parameters in which we operate. By combining these two technologies, you’ll be able to add in additional lines to robots.txt using edge tools like Sloth and Spark.

Another problem with an uneditable robots.txt file is blocking pages that you do not want in the index. One workaround to exclude unwanted pages from being crawled is the addition of a “noindex” tag to the head section of your theme.liquid page. This way, the page will not be indexed. Below is an example of the code you would include in the header:

{%ifhandlecontains‘page-handle-you-want-to-exclude’%}

<meta name=”robots” content=”noindex”>

{%endif%}

Locked sitemap

The sitemap.xml is a file that lists the important pages on your website that you’d want to be indexed. The file can include collections, products, images, blog pages and more. Sitemaps make it easier for search engines to crawl and index your site. 

Like robots.txt, Shopify automatically generates Extensible Markup Language (XML) sitemaps and updates them for you, however, these cannot be altered. Shopify’s sitemaps have four primary categories: products, collections, blogs, and webpages. Because there is no ability to produce sitemaps for images and videos on Shopify, you may miss out on opportunities for Google to discover images and videos that could help enhance the customer journey. When it comes to SEO, you want to make sure that everything of value is indexed on your site, including images and videos. 

Fortunately, this can be fixed by creating your own external video/image sitemap. You can build a sitemap by creating a manual sitemap XML file and then submitting it to Google to crawl using the Search Console Sitemap tool. If you want to opt out of manual coding, you can use a third-party Shopify app like Image Sitemap and install the app to your account. 

Now that you know how to navigate around unalterable robots.txt and locked sitemaps on Shopify, and understand their importance in SEO, you can start taking control of how your site is crawled and indexed. However, this is just the start of making sure your site is optimized. Once you understand the common problems with Shopify and why they are essential to fix, you can correct them and start to see improvements to your site. We will explore more limitations with Shopify and provide workaround solutions to help improve your SEO performance. 

To learn more on Shopify’s SEO limitations, or to find out how Croud can help enhance your overall SEO strategy, get in touch!

by Cassandra Inocencio
16 October 2020

Categories:

Tags:

Related posts