Page Comparison

...

Feature Description

Many resorts use Resorts often have hidden categories and products that they do not want publicly available to guests that don’t have exclusive access. Additionally, many resorts want to first route exclusively for certain guests. To control access and guide customers through the marketing site in their research before directing them to shopping pages, for these reasons, it can be helpful to block , blocking the shop from search engine crawling and indexing is helpful.

RobotsUsing robots.txt essentially allows , resorts can set rules to setup rules that prevent search engines from newly new crawling of pages. However, however, if the a page has already been crawled and indexed, it can may still show up appear in search engine results. If To avoid this is not desirable, a "no index" rule can be added applied to the entire shop to prevent , preventing all shop pages from showing up in search results, regardless of whether it was previously crawled and indexed or not. If a product or category is being newly created and it should not show up prior crawling and indexing.

For newly created products or categories that shouldn't be in search results, you can add the page adding them to the set of "no index" rules in robots.txt and it will never appear beforepublishing prevents them from appearing in search results if done before publishing. Keeping up on with robots.txt rules can allow allows you to keep many most shop pages of your shop indexable and searchable , but will ensure while ensuring that newly created products and categories that should not be crawled do not appear in stay hidden from search engine results.

For more detailed information on the differences between search engine crawling and indexing, refer to Google's documentation.