Block Search Engine Crawling and Indexing

To prevent search engines from crawling shopping pages and indexing them so that they are available via a Google search, the newest version of nopCommerce has several out-of-the-box features that can be leveraged to both prevent crawling of your shop (robots.txt) and to prevent indexing (no index) of the shop if a site has previously been crawled and indexed and therefore is available via Google search unless a noindex rule is added.