| Table of Contents |
|---|
...
1. (Optional) Configure specific pages to restrict crawling using robots.txt
This is recommended to prevent search engines from crawling hidden categories, voucher product urls, and any page that should not be indexed by search engines.
To restrict only certain pages from being crawled by search engine bots, configure your robots.txt entry for your shop by going to Configuration > Settings > General Settings.
Locate the robots.txt section and expand it. It will be pre-loaded with some default restrictions setup by nopcommerce.
...
To restrict only certain pages from being crawled by search engine bots, configure your robots.txt entry for your shop by going to Configuration > Settings > General Settings.
Locate the robots.txt section and expand it. It will be pre-loaded with some default restrictions setup by nopcommerce.
To disable crawling of the entire site enter
/*at the end of the localizable disallow path.
...