During an audit, it was found that a WooCommerce website was causing high CPU usage, which was due to frequent bot accesses to non-cachable pages with the parameter ?add_to_wishlist= These pages are indexed by bots such as Googlebot and Ahref, which slows down the website. To prevent this, it is recommended to /robots.txtfile that prevent bots from indexing these links. Regardless of whether the "add-to-cart" function is executed via JavaScript or directly in HTML, disabling indexing of such parameters is a necessary measure.
Optimized robots.txt
#Block WooCommerce assets User-agent: * Disallow: /cart/ Disallow: /shopping cart/ Disallow: /checkout/ Disallow: /kasse/ Disallow: /my-account/ Disallow: /my-account/ Disallow: /*?orderby=price Disallow: /*?orderby=rating Disallow: /*?orderby=date Disallow: /*?orderby=price-desc Disallow: /*?orderby=popularity Disallow: /*?filter Disallow: /*add-to-cart=* Disallow: /*?add_to_wishlist=* #Block Search assets User-agent: * Disallow: /search/ Disallow: *?s=* Disallow: *?p=* Disallow: *&p=* Disallow: *&preview=* Disallow: /searchWith these parameters, crawlers will no longer index "add-to-cart" links and other non-cacheable pages. This results in CPU, memory, and bandwidth savings.
Conclusion
For hosting providers that base their pricing on CPU usage, this measure would significantly reduce monthly costs. Even with traditional web hosting, this would result in a noticeable reduction in CPU usage. Furthermore, these robots.txt settings allow you to save valuable crawl credits when using Ahrefs. Last but not least, the overall ranking by Google crawlers also benefits, as valuable crawl limits are saved and the Google crawler can focus on essential content.
English 
Deutsch