crawl-delay

How to control AhrefsBot on your website? Since AhrefsBot strictly follows the robots.txt file on your website, you're in full control of what it does. To change the frequency at which AhrefsBot visits your site, just specify the minimum acceptable delay between two consecutive requests in your...