I'm very surprised by how much these web crawlers scan sites.
That just shows you how truly massive and powerful those companies are.
For every search that you perform on Google, 12 cents of revenue is generated. Not to mention they have knowledge of most of the
www.Internet at their disposal.
I'm sure that this will come to light in the future, because in a sense they could potentially be stealing content from your site that isn't displayed to the general public and they are in fact stealing from you. Bandwidth that you pay good money for at the very least.
I have noticed a slowdown of the site this week too.