Several webcrawlers found on Magistream

[6and]

There are several webcrawling bots on magistream website scraping data.

[Katnilb]

The search engines ones have been here for a very long time.

I am a bit curious whether the AI bots are at least partly the source of the lag - I've heard about some of them being very aggressive with their scraping including some listed in the Who Is Online section.

[Kestrad]

The search engines ones have been here for a very long time.

I am a bit curious whether the AI bots are at least partly the source of the lag - I've heard about some of them being very aggressive with their scraping including some listed in the Who Is Online section.

The AI bots were, in fact, causing some of the lag before Rose was able to identify them as bots and tell the forum software to treat them as such, which, if I understood correctly, throttles them so that they stop being able to scrape so aggressively and take up so much server resources.

[6and]

Thanks for letting me know that they are already on top of it! I appreciate the replies!