Google Removes Crawl Rate Limiter Tool From Search Console

(CTN News) – Google has recently made an announcement regarding the discontinuation of the search console crawl rate limiter tool.

This tool, which was introduced to the search console in 2008, allowed publishers to manage the crawling activity in order to prevent server overload.

However, due to significant improvements in crawling technology, Google has deemed this tool unnecessary and has scheduled its removal on January 8, 2024.

At one point, certain publishers faced excessive crawling that led to server overload and inability to serve webpages to users.

This issue prompted numerous complaints, eventually leading to introduce a tool in the search console. The tool’s purpose was to gather data for Google, and it typically took a day for requests to limit crawling to take effect, with the limit remaining in place for 90 days.

The Removal of Google’s Rate Limiter Tool

The declaration indicated that the crawling algorithms have achieved a level where can detect on their own when a server is approaching its limit and promptly respond by reducing the crawl rate.

Additionally, Google mentioned that the tool was seldom utilized and when it was employed, the crawl rate was typically adjusted to the slowest setting.

In the future, the default minimum crawl rate will be set to a lower rate, resembling the rate that publishers commonly request.

Simplifying Search Console

To enhance the user experience of the search console, it is recommended to eliminate unnecessary tools that are seldom used. This will declutter the console and make it more user-friendly. As a result, the overall experience of using the search console will be improved.

In case publishers are still facing issues with Googlebot’s crawl rate, they can utilize the report form to provide feedback to Google.

SEE ALSO:

Spotify Wrapped 2023 Will Be Released Soon. Find Out How To See Yours.