Tosunkaya.com

Cloudflare WAF Block Azure & Google Cloud users without blocking Bing & Googlebot

Everyday there are so many (thousands of daily) attacks and bots coming from Microsoft Azure and Google Cloud which steals your bandwidth and slows down your website by hackers. Apart from these it’s easier to block by ASN other attackers (Azure has millions of IPs starting with 20.96.x.x). But Googlebot’s ASN is same as Google Cloud origin and Bingbot’s ASN is same as Microsoft Azure which makes hard to block it includes concern of blocking Googlebot & Bingbot mistakenly. Fortunaly Cloudflare WAF has filters to add rule. There are 2 ways to Block Azure & Google Cloud users without blocking Bing & Googlebot.

First is blocking Google Cloud & Azure ASNs

Google Cloud ASN: 15169 and 396982

Microsoft Azure ASN: Mostly 8075, (8068 8069 8070 8071 8072 8073 8074 8075)

Also i recommend checking ASN of IP by this open-source website IP.ME which given by IANA to ProtonTeam, example: https://ip.me/ip/20.96.29.54

i will show blocking Azure:

Bingbot’s hostname: search.msn.com or msnbot (April 2022 Update: it’s bingbot) https://www.bing.com/webmasters/help/how-to-verify-bingbot-3905dc26  https://blogs.bing.com/webmaster/april-2022/Announcing-user-agent-change-for-Bing-crawler-bingbot

Current user-agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/W.X.Y.Z Safari/537.36

Reverse DNS and IP Lookup: Forward Confirming Lookup

Cloudflare Rule For Blocking Azure:

Solution 1 (Block Azure but “Cloudflare bot check”) | Trustworthy

(ip.geoip.asnum in {8075} and not cf.client.bot)

A firewall rule like this will block all traffic from Microsoft ASNs, but allow known good bots.

want to use all azure ASNs: (ip.geoip.asnum in {8068 8069 8070 8071 8072 8073 8074 8075} and not cf.client.bot)

Solution 2 (Block Azure But Hostname | Fake bingbots may allowed)

(ip.geoip.asnum eq 8075 and not http.host contains “bingbot”)

Update: Microsoft changed user agent msnbot to bingbot in April 2022: https://blogs.bing.com/webmaster/april-2022/Announcing-user-agent-change-for-Bing-crawler-bingbot

Cloudflare Rule For Blocking GoogleCloud:

Solution 1 (Block Google Cloud but “Cloudflare bot check”) | Trustworthy

(ip.geoip.asnum in {15169} and not cf.client.bot)
or below Azure & Google Cloud & Amazon
(ip.geoip.asnum in {8075 396982 15169 16509} and not cf.client.bot)

A firewall rule like this will block all traffic from GoogleCloud ASNs, but allow known good bots.

Other ASN: AS396982

Solution 2 (Block Google But Hostname | Fake googlebots may allowed)

(ip.geoip.asnum eq 15169 and not http.host contains “googlebot”)

image.png

One Rule For Blocking Both and allow legit bots

How Cloudflare Filters Bots

Based on https://cloud.google.com/compute/docs/faq#find_ip_range (https://www.gstatic.com/ipranges/cloud.json), the AS that announces the prefixes is the same for both Google Cloud and their primary services:

Google Cloud listing
image

Googlebot IP

image

Although, you might notice that the “host” of the IP is different. For Google Cloud, the IP’s host ends in googleusercontent.com, while Googlebot ends in googlebot.com. This method is precisely how Google expects people to verify that content coming from Google’s network is legitimate traffic from Googlebot.

https://support.google.com/webmasters/answer/80553?hl=en 5

As for how this matters in a Firewall rule, CF does verify Googlebot using the method Google recommends, and will appropriately set “verified bot” to false since it’s a fake Googlebot.


So, to go about making sure fake Bingbot and fake Googlebot are blocked, you should modify it to be:

Your firewall rule above would only trigger if the ASN were both numbers at the same time, which wouldn’t ever trigger.

If you have the Pro plan or above with the WAF and the “Cloudflare Specials” ruleset enabled, the managed rule 100035 is already set up to block fake Googlebot.

IP.ME or ipinfo.io

Also i publish edmalicious bad ASNs to block list here. Until that you may check by this website: https://cleantalk.org/blacklists/asn

Cloudflare Known Bots list: https://radar.cloudflare.com/verified-bots

Additional info: DuckDuckGo bots use Microsoft Azure IPs too and it’s listed in Cloudflare’s known bots list, so DuckDuckBot will be allowed: https://help.duckduckgo.com/duckduckgo-help-pages/results/duckduckbot/

5/53526 oy

Emre Tosunkaya

Kendini İnternete adamış bir tekno kişi. Teknoloji ve İnternet adına; WordPress, Webmaster, Android, Google, mobil, oyun, yazılım hakkında insanlara yararlı makaleler yazar.

  • Kuponla.com İndirim Kuponları
  • Uzman Diyetisyen Semiye Tosunkaya
  • Güzel Hosting, 2008'den beri kaliteli ve ekonomik paylaşımlı hosting, kiralık sunucu, sanal sunucu ve co-location hizmetleri sağlamaktadır.
  • Turhost: Türkiye'nin Lider Hosting ve Domain Servis Sağlayıcısı
  • n11.com - Alışverişin Uğurlu Adresi
  • En Trend Ürünler Türkiye'nin Online Alışveriş Sitesi Amazon'da
Kuponla.com İndirim Kuponları Binance %10 Komisyon İndirimi TRBinance %10 Komisyon İndirimi Amazon Amazon 50₺ indirim kodu Trendyol Güncel İndirim Kodları