joao.goncalves
/
NgixProxy_Pathfinder
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat: Bloqueio massivo de Scraping Libraries e Headless Browsers

This commit is contained in:
João Pedro Toledo Goncalves 2026-02-04 20:02:17 -03:00
parent 0634f09113
commit 2a27e07359
1 changed files with 6 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
snippets/security_maps.conf
View File

@ -11,7 +11,12 @@ map $http_user_agent $is_bad_bot {
# Crawlers Agressivos e Scrapers de Conteúdo # Crawlers Agressivos e Scrapers de Conteúdo
~*(HTTrack|ia_archiver|mj12bot|AhrefsBot|DotBot|SemrushBot|MJ12bot|DataForSeoBot|PetalBot|QuerySeekerSpider) 1; ~*(HTTrack|ia_archiver|mj12bot|AhrefsBot|DotBot|SemrushBot|MJ12bot|DataForSeoBot|PetalBot|QuerySeekerSpider) 1;
~*(SEO-Crawler|SEOstats|SpyFu|Lighthouse|PageSpeed|SiteAudit) 1; ~*(SEO-Crawler|SEOstats|SpyFu|Lighthouse|PageSpeed|SiteAudit|Screaming|MegaIndex|ZoominfoBot) 1;
~*(BLEXBot|WinHTTP|Xenu|Scrap|extract|grab|Crawlspace|WebCopier|TeleportPro|OfflineExplorer) 1;
# Bibliotecas de Scraping e Automação (MCPs, Frameworks)
~*(Scrapy|BeautifulSoup|selenium|puppeteer|playwright|phantomjs|HeadlessChrome|headless) 1;
~*(GuzzleHttp|axios|requests|urllib|libwww-perl|WinHTTP|Go-http-client|node-fetch|Faraday|Typhoeus) 1;
# Bloqueio Total de IA Crawlers (Treinamento e Coleta) # Bloqueio Total de IA Crawlers (Treinamento e Coleta)
~*(GPTBot|ChatGPT-User|OAI-SearchBot|anthropic-ai|ClaudeBot|Claude-Web|Claude-User|Claude-SearchBot) 1; ~*(GPTBot|ChatGPT-User|OAI-SearchBot|anthropic-ai|ClaudeBot|Claude-Web|Claude-User|Claude-SearchBot) 1;