Skip to content
SPECIFICTION
Get listed

Crawler Log Analyser

github.com

LinuxSelf-hosted
Visit github.com
Spec score 7.7 #9 of 27 in SEO Log File Analysis Tools

Spec sheet

Purpose
The project describes itself as operational SEO observability for self-hosted static sites running nginx.Read 10 Oct 2026 · github.com
Intended users
The maker positions it for operators of self-hosted static sites who need to inspect real crawler behavior in nginx access logs.Read 4 Oct 2026 · github.com
Integrations
The README says JSON output is the integration boundary and lists no Slack, Prometheus, Elasticsearch, or SaaS integrations.Read 10 Oct 2026 · github.com
Support
The README welcomes pull requests and says project review may be slow because it is maintained as time permits.Read 4 Oct 2026 · github.com
License
The project is licensed under AGPL-3.0-or-later; network services based on modified versions must make corresponding source code available under the AGPL.Read 10 Oct 2026 · github.com
Headquarters
Inverness, Scottish Highlands, UKRead 28 Sep 2026 · github.com
Maker
SpeyTech identifies Spey Systems Ltd as the company behind its repositories and describes it as a Scottish software and systems engineering company based in Inverness.Read 10 Oct 2026 · github.com
Additional detections
It can detect sitemap-path user-agent spoofing and rotating bot identity claims from a single IP.Read 4 Oct 2026 · github.com
Anomaly detection
It detects crawler-visible deploy-window issues including robots.txt failures, content-404 bursts, and post-404 crawl gaps.Read 10 Oct 2026 · github.com
Configuration
A TOML config file can specify ignored source IPs, site launch dates, and an AI crawler discovery window.Read 10 Oct 2026 · github.com
Crawler analysis
It classifies search and AI crawlers and scores Googlebot and broader search-crawler health from observed responses.Read 4 Oct 2026 · github.com
Crawler insights
It classifies search and AI crawlers and scores Googlebot crawl health from observed responses.Read 10 Oct 2026 · github.com
Data handling
The README says the tool has no database, telemetry, or external services.Read 10 Oct 2026 · github.com
Deploy anomalies
It detects crawler-visible deploy-window issues such as robots.txt failures, content-404 bursts, and post-404 crawl gaps.Read 4 Oct 2026 · github.com
Feature status
The feature-freeze page says no new feature work was planned through 31 August 2026 and that bug fixes and security fixes remain welcome.Read 10 Oct 2026 · github.com
Log analysis
It parses nginx access logs, including rotated .gz logs, and supports combined and extended seo_crawl formats.Read 10 Oct 2026 · github.com
Log parsing
It parses nginx access logs in combined or extended seo_crawl format, including rotated .gz logs.Read 4 Oct 2026 · github.com
Privacy and architecture
The maker describes it as a single-file Python script with no telemetry, dashboards, database, external services, or external dependencies.Read 4 Oct 2026 · speytech.com
Project status
The repository identifies v1.10.2 as its status and says feature work was frozen through 31 August 2026, while bug fixes and security fixes remain welcome.Read 4 Oct 2026 · github.com
Reports
It emits text, Markdown, or JSON reports and can exit non-zero on critical findings for cron and CI use.Read 4 Oct 2026 · github.com
Reports and automation
It emits text, Markdown, or JSON reports and can exit non-zero on critical findings for cron and CI use.Read 10 Oct 2026 · github.com
Requirements
It requires Python 3.11 or later and has no external dependencies.Read 10 Oct 2026 · github.com
Support and maintenance
The maintainer says review may be slow because the project is maintained as time permits.Read 10 Oct 2026 · github.com

Where Specifiction places it

Alternatives to Crawler Log AnalyserAll alternatives →