Crawler Log Analyser vs Spider-Lens
Crawler Log Analyser
github.comSpider-Lens
github.comSide by side
| Figure | Crawler Log Analyser | Spider-Lens |
|---|---|---|
| Price from | — | Free |
| Free plan | — | Yes |
| Free trial | — | — |
| Open source | — | — |
| Web | — | Yes |
| Windows | — | Yes |
| macOS | — | Yes |
| Linux | Yes | Yes |
| Self-hosted | Yes | Yes |
| Plans published | 0 | 1 |
| Spec score · SEO Log File Analysis Tools | 7.7 (#9) | 7.6 (#15) |
| Headquarters | Inverness, Scottish Highlands, UK | Caen, France |
| Integrations | The README says JSON output is the integration boundary and lists no Slack, Prometheus, Elasticsearch, or SaaS integrations. | The project documents Discord and Slack webhooks and an MCP server for Claude Code or other MCP-compatible clients. |
| Maker | SpeyTech identifies Spey Systems Ltd as the company behind its repositories and describes it as a Scottish software and systems engineering company based in Inverness. | The repository identifies the maker as GDM-Pixel and lists Caen, France. |
| Purpose | The project describes itself as operational SEO observability for self-hosted static sites running nginx. | Spider-Lens analyzes Apache and Nginx server logs to monitor traffic, HTTP errors, crawler bots, and server performance. |
| Requirements | It requires Python 3.11 or later and has no external dependencies. | The Node.js application requires Node.js 18 or later, npm 9 or later, and access to Apache or Nginx log files. |
Plans
In their own words
Crawler Log Analyser
Crawler Log Analyser has no write-up on Specifiction yet; its figures are above.
Spider-Lens
Spider-Lens is a free, open-source application for analyzing Apache and Nginx server logs. It monitors traffic, HTTP errors, crawler bots and server performance through a dashboard covering requests, human visits, error rates, HTTP status trends, bot distribution and 12-week patterns. Crawler monitoring detects 16 or more bots and shows crawl budget and crawler volume. Performance reports include Time to First Byte by URL, configurable slow thresholds and CSV or Excel export. A sitemap crawler follows internal links and records titles, H1 headings, word counts, canonicals, robots directives and crawl depth. SMTP alerts and webhooks can report 404 spikes, server errors, missing Googlebot and weekly summaries. The project documents Discord, Slack and an MCP server integration. It runs as a Node.js application or native WordPress plugin, with no Node.js required for the plugin. Node deployment requires Node.js 18+, npm 9+ and log access. The interface lists 16 languages and automatic browser-language detection. Nova uses Gemini when configured.