28 projects
Web crawling · 3
Show allNo matching projects.
A command-line tool for finding broken links, incorrect redirects and indexing conflicts on a live or preview website. It discovers URLs, checks what your server returns and gives you findings tied to specific pages, with the evidence behind them.
ILCrawler is the technical SEO crawler I use for my own websites and client audits. It brings page checks, links, screenshots, performance samples and reporting into one place. It’s also the project that repeatedly convinced me I was only one rewrite away from being happy with it.
A command-line tool for catching crawl and indexing problems when a website changes. It checks built files or pages fetched from a live or preview site, and compares scans so you can spot problems before a release.