Website Crawlers
Nine website crawlers, each screen-recorded crawling the same test sites.
- Beam Us Up SEO CrawlerCompletely free, no limits, and easy interface, automatic error detection.
- GNU WgetGNU Wget is a free software package for retrieving files using HTTP, HTTPS, FTP and FTPS, the most widely used Internet protocols.
- InterroBotInterroBot provides the website crawler, search, and analysis you need to understand your website.
- KatanaA next-generation crawling and spidering framework
- Screaming Frog SEO SpiderThe Screaming Frog SEO Spider is a website crawler that audits your site for over 300 SEO issues to help improve both SEO, and UX.
- SEO MacroscopeA free and open source website broken link checking and scanning tool, with some technical SEO functionality for common website problems.
- SiteOne CrawlerThe free Swiss-Army-knife for your website.
- Visual SEO StudioSEO audit tool for web agencies, freelancers, in-house SEOs and e-commerce owners.
- Xenu's Link SleuthXenu's Link Sleuth (TM) checks Web sites for broken links.