At work I’ve been taking over a very content intensive website the last few months. Today we found some links that were invalid whereby the destination URL was non-existent and we had to correct the links.
For this I found LinkChecker, and open source utility for checking broken links in websites.
Some of the main features of this quick download is :
- recursive and multithreaded checking
- HTTP/1.1, HTTPS, FTP, mailto:, news:, nntp:, Telnet and local file links support
- proxy support
- username/password authorization for HTTP and FTP and Telnet
- honors robots.txt exclusion protocol
- Cookie support
- Different interfaces: command line, GUI and web interface
What is quite nice about this utility is the command line interface, which can be used to automate regular checking of different sites, which is quite useful when you manage large collections of websites.