SuperBot Review

Having used crawlers like the unix-commandline wget, Teleport and Internet Researcher on Windows I downloaded Spybot to try what seemed to be yet another run of the mill web crawling software.

I was very pleasantly surprised that it has a lot of very advanced features. For example it has MIME type filter. This way you can say download only flash objects off some sites without actually downloading the html. If you are not familiar with MIME types then you can use the file extension filter as well.

Another unique advanced feature was one of the address generator where you can generate sequences of addresses by using replacing some expression in a typical address. And the interface for this particular feature is very easy to use.

SuperBot also comes with a search and replace feature which can replace words in the text that it downloads with words you input. I dont know why I would have any use of that as I would very much like to read the original content. But just in case you need it, it is there.

A nice feature which is not found in other crawlers is a report generation button that will give you a detailed report after the download is done.

Other than these new features the usual stuff was also there like password lists, a built-in browser, multi-threading download support, clipboard monitoring, queueing and automation of jobs. I think the downloading performance was pretty average to above average and I dont have complaints there. It is also pretty lightweight (occupies only 11MB RAM in idle state).

The interface is a bit unfriendly and takes getting used to. There seem to be a per job and a global settings. But the per job settings are hidden and are not visible. It took me 1/2 hr to figure that they were hidden behind the common jobs button. But after the first few site crawls I was happily going from tab to tab configuring my jobs as I wanted. The user interface definitely needs a lot of work. And the inbuilt browser definitely is not something I would use.