crawler4j

Open Source Web Crawler for Java

Homepage POM file JAR file Javadoc

'edu.uci.ics:crawler4j:3.3'

<dependency>
	<groupId>edu.uci.ics</groupId>
	<artifactId>crawler4j</artifactId>
	<version>3.3</version>
</dependency>

<dependency org="edu.uci.ics" name="crawler4j" rev="3.3"/>

"edu.uci.ics", "crawler4j", "3.3"

Dependencies

Compile dependencies

com.sleepycat » je » 4.0.92
commons-codec » commons-codec » 1.5
commons-logging » commons-logging » 1.1.1
log4j » log4j » 1.2.14
org.apache.httpcomponents » httpclient » 4.1.2
org.apache.httpcomponents » httpcore » 4.1.4
org.apache.tika » tika-parsers » 1.0

Test dependencies

junit » junit » 4.10