Artifact 'us.codecraft:webmagic-core:0.4.0'

A crawler framework. It covers the whole lifecycle of crawler: downloading, url management, content extraction and persistent. It can simply the development of a specific crawler.

Homepage POM file JAR file Javadoc

'us.codecraft:webmagic-core:0.4.0'

<dependency>
	<groupId>us.codecraft</groupId>
	<artifactId>webmagic-core</artifactId>
	<version>0.4.0</version>
</dependency>

<dependency org="us.codecraft" name="webmagic-core" rev="0.4.0"/>

"us.codecraft", "webmagic-core", "0.4.0"

Dependencies

Compile dependencies

com.google.guava » guava » 15.0
commons-collections » commons-collections » 3.2.1
log4j » log4j » 1.2.17
net.sourceforge.htmlcleaner » htmlcleaner » 2.5
org.apache.commons » commons-io » 1.3.2
org.apache.commons » commons-lang3 » 3.1
org.apache.httpcomponents » httpclient » 4.3.1
org.jsoup » jsoup » 1.7.2
us.codecraft » xsoup » 0.1.0

Test dependencies

junit » junit » 4.7