-
Notifications
You must be signed in to change notification settings - Fork 151
Jobs
Tony Shen edited this page May 16, 2019
·
5 revisions
在 SpiderEngine 中支持添加定时任务 job。
虽然在 Spider 中支持简单的定时任务,但是不支持cron表达式。
SpiderEngine 的 job 是支持 cron 的,因为底层使用了 Quartz。
package cn.netdiscovery.example;
import cn.netdiscovery.core.Spider;
import cn.netdiscovery.core.SpiderEngine;
import cn.netdiscovery.core.domain.Request;
/**
* Created by tony on 2019-05-12.
*/
public class TestSpiderJob {
public static void main(String[] args) {
SpiderEngine spiderEngine = SpiderEngine.create();
Request request = new Request("http://www.163.com").checkDuplicate(false);
spiderEngine
.addSpider(Spider.create().name("tony1").request(request))
.httpd()
.run();
spiderEngine.addSpiderJob("tony1",request,"0 * * * * ?");
}
}
- Configuration
- Downloader
- vertx webclient
- urlconnection
- http client
- okhttp3
- file
- selenium
- Chrome
- Firefox
- IE
- Phantomjs
- htmlunit
- Queue
- ConcurrentLinkedQueue
- Disruptor
- Redis
- Kafka
- RabbitMQ
- RocketMQ
- Parser
- Pipeline
- Rate limiting
- RPC
- Jobs
- Vert.x EventBus
- User Agent Pools
- IP Pools
- Cookies Pools
- Monitor
- Spider
- SpiderEngine
- DSL
- Coroutines
- ChangeLog