Code

trough feed#180

Code

Merged

jkafader merged 10 commits intomasterinternetarchive/heritrix3:masterfrom aitfive-1176internetarchive/heritrix3:aitfive-1176Copy head branch name to clipboardSep 6, 2017

+354-1

Commits

Commits on Jun 8, 2017

• initial version of TroughCrawlLogFeed.java

  nlevitt

  committed
  d1452c1

  Copy full SHA for d1452c1

• batch trough writes

  nlevitt

  committed
  eed0210

  Copy full SHA for eed0210

• hacked up tarball with heritrix, contrib and dependencies

  nlevitt

  committed
  66fabf7

  Copy full SHA for 66fabf7

• oops, String.join is a java 8 feature

  nlevitt

  committed
  a1f59da

  Copy full SHA for a1f59da

• missed the maven tarball assembly file

  nlevitt

  committed
  b670c2f

  Copy full SHA for b670c2f

Commits on Jun 15, 2017

• update for new data model queued_url->uncrawled_url and some new fields

  nlevitt

  committed
  a1a7232

  Copy full SHA for a1a7232

Commits on Jun 27, 2017

• include http response payload when logging errors from trough

  nlevitt

  committed
  2f29949

  Copy full SHA for 2f29949

• missed a comma

  nlevitt

  committed
  698a02a

  Copy full SHA for 698a02a

• missed another comma

  nlevitt

  committed
  66ead24

  Copy full SHA for 66ead24

• realized this stuff needs to be thread safe

  nlevitt

  committed
  1c92be2

  Copy full SHA for 1c92be2