db_check does not ERROR when cannot connect to storage

[danstoner]

The "error" currently only appears in DEBUG lines. In this case, the recordset could not be downloaded (could not connect to storage) which probably should be a fatal error since the program couldn't do any work. At the very least, an ERROR logline should be written.

2020-10-21 19:42:16.526 INFO  idb.db-check.7232e59b-b0e1-49cd-9632-b8ae1f49f828჻ Starting db_check ingest: False
2020-10-21 19:42:16.529 DEBUG idb.storage჻ Initialized IDigBioStorage connection (boto.connect_s3) to host 's.idigbio.org'
2020-10-21 19:42:16.673 DEBUG idb.db-check.7232e59b-b0e1-49cd-9632-b8ae1f49f828჻ Exception in get_file
2020-10-21 19:42:16.675 INFO  idb.db-check჻ 7232e59b-b0e1-49cd-9632-b8ae1f49f828 writing summary json
2020-10-21 19:42:16.680 INFO  idb.db-check.7232e59b-b0e1-49cd-9632-b8ae1f49f828჻ Finished db_check in 0.152s
2020-10-21 19:42:16.681 DEBUG idb჻ db_check(...) took 1.8s

[danstoner]

The accompanying summary file is just a dummy / stub.

{
  "datafile_ok": false, 
  "records_count": 0, 
  "records_update": 0, 
  "paused": false, 
  "harvest_date": "", 
  "mediarecords_create": 0, 
  "mediarecords_update": 0, 
  "recordset_id": "7232e59b-b0e1-49cd-9632-b8ae1f49f828", 
  "commited": false, 
  "mediarecords_delete": 0, 
  "filemd5": null, 
  "filename": "7232e59b-b0e1-49cd-9632-b8ae1f49f828", 
  "records_delete": 0, 
  "records_create": 0, 
  "mediarecords_count": 0
}

[danstoner]

Are we catching these in suspects file?

records_count and mediarecords_count are both 0.

[danstoner]

It is potentially a transient error... seems like both error log AND suspects handling (if we don't already have that) would be good. 🤔

To test... maybe db-check a non-existent uuid?