Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AuditTechnicalMetadataFileList: only consider preserved files, touchups for recent data model changes #5048

Merged
merged 1 commit into from
May 29, 2024

Conversation

jmartin-sul
Copy link
Member

@jmartin-sul jmartin-sul commented May 29, 2024

Why was this change made? 🤔

part of sul-dlss/technical-metadata-service#515

How was this change tested? 🤨

  • re-ran the full report on QA. got 142 results instead of the 369 returned before this change.
  • ran the techMD rake task techmd:generate_for_moab for a druid in the new result list, and saw it get remediated (job got queued and executed in techMD service, audit detected no error when re-run after that).
  • confirmed that non-preserved file names were no longer fed to techMD's audit endpoint. e.g. for druid:wy216td0310 on QA, i originally got the result druid:wy216td0310: found technical-metadata-service database; inconsistencies with v1 cocina: {"missing_filenames"=>["example.jp2"]}. but after this change, the non-preserved example.jp2 was no longer sent to techMD, and the audit then detected no errors for that druid (before this change logs showed auditing {:druid=>"druid:wy216td0310", :file_list=>[{:filename=>"example.tiff", :md5=>"7c35bef13c1fdc201ff0c93a42085bc8"}, {:filename=>"example.jp2", :md5=>"8ff299eda08d7c506273840d52a03bf3"}]}, and after this change they showed auditing {:druid=>"druid:wy216td0310", :preserved_file_list=>[{:filename=>"example.tiff", :md5=>"7c35bef13c1fdc201ff0c93a42085bc8"}]}, and logged a response like druid:wy216td0310: audited technical-metadata-service; response status: 200; response body: {"missing_filenames":[],"unexpected_filenames":[],"mismatched_checksum_file_infos":[]}).
  • confirmed that percent complete messages still show up in the logs

⚡ ⚠ If this change has cross service impact, including data writes to shared file systems, run integration tests and/or test in [stage|qa] environment, in addition to specs. ⚡

…ps for recent data model changes

* since technical-metadata-service's database only tracks preserved files, filter non-preserved files from the list we sent to techMD service for auditing
* remove a straggler usage of Dro, since we've gotten rid of that model (follow on from #5013)
* comment updates for both of the above changes
@justinlittman justinlittman merged commit 5ead346 into main May 29, 2024
6 checks passed
@justinlittman justinlittman deleted the auditTechMDFixups branch May 29, 2024 19:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants