Troubleshooting

Common Errors

• If you want to clear a running import or export, you MUST stop the background worker before clearing the jobs. Both sidekiq and delayed job will "hold" jobs that are being run and requeue them if they fail. Stopping the workers and then removing any jobs in the retry or importer/exporter queue will allow you to stop an import or export that is not going well.

• Error: RuntimeError - File /app/samvera/hyrax-webapp/tmp/originals/kentuckymaps_Image/files/LouAtlas1884_Cover.jpg does not exist

  • How to resolve: It’s most likely because the value in the CSV's file column does not match the name of the file in the files directory. When the values are changed to match respectively, the importer will get past these errors. For this specific example, the value in the file column (originally LouAtlas1884_Cover.jpg) had to be changed to match the name of the file in the files directory, which was named LouAtlas1884_AtlasCover

• Error: StandardError - Missing required elements, required elements are: title, source_identifier

  • How to resolve: This error occurs when a CSV is missing required columns or the columns weren’t mapped correctly in the Bulkrax config. Add the required columns to resolve this issue.

• Error: ActiveFedora::ObjectNotFoundError - Couldn't find Collection with 'id'=218e37fd-c7a1-4677-ac20-e3415fc33349

  • How to resolve: This is a user error where the collection they specified does not actually exist. Create the required collection or use another collection to resolve this issue.

• Error: StandardError - Missing required elements, missing element(s) are: title

  • How to resolve: In the CSV, the title heading does not exist or it’s been left blank. Fix the required title to resolve this issue.

• Error: Zip::DestinationFileExistsError - Destination '/data/tmp/shared/imports/import_216_20211012173943_378/i67_original_2/files/.DS_Store' already exists

  • How to resolve: This is an error that occurs specifically in MacOS and should now be resolved in newer versions of Bulkrax. Try zipping your file on a windows machine or using the terminal to zip you files (ex. zip -r data.zip . -x ".DS_Store" -x "__MACOSX"). If this error is still occurring or you are uncomfortable using the terminal, please contact your dev/IT person.

• Error: any ldp error, e.g. ldp::GONE error

  • How to resolve: LDP errors are usually errors that occur when a connection to Fedora can’t be made or two items are being saved at the same time. To resolve this, check that your Fedora connection is working correctly. If it is and you are still receiving this error, contact your dev/IT person and give them that information.

• Hyrax::LockManager::UnableToAcquireLockError

  • Potential solution: This is tricky when adding lots of FileSets to the same work. One thing we have done (since actual file saving / derivatives take time to process) is to scatter the import list. If you have 400 file sets on 100 works, sort them by something besides which work they go to so that they get queued with fewer trying to save to the same work at the same time.

Importer Errors

Whilst Errors with Entry build are logged against the db record, errors for the Importer are not (yet, this work is imminent).

The following can be run in the rails console to help figure out the status of an Importer.

# Find the Importer
> i = Bulkrax::Importer.find(IMPORTER_ID)

# Is the Import valid? 
# If false, check what the valid_import? method is doing to see what might have gone wrong
> i.valid_import?

# View errors on the Importer
> i.last_error

# Get the parser
> p = i.parser

# Get the records to be imported
# If this is empty, check the records method for the parser class
# eg. BagItParser calls metadata_paths
# metadata_paths uses metadata_file_name provided in the form
#   to find all the metadata files in the bag
# if that's wrong, records will end up as an empty array
> p.records

# Try running the import
> if i.collections_created?
>   i.import_works
> else
>   i.import_collections
>   i.import_works
> end

# See if there are entries
> i.entries.count

# Check an entry
> e = i.entries.first
> e.last_error # if this is nil, the entry has run without error

# Re-build an entry
> e.build # if this is successful it should return the work id
> e.save

# Check the metadata
> e.build_metadata
> e.parsed_metadata # parsed_metadata is sent to Hyrax as the attributes hash for work creation/update

# Find the work for the entry
> e.factory.build

# Run the factory
#  the build method will run this
#  running directly like this will surface errors from Fedora/Hyrax for easier troubleshooting
> e.factory.run