Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

streamline parents lookup per run. #463

Merged
merged 5 commits into from
Apr 14, 2022
Merged

streamline parents lookup per run. #463

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5b3db36
remove database column for parents and just use relationships look up…
sephirothkod Apr 13, 2022
818756d
Merge branch 'main' into parents_fix
sephirothkod Apr 13, 2022
ad37121
remove now unneeded db change
sephirothkod Apr 13, 2022
de95964
remove now unneeded spec
sephirothkod Apr 13, 2022
d3582c2
update test app db schema
sephirothkod Apr 14, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion app/models/bulkrax/importer_run.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ class ImporterRun < ApplicationRecord
belongs_to :importer
has_many :statuses, as: :runnable, dependent: :destroy

serialize :parents, Array
def parents
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this get called now with importer.last_run.parents in the schedule relationships job?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this get called now with importer.last_run.parents in the schedule relationships job?

Copy link
Contributor Author

@sephirothkod sephirothkod Apr 13, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes; that is the idea at least.

PendingRelationship.where(bulkrax_importer_run_id: id).pluck(:parent_id).uniq
end
end
end
7 changes: 0 additions & 7 deletions app/models/concerns/bulkrax/import_behavior.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ def parent_jobs
self.parsed_metadata[related_parents_parsed_mapping].each do |parent_identifier|
next if parent_identifier.blank?

add_parent_to_import_run(parent_identifier, importerexporter.last_run)
PendingRelationship.create!(child_id: self.identifier, parent_id: parent_identifier, bulkrax_importer_run_id: importerexporter.last_run.id, order: self.id)
end
end
Expand All @@ -37,16 +36,10 @@ def child_jobs
self.parsed_metadata[related_children_parsed_mapping].each do |child_identifier|
next if child_identifier.blank?

add_parent_to_import_run(self.identifier, importerexporter.last_run)
PendingRelationship.create!(parent_id: self.identifier, child_id: child_identifier, bulkrax_importer_run_id: importerexporter.last_run.id, order: self.id)
end
end

def add_parent_to_import_run(parent_id, run)
run.parents << parent_id
run.save
end

def find_collection_ids
self.collection_ids
end
Expand Down
5 changes: 0 additions & 5 deletions db/migrate/20220301020307_add_parents_to_bulkrax_importer_runs.rb
View file
Open in desktop

This file was deleted.

5 changes: 0 additions & 5 deletions db/migrate/20220330165510_remove_array_true_from_importer_run_parents_column.rb
View file
Open in desktop

This file was deleted.

14 changes: 0 additions & 14 deletions spec/models/bulkrax/csv_entry_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -670,20 +670,6 @@ module Bulkrax
end
end

describe '#add_parent_to_import_run' do
subject(:entry) { described_class.new(importerexporter: importer) }
let(:importer) { FactoryBot.build(:bulkrax_importer_csv, importer_runs: [importer_run]) }
let(:importer_run) { build(:bulkrax_importer_run) }

it 'adds the parent_id to the run' do
expect(importer_run.parents).to eq([])

entry.add_parent_to_import_run('dummy', importer_run)

expect(importer_run.parents).to eq(['dummy'])
end
end

describe '#build_relationship_metadata' do
subject(:entry) { described_class.new(importerexporter: exporter) }
let(:exporter) { create(:bulkrax_exporter, :with_relationships_mappings) }
Expand Down
3 changes: 1 addition & 2 deletions spec/test_app/db/schema.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
#
# It's strongly recommended that you check this file into your version control system.

ActiveRecord::Schema.define(version: 2022_03_30_165510) do
ActiveRecord::Schema.define(version: 2022_03_03_212810) do

create_table "bookmarks", force: :cascade do |t|
t.integer "user_id", null: false
Expand Down Expand Up @@ -90,7 +90,6 @@
t.integer "total_file_set_entries", default: 0
t.integer "processed_works", default: 0
t.integer "failed_works", default: 0
t.text "parents"
t.index ["importer_id"], name: "index_bulkrax_importer_runs_on_importer_id"
end

Expand Down