Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set default encoding of minter state files and allow overrides. #16

Merged
merged 1 commit into from
Aug 10, 2015
Merged

Set default encoding of minter state files and allow overrides. #16

merged 1 commit into from
Aug 10, 2015

Conversation

mjgiarlo
Copy link
Member

@mjgiarlo mjgiarlo commented Aug 7, 2015

While troubleshooting samvera-deprecated/sufia#1037, I found that the AF::Noid code that writes the output of Marshal.dump(minter.dump) to a state file is trying to write an ASCII-8BIT string. This causes Sufia to fail miserably, and here's an example:

  1) dashboard/index.html.erb main with transfers renders received and sent transfer requests
     Failure/Error: f.save!
     Encoding::UndefinedConversionError:
       "\xAC" from ASCII-8BIT to UTF-8
     # /home/mjg/workspace/active_fedora-noid/lib/active_fedora/noid/synchronized_minter.rb:50:in `write'
     # /home/mjg/workspace/active_fedora-noid/lib/active_fedora/noid/synchronized_minter.rb:50:in `block in next_id'
     # /home/mjg/workspace/active_fedora-noid/lib/active_fedora/noid/synchronized_minter.rb:43:in `open'
     # /home/mjg/workspace/active_fedora-noid/lib/active_fedora/noid/synchronized_minter.rb:43:in `next_id'
     # /home/mjg/workspace/active_fedora-noid/lib/active_fedora/noid/synchronized_minter.rb:15:in `block in mint'
     # /home/mjg/workspace/active_fedora-noid/lib/active_fedora/noid/synchronized_minter.rb:13:in `synchronize'
     # /home/mjg/workspace/active_fedora-noid/lib/active_fedora/noid/synchronized_minter.rb:13:in `mint'
     # /home/mjg/workspace/active_fedora-noid/lib/active_fedora/noid/service.rb:17:in `mint'
     # ./sufia-models/app/services/sufia/noid.rb:10:in `assign_id'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/active-fedora-9.3.0/lib/active_fedora/persistence.rb:179:in `assign_rdf_subject'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/active-fedora-9.3.0/lib/active_fedora/persistence.rb:145:in `create_record'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/active-fedora-9.3.0/lib/active_fedora/indexing.rb:45:in `create_record'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/active-fedora-9.3.0/lib/active_fedora/callbacks.rb:237:in `block (2 levels) in create_record'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:115:in `call'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:115:in `call'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:553:in `block (2 levels) in compile'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:503:in `call'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:503:in `call'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:88:in `run_callbacks'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/active-fedora-9.3.0/lib/active_fedora/callbacks.rb:237:in `block in create_record'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:115:in `call'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:115:in `call'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:553:in `block (2 levels) in compile'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:503:in `call'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:503:in `call'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:88:in `run_callbacks'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/active-fedora-9.3.0/lib/active_fedora/callbacks.rb:236:in `create_record'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/active-fedora-9.3.0/lib/active_fedora/persistence.rb:139:in `create_or_update'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/active-fedora-9.3.0/lib/active_fedora/persistence.rb:31:in `save!'
     # /home/mjg/.rvm/gems/ruby-2.2.1@sufia/gems/active-fedora-9.3.0/lib/active_fedora/validations.rb:56:in `save!'
     # ./spec/views/dashboard/index_spec.rb:140:in `block (5 levels) in <top (required)>'
     # ./spec/views/dashboard/index_spec.rb:138:in `tap'
     # ./spec/views/dashboard/index_spec.rb:138:in `block (4 levels) in <top (required)>'

@mjgiarlo mjgiarlo mentioned this pull request Aug 7, 2015
Merged
@awead
Copy link
Contributor

awead commented Aug 10, 2015

Should we have a short test that verifies we're sending force_encoding?

@mjgiarlo
Copy link
Member Author

@awead I've got a better PR in now, thanks to @grosscol

awead added a commit that referenced this pull request Aug 10, 2015
@awead
Merge pull request #16 from projecthydra-labs/write_to_binary_file
1e6706e 
Force UTF-8 encoding of minter state files
@awead awead merged commit 1e6706e into master Aug 10, 2015
@awead awead deleted the write_to_binary_file branch August 10, 2015 20:01
@awead
Copy link
Contributor

awead commented Aug 10, 2015

Nice!

@mjgiarlo mjgiarlo changed the title Force UTF-8 encoding of minter state files Set default encoding of minter state files and allow overrides. Aug 10, 2015
jcoyne
jcoyne reviewed Aug 10, 2015
View reviewed changes
lib/active_fedora/noid/synchronized_minter.rb
def next_id
id = ''
::File.open(statefile, ::File::RDWR|::File::CREAT, 0644) do |f|
::File.open(statefile, ::File::RDWR|::File::CREAT, 0644, file_opts) do |f|
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting this to ::File.open(statefile, 'a+b', 0644) do |f| ... is how I'd do this. You should be able to avoid lines 54 & the file_opts method then.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added |::File::BINARY to the above modes and that didn't fix this issue in the Rails environment, so I'm not sure binary is the fix here.

jcoyne added a commit that referenced this pull request Aug 11, 2015
@jcoyne
Merge pull request #17 from projecthydra-labs/revert_16
ec80a61 
Append to minter state files in binary mode. Reverts #16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@mjgiarlo @awead @jcoyne