Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PicaDecoder uses static members for instance-related information #109

Closed
cboehme opened this issue Jul 11, 2013 · 2 comments
Closed

PicaDecoder uses static members for instance-related information #109

cboehme opened this issue Jul 11, 2013 · 2 comments
Assignees
@cboehme
Labels
Bug Stream modules
Milestone
Version 2

Comments

@cboehme
Copy link
Member

cboehme commented Jul 11, 2013

This is not obvious and prone to produce unexpected behaviour.
@cboehme cboehme mentioned this issue Jul 12, 2013
Closed
@ghost ghost assigned cboehme Jul 14, 2013
cboehme added a commit to cboehme/metafacture-core that referenced this issue Jul 14, 2013
@cboehme
Re-implemented PicaDecoder based on a state machine.
3c75b41 
The old PicaDecoder used regular expressions to parse PICA+ records. 
This let to two problems:

 * Errors in the data resulted in exceptions which did not refer to the    
   portion of the data that caused the problem (e.g. a character index)
 * Due to the use of String.substring() for extracting data from the  
   record the full record was kept in memory (see issue metafacture#51)

The new PicaDecoder was written to solve these problems. The first one
was addressed by constructing the parser so that it only fails in two
clearly defined situations (missing id field and unexpected end of
record). The second one was solved by copying the parsed data portions
into new strings. 

In addition to the problems listed above, the following issues were
addressed:
 
 * metafacture#109 -- removed support for static usages of the encoder
 * metafacture#112 -- removed support for appendControlSubField. If Metamorph is  
   extended to pass data through (issue metafacture#107), this functionality can 
   easily be implemented in a script. It is also not clear how widely it 
   is used at all.
 
While having removed support for control subfields the new decoder
introduces a range of new options:

 * ignore missing id -- do not fail on missing ids but use an empty 
   string as record id
 * skip empty fields -- do not output fields without subfields or empty
   subfields only (i.e. subfields without name and value)
 * fix unexpected end of record -- if a record does not end with a field 
   delimiter one will be automatically added.
 * normalize UTF8 -- automatically performs UTF8 normalization of values
 
The unit tests have been rewritten to match the new options and to be
more useful for debugging.
@cboehme cboehme mentioned this issue Jul 14, 2013
Merged
@cboehme
Copy link
Member Author

cboehme commented Jul 15, 2013

The new PicaDecoder (pull request #113) solves this issue by removing support for static use of the decoder completely.

@cboehme
Copy link
Member Author

cboehme commented Nov 2, 2013

Fixed in pul lrequest #113

@cboehme cboehme closed this as completed Nov 2, 2013
blackwinter added a commit that referenced this issue Dec 13, 2024
@blackwinter
Produce Value.Array for marked array entities. (#109)
a42943d
blackwinter added a commit that referenced this issue Dec 13, 2024
@blackwinter
Require explicit array initialization. (#109)
7d9b2a4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cboehme cboehme

Labels
Bug Stream modules
Projects
None yet
Milestone
Version 2
Development

No branches or pull requests
1 participant
@cboehme