Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial support for OM System MakerNote #2167

Merged
merged 2 commits into from
Mar 28, 2022
Merged

Conversation

kmilos
Copy link
Collaborator

@kmilos kmilos commented Mar 25, 2022

Addresses #2126 in a "dumb" way by duplicating the header class.

@codecov
Copy link

codecov bot commented Mar 25, 2022

Codecov Report

Merging #2167 (18e2b9a) into 0.27-maintenance (86c25e2) will increase coverage by 0.07%.
The diff coverage is 68.96%.

@@                 Coverage Diff                  @@
##           0.27-maintenance    #2167      +/-   ##
====================================================
+ Coverage             58.73%   58.81%   +0.07%     
====================================================
  Files                   148      148              
  Lines                 23154    23161       +7     
  Branches              12689    12696       +7     
====================================================
+ Hits                  13600    13621      +21     
+ Misses                 6720     6703      -17     
- Partials               2834     2837       +3     
Impacted Files Coverage Δ
src/makernote_int.cpp 69.33% <68.96%> (-0.03%) ⬇️
samples/Jzon.h 34.88% <0.00%> (-0.12%) ⬇️
src/basicio.cpp 49.87% <0.00%> (+0.06%) ⬆️
src/exiv2.cpp 56.28% <0.00%> (+0.06%) ⬆️
src/rafimage.cpp 19.91% <0.00%> (+0.08%) ⬆️
src/xmp.cpp 77.46% <0.00%> (+0.15%) ⬆️
src/tags_int.cpp 77.21% <0.00%> (+0.19%) ⬆️
src/datasets.cpp 50.28% <0.00%> (+0.28%) ⬆️
src/tags.cpp 51.16% <0.00%> (+0.29%) ⬆️
src/cr2image.cpp 8.86% <0.00%> (+0.32%) ⬆️
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 86c25e2...18e2b9a. Read the comment docs.

@kmilos kmilos added makerNote Anything related to one of the various supported MakerNote formats enhancement feature / functionality enhancements labels Mar 25, 2022
@piponazo
Copy link
Collaborator

@kmilos In #2126 there is a sample image provide by the reporter of the issue. Is there any chance to extract the metadata from that image into an .exv file and add a new test in this branch to make sure that we are solving the issue?

@kmilos
Copy link
Collaborator Author

kmilos commented Mar 25, 2022

Is there any chance to extract the metadata from that image into an .exv file and add a new test in this branch to make sure that we are solving the issue?

Will do when I find some time again @piponazo

@sarunasb
Copy link

@kmilos In #2126 there is a sample image provide by the reporter of the issue. Is there any chance to extract the metadata from that image into an .exv file and add a new test in this branch to make sure that we are solving the issue?

Glad to provide any .exv files, but would need to know if any other switches beside -ex should be used.

@clanmills
Copy link
Collaborator

I think we had a discussion about how to manually create the .EXV when a segment (XMP or Exif) > 64k. #2126 (comment)

In the first instance, use $exiv2 -ea --force --verbose path to create a .EXV and you'll probably be happy with the result. If however the XMP or Exif metadata exceeds one JPEG segment (of 64k max), you'll have to apply some effort to create a suitable .EXV. You can inspect the structure of an image with the command $ exiv2 -pS path.

@sarunasb
Copy link

sarunasb commented Mar 25, 2022

$ exiv2 -ea --force --verbose OM100015.ORF
File 1/1: OM100015.ORF
Writing Exif data from OM100015.ORF to ./OM100015.exv
Writing XMP data from OM100015.ORF to ./OM100015.exv
Warning: No image data to encode Exif.OlympusCs.PreviewImageStart.
Warning: Exif tag Exif.Olympus2.ThumbnailImage not encoded
Warning: Exif tag Exif.OlympusCs.PreviewImageLength not encoded
Warning: Exif tag Exif.OlympusCs.PreviewImageStart not encoded
Warning: Exif tag Exif.OlympusCs.PreviewImageValid not encoded
Warning: Exif tag Exif.Photo.MakerNote not encoded
Warning: Exif tag Exif.OlympusIp.0x0635 not encoded
Warning: Exif tag Exif.OlympusIp.0x0636 not encoded
Warning: Exif tag Exif.OlympusIp.0x063a not encoded
Warning: Exif tag Exif.OlympusIp.0x063b not encoded
Warning: Exif tag Exif.OlympusIp.0x1103 not encoded
Warning: Exif tag Exif.OlympusIp.0x1104 not encoded
$ exiv2 -pS OM100015.ORF
ORF IMAGE
STRUCTURE OF TIFF FILE (II): OM100015.ORF
 address |    tag                              |      type |    count |    offset | value
      10 | 0x0100 ImageWidth                   |      LONG |        1 |           | 5220
      22 | 0x0101 ImageLength                  |      LONG |        1 |           | 3912
      34 | 0x0102 BitsPerSample                |     SHORT |        1 |           | 16
      46 | 0x0103 Compression                  |     SHORT |        1 |           | 1
      58 | 0x0106 PhotometricInterpretation    |     SHORT |        1 |           | 1
      70 | 0x010e ImageDescription             |     ASCII |       32 |      2484 | ...............................
      82 | 0x010f Make                         |     ASCII |       24 |      2516 | OM Digital Solutions   
      94 | 0x0110 Model                        |     ASCII |       17 |      2540 | OM-1            
     106 | 0x0111 StripOffsets                 |      LONG |        1 |           | 1817600
     118 | 0x0112 Orientation                  |     SHORT |        1 |           | 1
     130 | 0x0115 SamplesPerPixel              |     SHORT |        1 |           | 1
     142 | 0x0116 RowsPerStrip                 |      LONG |        1 |           | 3912
     154 | 0x0117 StripByteCounts              |      LONG |        1 |           | 17892967
     166 | 0x011a XResolution                  |  RATIONAL |        1 |      1064 | 350/1
     178 | 0x011b YResolution                  |  RATIONAL |        1 |      1072 | 350/1
     190 | 0x011c PlanarConfiguration          |     SHORT |        1 |           | 1
     202 | 0x0128 ResolutionUnit               |     SHORT |        1 |           | 2
     214 | 0x0131 Software                     |     ASCII |       32 |      2558 | Version 1.0                    
     226 | 0x0132 DateTime                     |     ASCII |       20 |      2400 | 2022:03:14 15:05:11
     238 | 0x013b Artist                       |     ASCII |       64 |      2420 | Sarunas Burdulis                 ...
     250 | 0x02bc XMLPacket                    |      BYTE |     2301 |      2590 | <?xpacket begin="..." id="W5M0Mp ...
     262 | 0x4746 Rating                       |     SHORT |        1 |           | 0
     274 | 0x8298 Copyright                    |     ASCII |       64 |      4894 | Sarunas Burdulis                 ...
     286 | 0x8769 ExifTag                      |      LONG |        1 |           | 314
     298 | 0x8825 GPSTag                       |      LONG |        1 |           | 806
END OM100015.ORF

OM100015.exv is here:
https://math.dartmouth.edu/nextcloud/s/mBPbcPNsJAY8tQW

@clanmills
Copy link
Collaborator

Gosh, I don't want to get sucked into this. Where's the "original" file OM100015.ORF?

Sometimes, when you're up to your neck in alligators, it's hard to remember that this mission was to drain the swamp! All we want is a small file that contains the metadata in which you are interested. We prefer to add small files to the test suite.

Can you get ExifTool to create a .EXV from OM100015.ORF?

@piponazo
Copy link
Collaborator

I thought it might convenient to bring here the topic about the usage of Git LFS. If we move on with the proposal I presented in #2159 we could eventually upload large binary files for testing.

@sarunasb
Copy link

Here is the original RAW OM100015.ORF (19MB), from which the .exv was created:
https://math.dartmouth.edu/nextcloud/s/n4S2WtaRWRXpoMw

I have also created .exv from the JPEG sample attached in #Support for files from OM System cameras #2126:
https://math.dartmouth.edu/nextcloud/s/tya7CdrTb8Neb2Q

I do have a local version of ExifTool, modified to read OM System MakerNote. Here is an .EXV created with
exiftool -tagsfromfile OM100015.ORF OM100015_exiftool.EXV:
https://math.dartmouth.edu/nextcloud/s/DpYAxCqQoPEyd4N

@kmilos
Copy link
Collaborator Author

kmilos commented Mar 28, 2022

Good to go I think, unless @clanmills or @hassec figure out a more compact/elegant way of implementing this.

Copy link
Collaborator

@clanmills clanmills left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this, @kmilos.

I'm going to approve this because it works and fits the existing architecture. It's amazing that we need 100+ lines of additional boiler plate code when tvisitor does the same job in 2 lines (which I suspect I can reduce to 1 line!).

The inflexibility of the __MnHeader::__Signature() methods is because they are baked into the code. I feel we have failed to discover how this can be detected and fixed by read(). Having created the class with confidence and proceeded to read the metadata, we should not need rigid static strings and size_t variables in different classes.

The other architectural issue is the makernote classes. tvisitor.cpp has no maker note classes at all. None. I never discovered a need for any. However the effort and risk involved in removing the makernote classes from Exiv2 cannot be justified. So, let's leave alone. If anybody ever rewrites Exiv2, I hope they will appreciate the simpler (and more robust and consistent) approach used by tvisitor to deal with makernotes, ICC, IPTC and format specific metadata such as PNG/textual-information. Less can be more!

None-the-less, the code is correct and does its job. Thank You to @kmilos for putting in the effort to add this to Exiv2.

@hassec
Copy link
Member

hassec commented Mar 28, 2022

Should we maybe just create a small follow-up issue to look into cleaning up the duplication? Just so that it doesn't get forgotten?

Otherwise, I'm happy for this to go in 👍

@clanmills
Copy link
Collaborator

@hassec The duplication? Are you referring to the boiler plate class code being mostly duplicated in the _MnHeader classes? If that's what you're thinking, my inclination is to allow it as the effort/risk to re-engineer these classes is considerable.

@kmilos
Copy link
Collaborator Author

kmilos commented Mar 28, 2022

Should we maybe just create a small follow-up issue to look into cleaning up the duplication? Just so that it doesn't get forgotten?

Sure, feel free!

@kmilos kmilos merged commit 8c27e23 into 0.27-maintenance Mar 28, 2022
@kmilos kmilos deleted the 027_omsystem_mn branch March 28, 2022 17:39
@kmilos kmilos added the forward-to-main Forward changes in a 0.28.x PR to main with Mergify label Mar 28, 2022
@kmilos kmilos added this to the v0.27.6 milestone Mar 28, 2022
piponazo added a commit that referenced this pull request Mar 31, 2022
Initial support for OM System MakerNote (backport #2167)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement feature / functionality enhancements forward-to-main Forward changes in a 0.28.x PR to main with Mergify makerNote Anything related to one of the various supported MakerNote formats
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants