Port redmine issues from 1024 to 1170

[piponazo]

In this PR I'm porting most of the remaining redmine issues. However I skipped few of them:

• 1043 & 1074 Need further analysis.
• 1137 Is testing the piping of two exiv2 executions and our python test infrastructure is not prepared for that.

I just want to contribute what I have done until now. In the next weeks I will be less available and I did not want to block the removal of the printStructure function.

[piponazo]

@D4N Did you have the chance to look at these changes?

[D4N]

Yes, I started yesterday but didn't have the time to go through everything.

[D4N]

Have you written the file names here for a reason? Since you can create both lists via:

    filenames = [
        "$data_path/exiv2-bug1153{E}{i}.exv".format(E=E, i=i)
        for E, i in itertools.product(
                ['A', 'J'],
                "a b c d e f g h i j k".split()
        )
    ]

    commands = [
        "$exiv2 -pa -g Lens {!s}".format(fname) for fname in filenames
    ]

[piponazo]

Laziness. I will follow your proposal ;)

[D4N]

Could you try to re-create 15727cf with a different diff algorithm or change the commit message? To be honest, I haven't got a clue what got changed.

[D4N]

On my machine test 1054 fails with this diff:

-                       "DateTime": "2005:09:07 22:09:51",
?                                             ^^^^
+                       "DateTime": "2005:09:08 00:09:51",
?                                             ^^^^
                        "ImageDescription": "XMPFiles BlueSquare test file, created in Photoshop CS2, saved as .psd, .jpg, and .tif.",
                        "Make": "Nikon"
                },
                "Photo": {
                        "ColorSpace": 1,
                        "PixelXDimension": 360,
                        "PixelYDimension": 216,
-                       "DateTimeDigitized": "2005:09:07 22:07:40"
?                                                      ^^^^
+                       "DateTimeDigitized": "2005:09:08 00:07:40"
?                                                      ^^^^
                }

I'll take a look at that later, maybe it's just a local issue.

[D4N]

I have spent a while in gdb debugging the exiv2json call and found out that it is indeed a timezone issue. It seems that during metadata conversion, the dates are converted from the original timezone (which seems to be saved in xmp) into the local time zone (afaik exif metadata have no clue about timezones).

Anyway, long story short, in the bash testsuite there is a sneaky: export TZ=BST-1 in line 434 in functions.source, which explains why it works in the bash test suite, but not in the python one. I'll try to come up with a fix for this (I guess a new section in the test suite would be suitable) and then report back.

[piponazo]

Yes, I noticed this problem about the timezones, and at the end I decided to leave the code as in the original test. Maybe we can investigate this issue in the future (And create an issue to not forget about it). Otherwise the printStructure EPIC will be delayed.

[piponazo]

Could you try to re-create 15727cf with a different diff algorithm or change the commit message? To be honest, I haven't got a clue what got changed.

The main() function did not have curly brackets to delimit its body. It was the only change ... I do not know why the commit diff looks so odd. I think it is caused because my editor also fixed some indentation issues.

[D4N]

Maybe we can investigate this issue in the future

The problem with that is, that the python test suite will only work on local machines where the timezone is set to UTC (mine isn't). I would suggest that we temporarily skip the test unless os.getenv('TZ') == "UTC"? We could export TZ='UTC' on travis, so that we still retain the test coverage.

I have a rough idea how we could add this into the test suite, which would also fix the issue with piping exiv2 commands together. In a nutshell, the user could provide a Command class instance to a system test:

# -*- coding: utf-8 -*-

import system_tests


class aTest(metaclass=system_tests.CaseMeta):

    commands = [
        system_tests.Command(
            "$exiv2 $filename", stdin=Variable # or string,
            stdou=other_variable_where_stdout_is_written_to,
            env={"TZ": "UTC", "LANG": "C"}
        )
    ]
    # rest as expected

That will require some additional testing & fiddling with the metaclass, but it should be possible. I can create a new issue for a discussion of this.

The main() function did not have curly brackets to delimit its body. It was the only change ... I do not know why the commit diff looks so odd. I think it is caused because my editor also fixed some indentation issues.

Ah, I see. I am afraid there is not much that can be done, such diffs tend to look ugly. You can retry it with the patience or histogram diff algorithms, but I doubt they'll improve the situation.

[D4N]

@piponazo Could you squash your commits together, disable the test for issue 1054 unless os.getenv('TZ') == "UTC" and add an export TZ=UTC to travis? I'd merge this PR then.

[piponazo]

disable the test for issue 1054 unless os.getenv('TZ') == "UTC"

@D4N Is there an existing way to disable tests easily? I am not sure how to do this.

[D4N]

@D4N Is there an existing way to disable tests easily? I am not sure how to do this.

unittest.skip should work in this case. I think something like this should do the trick:

import os
import unittest

import system_tests

@unittest.skipUnless(os.getenv('TZ') == 'UTC', "Testcase only works with the timezone set to UTC")
class Exiv2jsonRecursiveJsonTreeWithXMP(metaclass=system_tests.CaseMeta):
    # rest here

[D4N]

Thanks for the grind work!