openpyexcel upgrade

[czue]

this makes it substantially easier to run commcare hq and commcare-export in the same python environment (and generally seems good to upgrade libraries).

@snopoke I tested basic query file reading and writing to an .xlsx file. do you know what else I should be testing or if there are other gotchas you might expect with this change?

EDIT: It looks like at least one change is that Excel 2007+ files are required to have a valid '.xlsx' extension even if they were previously valid files with a different name... Not sure if that's a problem.

[czue]

(looking at tests)

[emord]

this makes it substantially easier to run commcare hq and commcare-export in the same python environment (and generally seems good to upgrade libraries).

Using == will make this much harder as we then need to upgrade both of these libraries.

as an aside: I think its a bad workflow to use the same virtualenv anyways unless we explicitly want to support that and write tests for it or include commcare-export as a dependency in commcare-hq. different projects with different environments is what virtualenv is made for.

[czue]

Using == will make this much harder as we then need to upgrade both of these libraries.

Oops. What's the right thing to do? ~=?

I think its a bad workflow to use the same virtualenv anyways unless we explicitly want to support that and write tests for it or include commcare-export as a dependency in commcare-hq. different projects with different environments is what virtualenv is made for.

Totally agree. I'm working on a project that explicitly needs to run them in the same env and so I'm looking for a reasonable way to do it and this seemed like the approach that made the most sense (can ping you on that PR when it is ready if you want). I'm not planning on making any dependencies explicit since it's a bit of a one-off.

[czue]

my local machine:

$ ls ~/.virtualenvs/ | wc
     41      41     609

[emord]

Oops. What's the right thing to do? ~=?

That's hard to say and its a library by library thing. Generally if you want libraries to work together robustly, you should make them as loose as possible. Judging from https://openpyxl.readthedocs.io/en/2.6/changes.html you may want >=2.1,<3 whatever the minimum version that's required is.

I think the way most libraries test this out is to use tox to create travis builds for each version, so in this example it would test openpyxl 2.1, 2.2, 2.3, 2.4, 2.5, 2.6.

[czue]

Gotcha. Am guessing I can figure out what versions are compatible / incompatible some way and change it. Before I go down that road, I'm just trying to understand what the value proposition of that effort is and make sure it's a good use of time.

Does your comment still apply if we continue to expect people to (mostly) use commcare-export as a standalone tool? I'm not totally sure where the upgrade dependencies/conflicts would arise.

I guess I'm tempted to just change it to >= and call it a day, but maybe I'm missing something.

Edit: I agree with Simon and think that's actually a case for leaving it as-is.

[emord]

Yeah. I think that's fine if this is a one-off and does not need to be maintained. Just wanted to make a note that the discrepancy between making two repository's libraries compatible and using ==

[snopoke]

Looks ok to me (Not sure on our the == vs ~= question). If the tests are passing then I don't think there's any other concern.

[snopoke]

This isn't generally used as a library so I think it's better to fix it to a smaller range to avoid potential issues if someone installs the tool with an untested version that breaks.

[czue]

@snopoke is there anything else you want me to do/test on this or is it ok to merge in? Also, let me know if you'd like me to cut a new release

[snopoke]

no need to cut a new release unless now unless it makes things easier for you

[czue]

nope all good for now, thanks!