Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deepcopy in py3k evaluates interpolated strings #188

Open
WeatherGod opened this issue Jul 10, 2019 · 7 comments · May be fixed by #189
Open

deepcopy in py3k evaluates interpolated strings #188

WeatherGod opened this issue Jul 10, 2019 · 7 comments · May be fixed by #189

Comments

@WeatherGod
Copy link

Consider the following code example:

from __future__ import print_function
from copy import deepcopy
from configobj import ConfigObj
conf = ConfigObj()
conf['foo'] = 'bar'
conf['baz'] = '%(foo)s.txt'
conf_new = deepcopy(conf)
conf_new['foo'] = 'abc'
print(conf_new['bar'])

In python 2.7, the output is "abc.txt". In python 3.7, the output is 'bar.txt', which is the value of the 'bar' key in the original config object.

@WeatherGod
Copy link
Author

Continuing the investigate this. I compared the execution paths between py2.7 and py3.7. The point where it diverges is in def __reduce__(self): when it calls dict(self). From that, I was able to get a slightly simplified example that demonstrates the problem:

from __future__ import print_function
from configobj import ConfigObj
conf = ConfigObj()
conf['foo'] = 'bar'
conf['baz'] = '%(foo)s.txt'
print(dict(conf))

In py2.7, this prints out
{'foo': 'bar', 'baz': '%(foo)s.txt'}

In py3.7, this prints out:
{'foo': 'bar', 'baz': 'bar.txt'}

@WeatherGod
Copy link
Author

having a hard time debugging past this point. It seems like the behavior of dict() has changed in subtle ways. I am going to see if earlier py3k releases had this problem.

@WeatherGod
Copy link
Author

Ah ha! this all works as expected in python 3.5! So, a change introduced in py3.6 broke things. Perhaps the reworked dictionary?

@WeatherGod
Copy link
Author

Apparently matplotlib ran into this problem last year with their rcparams implementation and I completely missed this: matplotlib/matplotlib#12604

So, this would have been broken on a minor release from cpython. Looking into seeing how configobj can be adapted to fix this problem.

@WeatherGod
Copy link
Author

Alright, I figured out a solution to the following cases: copy.copy(conf), copy.deepcopy(conf), and conf.copy(), but I haven't figured out dict(conf). Basically, dict() will now use conf.items(), which has always triggered string interpolation. Previously, it used dict.items(), which didn't.

@WeatherGod
Copy link
Author

while writing up unit tests, I discovered an interesting inconsistency in the existing code in pre-py3.6 versions.

conf = ConfigObj(['[a]', 'foo = bar', 'baz = $foo'], interpolation=True)
assert dict(conf) == {'a': {'foo': 'bar', 'baz': 'bar'}}
assert dict(conf['a']) == {'foo': 'bar', 'baz': '$foo'}

So, it seems that calling dict() on a ConfigObj is different than calling dict() on a Section? Not exactly sure why this is happening.

WeatherGod added a commit to WeatherGod/configobj that referenced this issue Jul 11, 2019
@WeatherGod WeatherGod linked a pull request Jul 11, 2019 that will close this issue
@robdennis
Copy link
Member

robdennis commented Jan 8, 2023

I appreciate the investigation here and I'm inclined to get this into 5.1.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants