[question] Packaging python pyc files

[keithrob91]

I've just discovered that packaging of pyc files (within __pycache__ directories) doesn't seem to work.

I believe its because of the checks within discarded_file() and gather_file() within manifest.py that deliberately exclude files called .pyc and directories called __pycache__

Is there anyway to make this work or is it a feature request?

[memsharded]

This is deliberate. .pyc files are not intended to be portable, as they depend on the python version, etc. which is something not possible to control with Conan, as Conan itself could be running under different python versions, and that is not controlled.

https://stackoverflow.com/questions/11368304/what-are-the-limitations-of-distributing-pyc-files

[memsharded]

quick question: what would be the use case? Even for pretty large python codebases, it is not a big issue to distribute .py and let python bytecompile them the first run.

[keithrob91]

quick question: what would be the use case? Even for pretty large python codebases, it is not a big issue to distribute .py and let python bytecompile them the first run.

We create 'secure appliances' which means we lock down the application and O/S in certain ways to protect it from attack. One of these protection mechanisms is to use Advanced Intrusion Detection Environment (AIDE) to protect files and directories from being changed maliciously.

This causes a problem when .pyc files are generated as this would be a change to a directory we monitor, which would trigger AIDE, which would shut the system down! Also, as it is the .pyc file that would subsequently be used by Python, not the original .py file, we would want to protect both the .py and .pyc files from modification.

In short, we want to distribute the .pyc files on install so that AIDE can (1) protect them and (2) not have to care about them suddenly appearing.

[keithrob91]

There are of course ways we can get around this without using Conan, but, we liked the idea of using Conan!

[keithrob91]

One last thing! If you look at Redhat supplied Python RPMs, they ship the .pyc files, so it isn't a totally crazy thing to do.

[memsharded]

Ok, thanks for the clarification. It doesn't seem possible, it is too deep in conan code, and distributed in different places, it would be too breaking. I am having a look, but it seems you would need some kind of workaround, like zipping them first or something like that.

I don't know if it could be another type of workaround, but if generating .pyc files is the thing breaking in the target, then maybe sys.dont_write_bytecode = True would avoid that completely and could help too?

[memsharded]

Also, this would be a duplicate of: #4961. I am closing this one, please follow up there. Thanks!

[keithrob91]

I don't know if it could be another type of workaround, but if generating .pyc files is the thing breaking in the target, then maybe sys.dont_write_bytecode = True would avoid that completely and could help too?

Thank you for the suggestion, we considered that, but have discounted it. I'm now following #4961 with interest! For now we will go with a workaround like the zip option or something similar.