Is it possible to use pytmx along with pygame_sdl2 for easy android deployment?

[umbe1987]

Hi @bitcraft.
First of all, thanks for this great project!
I've been experimenting with it and have been very satisfied so far.
I just discovered pygame_sdl2 and I would really like to use it along with rapt for testing my game on my android device.
However, when I import pygame_sdl2 the game does not work anymore like before (it does not even inizialize) giving error like "no mode has been set". Apparently, this is happening during the call to tiled_map = load_pygame('map.tmx') as I explained in this issue.
Does anybody know of a succesfull port of a pygame_sdl2 to android using pytmx?
Thanks!

[bitcraft]

I am assuming that there is some issue with how pytmx imports pygame....maybe? Pytmx makes heavy use of software surfaces and the mask module, and that may also have some conflicts. I don't have the test environment to work with this issue, but if you are willing to try a few things, then we can maybe pinpoint the issue and work something out.

My first request is to find a more specific traceback, if possible. That call to load_pygame is just a function call to more python code, the actual cause of the crash is likely somewhere else.

[umbe1987]

Ehi, thanks for your feedback!
I'll come back to this as I am really interested in make pytmx work with pygame_sdl2.
I'll try to debug the code to hopefully pinpoint the prooblem or at least post the error I was receiving.
Just few informations to start with on the issue of the interaction between pytmx, pygame and pygame_sdl2:

1. pygame module is "overwritten" by pygame_sdl2 by writing this code at the top of each scripts (source):

try:
    import pygame_sdl2
    pygame_sdl2.import_as_pygame()
except ImportError:
    pass

2. These are pygame_sdl2 omissions (source):

Current omissions include:

Modules not listed above.
APIs that expose pygame data as buffers or arrays.
Support for non-32-bit surface depths. Our thinking is that 8, 16, and (to some extent) 24-bit surfaces are legacy formats, and not worth duplicating code four or more times to support. This only applies to in-memory formats - when an image of lesser color depth is loaded, it is converted to a 32-bit image.
Support for palette functions, which only apply to 8-bit surfaces.

[MyreMylar]

Not certain that this is related but I've also found that pgame_sdl2 doesn't play nicely with pygame on windows after freezing a program including both.

Do you think it would be possible to change the init file of PyTMX so that it doesn't always include pygame (and consequently bring along all of pygame's dlls) whether you are using it or not?

To explain a bit more I'm using both PyTMX and pygame_sdl2 to try and create an isometric tiled prototype rendering engine (it runs really nicely by the way!). The problem comes when I try to freeze the program on windows. Even though I'm using a custom image loader and not actually using any of the pygame code, the __init__.py file for pyTMX includes these lines:

try:
    from pytmx.util_pygame import load_pygame
except ImportError:
    logger.debug('cannot import pygame tools')

Which, following the chain of imports, means that freezing tools like py2exe, pyInstaller and cx_freeze are going to try and pull pygame and all it's associated dlls into your program.

I think this pulling in of unnecessary dlls is probably happening with pyglet using programs and well as sdl2 using ones, but it probably only causes a crash with sdl2 - or at least with pygame_sdl2.

As far as I can tell those lines are just there as a debug warning, which may now be out of date because there are several official image loaders.

[bitcraft]

@MyreMylar
TBH, i'm not sure how to move forward with this, because pytmx is widely used at this point with pygame. I think honestly, the best solution would be to blacklist pygame in your build script or to use a venv without pygame to build. Because IMO, just because pygame_sdl2 breaks when pygame is also imported....I don't see how that's a problem I need to fix (sorry if that sounds harsh!)

Nice work btw, on the iso view game. Are you using the SDL2 Renderer?

[MyreMylar]

Nice work btw, on the iso view game. Are you using the SDL2 Renderer?

Yep. I'm also trying to improve the library to work better for Tiled/TMX games in general. I have the basics of a custom image loader for pyTMX written that uses the Renderer module's TextureNodes. Ends up running about eight times faster than nearly the same code in pygame (about 350FPS vs 45FPS in my current prototypes).

Because IMO, just because pygame_sdl2 breaks when pygame is also imported....I don't see how that's a problem I need to fix (sorry if that sounds harsh!)

Well I believe the reason it breaks is because SDL2 and SDL1 are incompatible at the c library level - you should get the same problems with the pysdl2 importer. I was initially pursuing fixing this problem in the pygame_sdl2 library but other than pygame_sdl2 moving to sdl1 (which would rather undo the point of the library) there is nothing that can be done over there.

I don't think that removing the util_pygame import from the __init__ file should break pygame users very much because they should be importing util_pygame themselves already if they are following the documentation. Nor is it a hard fix to add import pytmx.util_pygame if their code does break.

The only other proper long term option I can see is creating a permanent PyTMX_sdl2 fork that would be basically the same as the root but with proper sdl2 support by removing this import. Perhaps we could add a deprecation warning to the init.py file mentioning that pygame users will have to import util_pygame in the future and leave it there for a few months?

At the least if it is not going to be fixed then we should probably document this problem because otherwise people will keep running into it and it is in no way immediately obvious what the problem is (it took me several days to track the crash down to PyTMX and then on to pygame).

Happy to create some pull requests for any of the three methods:

1. Deprecation warning -> followed by code fix in X months

2. Code fix

3. Document incompatibility between PyTMX and SDL2 due to dependence on pygame/SDL1 and some possible end user solutions.

[MyreMylar]

I've made some branches with the three proposed options over on my fork of PyTMX:

1. decrement warning for pygame dependency - This means users running their code with the -Wd flag will get a warning when running.

2. removing pygame dependency - This just takes out the pygame import from the __init__.py file. It could be expanded on by moving the 'you don't have pygame installed' logging error to the util_pygame.py file if you want to keep that functionality, I've just kept it simple for now. I've tested this fix and it solves the crash I was having and still allows both pygame and pygame_sdl2 versions of my program to run using PyTMX in the fashion described in the documentation.

3. document the incompatibility - This just describes the issue in the readme file and describes a way for users to try and solve it through stripping pygame out themselves. This fix also solves the crash I was having.

My preference would be for the library to move towards solution 2 just because I think it'll create less issues going forward into the future as more people migrate to using pygame alternatives. Plus, it's a lot easier to fix any issues that do crop up with pygame users by saying 'import pytmx.util_pygame' or 'refer to the docs' than it is for users that get these crashes to figure out they are coming from PyTMX and then how to solve them.

Anyway, let me know which you prefer and I'll convert it to a pull request. Then hopefully I'll be back in a few weeks/months (however long it takes me to get pygame_sdl2 updated) with a util_pygame_sdl2.py that runs a bit faster.

[bitcraft]

Thanks for thinking about the problem @MyreMylar. PyTMX is pretty old at this point, and I've made a few mistakes along the way. I really don't want to break people's projects by changing too much.

Pygame is not required for pytmx, as you know. It will happily run if it cannot find it. The problem is, that as it is checking for pygame, other things beyond my control are breaking....why is it my responsibility to make sure pysinstaller, pygame_sdl2, etc don't break if pygame is imported? I'm not convinced its my problem that things break when i do a reasonable attempt to import pygame and fail gracefully if not found (using try/except).

To summarize, you've got a problem with your build environment where two libraries will not import at the same time. I don't see how it is the responsibility of pytmx to be aware of that. Who knows, maybe next week pygame_sdl2 will work with both imported and then I would have wasted all this time developing a solution to a problem that has effected just one person at this point.

Blacklist pygame in your build script, or use a venv, both are very reasonable solutions. Adding documentation stating the incompatibility is fine, i guess, but i don't think anything should be done really; its not an issue with pytmx.

[MyreMylar]

OK, I will submit the documentation pull request.

I do think it is worth trying to make your library work with other commonly used libraries if possible, but I can understand losing enthusiasm for an old project. Heck knows I've done that enough in the past.

You would know better than I if using PyTMX with SDL2 is unpopular and not worth supporting.

[bitcraft]

I haven't lost enthusiasm; I'm stating that it is old and stable, and I wouldn't want to break it for the other people who use it by fiddling with how it checks for pygame, or dropping the statement where it tries to import it. It works great with other commonly used libraries, pyglet, kivy, even pygame_sdl2...but that last one doesn't play well with pygame and that's not my problem....at all. Look, I'll be honest, I'm not sure why you are pushing me so hard to change this. It works great with pygame_sdl2 (you know that already), but fails in your dev environment. Think about it, why should I make changes to something that is not my fault at all? You're submitting patches to pygame_sdl2, so why don't you fix that, the root of the problem?

…

On Tue, Apr 17, 2018 at 11:48 AM, Dan Lawrence ***@***.***> wrote: OK, I will submit the documentation pull request. I do think it is worth trying to make your library work with other commonly used libraries if possible, but I can understand losing enthusiasm for an old project. Heck knows I've done that enough in the past. You would know better than I if using PyTMX with SDL2 is unpopular and not worth supporting. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#95 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAh-p6Jf9waDXZBlb5opJubGVfQb6wuzks5tphz5gaJpZM4Rl3uz> .