Bad MAGIC (0x0A0D0DA8)

[lmachens]

I have a pyc file, which has a unknown magic.
3.11 has 0x0A0D0DA7, my file has 0x0A0D0DA8 and 3.12 has 0x0A0D0DCB. I assume, it's something in between?
But opcodes like STORE_FAST_LOAD_FAST are in use (which are available in #498)

Some sample files:
bad_magic.zip

Maybe someone has an idea?

[greenozon]

0xDA7 (dec: 3495) is the last one in 3.11, not sure how did you get a pyc module with 3496.... maybe some private custom build?

https://github.com/python/cpython/blob/8303d32ff55945c5b38eeeaf1b1811dbcf8aa9be/Lib/importlib/_bootstrap_external.py#L421

[greenozon]

you could make a hack:
change the existing magic for python 3.11 and rebuild your binaries:
MAGIC_3_11 = 0x0A0D0DA7, -> 0x0A0D0DA8,
and see if it helps to decode your pyc*

[lmachens]

@greenozon
Thx!
I already did that and get Unsupported opcode: STORE_FAST_LOAD_FAST error.
Then I tried out your 3.13 PR (thx !) because it has STORE_FAST_LOAD_FAST, but it throws Unsupported opcode: INPLACE_ADD

[greenozon]

to be honest, I have some doubts you are using 3.13 version of python which is still in beta phase...

could you tell more about your app/sw/env you are using?

it's very suspicious you have some unsupported MAGIC which is in between of 3.11 and 3.12 versions...

[lmachens]

@greenozon
It's very possible, that it's a custom build.
These files are from the game Once Human, which is using Neox game engine.

[greenozon]

OK, could you identify what python is being in use over there?
some python related dlls nearby or so...

if the MAGIC sig was changed most probably the opcodes/whatever might be hacked as well!?

[lmachens]

Sadly, I couldn't find a version number.
Yes, it's possible that they hacked them as well.
Did you take a look at the files?

[greenozon]

yeah, hacked pycdas just out of curiosity...
results are "bad":

    [Disassembly]
        0       <INVALID>
        2       <INVALID>
        4       <INVALID>
        6       <INVALID>
        8       STORE_NAME                      0: bindict
        10      <INVALID>
        12      <INVALID>
        14      <INVALID>
        34      <INVALID>
        36      KW_NAMES                        1: None
        40      <INVALID>
        50      STORE_NAME                      1: data
        52      <INVALID>
        54      <INVALID>

try to inspect other *.exe; *.dll in your game folder (eg using ExeinfoPE or Die tools or whatever you got used to...)

[greenozon]

ideally if you find some python engine dll (most probably modified cpython)

[greenozon]

Basically protected/obfuscated .pyc module is a advanced topic
if you up to it and ready to spend some days/weeks, there are some nice writeups, eg:
https://www.slideshare.net/slideshow/reversingobfuscatedpythonapplications-dropbox140819110311phpapp01/38200072

[lmachens]

Thx for your time and feedback @greenozon
Are you able to extract the bindict? This is the most important variable for me.

[greenozon]

jokes aside,
figure out what protection/obfuscation applied and where is the execution environment

[rockingdice]

@lmachens as far as I know, the neox engine always does python opcode mapping for security reasons. So it's not regular pyc file to deal with.

BTW I'm surprised someone is also interested in that game as I did. Are you interested in talking in private to discuss more details about it? You can reach me: mhwo_feedback@outlook.com

[lguilhermee]

Don't know if you guys already started, but i'm reversing the game, i've never messed with python scripting built into the game only lua.

Related to code, it feels that they changed a bit the opcodes, gonna try to map it and use this project to decrypt it

[lmachens]

@lguilhermee Yes, we already started. Feel free to DM me on Discord
devleon

[lguilhermee]

@lguilhermee Yes, we already started. Feel free to DM me on Discord

devleon

I actually already got it

# Source Generated with Decompyle++
# File: GWLogin.pyc (Python 3.13)

__author__ = 'lgfn1473'
from GameWorld import GameWorld
import client_const
import client_utility
from dcs_core.proxy.VisualProxyMgr import VisualProxyMgr
from dcs_extend.const.visual_const import UICmd
from dcs_extend.const.visual_const import EngineCmd
from unisdk_helper.CommonSDKManager import CommonSDKManager

class GWLogin(GameWorld):
    GW_TYPE = client_const.GAME_WORLD_LOGIN

    def _init(self, scene_data_dict, last_world_info = (None, None)):
        if not client_const.DESTROY_BEFORE_SCENE:
            if not last_world_info:
                self.scene_file = client_const.LOGIN_SCENE_PATH
                VisualProxyMgr.engine_command(EngineCmd.LOAD_SCENE, {
                    'scene_path': self.scene_file,
                    'scene_owner_id': self._scene_owner_id })
            else:
                VisualProxyMgr.engine_command(EngineCmd.UPDATE_SCENE_OWNER, {
                    'scene_owner_id': self._scene_owner_id })
                self.scene_file = last_world_info['scene_file']
        else:
            self.scene_file = client_const.LOGIN_SCENE_PATH
            VisualProxyMgr.engine_command(EngineCmd.LOAD_SCENE, {
                'scene_path': self.scene_file,
                'scene_owner_id': self._scene_owner_id })
        self.set_loaded()


    def _show(self):
        if not client_utility.is_mobile_like_platform():
            LocalInfoHelper = LocalInfoHelper
            import helper.LocalInfoHelper
            if not LocalInfoHelper.get('inited_brightness', False):
                VisualProxyMgr.pre_ui_command(UICmd.UI_CMD, {
                    'func': 'show_ui',
                    'ui_name': 'PanelSetLight' })
                CommonSDKManager().game_related.channel_entergameprocess_log({ }, 'light_option')
            else:
                VisualProxyMgr.pre_ui_command(UICmd.UI_CMD, {
                    'func': 'show_ui',
                    'ui_name': 'PanelLogin' })
                CommonSDKManager().game_related.channel_login_ui_log({ })
        else:
            VisualProxyMgr.pre_ui_command(UICmd.UI_CMD, {
                'func': 'show_ui',
                'ui_name': 'PanelLogin' })
            CommonSDKManager().game_related.channel_login_ui_log({ })
        if not client_const.IS_PUBLISH_VERSION:
            VisualProxyMgr.pre_ui_command(UICmd.UI_CMD, {
                'func': 'show_ui',
                'ui_name': 'PanelDebugBtn' })
            return None

[lmachens]

Yes, we have the same result. but some of them are not complete like GWLoadingBase.py

# Source Generated with Decompyle++
# File: GWLoadingBase.pyc (Python 3.20)

import client_const
from dcs_extend.const.common_const import LoadingState
from game_world.GameWorld import GameWorld
from dcs_core.proxy.VisualProxyMgr import VisualProxyMgr
from dcs_extend.attribute.AttrDefine import AttrLocaltion, AttrVisual, AttrMovementServer
from dcs_extend.const.visual_const import EngineQuery, VisualQuery
from entities.ClientAvatar import ClientAvatar
from dcs_core.proxy import HolderProxy
from dcs_core.proxy.CmdProxyMgr import CmdProxyMgr
from dcs_core.pool.EntityPool import EntityPool
from dcs_extend.const.entity_const import EntityType
from dcs_extend.const.cmd_const import Cmd
from dcs_extend.const.comp_const import CompType
from dcs_core.Env import Env
from entities.ClientEntity import WaitEvtMgr
from game_common.helper.DataMgr import DataMgr
from unisdk_helper.CommonSDKManager import CommonSDKManager
from dcs_engine.montage import MontageSDK
_logger = Env.get_logger('GWLoadingBase')
LOADING_TICK_COUNTER = 100
VISUAL_PREPARE = 'VISUAL_PREPARE'
RELATIVE_VISUAL_PREPARE = 'RELATIVE_VISUAL_PREPARE'
WILD_LEVEL = 'WILD_LEVEL'
FOLIAGE_LOADED = 'FOLIAGE_LOADED'
SHADER_COMPILED = 'SHADER_COMPILED'
ASYNC_LOADED = 'ASYNC_LOADED'

class GWLoadingBase(GameWorld):
    pass
# WARNING: Decompyle incomplete

But our main issue are decryption of the bindict (e.g. client_data\accurate_map_info_data.py)

[wilson0x4d]

Yes, we have the same result. but some of them are not complete [...]

Is it opcode substitutions or something more complex?

[lguilhermee]

Yes, we have the same result. but some of them are not complete [...]

Is it opcode substitutions or something more complex?

The issue is not the opcodes, I've mapped them already, but the lake of support of pycdc for full decompilation of version 3.11. I have implemented a few more opcodes and the decompilation it's about 70% better

[greenozon]

feel free to make some contribution (eg open PRs) and make the progress running! :)

[lguilhermee]

feel free to make some contribution (eg open PRs) and make the progress running! :)

I wish my implementation were better, but it may produce incorrect results because I haven't tested it thoroughly in all cases. For me, decompiling the scripts works, but sometimes, while decompiling, I get warnings indicating that there are still items left on the stack

[wilson0x4d]

@lguilhermee ... a fork on your gh with the changes/hacks would be nice since i'm running through this now as well, and, it seems like such a huge waste of human effort to duplicate what someone else has already done or is in the process of doing. i couldn't care if it looks like a steamy pile of dook if it would let me read through some of the pycs i've dumped that'd be a HUGE win.

[Niocas]

feel free to make some contribution (eg open PRs) and make the progress running! :)

I wish my implementation were better, but it may produce incorrect results because I haven't tested it thoroughly in all cases. For me, decompiling the scripts works, but sometimes, while decompiling, I get warnings indicating that there are still items left on the stack

Hey, currently looking into the same problem, how did you get the changed opcodes with any dll? Do you have the python_3_11_neox.cpp by any chance? Would greatly appreciate it

[DefiantDSX]

The files are encoded with PyCrypto 2.6.1 so when decompiled they decompile encrypted, however the decryption key is not known.

[DefiantDSX]

feel free to make some contribution (eg open PRs) and make the progress running! :)

I wish my implementation were better, but it may produce incorrect results because I haven't tested it thoroughly in all cases. For me, decompiling the scripts works, but sometimes, while decompiling, I get warnings indicating that there are still items left on the stack

Hey, currently looking into the same problem, how did you get the changed opcodes with any dll? Do you have the python_3_11_neox.cpp by any chance? Would greatly appreciate it

File with updated OPCodes for Once Human, you will need to edit the MAGIC sig prior to building from A7 to A8, but some OPCodes are not fully implimented for 3.11 yet so you will run into issues with files not fully decompiling.

python_3_11.zip