Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up builds: bypass asm-processor #1719

Closed
Dragorn421 opened this issue Feb 5, 2024 · 1 comment
Closed

Speed up builds: bypass asm-processor #1719

Dragorn421 opened this issue Feb 5, 2024 · 1 comment

Comments

@Dragorn421
Copy link
Collaborator

I've been looking into how to speed up build times. I found that asm-processor adds non negligible overhead (due to python startup and imports, the asm-processor code itself runs in negligible time)


I wrote a C program "crwasmp" https://github.com/Dragorn421/DragoStuff/tree/crwasmp that performs what we currently use asm-processor for, that is

  • replace floats by the IEEE-754 binary representation ("in hex") in CutsceneData
  • convert encoding from utf-8 to euc-jp
  • run ido on the processed source

Note: oot is not using GLOBAL_ASM, the main feature of asm-processor, since gc-eu-mq-dbg was matched

it saves ~25% of time for building src/
(from built, rm -rf build/gc-eu-mq-dbg/src && time make)
in details, on my machine:
make -j1: 71s -> 54s (-24%)
make -j4: 27s -> 20s (-26%)
make -j8: 22s -> 17s (-23%)

This shows we have a lot of time to gain from not using asm-processor in the build process


An option "option-crwasmp" is to introduce above program "crwasmp" into the repo, replacing $(PYTHON) tools/asm_processor/build.py in the Makefile with it.

Another option "option-justusehex" would be to have ZAPD write CutsceneData floats directly as IEEE-754 (possibly with the float value as a comment like 0x42142142 /* 421.0f */).
Then all that's left to do before calling ido is the utf-8 to euc-jp encoding conversion, which is easily done with the iconv program. As a bonus this allows removing #pragma asmproc recurse.

Either way we would still keep asm-processor in the repo, which could selectively be used on specific files (cherry-pick in the makefile), if we do end up reintroducing GLOBAL_ASMs in the repo.

The two options are outlined in more details below

option-crwasmp has been tested, it's what I have right now.
There's no reason to believe option-justusehex would be slower (should be (marginally) faster, even)

option-crwasmp

option-justusehex

  • modify ZAPD to write hex IEEE-754 instead of floats in CutsceneData (e.g. 0x42142142 or 0x42142142 /* 421.0f */ instead of 421.0f), toggled by a config option
    • patch ZAPD upstream
    • pull ZAPD into zeldaret/oot
    • change config to write IEEE-754
  • remove asm-processor usage, instead use the iconv program (pseudo-command: preproc=`mktemp`; iconv -f utf-8 -t euc-jp -o input $preproc; ido FLAGS -o output $preproc; rm -f $preproc)
  • remove no-longer-needed #pragma asmproc recurse

option-donothing

it is an option but please not this one

@Dragorn421
Copy link
Collaborator Author

Implemented by #1824

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant