Add memory performance tests

[mikiw]

It would be nice to have script or memory performance tests to detect memory leaks.

Maybe something similar to crates/starknet-server/tests/test_memory_test.rs in #276

[mikiw]

Related to #272

[hhamud]

@ivpavici can I take this one?

[FabijanC]

@hhamud What is your vision of tackling this? What exactly would you to resolve it?

[hhamud]

@hhamud What is your vision of tackling this? What exactly would you to resolve it?

Currently to add in memory profiling tools and tests and explore the issue further on. I can see another issue here about the problem of memory leaks. Though I would like to be assigned this issue so I can fully dive in and resolve this specific issue

[FabijanC]

Not sure that's what the issue creator had in mind; can you elaborate @mikiw ?

[ivpavici]

@hhamud good luck!
For the beginning - you can profile the memory and find any potential leaks! Please also check devnet with state archive capacity set to full, there for sure we observe some issues:
https://github.com/0xSpaceShard/starknet-devnet-rs?tab=readme-ov-file#querying-old-state-by-specifying-block-hash-or-number

[mikiw]

I had in mind 2 ways of doing that:

1. [Harder - requires test and fix of Memory exhaustion  #272] Add test for 10000 invoke transactions (for example minting) and run it with state-archive-capacity full, and measure memory and tweak some numbers like we expect 10000 to have less than 1 GB of RAM or 1000/100mb or something like this. Later after fixing the memory performance problem, the test should pass (TDD approach).
2. [Easier - just test] Add test for 1000/10000 invoke transactions (for example minting) and test this with 2 modes --state-archive-capacity full and --state-archive-capacity full none. For now, we expect that none test will pass and full will not so for now we can add ignore like this '#[ignore = "Requires fix of performance problem in --state-archive-capacity full mode"]'

I would recommend you @hhamud to go easier way :)