Cli Entrypoints and execution time recording

[saleha-muzammil]

• ./PROBE record does both execution and transcription
• ./PROBE record --make --no-transcribe , only executes
• ./PROBE transcribe-only /path/to/raw_probe_dir --output probe_log does only transcription

To view execution timings of each, we add time :
time ./PROBE record --make head ../flake.nix

The python function (hyperfine replacement) is in the probe_src/performance_test.py

• Write microbenchmarking
• Run performance test-cases in two steps: one with just libprobe record and one with just transcription. (3 new CLI entrypoints, described in comments in CLI.py)
• Python script that runs ./PROBE record --no-transcribe and ./PROBE transcribe for several tests. probe_src/performance_test.py

[charmoniumQ]

If you resolve the conflict, I can merge this in.

[saleha-muzammil]

Resolved the conflict

[charmoniumQ]

Can we separate this into record --no-transcribe and transcribe and structure the code such that we can try 3 or 5 different commands (a list).

[saleha-muzammil]

Yes, I will do this, though I think we can't add the transcribe command in the list since it requires the path to the raw file.

[saleha-muzammil]

Added more commands in the list, reduced the warmup and benchmark iterations just to get results earlier

[charmoniumQ]

I'm a bit confused by "we can't add transcribe ...". Will something like this work?

for each test_case:
    time "PROBE record --no-transcribe test_case"
    output times to CSV
    time "PROBE transcribe test_case"
    output times to CSV

[saleha-muzammil]

That is because transcribe will require a path to the raw data which it needs to transcribe. The --no-transcribe command stores this raw data, and the transcribe command later processes it. Each transcription refers to the raw data directory that was created during the recording phase.
So, the --no-transcribe command after execution also tells us the path to the raw directory, and we give that path to the transcribe command to process the raw data.

./PROBE record does both execution and transcription
./PROBE record --make --no-transcribe , only executes
./PROBE transcribe-only /path/to/raw_probe_dir --output probe_log does only transcription

So, i meant to say that it might not be possible to hard code a transcribe command in the commands list since we don't know the path to the raw directory.

The code you shared can work but we will have to store the path to the raw directory somewhere after each --no-transcribe command.

for each test_case:
    time "PROBE record --no-transcribe test_case"
    output times to CSV
    time "PROBE transcribe test_case"
    output times to CSV

[charmoniumQ]

The code you shared can work but we will have to store the path to the raw directory somewhere after each --no-transcribe command.

Let's go ahead and do that, if it's simple. That will allow us to have better performance timing because we can separate the overhead of recording from transcribing.

[saleha-muzammil]

Donee

[charmoniumQ]

Please rebase onto main when you get a chance and ensure just pre-commit passes.

[saleha-muzammil]

Hey, this pr has become very messy when i tried to rebase it, I will open a new pr for the cli points and performance_tests

[saleha-muzammil]

The work has been shifted to PR(#33)