Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize text output file #2

Open
benjamin051000 opened this issue Mar 29, 2023 · 6 comments
Open

Optimize text output file #2

benjamin051000 opened this issue Mar 29, 2023 · 6 comments
Labels
enhancement New feature or request

Comments

@benjamin051000
Copy link
Member

Currently the output files are on the order of 50-100MB in size. This is hugely space-inefficient. Consider a way to optimize this!

@benjamin051000 benjamin051000 added enhancement New feature or request good first issue Good for newcomers and removed good first issue Good for newcomers labels Mar 29, 2023
@benjamin051000
Copy link
Member Author

I believe most of the records in the file are duplicates, especially since in our tests we are painting the same color (black) to the screen 90% of the time or so.

A solution to drastically reduce filesize and increase performance would be to only record signal events. This would honestly be pretty easy to do.

@benjamin051000
Copy link
Member Author

Lines that have the potential to be reduced:

  • 1,989,740 lines of "ns 1 1 000 000 00" (53,246,087 chars on those lines). (This is ~75% of the entire file!)
  • ~300k lines of vsync/hsync (drop in the bucket, but can be optimized)

@benjamin051000
Copy link
Member Author

The log would work like this: Each entry is when a signal changed.

In the simulator, only update the values when we reach the timestamp.

So what we can do is something like this:

  1. Read two lines to determine the first time interval
  2. Use the signals of the first one until the interval expires
  3. Advance both lines to the next interval/set of signals

This requires us to know the time interval. This may need to be supplied by a CLI flag.

@benjamin051000
Copy link
Member Author

benjamin051000 commented Mar 30, 2023

Our output file is 98.4758% smaller! Let's see how difficult it is to integrate into the script

-rw-r--r--. 1 benjamin benjamin  1088541 Mar 30 13:52 newout.txt
-rw-r--r--. 1 benjamin benjamin 71415293 Mar 29 23:04 vga_output.txt

@benjamin051000
Copy link
Member Author

Ideally we could just have the file output two times to define a range... but I really don't care to learn VHDL text IO functions.

@benjamin051000
Copy link
Member Author

benjamin051000 commented May 24, 2023

This is super close to being finished. Someone should pick this up and finish the algorithm. As displayed above, it will result in way smaller log files, and likely will result in much faster image displaying. This is being worked on on the optimize-log branch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

When branches are created from issues, their pull requests are automatically linked.

1 participant