Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

modules/zstd: Rework ZSTD Decoder #1654

Draft
wants to merge 38 commits into
base: main
Choose a base branch
from
Draft

modules/zstd: Rework ZSTD Decoder #1654

wants to merge 38 commits into from

Commits on Nov 22, 2024

  1. modules/zstd/cocotb: Add cocotb testing utilities 

    - XLSStruct for easier handling and serializing/deserializing XLS structs
- XLSChannel that serves as a dummy receiving channel
- XLSMonitor that monitors transactions on an XLS channel
- XLSDriver that can send data on an XLS channel
- LatencyScoreboard that can measure latency between corresponding transactions on input and output buses
- File-backed AXI memory python model

Internal-tag: [#64075]
Signed-off-by: Krzysztof Obłonczek <koblonczek@antmicro.com>
    @koblonczek @lpawelcz
    koblonczek authored and lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    603a5c4 View commit details
    Browse the repository at this point in the history

  2. modules/zstd/memory/MemReader: Add cocotb tests 

    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    cc7a81a View commit details
    Browse the repository at this point in the history

  3. modules/zstd/memory/AxiWriter: Add cocotb test 

    Co-authred-by: Pawel Czarnecki <pczarnecki@antmicro.com>
Co-authred-by: Robert Winkler <rwinkler@antmicro.com>
Signed-off-by: Michal Czyz <mczyz@antmicro.com>
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
    @mczyz-antmicro @lpawelcz
    mczyz-antmicro authored and lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    9845744 View commit details
    Browse the repository at this point in the history

  4. modules/zstd/memory/MemWriter: Add cocotb test 

    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    ece47bd View commit details
    Browse the repository at this point in the history

  5. modules/zstd/memory/README: Describe verilog simulation 

    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    fffbfb3 View commit details
    Browse the repository at this point in the history

  6. CI/modules-zstd: Add calls to verilog simulation targets 

    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    26d3208 View commit details
    Browse the repository at this point in the history

  7. modules/zstd/BUILD: increase pipeline_stages for DecoderMux proc 

    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    f2e9dab View commit details
    Browse the repository at this point in the history

  8. modules/rle/common: Add RLE output struct 

    Signed-off-by: Maciej Torhan <mtorhan@antmicro.com>
    @m-torhan @lpawelcz
    m-torhan authored and lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    72cd947 View commit details
    Browse the repository at this point in the history

  9. modules/zstd: Remove MagicNumberDecoder 

    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    392616b View commit details
    Browse the repository at this point in the history

  10. modules/zstd: Remove BlockDecoder proc 

    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    3bb434f View commit details
    Browse the repository at this point in the history

  11. modules/zstd: Remove DecDemux proc 

    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    3d216d4 View commit details
    Browse the repository at this point in the history

  12. modules/zstd/block_header: Specify new type for the block size 

    Signed-off-by: Maciej Torhan <mtorhan@antmicro.com>
    @m-torhan @lpawelcz
    m-torhan authored and lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    f9ce3a3 View commit details
    Browse the repository at this point in the history

  13. modules/zstd: Cleanup BlockHeader 

    Remove references to buffer structs as those are not used anywhere

Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    8dd1ba4 View commit details
    Browse the repository at this point in the history

  14. modules/zstd/BUILD: Introduce common codegen args 

    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    f1c8346 View commit details
    Browse the repository at this point in the history

  15. modules/zstd: Add AxiCsrAccessor 

    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    d42ccde View commit details
    Browse the repository at this point in the history

  16. modules/zstd: Add CsrConfig 

    Co-authored-by: Pawel Czarnecki <pczarnecki@antmicro.com>
Co-authored-by: Robert Winkler <rwinkler@antmicro.com>
Signed-off-by: Maciej Torhan <mtorhan@antmicro.com>
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
    @m-torhan @lpawelcz @rw1nkler
    3 people committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    fe4cf70 View commit details
    Browse the repository at this point in the history

  17. modules/zstd: Add FrameHeaderDecoder 

    Signed-off-by: Krzysztof Oblonczek <koblonczek@antmicro.com>
    @koblonczek @lpawelcz
    koblonczek authored and lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    96c67b2 View commit details
    Browse the repository at this point in the history

  18. modules/zstd: Add BlockHeaderDecoder 

    Signed-off-by: Maciej Torhan <mtorhan@antmicro.com>
    @m-torhan @lpawelcz
    m-torhan authored and lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    f2220fb View commit details
    Browse the repository at this point in the history

  19. modules/zstd: Add RawBlockDecoder 

    Co-authored-by: Pawel Czarnecki <pczarnecki@antmicro.com>
Signed-off-by: Maciej Torhan <mtorhan@antmicro.com>
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @m-torhan @lpawelcz
    m-torhan and lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    872e528 View commit details
    Browse the repository at this point in the history

  20. modules/zstd: Add RleBlockDecoder 

    Co-authored-by: Pawel Czarnecki <pczarnecki@antmicro.com>
Co-authored-by: Robert Winkler <rwinkler@antmicro.com>
Signed-off-by: Maciej Torhan <mtorhan@antmicro.com>
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
    @m-torhan @lpawelcz @rw1nkler
    3 people committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    3fce60e View commit details
    Browse the repository at this point in the history

  21. modules/zstd: Add ZstdDecoder 

    Co-authored-by: Pawel Czarnecki <pczarnecki@antmicro.com>
Signed-off-by: Robert Winkler <rwinkler@antmicro.com>
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @rw1nkler @lpawelcz
    rw1nkler and lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    87691da View commit details
    Browse the repository at this point in the history

  22. modules/zstd/cocotb: Add ZSTD frame generator library 

    This reverts commit 04ad379225b706ddf492d440c673e77348d7a409.
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    af84d2c View commit details
    Browse the repository at this point in the history

  23. dependency_support: Add zstandard python library 

    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    ffa45ff View commit details
    Browse the repository at this point in the history

  24. modules/zstd: Add verilog simulation of the ZstdDecoder 

    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    ee6eb0d View commit details
    Browse the repository at this point in the history

  25. modules/zstd: Update documentation 

    Internal-tag: [#67096]
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    4e8dc37 View commit details
    Browse the repository at this point in the history

  26. modules/zstd: Add AxiRamReader implementation 

    Signed-off-by: Maciej Torhan <mtorhan@antmicro.com>
Signed-off-by: Krzysztof Obłonczek <koblonczek@antmicro.com>
    @m-torhan @lpawelcz
    m-torhan authored and lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    87fc4e4 View commit details
    Browse the repository at this point in the history

  27. modules/zstd/zstd_dec: Add DSLX tests for ZstdDecoder 

    Signed-off-by: Krzysztof Obłonczek <koblonczek@antmicro.com>
    @koblonczek @lpawelcz
    koblonczek authored and lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    07b2d40 View commit details
    Browse the repository at this point in the history

  28. modules/zstd/zstd_dec: handle contents of the Status CSR 

    Internal-tag: [#66955]
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    a871c8f View commit details
    Browse the repository at this point in the history

  29. modules/zstd/data_generator: fix formatting 

    Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    bbcb4e7 View commit details
    Browse the repository at this point in the history

  30. modules/zstd/memory:axi_stream_remove_empty: Fix byte ordering 

    * Fix byte ordering when receiving a series of non-empty packets
* Adjust MemReader DSLX tests

Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    1d5b4a8 View commit details
    Browse the repository at this point in the history

  31. modules/zstd/memory/axi_stream_remove_empty: Extract remove_empty_byt… 

    …es function into a separate proc

* Extract the operation of removing not-strobed bytes from input frames
  to a separate proc
* Extract control logic to AxiStreamRemoveEmptyInternal proc
* Optimize strobe calculation

Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    7872c78 View commit details
    Browse the repository at this point in the history

  32. modules/zstd/memory/axi_writer: Assign parameterized max lane value 

    Fix paramaterization of the proc

Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    c774bdb View commit details
    Browse the repository at this point in the history

  33. modules/zstd/memory/mem_writer: Add support for not-full input data p… 

    …ackets

Add AxiStreamRemoveEmpty proc to the processing pipeline. It removes non-strobed
bytes from the input AXI Stream frames and forms full frames (ensures that only
the last input data packet won't be full).

Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    a96eda6 View commit details
    Browse the repository at this point in the history

  34. modules/zstd/memory/mem_writer: Add MemWriterInternal proc 

    * Extract control logic to MemWriterInternal proc
* Create alias for the MemWriter response type

Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    a85da0e View commit details
    Browse the repository at this point in the history

  35. modules/zstd/zstd_dec: Write decoded data to the memory 

    SequenceExecutor:
  * Add output channel in the format compliant with MemWriter input data channel type

ZstdDecoder:
  * Add MemWriter proc:
    * Write request formed based on the address of the OutputBuffer CSR and
      FrameContentSize field from the Frame Header
    * Data to write is sent out to the proc by the SequenceExecutor
  * Transition to the FINISH state (and triggers notify channel) only after
    receiving the response from the MemWriter
  * DSLX tests:
    * Receive decoded data sent out on the AXI interface by the MemWriter proc
    * Mock the output memory buffer as a DSLX array
  * Cocotb tests:
    * Move third-party verilog modules (AXI Interconnect) to external directory
    * Replace AXI Interconnect with AXI Crossbar that handles simultaneous AXI Read
      and Write transactions
    * Add reference memory and fill it with expected data for comparison
      against testbench memory at the end of the decoding

Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    58b0a79 View commit details
    Browse the repository at this point in the history

  36. modules/zstd/zstd_dec: Remove stream-based output interface 

    * Remove Repacketizer proc
* Remove stream-based output channels from
  * SequenceExecutor
  * ZstdDecoder

Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    2199a2d View commit details
    Browse the repository at this point in the history

  37. modules/zstd/zstd_dec_cocotb_test: Improve Verilog simulation 

    * Decode multiple ZSTD frames in a single cocotb testbench
* Add one cocotb testbench per type of the ZSTD frames:
  * Frames with RAW blocks only
  * Frames with RLE blocks only
  * Frames with Compressed blocks only (disabled)
  * Frames with mixed blocks (disabled)

Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    864a733 View commit details
    Browse the repository at this point in the history

  38. modules/zstd/README: Update output interface description 

    Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <pczarnecki@antmicro.com>
    @lpawelcz
    lpawelcz committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    998a462 View commit details
    Browse the repository at this point in the history