[WIP] Vizier integration

[lpawelcz]

Depends on #1137

This PR integrates Vizier Black-Box Optimization tool with XLS workspace.
It allows running parameter optimization tasks on designs written in DSLX.

The optimization task relies on:

• performance metrics which define the objective of the optimization
• set of parameters to optimize in order to fulfill the objective
• evaluation function used for obtaining performance metrics values based on given parameters values

The optimization engine explores the parameter search space in iterative fashion composing suggestions of parameter values which are then used in evaluation function to obtain performance metrics. The evaluation function evaluates the sets of parameter values in the process of implementing and testing DSLX designs.

The implementation consists of 3 components:

• vizier_runner.py - python script utilizing vizier library, which is responsible for:

  • Defining:
    • Parameters for the optimization process
    • Performance Metrics used for search space exploration
    • Evaluation Function used for testing suggested parameter values and obtaining performance metrics by implementing and testing DSLX designs (by calling generic bazel rules instatiated by generate_compression_block_parameter_optimization())
  • Setting up the search space exploration engine
  • Performing the optimization of defined parameters

• vizier_optimize - top-level bazel rule used for starting vizier_runner

• generate_compression_block_parameter_optimization() - helper bazel macro which purpose is to instantiate a set of generic bazel rules necessary for using suggestions for implementing and testing DSLX design in order to get performance metrics. Generic rules instantiated by this macro are used in the evaluation function or serve as the entry point for starting the optimization task (vizier_optimize rule). Parameter values from the suggestions are passed to those rules by environment variables and bazel argument --action_env.

This PR purpose is to showcase possible way of adding support for parameter optimization and handling of the suggestions in the evaluation function. For the purpose of the showcase, the parameters and performance metrics available for the optimization tasks are limited to:

• parameters:
  • pipeline_stages
  • clock_period (added as parameter to sweep but not used in evaluation)
  • target_die_utilization_percentage (added as parameter to sweep but not used in evaluation)
  • placement_density (added as parameter to sweep but not used in evaluation)
• performance metrics:
  • delay (latency)
  • throughput
  • chip_area (added as performance metric but not used in optimization task)

Additionally, PR provides an example which showcases the usage of integration components. The example is based on rle_enc module. It optimizes delay and throughput metrics based on single pipeline_stages parameter. The example can be launched with:

bazel run //xls/modules/rle:rle_enc_compression_block_parameter_optimization

This PR requires a number of additional changes which include:

• passing ctx.configuration.default_shell_env to bazel action in codegen rule in order to facilitate passing environment variables through bazel into scripts or binaries lying under called bazel rules.
• passing ctx.configuration.default_shell_env to bazel action in place_and_route rule in bazel_rules_hdl (temporarily use forked bazel_rules_hdl repository)
• bumping python to version 3.10 because google-vizier modules requires kw_only parameter of dataclass which was introduced in python 3.10
• bumping rules_python to version 0.26.0 to allow correct installation of modules which depend on modules like setuptools

@proppy please take a look
Please note that since this work is based on #1137, it has commits from that PR in commit list and the changes from those are visible in Files changed. The commits relevant to this PR are top 8 commits, starting from [TEMP] use forked bazel_rules_hdl

[proppy]

Some initial feedback

parameters

It would be nice to also have design based parameters, you can parameters DSLX based design by generating a dslx from from a textproto file using

xls/xls/tools/proto_to_dslx_main.cc

Line 61 in 60a8f20

int main(int argc, char* argv[]) {

generate_compression_block_parameter_optimization() - helper bazel macro which purpose is to instantiate a set of generic bazel rules

I think it would easier to decouple the flow from bazel (currently it does bazel -> python -> shell(bazel)), at least initially until we converge on the design discussion :

• for synthesize you could reuse https://github.com/hdl/bazel_rules_hdl/blob/main/synthesis/synth.tcl as it
• for sta you could reuse https://github.com/hdl/bazel_rules_hdl/blob/main/static_timing/sta.tcl
• for place and route you could generate the tcl placeholder with the bazel rules locally and then make it parameterizable

Another alternative is to rely on the python bindings for yosys and openroad and drive the tools directly as library rather than launching them as commands with a tcl script.

@QuantamHD and @grotival are also interested into a finding a more sustainable to do without an hard dependencies on shelling out to bazel, so it'd be good to get their perspective as well.

performance metrics:

• throughput

do you currently extract throughput data from simulation data using realistic payload? if not, it @grebe could provide an example he put together for https://github.com/google/xls/tree/main/xls/examples/protobuf

[lpawelcz]

@proppy parameterization through DSLX files generated with proto_to_dslx_main indeed looks like a good solution for passing DSLX design parameters, thank you for that suggestion, I will look into this.

My biggest concern is to figure out a solution that would not require reimplementing flows that are already defined in bazel rules. This is because I want to prevent a situation where there is a need to maintain 2 separate flows that perform the same task. For example, in synthesis rule we gather the files required for the synthesis, construct args and environment variables and only after that the rule calls the synthesis tool.
This creates a problem if we would like to reuse only the synth.tcl script. In such case we would have to reimplement the whole environment preparation part and make sure it is always the same as the original implementation in bazel rule. That is why I think it is worth to take a closer look at rule synthesis_binary which takes our bazel synthesize_rtl rule as input and generates a shell script which should perform the same steps as synthesise_rtl would. I think we could benefit from this approach in synthesis and place_and_route. We could make vizier_optimize depend on shell script generation rules and this way we could modify the flow to look more like: bazel -> python -> shell(yosys, openroad, etc.).

Your idea with python bindings for yosys and openroad is also very interesting and I will surely investigate more on that but I believe that apart from the bindings for the tools we should also have a way of generating the preparation steps before those tools are called based on the implementation of bazel rules.

do you currently extract throughput data from simulation data using realistic payload?

No, current example simulates RLE encoder with some trivial payload which does not represent realistic data

[proppy]

My biggest concern is to figure out a solution that would not require reimplementing flows that are already defined in bazel rules. This is because I want to prevent a situation where there is a need to maintain 2 separate flows that perform the same task.

While I agree it would be best to have maintain only one flow, I think that's orthogonal to what we are trying to achieve here (the Vizier driven parameter exploration). I think we should tackle that as a separate effort.

In such case we would have to reimplement the whole environment preparation part and make sure it is always the same as the original implementation in bazel rule.

Another way to look at this, is that if we come with a good way to tackle the flow preparation outside of bazel (in C++, Python) we might be able to integrate it as a tool back into the bazel rules and drop a bunch of non-reusable skylark logic.

[proppy]

what do you think of moving this (along side #1137) in separate repo similar to https://github.com/antmicro/xls-cosimulation-demonstrator ?