ASAP7 tutorial power-par error #1173
Comments
|
At first glance, your error dump seems weirdly incomplete... can you post a few files first:
|
|
Hi thank you very much for the reply. I am posting the files you requested |
|
Nothing appears to be wrong with the input files. Can you post the |
|
Of course. These are the last two log files that where output before the error (no newer log files were produced) |
|
I just ran the command again (make power-par CONFIG=TinyRocketConfig BINARY=$RISCV/riscv64-unknown-elf/share/riscv-tests/isa/rv32ui-p-simple) after doing a make clean. The error message seems to be clearer "sim.inputs.execution_flags": [ I am attaching all the .log files generated in this fresh run |
|
I see. It looks like Python is choking up trying to write the I think this is a case of old sloppy programming. Can you add |
|
I made the change you suggested but got the same error I am attaching the log files If this is due to a buffer being full is there a way to increase the buffer size (e.g with some kind of ulimit command ?) |
|
Looks like it got a little farther in the output print... but I see, the issue is actually not with the file write ( Based on this: https://stackoverflow.com/questions/65876753/how-to-make-pythons-print-non-blocking it looks like you can bypass output buffering with a Regardless, I think the |
|
This seems to have fixed it. Thank you very much. Unfortunately I am getting an other error. These are the logs This is the vcd file in question |
|
Glad to see that fixed it - I'll make a PR to Hammer. Hm, unfortunately the logs do not capture the Also, you're using Voltus 21.12. Do you know if the |
@odxa20 reported in ucb-bar/chipyard#1173 that an X-to-Y step caused an BlockingIOError. Replacing printing the entire output config to stdout with just a message pointing to where the output config was written solves this + makes Hammer much less verbose.
|
vcs did not produce an error when running. Is there a way to check the validity of the .vpd file ? We unfortunately don't currently have access to older versions of Voltus. Is there a way to check if the read_activity_file command in this newer version distinguishes vpd and vcd formats ? |
|
You can try to read the VPD into a waveform viewer. You have a couple options here:
Yes, you can pull up either the Voltus Text Command Reference or in the Voltus terminal you can type |
|
Hi. I just checked the Voltus Text Command Reference and there is no mention of vpd files. The file types that are supported are VCD | TCF | SAIF | FSDB | PHY | SHM. I am attaching the documentation file below. I am in the process of getting DVE working since our installation did not have it installed by default so that I can check that the vpd trace file is valid. Again I am very grateful for your help and hope we can get this working |
In Voltus, VPD files are supposed to be read as VCD format. It would be odd if it no longer supports VPD... |
|
In the attached file which I exported from the Voltus Text Command Reference there is no mention of vpd (maybe it is implied they are treated the same as vcd ?). Does your version explicilty mention vpd files ? |
|
Right, for many versions now it is implied they are treated the same as VCD. I'm pretty sure it's built on VCD but is a compressed binary format. You could also try dumping in FSDB format instead. However, I have noticed that Voltus' FSDB reader is a few versions behind VCS' FSDB writer, so you'd need to use an older version of VCS. |
|
I just opened the vpd file with wv and it seems fine. No errors and signals are showing. What would be the next step to find the issue ? |
|
I see. Can you try converting it to VCD and reading that into Voltus instead (override the waveforms key) |
|
I just managed to get vpd2vcd working (was missing some 32-bit libraries) and converted the vpd file to vcd. What do you mean by override the waveforms key ? (sorry for my inexperience. this is the first time I am working with industrial cad tools) |
|
The |
|
So I change line 189 of the vlsi/Makefile from |
|
No, edit |
|
Ok got it. Changed that and will run the power-par command. Once it's finished I will post the results |
|
For some reason after running |
|
You can run the |
|
I managed to run the I am attaching the |
|
Ok, so it looks like it was able to run the From the log, it looks like it was able to analyze the power (it says it finished calculating) but I think the warning could reveal something. Your simulation seems excessively long (0.01 seconds, or 2e7 time steps) - did the simulation timeout, or do you have a custom program that you're running? I have a suspicion that Voltus is unable to process the toggles for so many instances over so many timestamps and may have run out of resources, hence the error. |
@odxa20 reported in ucb-bar/chipyard#1173 that an X-to-Y step caused an BlockingIOError. Replacing printing the entire output config to stdout with just a message pointing to where the output config was written solves this + makes Hammer much less verbose.
|
Hi. I created a new chipyard installation (v1.7.0) which contains the latest versions of hammer and hammer-plugins which you specified. I ran the sim-par-debug command and the simulation shows a PASS After doing the hack with the .vpd to .vcd conversion I ran power-par but unfortunately the same error occurs (ram gets filled up and voltus exits after a few seconds). Here is the output I would greatly appreciate your help when you get the opportunity. If you are attending ISCA2022 I wish you the best. I watched the Chipyard tutorial remotely and am looking forward to watching the paper presentations |
|
When looking at the vpd file in waveviewer I can see that its 5915.5ns long which matches with the output of VCS which indicates that the simulation ran for 5916 simulation cycles |
|
I see that your VCS simulation is now not timed out. However, it looks like Voltus still interprets the waveform as having an end time of 0.01s, like the original error. Are you sure you converted the correct VPD to VCD? If so, are there any options in the |
|
Thank you very much for following up on this. I just ran the flow from scratch to make sure I had the correct vpd file. I opened both vcd and vpd files on waveviewer and they both seem to be 5915.5 ns long. I am attaching the files so you can also confirm this |
|
Oh! I think it might be caused by this: https://github.com/ucb-bar/chipyard/blob/main/vlsi/Makefile#L215 getting turned into an |
|
Of course. I will try this now and report back |
|
The run just finished with no errors :) Thank you very much for helping me debug this. Is there a way to edit the Makefile so that it automatically sets -end to the simulation actual time ? |
|
I was also thinking of making a hook according to the documentation to automatically convert the vpd file to vcd |
|
I was going through the log files produced and noticed this After looking in the rail reports folders they are empty. Could this be due to my version of Voltus being newer or is this a known issue ? |
|
We'll take a look at intelligently specifying the This looks like something went wrong in the power grid library generation prior to the power/rail analysis. Can you check if |
|
The directory does exist and has 2 sub-directories for the two different ASAP7 operating points |
|
Ok. Inside each of the 2 sub-directories, is there a |
|
techonly.cl exists so here is the log I am also attaching the entire log file so you can look at it in more detail |
|
Huh. Something might be wrong with generating
|
|
I deleted the generated libraries and rerun power-par with redo-power-par target. I am attaching the log file containing |
|
I'm stumped. The library generation looks correct and successful from the log, and they are specified properly for rail analysis. This was also verified working back with Voltus 19.17. Can you help debug this with the suggested
|
|
The output of |
|
I noticed in the power-par-rundir directory there is a text file named |
|
Hm, very weird. Looking through your voltus.log again, here's what I see:
For the ptiavg file case - can you look in your As for the library merge failure - I literally have no clue what's happening. Do you have access to an older Voltus version? |
|
Indeed this is very weird :) There are ptiavg files in all |
|
Hi. I'm very sorry for the delay. There were some technical issues with installing the older version on our EDA workstation which have now been resolved. I managed to run power analysis successfully on version 20.11 without any errors so there must be some compatibility problem with newer versions. The only problem is that version 20.11 of Innovus (needed to run power analysis with Voltus 20.11) is extremely slow. The place and route step with Innovous 21.12 finished in 30 minutes whereas Innovus 20.11 finishes in a whopping 6 hours and 36 minutes. During the run I noticed in the performance monitor that only one core was used out of 24. The newer version had no issues using all 24. The tcl script that runs in Innovus has the following command |
|
Wow, that's very odd. I have no idea why that is happening. Is there anything in the log to suggest that the Good to know that older version of Voltus worked... but it's frustrating that these tool versions have bugs like this. |
|
Again I’m very sorry for the delay. We migrated our EDA tools to a new machine and wasn’t able to run any more tests until now. There is no indication that set_multi_cpu_usage failed. We have now reverted to using the newer versions of the tools for faster design space exploration and using the older version only when rail analysis is needed. This is a temporary workaround. I hope we can fix rail analysis so that it also works with the newer tool versions. If you have any tests in mind that I should run please let me know |
|
Re: the Innovus times, I suspect this thread is related: https://groups.google.com/g/chipyard/c/-8JUSeCx5qs. Can you check your Innovus runs to see if there's a difference in the enclosure violations between the two versions? As for the Voltus bugs, I don't have any tests in mind - we have in the past just made sure to use different versions. |
|
I have started two runs of the same config and will let you know as soon as they finish |
* Support relative directory meta application inside hierarchical constraints (#626) * Fix ASAP7 SRAMs (#645) * If prependlocal is applied to list prepend to list entries (#619) Co-authored-by: Harrison Liew <harrisonliew@gmail.com> * Assume a string in custom constraints is length 1 list (#618) Co-authored-by: Harrison Liew <harrisonliew@gmail.com> * Formal Verification (LEC at first) (#651) * Cadence plugins now public (#652) * Update to support Python 3.10 - Update typing-related code for newer Python/mypy - Old pyyaml uses dead code File "/buildbot/run_unittests/build/src/tools/pyyaml/lib3/yaml/constructor.py", line 126, in construct_mapping if not isinstance(key, collections.Hashable): AttributeError: module 'collections' has no attribute 'Hashable' * Remove vendored pyyaml * add type-checking to configuration files (#655) * Add specify block in SRAM compiler for vcs sdf back-annotation (#658) * Integrate type checking into tool (#660) * DRC w/ Magic, LVS w/ netgen (#657) * magic DRC works * hier DRC count doesn't work in batch mode, LVS runs but layout -> spice extraction fails with Illegal overlap (types to not connect) error * type checking * move .ext files into separate rundir * extract improvements but still bad netlist * clarify some comments * No BYTEMASK for SRAM1RW256x64/128, Change SITE in LEFs (#653) * Don't print output config to stdout (#661) @odxa20 reported in ucb-bar/chipyard#1173 that an X-to-Y step caused an BlockingIOError. Replacing printing the entire output config to stdout with just a message pointing to where the output config was written solves this + makes Hammer much less verbose. * Yosys + Openroad Plugins (#648) * added a few imports * adding synthesis.yosys.binary key * splitting sky130 tiehilocell into tiehicell and tielocell * first pass on Yosys plugin, based on OpenLANE TCL scripts * cleaned up plugin, manually write sdc file * finished basic RTL to GDS flow for openroad plugin * adding klayout technology file * adding openroad-specific files for sky130 * adding openroad-specific files for sky130 * sky130 plugin modifications for openroad * yosys plugin * hammer changes to support openroad/yosys * implemented open_chip script and starting at a par step, cleaned up many things * openroad pnr fails at write gds step, something wrong with klayout settings * fixes from PR comments, chipyard design runs through synthesis * sky130 plugin updates to support openroad + new PDK updates * adding mypy tests, changed buffer to driver * small changes * openroad pnr flow complete (write GDS works) * cleaned up openroad plugin * removed hard-coded paths and other cleanup * removed hard-coded cell names * clarified unit conversion * removing more hard-coded values * place pins side functionality, hook for set wire rc * fixes for unit tests * fixed most of tests, def2stream.py is failing * seems to be passing all tests * excluding def2stream.py from mypy tests * fixing special cells test * Update defaults_types.yml to fix type error for vlsi.core.node (#663) * Update defaults_types.yml Changed default type of vlsi.core.node to int from str since in all the technology files defaults.yml the value is an int. Without this change the following error arises Expected primary type str for vlsi.core.node, got type int * Update defaults.yml Changed default vlsi.core.node to conform with recently implemented type checks * Fix small error in tech defaults (#664) `vlsi.core.technology` is a str and `vlsi.core.node` is an int * Trying bash test run * Trying bash test run again Co-authored-by: Colin Schmidt <colin.schmidt@sifive.com> Co-authored-by: Harrison Liew <harrisonliew@gmail.com> Co-authored-by: Edward Wang <edward.c.wang@compdigitec.com> Co-authored-by: Bryan Ngo <48371418+bdngo@users.noreply.github.com> Co-authored-by: Waxpple <a0910618112@gmail.com> Co-authored-by: Nayiri <38256927+nayiri-k@users.noreply.github.com> Co-authored-by: Odysseas Chatzopoulos <odxa20@users.noreply.github.com> Co-authored-by: edwardcwang <edwardcwang@users.noreply.github.com>
|
Superceded by discussion in #1185 |
Background Work
Chipyard Version and Hash
Release: 1.6.2
Hash: 481398b
OS Setup
Linux edabox 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 x86_64 x86_64 x86_64 GNU/Linux
LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: RedHatEnterprise
Description: Red Hat Enterprise Linux release 8.6 (Ootpa)
Release: 8.6
Codename: Ootpa
Other Setup
Using genus 21.10, innovus 21.11 and voltus 21.12
Current Behavior
When running the power-par build target (make power-par CONFIG=TinyRocketConfig BINARY=$RISCV/riscv64-unknown-elf/share/riscv-tests/isa/rv32ui-p-simple) I get the following error
"sim.inputs.execution_flags": [
"+verbose",
"+vcdplusfile=/home/odxa20/chipyard/vlsi/output/chipyard.TestHarness.TinyRocketConfig/rv32ui-p-simple.vpd",
"+dramsim",
"+dramsim_ini_dir=/home/odxa20/chipyard/generators/testchipip/src/main/resources/dramsim2_ini",
"+max-cycles=10000000"
],
"sim.inpmake: *** [/home/odxa20/chipyard/vlsi/build/chipyard.TestHarness.TinyRocketConfig-ChipTop/hammer.d:68: /home/odxa20/chipyard/vlsi/build/chipyard.TestHarness.TinyRocketConfig-ChipTop/power-sim-par-input.json] Error 1
Expected Behavior
Should finish normally
Other Information
No response
The text was updated successfully, but these errors were encountered: