Incremental DRT crash after antenna fixing

[gatecat]

Describe the bug

As per #5507 (comment), I am looking at debugging post-DRT antenna violation fixing and incremental DRT. I can reproduce the problems I see on our proprietary PDK with sky130 and the default flow.tcl; modified to remove post-GRT antenna fixing so there are more antenna violations after DRT (but a similar number to what we see on our proprietary PDK).

The modified flow.tcl does work on aes_sky130hd.tcl, albeit that post-DRT antenna fixing takes a disproportionately long time, mostly spent in grt::FastRouteCore::mazeRouteMSMD (a side issue, but one we also see on the actual target design).

On ibex_sky130hd.tcl, however, the bigger issue is that incremental DRT eventually crashes:

    Completing 100% with 203 violations.
    elapsed time = 00:00:30, memory = 3869.86 (MB).
Error: _05824_ 1 pin not visited #guides = 16
Error: _08891_ 1 pin not visited #guides = 3
Error: _08181_ 1 pin not visited #guides = 5
Error: _08190_ 1 pin not visited #guides = 10
Error: _09589_ 1 pin not visited #guides = 5Error: _13267_ 
1 pin not visited #guides = 9
Error: _01459_ 1 pin not visited #guides = 3
Error: _01529_ 1 pin not visited #guides = 3
Error: gen_regfile_ff.register_file_i.rf_reg\[972\] 1 pin not visited #guides = 6
Error: net515 1 pin not visited #guides = 6
Error: net510 1 pin not visited #guides = 2
Error: _10685_ 1 pin not visited #guides = 6
Error: gen_regfile_ff.register_file_i.rf_reg\[425\] 1 pin not visited #guides = 17
Error: _02641_ 1 pin not visited #guides = 29
Error: _08976_ 1 pin not visited #guides = 4
Error: _13427_ 1 pin not visited #guides = 31
Error: _09873_ 1 pin not visited #guides = 3
Error: _13472_ 1 pin not visited #guides = 5
Error: net874 1 pin not visited #guides = 10
Error: net869 1 pin not visited #guides = 11
Error: _08276_ 1 pin not visited #guides = 8
Error: _09913_ 1 pin not visited #guides = 4
Error: _10849_ 1 pin not visited #guides = 5
Error: _10850_ 1 pin not visited #guides = 9
Error: _10857_ 1 pin not visited #guides = 18
Error: _10858_ 1 pin not visited #guides = 10
Error: _11221_ 1 pin not visited #guides = 3
Error: _02083_ 1 pin not visited #guides = 6
Error: _09174_ 1 pin not visited #guides = 5
Error: _08310_ 1 pin not visited #guides = 5
Error: _10918_ 1 pin not visited #guides = 13
Error: _09317_ 1 pin not visited #guides = 2
Error: _09337_ 1 pin not visited #guides = 10
Error: _11083_ 1 pin not visited #guides = 11
Error: _11089_ 1 pin not visited #guides = 6
Error: id_stage_i.controller_i.lsu_addr_last_i\[6\] 1 pin not visited #guides = 8
Error: _04431_ 1 pin not visited #guides = 4
Error: _09965_ 1 pin not visited #guides = 7
Error: net762 1 pin not visited #guides = 14
Error: _11369_ 1 pin not visited #guides = 15
Error: _11115_ 1 pin not visited #guides = 11
Error: _09502_ 1 pin not visited #guides = 5
Error: _04515_ 1 pin not visited #guides = 7
Error: _10037_ 1 pin not visited #guides = 3
Error: data_wdata_o[1] 1 pin not visited #guides = 3
Error: data_wdata_o[22] 1 pin not visited #guides = 25
Error: data_wdata_o[23] 1 pin not visited #guides = 9
Error: _09562_ 1 pin not visited #guides = 6
Error: data_wdata_o[27] 1 pin not visited #guides = 18
Error: _10171_ 1 pin not visited #guides = 13
Error: _11600_ 1 pin not visited #guides = 4
Error: _11618_ 1 pin not visited #guides = 15
Error: net725 1 pin not visited #guides = 14
Error: load_store_unit_i.rdata_q\[15\] 1 pin not visited #guides = 4
Error: load_store_unit_i.rdata_q\[17\] 1 pin not visited #guides = 7
Error: load_store_unit_i.rdata_q\[25\] 1 pin not visited #guides = 4
Error: _02411_ 1 pin not visited #guides = 39
Error: _11142_ 1 pin not visited #guides = 3
Error: _10371_ 1 pin not visited #guides = 4
Error: net781 1 pin not visited #guides = 19
Error: _03297_ 1 pin not visited #guides = 5
Error: _03309_ 1 pin not visited #guides = 6
Error: _12678_ 1 pin not visited #guides = 8
Error: net712 1 pin not visited #guides = 19
Error: _08643_ 1 pin not visited #guides = 11
Error: ex_block_i.genblk3.gen_multdiv_fast.multdiv_i.op_numerator_q\[3\] 1 pin not visited #guides = 8
Error: _11928_ 1 pin not visited #guides = 7
Error: _03780_ 1 pin not visited #guides = 4
Error: _04072_ 1 pin not visited #guides = 14
Error: _04171_ 1 pin not visited #guides = 6
Error: gen_regfile_ff.register_file_i.rf_reg\[137\] 1 pin not visited #guides = 4
Error: _08751_ 1 pin not visited #guides = 2
Error: net557 1 pin not visited #guides = 8
Error: _13002_ 1 pin not visited #guides = 6
Error: _13037_ 1 pin not visited #guides = 6
Error: _13085_ 1 pin not visited #guides = 4
Error: _13094_ 1 pin not visited #guides = 16
Error: _13107_ 1 pin not visited #guides = 4
Error: _13242_ 1 pin not visited #guides = 24
Error: _05639_ 1 pin not visited #guides = 5
Error: _05677_ 1 pin not visited #guides = 3
Error: _06424_ 1 pin not visited #guides = 17
Error: _06451_ 1 pin not visited #guides = 6
Error: checkConnectivity break, net _01459_
Objs not visited:
frPathSeg: begin (310270 529210 ) end ( 310270 531300 ) layerNum 6
	beginStyle: 1
	endStyle: 1
frVia: at ( 310270 529210 )
VIA DEF:
VIA L1M1_PR_MR DEFAULT
  RECT -85 -85 85 85
  RECT -85 -85 85 85
  RECT -145 -115 145 115
frVia: at ( 310270 529210 )
VIA DEF:
VIA M1M2_PR DEFAULT
  RECT -160 -130 160 130
  RECT -75 -75 75 75
  RECT -130 -160 130 160
INSTTERM: (INST/CELL/TERM/NET) _28847_ sky130_fd_sc_hd__dfrtp_1 D _01459_
[1]    71729 segmentation fault (core dumped)  ../build/src/openroad ibex_sky130hd.tcl

Expected Behavior

It's possible to repair antenna violations and re-run DRT as many times as needed without it crashing.

Environment

Git commit: 8a8e29fbbf13ef46e805a326ea91a3705dc82a54
kernel: Linux 6.10.2-arch1-2
os: Arch Linux 
cmake version 3.30.2
CMake Warning at CMakeLists.txt:106 (message):
  OpenROAD git describe failed, using sha1 instead


-- The CXX compiler identification is GNU 14.2.1

To Reproduce

Apply the following patch to test/flow.tcl:

diff --git a/test/flow.tcl b/test/flow.tcl
index 78f985288..9595fd1ad 100644
--- a/test/flow.tcl
+++ b/test/flow.tcl
@@ -206,7 +206,7 @@ write_verilog -remove_cells $filler_cells $verilog_file
 ################################################################
 # Antenna repair
 
-repair_antennas -iterations 5
+# repair_antennas -iterations 5
 
 check_antennas
 utl::metric "GRT::ANT::errors" [ant::antenna_violation_count]
@@ -234,7 +234,7 @@ detailed_route -output_drc [make_result_file "${design}_${platform}_route_drc.rp
                -save_guide_updates \
                -bottom_routing_layer $min_routing_layer \
                -top_routing_layer $max_routing_layer \
-               -verbose 0
+               -verbose 1
 
 write_guides [make_result_file "${design}_${platform}_output_guide.mod"]
 set drv_count [detailed_route_num_drvs]
@@ -249,6 +249,31 @@ write_db $routed_db
 set routed_def [make_result_file ${design}_${platform}_route.def]
 write_def $routed_def
 
+## ****
+
+remove_fillers
+
+global_route -start_incremental -allow_congestion
+repair_antennas -iterations 5
+global_route -end_incremental -allow_congestion
+
+filler_placement $filler_cells
+
+set drt_ant_repair_db [make_result_file ${design}_${platform}_drt_ant_repair.db]
+write_db $drt_ant_repair_db
+
+detailed_route -output_drc [make_result_file "${design}_${platform}_ant_fix_route_drc.rpt"] \
+               -output_maze [make_result_file "${design}_${platform}_ant_fix_maze.log"] \
+               -save_guide_updates \
+               -bottom_routing_layer $min_routing_layer \
+               -top_routing_layer $max_routing_layer \
+               -verbose 1
+
+set repaired_db [make_result_file ${design}_${platform}_repaired_drt.db]
+write_db $repaired_db
+
+## ****
+
 ################################################################
 # Extraction

Run otherwise unmodified test/ibex_sky130hd.tcl

Relevant log output

No response

Screenshots

No response

Additional Context

No response

[eder-matheus]

@gatecat Could you try the latest master branch in your private design? My merged PR fixed the crash in the ibex_sky130hd design.

[gatecat]

Yep, the crash is fixed, thanks! The limiting factor now is repair_antennas hanging with multiple iterations, see #5668