Solver generic_search fail: ConvHipImplicitGemmBwdDataV1R1Xdlops and ConvHipImplicitGemmForwardV4R4Xdlops #427

alexandraBara · 2020-09-11T15:05:46Z

After this issue #411 was solved i pulled in the changes and ran tuning again.
Ran into another issue that seems to be generic_search failing for 2 solvers. I will attach logs bellow

alexandraBara · 2020-09-11T15:06:41Z

ConvHipImplicitGemmBwdDataV1R1Xdlops

INFO - Executing: sudo docker run --device='/dev/kfd' --device='/dev/dri' -w /home/miopenpdb -v /home/miopenpdb:/home/miopenpdb --user=root --group-add video --privileged=true --rm miopentuna bash  -c "export MIOPEN_LOG_LEVEL=7 && export MIOPEN_FIND_ENFORCE=3 && export HIP_VISIBLE_DEVICES=7 && MIOpenDriver conv -V 0 -i 1 --forw 2 --pad_h 2 --out_channels 128 --fil_w 5 --dilation_w 1 --fil_h 5 --in_h 7 --conv_stride_w 1 --group_count 1 --in_channels 32 --in_w 7 --dilation_h 1 --conv_stride_h 1 --pad_w 2 --batchsize 128 --pad_mode default --mode conv --fil_d 1 --in_d 1 --spatial_dim 2 --conv_stride_d 1 --dilation_d 1 --pad_d 0 --trans_output_pad_d 0 2>&1 "
INFO - Setting job id 11099079 state to running
INFO - MIOpenDriver conv -V 0 -i 1 --forw 2 --pad_h 2 --out_channels 128 --fil_w 5 --dilation_w 1 --fil_h 5 --in_h 7 --conv_stride_w 1 --group_count 1 --in_channels 32 --in_w 7 --dilation_h 1 --conv_stride_h 1 --pad_w 2 --batchsize 128 --pad_mode default --mode conv --fil_d 1 --in_d 1 --spatial_dim 2 --conv_stride_d 1 --dilation_d 1 --pad_d 0 --trans_output_pad_d 0
INFO - MIOpen(HIP): Info [Handle] stream: 0x3a98f70, device_id: 0
INFO - MIOpen(HIP): Info [BackwardDataGetWorkSpaceSize]
INFO - MIOpen(HIP): Info2 [HipCompilerVersionImpl] Read version information from HIP package...
INFO - MIOpen(HIP): Info [HipCompilerVersionImpl] 3.6.20263
INFO - MIOpen(HIP): Info [AmdRocmMetadataVersionDetect] ROCm MD version AMDHSA_COv3, MIOpen version 2.7.0.8186-ab68183b
INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] Running: '/opt/rocm/llvm/bin/clang --version'
INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] clang version 11.0.0 (/data/jenkins_workspace/compute-rocm-rel-3.6/external/llvm-project/clang f7b7e21a21d08df6971d2c77315a0e41b7639334)
INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] Target: x86_64-unknown-linux-gnu
INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] Thread model: posix
INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] InstalledDir: /opt/rocm/llvm/bin
INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl]
INFO - MIOpen(HIP): Info2 [SQLiteBase] Initializing system database file /opt/rocm/miopen/share/miopen/db/miopen.db
INFO - MIOpen(HIP): Trace [Exec] 140138148409728:PRAGMA table_info(config);
INFO - MIOpen(HIP): Trace [Exec] 140138148409728:PRAGMA table_info(perf_db);
INFO - MIOpen(HIP): Info2 [SQLiteBase] Initializing user database file /home/miopenpdb/.config/miopen/miopen_1.0.0.udb
INFO - MIOpen(HIP): Trace [Exec] 140138148409728:SELECT name FROM sqlite_master WHERE type = 'table' AND (name = 'config');
INFO - MIOpen(HIP): Trace [Exec] 140138148409728:SELECT name FROM sqlite_master WHERE type = 'table' AND (name = 'perf_db');
INFO - MIOpen(HIP): Trace [SQLitePerfDb] Database created successfully
INFO - MIOpen(HIP): Trace [Exec] 140138148409728:PRAGMA table_info(config);
INFO - MIOpen(HIP): Trace [Exec] 140138148409728:PRAGMA table_info(perf_db);
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinograd3x3U: Not applicable
INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxSf3x2 (not searchable)
INFO - MIOpen(HIP): Info2 [GetSolution]  N=128 C=128 H=7 W=7 K=32 n_groups=120 flags=7 R=5 S=5 pad_H=2 pad_W=2 out_H=7 out_W=7
INFO - MIOpen(HIP): Info2 [GetSolution] ...flags=519 d_N_stride=25088 d_C_stride=196 f_K_stride=100 f_C_stride=3200 o_N_stride=6272 o_K_stride=196
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxSf3x2: Success.
INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxSf2x3 (db access disabled)
INFO - MIOpen(HIP): Info [GetPerformanceConfig] 120
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxSf2x3: Success.
INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxS (not searchable)
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxS: Success.
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<3-3>: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<4-3>: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<5-3>: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<6-3>: Not applicable
INFO - MIOpen(HIP): Info [GetFindModeValueImpl] MIOPEN_FIND_MODE enforced to NORMAL(1) due to MIOPEN_FIND_ENFORCE
INFO - MIOpen(HIP): Info [GetFindModeValueImpl] MIOPEN_FIND_MODE = NORMAL(1)
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm3x3U: Not applicable
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm1x1U: Not applicable
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm1x1UV2: Not applicable
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm5x10u2v2f1: Not applicable
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm7x7c3h224w224k64u2v2p3q3f1: Not applicable
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm5x10u2v2b1: Not applicable
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvOclDirectFwd11x11: Not applicable
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvOclDirectFwdGen: Not applicable
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvOclDirectFwd3x3: Not applicable
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvOclDirectFwd1x1: Not applicable
INFO - MIOpen(HIP): Info2 [GetPerformanceConfig] Returns: 8,8,8,8,1,1,8,2,1
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvHipImplicitGemmForwardV4R4Xdlops: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvHipImplicitGemmV4R4GenXdlopsFwdFp32: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvHipImplicitGemmV4R4GenFwdXdlops: Not applicable
INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvHipImplicitGemmBwdDataV1R1Xdlops (db access disabled)
INFO - MIOpen(HIP): Info [EuristicInit] 32,128,8,32,64,4,0,1
INFO - MIOpen(HIP): Info [GetPerformanceConfigBase] 32,128,8,32,64,4,0,1
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvHipImplicitGemmBwdDataV1R1Xdlops: Success.
INFO - MIOpen(HIP): Info2 [BackwardDataGetWorkSpaceSize] 27697152
INFO - MIOpen(HIP): Info [FindConvBwdDataAlgorithm] requestAlgoCount = 2, workspace = 27697152
INFO - MIOpen(HIP): Info2 [FindRecordUnsafe] Looking for key 128-7-7-5x5-32-7-7-128-2x2-1x1-1x1-0-NCHW-FP32-B in file /home/miopenpdb/.config/miopen/gfx90878.HIP.2_7_0_8186-ab68183b.ufdb.txt
INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 0.290088 ms
INFO - MIOpen(HIP): Info [TryLoad] Find-db regenerating.
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinograd3x3U: Not applicable
INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxSf3x2 (not searchable)
INFO - MIOpen(HIP): Info2 [GetSolution]  N=128 C=128 H=7 W=7 K=32 n_groups=120 flags=7 R=5 S=5 pad_H=2 pad_W=2 out_H=7 out_W=7
INFO - MIOpen(HIP): Info2 [GetSolution] ...flags=519 d_N_stride=25088 d_C_stride=196 f_K_stride=100 f_C_stride=3200 o_N_stride=6272 o_K_stride=196
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxSf3x2: Success.
INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxSf2x3
INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP32,B,2,128,7,7,1,5,5,1,32,128,2,2,0,1,1,0,1,1,0,0,1]
INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP32,B,2,128,7,7,1,5,5,1,32,128,2,2,0,1,1,0,1,1,0,0,1]
INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvOclDirectFwd:8,8,8,8,1,1,2,1,2
INFO - MIOpen(HIP): Info [GetValues] =ConvBinWinogradRxSf2x3:<values not found>
INFO - MIOpen(HIP): Info2 [Measure] Db::Load time: 138.718 ms
INFO - MIOpen(HIP): Info [FindSolutionImpl] Perf Db: record not found for: ConvBinWinogradRxSf2x3
INFO - MIOpen(HIP): Info [FindSolutionImpl] Starting search: ConvBinWinogradRxSf2x3, enforce: SEARCH(3), ALL(1)
INFO - MIOpen(HIP): Info [GetPerformanceConfig] 120
INFO - MIOpen(HIP): Warning [GenericSearch] ConvBinWinogradRxSf2x3: Searching the best solution among 120...
INFO - MIOpen(HIP): Info2 [GenericSearch] #0/0/120 1
INFO - MIOpen(HIP): Info2 [SQLiteBase] Initializing system database file
INFO - MIOpen(HIP): Info [KernDb] database not present
INFO - MIOpen(HIP): Info2 [SQLiteBase] Initializing user database file /home/miopenpdb/.cache/2.7.0.8186-ab68183b/gfx90878.ukdb
INFO - MIOpen(HIP): Trace [Exec] 140138148409728:CREATE TABLE IF NOT EXISTS `kern_db` (`id` INTEGER PRIMARY KEY ASC,`kernel_name` TEXT NOT NULL,`kernel_args` TEXT NOT NULL,`kernel_blob` BLOB NOT NULL,`kernel_hash` TEXT NOT NULL,`uncompressed_size` INT NOT NULL);CREATE UNIQUE INDEX IF NOT EXISTS `idx_kern_db` ON kern_db(kernel_name, kernel_args, kernel_hash, uncompressed_size);
INFO - MIOpen(HIP): Info2 [KernDb] Database created successfully
INFO - MIOpen(HIP): Trace [Exec] 140138148409728:PRAGMA table_info(kern_db);
INFO - MIOpen(HIP): Info2 [LoadBinary] Loading binary for: Conv_Winograd_v21_1_0_gfx9_fp32_stride1.s ;args:  -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [Prepare] SELECT kernel_blob, kernel_hash, uncompressed_size FROM kern_db WHERE (kernel_name = 'Conv_Winograd_v21_1_0_gfx9_fp32_stride1.s.o') AND (kernel_args = ' -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908');
INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 1.48071 ms
INFO - MIOpen(HIP): Info2 [LoadBinary] Sucessfully loaded binary for: Conv_Winograd_v21_1_0_gfx9_fp32_stride1.s ;args:  -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  0/0/120 elapsed_time: 6.81911, best_time: 3.40282e+38, 1
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 6.81911 / 3.40282e+38 = 2.00396e-38
INFO - MIOpen(HIP): Info [GenericSearch] #0/0/120 6.81316 < 3.40282e+38 1
INFO - MIOpen(HIP): Info2 [GenericSearch] #1/0/120 2
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  1/0/120 elapsed_time: 3.39276, best_time: 6.81316, 2
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 3.39276 / 6.81316 = 0.497971
INFO - MIOpen(HIP): Info [GenericSearch] #1/0/120 3.40079 < 6.81316 2
INFO - MIOpen(HIP): Info2 [GenericSearch] #2/0/120 3
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  2/0/120 elapsed_time: 2.34317, best_time: 3.40079, 3
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 2.34317 / 3.40079 = 0.689008
INFO - MIOpen(HIP): Info [GenericSearch] #2/0/120 2.33082 < 3.40079 3
INFO - MIOpen(HIP): Info2 [GenericSearch] #3/0/120 4
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  3/0/120 elapsed_time: 1.70862, best_time: 2.33082, 4
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 1.70862 / 2.33082 = 0.733055
INFO - MIOpen(HIP): Info [GenericSearch] #3/0/120 1.69924 < 2.33082 4
INFO - MIOpen(HIP): Info2 [GenericSearch] #4/0/120 5
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  4/0/120 elapsed_time: 1.38638, best_time: 1.69924, 5
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 1.38638 / 1.69924 = 0.815883
INFO - MIOpen(HIP): Info [GenericSearch] #4/0/120 1.38174 < 1.69924 5
INFO - MIOpen(HIP): Info2 [GenericSearch] #5/0/120 6
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  5/0/120 elapsed_time: 1.17054, best_time: 1.38174, 6
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 1.17054 / 1.38174 = 0.847151
INFO - MIOpen(HIP): Info [GenericSearch] #5/0/120 1.16971 < 1.38174 6
INFO - MIOpen(HIP): Info2 [GenericSearch] #6/0/120 7
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  6/0/120 elapsed_time: 1.06495, best_time: 1.16971, 7
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 1.06495 / 1.16971 = 0.910434
INFO - MIOpen(HIP): Info [GenericSearch] #6/0/120 1.05541 < 1.16971 7
INFO - MIOpen(HIP): Info2 [GenericSearch] #7/0/120 8
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  7/0/120 elapsed_time: 0.84447, best_time: 1.05541, 8
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.84447 / 1.05541 = 0.800134
INFO - MIOpen(HIP): Info [GenericSearch] #7/0/120 0.843574 < 1.05541 8
INFO - MIOpen(HIP): Info2 [GenericSearch] #8/0/120 9
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  8/0/120 elapsed_time: 0.84287, best_time: 0.843574, 9
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.84287 / 0.843574 = 0.999166
INFO - MIOpen(HIP): Info [GenericSearch] #8/0/120 0.84351 < 0.843574 9
INFO - MIOpen(HIP): Info2 [GenericSearch] #9/0/120 10
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  9/0/120 elapsed_time: 0.740151, best_time: 0.84351, 10
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.740151 / 0.84351 = 0.877466
INFO - MIOpen(HIP): Info [GenericSearch] #9/0/120 0.739575 < 0.84351 10
INFO - MIOpen(HIP): Info2 [GenericSearch] #10/0/120 11
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  10/0/120 elapsed_time: 0.635832, best_time: 0.739575, 11
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.635832 / 0.739575 = 0.859726
INFO - MIOpen(HIP): Info [GenericSearch] #10/0/120 0.63516 < 0.739575 11
INFO - MIOpen(HIP): Info2 [GenericSearch] #11/0/120 12
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  11/0/120 elapsed_time: 0.634712, best_time: 0.63516, 12
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.634712 / 0.63516 = 0.999295
INFO - MIOpen(HIP): Info [GenericSearch] #11/0/120 0.634904 < 0.63516 12
INFO - MIOpen(HIP): Info2 [GenericSearch] #12/0/120 13
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  12/0/120 elapsed_time: 0.531194, best_time: 0.634904, 13
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.531194 / 0.634904 = 0.836652
INFO - MIOpen(HIP): Info [GenericSearch] #12/0/120 0.53097 < 0.634904 13
INFO - MIOpen(HIP): Info2 [GenericSearch] #13/0/120 14
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  13/0/120 elapsed_time: 0.531034, best_time: 0.53097, 14
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.531034 / 0.53097 = 1.00012
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.531066 >= 0.53097
INFO - MIOpen(HIP): Info2 [GenericSearch] #14/0/120 15
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  14/0/120 elapsed_time: 0.530714, best_time: 0.53097, 15
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.530714 / 0.53097 = 0.999518
INFO - MIOpen(HIP): Info [GenericSearch] #14/0/120 0.53065 < 0.53097 15
INFO - MIOpen(HIP): Info2 [GenericSearch] #15/0/120 16
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  15/0/120 elapsed_time: 0.425915, best_time: 0.53065, 16
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.425915 / 0.53065 = 0.802629
INFO - MIOpen(HIP): Info [GenericSearch] #15/0/120 0.425851 < 0.53065 16
INFO - MIOpen(HIP): Info2 [GenericSearch] #16/0/120 17
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  16/0/120 elapsed_time: 0.426395, best_time: 0.425851, 17
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.426395 / 0.425851 = 1.00128
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.427291 >= 0.425851
INFO - MIOpen(HIP): Info2 [GenericSearch] #17/0/120 18
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  17/0/120 elapsed_time: 0.426235, best_time: 0.425851, 18
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.426235 / 0.425851 = 1.0009
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.426971 >= 0.425851
INFO - MIOpen(HIP): Info2 [GenericSearch] #18/0/120 19
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  18/0/120 elapsed_time: 0.427355, best_time: 0.425851, 19
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.427355 / 0.425851 = 1.00353
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.427355 >= 0.425851
INFO - MIOpen(HIP): Info2 [GenericSearch] #19/0/120 20
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  19/0/120 elapsed_time: 0.427195, best_time: 0.425851, 20
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.427195 / 0.425851 = 1.00316
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.427355 >= 0.425851
INFO - MIOpen(HIP): Info2 [GenericSearch] #20/0/120 21
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  20/0/120 elapsed_time: 0.429435, best_time: 0.425851, 21
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.429435 / 0.425851 = 1.00842
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.429531 >= 0.425851
INFO - MIOpen(HIP): Info2 [GenericSearch] #21/0/120 22
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  21/0/120 elapsed_time: 0.323516, best_time: 0.425851, 22
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.323516 / 0.425851 = 0.759693
INFO - MIOpen(HIP): Info [GenericSearch] #21/0/120 0.32358 < 0.425851 22
INFO - MIOpen(HIP): Info2 [GenericSearch] #22/0/120 23
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  22/0/120 elapsed_time: 0.322876, best_time: 0.32358, 23
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.322876 / 0.32358 = 0.997824
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.32358 >= 0.32358
INFO - MIOpen(HIP): Info2 [GenericSearch] #23/0/120 24
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  23/0/120 elapsed_time: 0.323996, best_time: 0.32358, 24
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.323996 / 0.32358 = 1.00129
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.324348 >= 0.32358
INFO - MIOpen(HIP): Info2 [GenericSearch] #24/0/120 25
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  24/0/120 elapsed_time: 0.322556, best_time: 0.32358, 25
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.322556 / 0.32358 = 0.996835
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.324348 >= 0.32358
INFO - MIOpen(HIP): Info2 [GenericSearch] #25/0/120 26
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  25/0/120 elapsed_time: 0.323996, best_time: 0.32358, 26
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.323996 / 0.32358 = 1.00129
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.324476 >= 0.32358
INFO - MIOpen(HIP): Info2 [GenericSearch] #26/0/120 27
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  26/0/120 elapsed_time: 0.325116, best_time: 0.32358, 27
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.325116 / 0.32358 = 1.00475
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.324732 >= 0.32358
INFO - MIOpen(HIP): Info2 [GenericSearch] #27/0/120 28
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  27/0/120 elapsed_time: 0.327196, best_time: 0.32358, 28
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.327196 / 0.32358 = 1.01118
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.325244 >= 0.32358
INFO - MIOpen(HIP): Info2 [GenericSearch] #28/0/120 29
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  28/0/120 elapsed_time: 0.326236, best_time: 0.32358, 29
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.326236 / 0.32358 = 1.00821
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.3255 >= 0.32358
INFO - MIOpen(HIP): Info2 [GenericSearch] #29/0/120 30
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  29/0/120 elapsed_time: 0.326236, best_time: 0.32358, 30
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.326236 / 0.32358 = 1.00821
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.32582 >= 0.32358
INFO - MIOpen(HIP): Info2 [GenericSearch] #30/0/120 31
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  30/0/120 elapsed_time: 0.324796, best_time: 0.32358, 31
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.324796 / 0.32358 = 1.00376
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.326492 >= 0.32358
INFO - MIOpen(HIP): Info2 [GenericSearch] #31/0/120 32
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  31/0/120 elapsed_time: 0.220478, best_time: 0.32358, 32
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.220478 / 0.32358 = 0.681371
INFO - MIOpen(HIP): Info [GenericSearch] #31/0/120 0.22067 < 0.32358 32
INFO - MIOpen(HIP): Info2 [GenericSearch] #32/0/120 33
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  32/0/120 elapsed_time: 0.223198, best_time: 0.22067, 33
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223198 / 0.22067 = 1.01146
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.221982 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #33/0/120 34
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  33/0/120 elapsed_time: 0.219998, best_time: 0.22067, 34
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.219998 / 0.22067 = 0.996956
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.221278 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #34/0/120 35
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  34/0/120 elapsed_time: 0.223838, best_time: 0.22067, 35
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223838 / 0.22067 = 1.01436
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.22243 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #35/0/120 36
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  35/0/120 elapsed_time: 0.221278, best_time: 0.22067, 36
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.221278 / 0.22067 = 1.00276
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.22179 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #36/0/120 37
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  36/0/120 elapsed_time: 0.223838, best_time: 0.22067, 37
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223838 / 0.22067 = 1.01436
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.222718 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #37/0/120 38
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  37/0/120 elapsed_time: 0.221598, best_time: 0.22067, 38
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.221598 / 0.22067 = 1.00421
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.222206 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #38/0/120 39
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  38/0/120 elapsed_time: 0.224158, best_time: 0.22067, 39
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.224158 / 0.22067 = 1.01581
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.22291 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #39/0/120 40
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  39/0/120 elapsed_time: 0.220798, best_time: 0.22067, 40
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.220798 / 0.22067 = 1.00058
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.222974 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #40/0/120 41
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  40/0/120 elapsed_time: 0.223518, best_time: 0.22067, 41
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223518 / 0.22067 = 1.01291
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.222782 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #41/0/120 42
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  41/0/120 elapsed_time: 0.223678, best_time: 0.22067, 42
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223678 / 0.22067 = 1.01363
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.222846 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #42/0/120 43
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  42/0/120 elapsed_time: 0.223518, best_time: 0.22067, 43
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223518 / 0.22067 = 1.01291
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.223102 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #43/0/120 44
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  43/0/120 elapsed_time: 0.220958, best_time: 0.22067, 44
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.220958 / 0.22067 = 1.00131
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.222942 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #44/0/120 45
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  44/0/120 elapsed_time: 0.221918, best_time: 0.22067, 45
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.221918 / 0.22067 = 1.00566
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.22323 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #45/0/120 46
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  45/0/120 elapsed_time: 0.223358, best_time: 0.22067, 46
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223358 / 0.22067 = 1.01218
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.223166 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #46/0/120 47
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  46/0/120 elapsed_time: 0.223998, best_time: 0.22067, 47
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223998 / 0.22067 = 1.01508
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.22339 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #47/0/120 48
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  47/0/120 elapsed_time: 0.224158, best_time: 0.22067, 48
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.224158 / 0.22067 = 1.01581
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.223518 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #48/0/120 49
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  48/0/120 elapsed_time: 0.224478, best_time: 0.22067, 49
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.224478 / 0.22067 = 1.01726
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.223934 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #49/0/120 50
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  49/0/120 elapsed_time: 0.224798, best_time: 0.22067, 50
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.224798 / 0.22067 = 1.01871
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.223838 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #50/0/120 51
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  50/0/120 elapsed_time: 0.223678, best_time: 0.22067, 51
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223678 / 0.22067 = 1.01363
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.223582 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #51/0/120 52
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  51/0/120 elapsed_time: 0.222078, best_time: 0.22067, 52
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.222078 / 0.22067 = 1.00638
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.223966 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #52/0/120 53
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  52/0/120 elapsed_time: 0.223678, best_time: 0.22067, 53
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223678 / 0.22067 = 1.01363
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.223678 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #53/0/120 54
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  53/0/120 elapsed_time: 0.224798, best_time: 0.22067, 54
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.224798 / 0.22067 = 1.01871
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.224414 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #54/0/120 55
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  54/0/120 elapsed_time: 0.224958, best_time: 0.22067, 55
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.224958 / 0.22067 = 1.01943
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.224894 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #55/0/120 56
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  55/0/120 elapsed_time: 0.226558, best_time: 0.22067, 56
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.226558 / 0.22067 = 1.02668
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.225438 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #56/0/120 57
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  56/0/120 elapsed_time: 0.225118, best_time: 0.22067, 57
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.225118 / 0.22067 = 1.02016
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.225342 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #57/0/120 58
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  57/0/120 elapsed_time: 0.225918, best_time: 0.22067, 58
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.225918 / 0.22067 = 1.02378
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.22643 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #58/0/120 59
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  58/0/120 elapsed_time: 0.226238, best_time: 0.22067, 59
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.226238 / 0.22067 = 1.02523
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.226814 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #59/0/120 60
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  59/0/120 elapsed_time: 0.226878, best_time: 0.22067, 60
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.226878 / 0.22067 = 1.02813
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.22691 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #60/0/120 61
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  60/0/120 elapsed_time: 0.229438, best_time: 0.22067, 61
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.229438 / 0.22067 = 1.03973
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.228222 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #61/0/120 62
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  61/0/120 elapsed_time: 0.226878, best_time: 0.22067, 62
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.226878 / 0.22067 = 1.02813
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.229534 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #62/0/120 63
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  62/0/120 elapsed_time: 0.230558, best_time: 0.22067, 63
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.230558 / 0.22067 = 1.04481
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.230558 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #63/0/120 64
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  63/0/120 elapsed_time: 0.122239, best_time: 0.22067, 64
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.122239 / 0.22067 = 0.553945
INFO - MIOpen(HIP): Info [GenericSearch] #63/0/120 0.122303 < 0.22067 64
INFO - MIOpen(HIP): Info2 [GenericSearch] #64/0/120 65
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  64/0/120 elapsed_time: 0.124479, best_time: 0.122303, 65
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.124479 / 0.122303 = 1.01779
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.125151 >= 0.122303
INFO - MIOpen(HIP): Info2 [GenericSearch] #65/0/120 66
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  65/0/120 elapsed_time: 0.122239, best_time: 0.122303, 66
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.122239 / 0.122303 = 0.999477
INFO - MIOpen(HIP): Info [GenericSearch] #65/0/120 0.121887 < 0.122303 66
INFO - MIOpen(HIP): Info2 [GenericSearch] #66/0/120 67
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  66/0/120 elapsed_time: 0.123199, best_time: 0.121887, 67
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.123199 / 0.121887 = 1.01076
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.122623 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #67/0/120 68
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  67/0/120 elapsed_time: 0.122719, best_time: 0.121887, 68
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.122719 / 0.121887 = 1.00683
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.122463 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #68/0/120 69
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  68/0/120 elapsed_time: 0.123039, best_time: 0.121887, 69
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.123039 / 0.121887 = 1.00945
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.123039 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #69/0/120 70
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  69/0/120 elapsed_time: 0.123039, best_time: 0.121887, 70
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.123039 / 0.121887 = 1.00945
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.123263 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #70/0/120 71
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  70/0/120 elapsed_time: 0.123999, best_time: 0.121887, 71
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.123999 / 0.121887 = 1.01733
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.124319 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #71/0/120 72
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  71/0/120 elapsed_time: 0.125599, best_time: 0.121887, 72
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.125599 / 0.121887 = 1.03045
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.124415 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #72/0/120 73
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  72/0/120 elapsed_time: 0.126239, best_time: 0.121887, 73
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.126239 / 0.121887 = 1.03571
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.125919 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #73/0/120 74
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  73/0/120 elapsed_time: 0.124639, best_time: 0.121887, 74
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.124639 / 0.121887 = 1.02258
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.125535 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #74/0/120 75
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  74/0/120 elapsed_time: 0.125919, best_time: 0.121887, 75
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.125919 / 0.121887 = 1.03308
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.126495 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #75/0/120 76
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  75/0/120 elapsed_time: 0.125919, best_time: 0.121887, 76
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.125919 / 0.121887 = 1.03308
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.125823 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #76/0/120 77
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  76/0/120 elapsed_time: 0.125919, best_time: 0.121887, 77
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.125919 / 0.121887 = 1.03308
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.126463 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #77/0/120 78
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  77/0/120 elapsed_time: 0.126559, best_time: 0.121887, 78
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.126559 / 0.121887 = 1.03833
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.127487 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #78/0/120 79
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  78/0/120 elapsed_time: 0.127359, best_time: 0.121887, 79
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.127359 / 0.121887 = 1.04489
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.127455 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #79/0/120 80
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  79/0/120 elapsed_time: 0.129759, best_time: 0.121887, 80
INFO - MIOpen(HIP): Info2 [GenericSearch] #80/0/120 81
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  80/0/120 elapsed_time: 0.129439, best_time: 0.121887, 81
INFO - MIOpen(HIP): Info2 [GenericSearch] #81/0/120 82
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  81/0/120 elapsed_time: 0.130399, best_time: 0.121887, 82
INFO - MIOpen(HIP): Info2 [GenericSearch] #82/0/120 83
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  82/0/120 elapsed_time: 0.127839, best_time: 0.121887, 83
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.127839 / 0.121887 = 1.04883
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.129343 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #83/0/120 84
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  83/0/120 elapsed_time: 0.128799, best_time: 0.121887, 84
INFO - MIOpen(HIP): Info2 [GenericSearch] #84/0/120 85
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  84/0/120 elapsed_time: 0.129119, best_time: 0.121887, 85
INFO - MIOpen(HIP): Info2 [GenericSearch] #85/0/120 86
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  85/0/120 elapsed_time: 0.130399, best_time: 0.121887, 86
INFO - MIOpen(HIP): Info2 [GenericSearch] #86/0/120 87
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  86/0/120 elapsed_time: 0.129279, best_time: 0.121887, 87
INFO - MIOpen(HIP): Info2 [GenericSearch] #87/0/120 88
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  87/0/120 elapsed_time: 0.131039, best_time: 0.121887, 88
INFO - MIOpen(HIP): Info2 [GenericSearch] #88/0/120 89
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  88/0/120 elapsed_time: 0.131679, best_time: 0.121887, 89
INFO - MIOpen(HIP): Info2 [GenericSearch] #89/0/120 90
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  89/0/120 elapsed_time: 0.134879, best_time: 0.121887, 90
INFO - MIOpen(HIP): Info2 [GenericSearch] #90/0/120 91
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  90/0/120 elapsed_time: 0.134399, best_time: 0.121887, 91
INFO - MIOpen(HIP): Info2 [GenericSearch] #91/0/120 92
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  91/0/120 elapsed_time: 0.133919, best_time: 0.121887, 92
INFO - MIOpen(HIP): Info2 [GenericSearch] #92/0/120 93
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  92/0/120 elapsed_time: 0.133759, best_time: 0.121887, 93
INFO - MIOpen(HIP): Info2 [GenericSearch] #93/0/120 94
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  93/0/120 elapsed_time: 0.136959, best_time: 0.121887, 94
INFO - MIOpen(HIP): Info2 [GenericSearch] #94/0/120 95
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  94/0/120 elapsed_time: 0.137439, best_time: 0.121887, 95
INFO - MIOpen(HIP): Info2 [GenericSearch] #95/0/120 96
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  95/0/120 elapsed_time: 0.137759, best_time: 0.121887, 96
INFO - MIOpen(HIP): Info2 [GenericSearch] #96/0/120 97
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  96/0/120 elapsed_time: 0.136159, best_time: 0.121887, 97
INFO - MIOpen(HIP): Info2 [GenericSearch] #97/0/120 98
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  97/0/120 elapsed_time: 0.138559, best_time: 0.121887, 98
INFO - MIOpen(HIP): Info2 [GenericSearch] #98/0/120 99
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  98/0/120 elapsed_time: 0.138239, best_time: 0.121887, 99
INFO - MIOpen(HIP): Info2 [GenericSearch] #99/0/120 100
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  99/0/120 elapsed_time: 0.138239, best_time: 0.121887, 100
INFO - MIOpen(HIP): Info2 [GenericSearch] #100/0/120 101
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  100/0/120 elapsed_time: 0.138239, best_time: 0.121887, 101
INFO - MIOpen(HIP): Info2 [GenericSearch] #101/0/120 102
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  101/0/120 elapsed_time: 0.139679, best_time: 0.121887, 102
INFO - MIOpen(HIP): Info2 [GenericSearch] #102/0/120 103
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  102/0/120 elapsed_time: 0.139039, best_time: 0.121887, 103
INFO - MIOpen(HIP): Info2 [GenericSearch] #103/0/120 104
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  103/0/120 elapsed_time: 0.140639, best_time: 0.121887, 104
INFO - MIOpen(HIP): Info2 [GenericSearch] #104/0/120 105
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  104/0/120 elapsed_time: 0.140959, best_time: 0.121887, 105
INFO - MIOpen(HIP): Info2 [GenericSearch] #105/0/120 106
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  105/0/120 elapsed_time: 0.141279, best_time: 0.121887, 106
INFO - MIOpen(HIP): Info2 [GenericSearch] #106/0/120 107
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  106/0/120 elapsed_time: 0.142399, best_time: 0.121887, 107
INFO - MIOpen(HIP): Info2 [GenericSearch] #107/0/120 108
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  107/0/120 elapsed_time: 0.142079, best_time: 0.121887, 108
INFO - MIOpen(HIP): Info2 [GenericSearch] #108/0/120 109
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  108/0/120 elapsed_time: 0.140799, best_time: 0.121887, 109
INFO - MIOpen(HIP): Info2 [GenericSearch] #109/0/120 110
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  109/0/120 elapsed_time: 0.141919, best_time: 0.121887, 110
INFO - MIOpen(HIP): Info2 [GenericSearch] #110/0/120 111
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  110/0/120 elapsed_time: 0.142239, best_time: 0.121887, 111
INFO - MIOpen(HIP): Info2 [GenericSearch] #111/0/120 112
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  111/0/120 elapsed_time: 0.144959, best_time: 0.121887, 112
INFO - MIOpen(HIP): Info2 [GenericSearch] #112/0/120 113
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  112/0/120 elapsed_time: 0.141919, best_time: 0.121887, 113
INFO - MIOpen(HIP): Info2 [GenericSearch] #113/0/120 114
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  113/0/120 elapsed_time: 0.142079, best_time: 0.121887, 114
INFO - MIOpen(HIP): Info2 [GenericSearch] #114/0/120 115
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  114/0/120 elapsed_time: 0.141919, best_time: 0.121887, 115
INFO - MIOpen(HIP): Info2 [GenericSearch] #115/0/120 116
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  115/0/120 elapsed_time: 0.143359, best_time: 0.121887, 116
INFO - MIOpen(HIP): Info2 [GenericSearch] #116/0/120 117
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  116/0/120 elapsed_time: 0.144159, best_time: 0.121887, 117
INFO - MIOpen(HIP): Info2 [GenericSearch] #117/0/120 118
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  117/0/120 elapsed_time: 0.144799, best_time: 0.121887, 118
INFO - MIOpen(HIP): Info2 [GenericSearch] #118/0/120 119
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  118/0/120 elapsed_time: 0.145279, best_time: 0.121887, 119
INFO - MIOpen(HIP): Info2 [GenericSearch] #119/0/120 120
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  119/0/120 elapsed_time: 0.145919, best_time: 0.121887, 120
INFO - MIOpen(HIP): Warning [GenericSearch] Done: 120/0/120, best #65 0.121887 66
INFO - MIOpen(HIP): Warning [GenericSearch] ...Score: 1.19848 (default time 0.146079)
INFO - MIOpen(HIP): Info2 [Prepare] INSERT OR IGNORE INTO config( layout,data_type,direction,spatial_dim,in_channels,in_h,in_w,in_d,fil_h,fil_w,fil_d,out_channels,batchsize,pad_h,pad_w,pad_d,conv_stride_h,conv_stride_w,conv_stride_d,dilation_h,dilation_w,dilation_d,bias,group_count ) VALUES( ?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?);
INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP32,B,2,128,7,7,1,5,5,1,32,128,2,2,0,1,1,0,1,1,0,0,1]
INFO - MIOpen(HIP): Info2 [UpdateUnsafe] 1 rows updated
INFO - MIOpen(HIP): Info2 [Prepare] INSERT OR REPLACE INTO perf_db(config, solver, params, arch, num_cu) VALUES((SELECT id FROM config WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) ) ) , ? , ? , ? , ?);
INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP32,B,2,128,7,7,1,5,5,1,32,128,2,2,0,1,1,0,1,1,0,0,1,ConvBinWinogradRxSf2x3,66,gfx908,120]
INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvBinWinogradRxSf2x3:66
INFO - MIOpen(HIP): Info2 [Measure] Db::Update time: 3.31869 ms
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxSf2x3: Success.
INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxS (not searchable)
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxS: Success.
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<3-3>: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<4-3>: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<5-3>: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<6-3>: Not applicable
INFO - MIOpen(HIP): Info2 [LoadBinary] Loading binary for: Conv_Winograd_v16_5_0_stride1.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [LoadBinary] Loading binary for: conv_3x3_wheel_alpha_v9_0_15.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [Prepare] SELECT kernel_blob, kernel_hash, uncompressed_size FROM kern_db WHERE (kernel_name = 'conv_3x3_wheel_alpha_v9_0_15.s.o') AND (kernel_args = '-Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908');
INFO - MIOpen(HIP): Info2 [Prepare] SELECT kernel_blob, kernel_hash, uncompressed_size FROM kern_db WHERE (kernel_name = 'Conv_Winograd_v16_5_0_stride1.s.o') AND (kernel_args = '-Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908');
INFO - MIOpen(HIP): Info2 [LoadBinary] Loading binary for: Conv_Winograd_v21_1_0_gfx9_fp32_stride1.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [Prepare] SELECT kernel_blob, kernel_hash, uncompressed_size FROM kern_db WHERE (kernel_name = 'Conv_Winograd_v21_1_0_gfx9_fp32_stride1.s.o') AND (kernel_args = '-Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908');
INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 3.09883 ms
INFO - MIOpen(HIP): Info2 [LoadBinary] Sucessfully loaded binary for: conv_3x3_wheel_alpha_v9_0_15.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 3.53101 ms
INFO - MIOpen(HIP): Info2 [LoadBinary] Sucessfully loaded binary for: Conv_Winograd_v21_1_0_gfx9_fp32_stride1.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 3.8889 ms
INFO - MIOpen(HIP): Info2 [LoadBinary] Sucessfully loaded binary for: Conv_Winograd_v16_5_0_stride1.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [PrepareInvoker] Preparing kernel: miopenSp3AsmConvRxSf3x2
INFO - MIOpen(HIP): Info [EvaluateInvokers] ConvBinWinogradRxSf3x2: miopenSp3AsmConvRxSf3x2: 0.444955 < 3.40282e+38
INFO - MIOpen(HIP): Info2 [PrepareInvoker] Preparing kernel: miopenSp3AsmConv_v21_1_0_gfx9_fp32_stride1
INFO - MIOpen(HIP): Info2 [GetSolution]  N=128 G=1 C=128 H=7 W=7 K=32 n_groups=66 flags=1543 R=5 S=5 pad_H=2 pad_W=2 out_H=7 out_W=7 d_buf.byte_stride.nk=25088 d_buf.byte_stride.c=196 d_buf.byte_stride.h=28 d_buf.byte_stride.w=4 f_buf.byte_stride.nk=100 f_buf.byte_stride.c=3200 f_buf.byte_stride.h=20 f_buf.byte_stride.w=4 o_buf.byte_stride.nk=6272 o_buf.byte_stride.c=196 o_buf.byte_stride.h=28 o_buf.byte_stride.w=4 d_buf.byte_stride.g=25088 o_buf.byte_stride.g=6272 f_buf.byte_stride.g=409600
INFO - MIOpen(HIP): Info [EvaluateInvokers] ConvBinWinogradRxSf2x3: miopenSp3AsmConv_v21_1_0_gfx9_fp32_stride1: 0.146079 < 0.444955
INFO - MIOpen(HIP): Info2 [PrepareInvoker] Preparing kernel: miopenSp3AsmConvRxSU
INFO - MIOpen(HIP): Info2 [GetSolution]  N=128 C=128 H=7 W=7 K=32 n_groups=120 flags=7 R=5 S=5 pad_H=2 pad_W=2 out_H=7 out_W=7
INFO - MIOpen(HIP): Info [EvaluateInvokers] ConvBinWinogradRxS: miopenSp3AsmConvRxSU: 0.184478 >= 0.146079
INFO - MIOpen(HIP): Info2 [Register] Invoker registered for algorithm 128x7x7x5x5x32x7x7x128xNCHWxFP32x2x2x1x1x1x1x1xB and solver ConvBinWinogradRxSf2x3
INFO - MIOpen(HIP): Info2 [SetAsFound1_0] Solver ConvBinWinogradRxSf2x3 registered as find 1.0 best for miopenConvolutionBwdDataAlgoWinograd in 128x7x7x5x5x32x7x7x128xNCHWxFP32x2x2x1x1x1x1x1xB
INFO - MIOpen(HIP): Info [EvaluateInvokers] Selected: ConvBinWinogradRxSf2x3: miopenSp3AsmConv_v21_1_0_gfx9_fp32_stride1: 0.146079, workspce_sz = 0
INFO - MIOpen(HIP): Info [SetValues] 128-7-7-5x5-32-7-7-128-2x2-1x1-1x1-0-NCHW-FP32-B, content inserted: miopenConvolutionBwdDataAlgoWinograd:ConvBinWinogradRxSf2x3,0.146079,0,miopenConvolutionBwdDataAlgoWinograd,<unused>
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm3x3U: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm1x1U: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm1x1UV2: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm5x10u2v2f1: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm7x7c3h224w224k64u2v2p3q3f1: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm5x10u2v2b1: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwd11x11: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwdGen: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwd3x3: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwd1x1: Not applicable
INFO - MIOpen(HIP): Info2 [GetPerformanceConfig] Returns: 8,8,8,8,1,1,8,2,1
INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvOclDirectFwd
INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP32,B,2,128,7,7,1,5,5,1,32,128,2,2,0,1,1,0,1,1,0,0,1]
INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvBinWinogradRxSf2x3:66
INFO - MIOpen(HIP): Info [GetValues] =ConvOclDirectFwd:<values not found>
INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP32,B,2,128,7,7,1,5,5,1,32,128,2,2,0,1,1,0,1,1,0,0,1]
INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvOclDirectFwd:8,8,8,8,1,1,2,1,2
INFO - MIOpen(HIP): Info [GetValues] =ConvOclDirectFwd:8,8,8,8,1,1,2,1,2
INFO - MIOpen(HIP): Info2 [Measure] Db::Load time: 135.471 ms
INFO - MIOpen(HIP): Info2 [FindSolutionImpl] Perf Db: record loaded: ConvOclDirectFwd
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwd: Success.
INFO - MIOpen(HIP): Info2 [LoadBinary] Loading binary for: MIOpenConvDirUni.cl ;args:  -DMLO_HW_WAVE_SZ=64 -DMLO_DIR_FORWARD=0 -DMLO_FILTER_SIZE0=5 -DMLO_FILTER_SIZE1=5 -DMLO_FILTER_PAD0=2 -DMLO_FILTER_PAD1=2 -DMLO_FILTER_STRIDE0=1 -DMLO_FILTER_STRIDE1=1 -DMLO_N_OUTPUTS=32 -DMLO_N_INPUTS=128 -DMLO_BATCH_SZ=128 -DMLO_OUT_WIDTH=7 -DMLO_OUT_HEIGHT=7 -DMLO_OUT_BATCH_STRIDE=1568 -DMLO_OUT_CHANNEL_STRIDE=49 -DMLO_OUT_STRIDE=7 -DMLO_IN_WIDTH=7 -DMLO_IN_HEIGHT=7 -DMLO_IN_BATCH_STRIDE=6272 -DMLO_IN_CHANNEL_STRIDE=49 -DMLO_IN_STRIDE=7 -DMLO_IN_TILE0=8 -DMLO_IN_TILE1=8 -DMLO_GRP_TILE0=8 -DMLO_GRP_TILE1=8 -DMLO_OUT_TILE0=1 -DMLO_OUT_TILE1=1 -DMLO_N_STACKS=1 -DMLO_N_OUT_TILES=2 -DMLO_N_OUT_TILES_PERSTACK=2 -DMLO_N_IN_TILES_PERSTACK=1 -DMLO_N_READ_PROCS=64 -DMLO_ALU_VTILE0=8 -DMLO_ALU_VTILE1=8 -DMIOPEN_USE_FP16=0 -DMIOPEN_USE_FP32=1 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -DMLO_CONV_BIAS=0 -DMIOPEN_USE_FP16=0 -DMIOPEN_USE_FP32=1 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [Prepare] SELECT kernel_blob, kernel_hash, uncompressed_size FROM kern_db WHERE (kernel_name = 'MIOpenConvDirUni.cl.o') AND (kernel_args = ' -DMLO_HW_WAVE_SZ=64 -DMLO_DIR_FORWARD=0 -DMLO_FILTER_SIZE0=5 -DMLO_FILTER_SIZE1=5 -DMLO_FILTER_PAD0=2 -DMLO_FILTER_PAD1=2 -DMLO_FILTER_STRIDE0=1 -DMLO_FILTER_STRIDE1=1 -DMLO_N_OUTPUTS=32 -DMLO_N_INPUTS=128 -DMLO_BATCH_SZ=128 -DMLO_OUT_WIDTH=7 -DMLO_OUT_HEIGHT=7 -DMLO_OUT_BATCH_STRIDE=1568 -DMLO_OUT_CHANNEL_STRIDE=49 -DMLO_OUT_STRIDE=7 -DMLO_IN_WIDTH=7 -DMLO_IN_HEIGHT=7 -DMLO_IN_BATCH_STRIDE=6272 -DMLO_IN_CHANNEL_STRIDE=49 -DMLO_IN_STRIDE=7 -DMLO_IN_TILE0=8 -DMLO_IN_TILE1=8 -DMLO_GRP_TILE0=8 -DMLO_GRP_TILE1=8 -DMLO_OUT_TILE0=1 -DMLO_OUT_TILE1=1 -DMLO_N_STACKS=1 -DMLO_N_OUT_TILES=2 -DMLO_N_OUT_TILES_PERSTACK=2 -DMLO_N_IN_TILES_PERSTACK=1 -DMLO_N_READ_PROCS=64 -DMLO_ALU_VTILE0=8 -DMLO_ALU_VTILE1=8 -DMIOPEN_USE_FP16=0 -DMIOPEN_USE_FP32=1 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -DMLO_CONV_BIAS=0 -DMIOPEN_USE_FP16=0 -DMIOPEN_USE_FP32=1 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908');
INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 0.077527 ms
INFO - MIOpen(HIP): Info2 [LoadBinary] Unable to load binary for: MIOpenConvDirUni.cl ;args:  -DMLO_HW_WAVE_SZ=64 -DMLO_DIR_FORWARD=0 -DMLO_FILTER_SIZE0=5 -DMLO_FILTER_SIZE1=5 -DMLO_FILTER_PAD0=2 -DMLO_FILTER_PAD1=2 -DMLO_FILTER_STRIDE0=1 -DMLO_FILTER_STRIDE1=1 -DMLO_N_OUTPUTS=32 -DMLO_N_INPUTS=128 -DMLO_BATCH_SZ=128 -DMLO_OUT_WIDTH=7 -DMLO_OUT_HEIGHT=7 -DMLO_OUT_BATCH_STRIDE=1568 -DMLO_OUT_CHANNEL_STRIDE=49 -DMLO_OUT_STRIDE=7 -DMLO_IN_WIDTH=7 -DMLO_IN_HEIGHT=7 -DMLO_IN_BATCH_STRIDE=6272 -DMLO_IN_CHANNEL_STRIDE=49 -DMLO_IN_STRIDE=7 -DMLO_IN_TILE0=8 -DMLO_IN_TILE1=8 -DMLO_GRP_TILE0=8 -DMLO_GRP_TILE1=8 -DMLO_OUT_TILE0=1 -DMLO_OUT_TILE1=1 -DMLO_N_STACKS=1 -DMLO_N_OUT_TILES=2 -DMLO_N_OUT_TILES_PERSTACK=2 -DMLO_N_IN_TILES_PERSTACK=1 -DMLO_N_READ_PROCS=64 -DMLO_ALU_VTILE0=8 -DMLO_ALU_VTILE1=8 -DMIOPEN_USE_FP16=0 -DMIOPEN_USE_FP32=1 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -DMLO_CONV_BIAS=0 -DMIOPEN_USE_FP16=0 -DMIOPEN_USE_FP32=1 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [SaveBinary] Saving binary for: MIOpenConvDirUni.cl ;args:  -DMLO_HW_WAVE_SZ=64 -DMLO_DIR_FORWARD=0 -DMLO_FILTER_SIZE0=5 -DMLO_FILTER_SIZE1=5 -DMLO_FILTER_PAD0=2 -DMLO_FILTER_PAD1=2 -DMLO_FILTER_STRIDE0=1 -DMLO_FILTER_STRIDE1=1 -DMLO_N_OUTPUTS=32 -DMLO_N_INPUTS=128 -DMLO_BATCH_SZ=128 -DMLO_OUT_WIDTH=7 -DMLO_OUT_HEIGHT=7 -DMLO_OUT_BATCH_STRIDE=1568 -DMLO_OUT_CHANNEL_STRIDE=49 -DMLO_OUT_STRIDE=7 -DMLO_IN_WIDTH=7 -DMLO_IN_HEIGHT=7 -DMLO_IN_BATCH_STRIDE=6272 -DMLO_IN_CHANNEL_STRIDE=49 -DMLO_IN_STRIDE=7 -DMLO_IN_TILE0=8 -DMLO_IN_TILE1=8 -DMLO_GRP_TILE0=8 -DMLO_GRP_TILE1=8 -DMLO_OUT_TILE0=1 -DMLO_OUT_TILE1=1 -DMLO_N_STACKS=1 -DMLO_N_OUT_TILES=2 -DMLO_N_OUT_TILES_PERSTACK=2 -DMLO_N_IN_TILES_PERSTACK=1 -DMLO_N_READ_PROCS=64 -DMLO_ALU_VTILE0=8 -DMLO_ALU_VTILE1=8 -DMIOPEN_USE_FP16=0 -DMIOPEN_USE_FP32=1 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -DMLO_CONV_BIAS=0 -DMIOPEN_USE_FP16=0 -DMIOPEN_USE_FP32=1 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [Prepare] INSERT OR REPLACE INTO kern_db(kernel_name, kernel_args, kernel_blob, kernel_hash, uncompressed_size) VALUES(?, ?, ?, ?, ?);
INFO - MIOpen(HIP): Info2 [Measure] Db::StoreRecord time: 6.39631 ms
INFO - MIOpen(HIP): Info2 [PrepareInvoker] Preparing kernel: MIOpenConvUni
INFO - MIOpen(HIP): Info [EvaluateInvokers] ConvOclDirectFwd: MIOpenConvUni: 0.388957 < 3.40282e+38
INFO - MIOpen(HIP): Info2 [Register] Invoker registered for algorithm 128x7x7x5x5x32x7x7x128xNCHWxFP32x2x2x1x1x1x1x1xB and solver ConvOclDirectFwd
INFO - MIOpen(HIP): Info2 [SetAsFound1_0] Solver ConvOclDirectFwd registered as find 1.0 best for miopenConvolutionBwdDataAlgoDirect in 128x7x7x5x5x32x7x7x128xNCHWxFP32x2x2x1x1x1x1x1xB
INFO - MIOpen(HIP): Info [EvaluateInvokers] Selected: ConvOclDirectFwd: MIOpenConvUni: 0.388957, workspce_sz = 0
INFO - MIOpen(HIP): Info [SetValues] 128-7-7-5x5-32-7-7-128-2x2-1x1-1x1-0-NCHW-FP32-B, content inserted: miopenConvolutionBwdDataAlgoDirect:ConvOclDirectFwd,0.388957,0,miopenConvolutionBwdDataAlgoDirect,<unused>
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvHipImplicitGemmForwardV4R4Xdlops: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvHipImplicitGemmV4R4GenXdlopsFwdFp32: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvHipImplicitGemmV4R4GenFwdXdlops: Not applicable
INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvHipImplicitGemmBwdDataV1R1Xdlops
INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP32,B,2,128,7,7,1,5,5,1,32,128,2,2,0,1,1,0,1,1,0,0,1]
INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvBinWinogradRxSf2x3:66
INFO - MIOpen(HIP): Info [GetValues] =ConvHipImplicitGemmBwdDataV1R1Xdlops:<values not found>
INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP32,B,2,128,7,7,1,5,5,1,32,128,2,2,0,1,1,0,1,1,0,0,1]
INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvOclDirectFwd:8,8,8,8,1,1,2,1,2
INFO - MIOpen(HIP): Info [GetValues] =ConvHipImplicitGemmBwdDataV1R1Xdlops:<values not found>
INFO - MIOpen(HIP): Info2 [Measure] Db::Load time: 151.451 ms
INFO - MIOpen(HIP): Info [FindSolutionImpl] Perf Db: record not found for: ConvHipImplicitGemmBwdDataV1R1Xdlops
INFO - MIOpen(HIP): Info [FindSolutionImpl] Starting search: ConvHipImplicitGemmBwdDataV1R1Xdlops, enforce: SEARCH(3), ALL(1)
INFO - MIOpen(HIP): Info [EuristicInit] 32,128,8,32,64,4,0,1
INFO - MIOpen(HIP): Info [GetPerformanceConfigBase] 32,128,8,32,64,4,0,1
INFO - MIOpen(HIP): Warning [GenericSearch] ConvHipImplicitGemmBwdDataV1R1Xdlops: Searching the best solution among 0 (spare)...
INFO - MIOpen(HIP): Warning [GenericSearch] Done: 0/0/0, best #0 3.40282e+38 4,4,1,4,4,1,0,0
**INFO - MIOpen(HIP): Error [FindSolutionImpl] Search failed for: ConvHipImplicitGemmBwdDataV1R1Xdlops: /root/dMIOpen/src/include/miopen/generic_search.hpp:575: Search failed
INFO - MIOpen(HIP): Info [EuristicInit] 32,128,8,32,64,4,0,1**

alexandraBara · 2020-09-11T15:07:25Z

ConvHipImplicitGemmForwardV4R4Xdlops

 INFO - Executing: sudo docker run --device='/dev/kfd' --device='/dev/dri' -w /home/miopenpdb -v /home/miopenpdb:/home/miopenpdb --user=root --group-add video --privileged=true --rm miopentuna bash  -c "export MIOPEN_LOG_LEVEL=7 && export MIOPEN_FIND_ENFORCE=3 && export HIP_VISIBLE_DEVICES=7 && MIOpenDriver convfp16 -V 0 -i 1 --forw 1 --pad_h 2 --out_channels 64 --fil_w 5 --dilation_w 1 --fil_h 5 --in_h 14 --conv_stride_w 1 --group_count 1 --in_channels 24 --in_w 14 --dilation_h 1 --conv_stride_h 1 --pad_w 2 --batchsize 128 --pad_mode default --mode conv --fil_d 1 --in_d 1 --spatial_dim 2 --conv_stride_d 1 --dilation_d 1 --pad_d 0 --trans_output_pad_d 0 2>&1 " 
 INFO - Setting job id 11100890 state to running
 INFO - MIOpenDriver convfp16 -V 0 -i 1 --forw 1 --pad_h 2 --out_channels 64 --fil_w 5 --dilation_w 1 --fil_h 5 --in_h 14 --conv_stride_w 1 --group_count 1 --in_channels 24 --in_w 14 --dilation_h 1 --conv_stride_h 1 --pad_w 2 --batchsize 128 --pad_mode default --mode conv --fil_d 1 --in_d 1 --spatial_dim 2 --conv_stride_d 1 --dilation_d 1 --pad_d 0 --trans_output_pad_d 0
 INFO - MIOpen(HIP): Info [Handle] stream: 0x2f7ff70, device_id: 0
 INFO - MIOpen(HIP): Info [ForwardGetWorkSpaceSize]
 INFO - MIOpen(HIP): Info2 [HipCompilerVersionImpl] Read version information from HIP package...
 INFO - MIOpen(HIP): Info [HipCompilerVersionImpl] 3.6.20263
 INFO - MIOpen(HIP): Info [AmdRocmMetadataVersionDetect] ROCm MD version AMDHSA_COv3, MIOpen version 2.7.0.8186-ab68183b
 INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] Running: '/opt/rocm/llvm/bin/clang --version'
 INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] clang version 11.0.0 (/data/jenkins_workspace/compute-rocm-rel-3.6/external/llvm-project/clang f7b7e21a21d08df6971d2c77315a0e41b7639334)
 INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] Target: x86_64-unknown-linux-gnu
 INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] Thread model: posix
 INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] InstalledDir: /opt/rocm/llvm/bin
 INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl]
 INFO - MIOpen(HIP): Info2 [SQLiteBase] Initializing system database file /opt/rocm/miopen/share/miopen/db/miopen.db
 INFO - MIOpen(HIP): Trace [Exec] 140589208588672:PRAGMA table_info(config);
 INFO - MIOpen(HIP): Trace [Exec] 140589208588672:PRAGMA table_info(perf_db);
 INFO - MIOpen(HIP): Info2 [SQLiteBase] Initializing user database file /home/miopenpdb/.config/miopen/miopen_1.0.0.udb
 INFO - MIOpen(HIP): Trace [Exec] 140589208588672:SELECT name FROM sqlite_master WHERE type = 'table' AND (name = 'config');
 INFO - MIOpen(HIP): Trace [Exec] 140589208588672:SELECT name FROM sqlite_master WHERE type = 'table' AND (name = 'perf_db');
 INFO - MIOpen(HIP): Trace [SQLitePerfDb] Database created successfully
 INFO - MIOpen(HIP): Trace [Exec] 140589208588672:PRAGMA table_info(config);
 INFO - MIOpen(HIP): Trace [Exec] 140589208588672:PRAGMA table_info(perf_db);
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinograd3x3U: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxSf3x2: Not applicable
 INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxSf2x3 (db access disabled)
 INFO - MIOpen(HIP): Info [GetPerformanceConfig] 120
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxSf2x3: Success.
 INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxS (not searchable)
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxS: Success.
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<3-3>: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<4-3>: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<5-3>: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<6-3>: Not applicable
 INFO - MIOpen(HIP): Info [GetFindModeValueImpl] MIOPEN_FIND_MODE enforced to NORMAL(1) due to MIOPEN_FIND_ENFORCE
 INFO - MIOpen(HIP): Info [GetFindModeValueImpl] MIOPEN_FIND_MODE = NORMAL(1)
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm3x3U: Not applicable
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm1x1U: Not applicable
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm1x1UV2: Not applicable
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm5x10u2v2f1: Not applicable
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm7x7c3h224w224k64u2v2p3q3f1: Not applicable
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm5x10u2v2b1: Not applicable
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvOclDirectFwd11x11: Not applicable
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvOclDirectFwdGen: Not applicable
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvOclDirectFwd3x3: Not applicable
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvOclDirectFwd1x1: Not applicable
 INFO - MIOpen(HIP): Info2 [GetPerformanceConfig] Returns: 8,8,16,16,2,2,8,2,1
 INFO - MIOpen(HIP): Info [EuristicInit] 64,256,1,64,64,8,0,1,8
 INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvHipImplicitGemmForwardV4R4Xdlops (db access disabled)
 INFO - MIOpen(HIP): Info [EuristicInit] 64,256,1,64,64,8,0,1,8
 INFO - MIOpen(HIP): Info [GetPerformanceConfig] 64,256,1,64,64,8,0,1,8
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvHipImplicitGemmForwardV4R4Xdlops: Success.
 INFO - MIOpen(HIP): Info2 [ForwardGetWorkSpaceSize] 235200
 INFO - MIOpen(HIP): Info [FindConvFwdAlgorithm] requestAlgoCount = 2, workspace = 235200
 INFO - MIOpen(HIP): Info2 [FindRecordUnsafe] Looking for key 24-14-14-5x5-64-14-14-128-2x2-1x1-1x1-0-NCHW-FP16-F in file /home/miopenpdb/.config/miopen/gfx90878.HIP.2_7_0_8186-ab68183b.ufdb.txt
 INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 0.909458 ms
 INFO - MIOpen(HIP): Info [TryLoad] Find-db regenerating.
 INFO - MIOpen(HIP): Info2 [GetKernels] 0 kernels for key: miopenIm2d2Col "c24i14_14w5_5p2_2s1_1d1_1t0"
 INFO - MIOpen(HIP): Info2 [AddKernel] Key: miopenIm2Col "c24i14_14w5_5p2_2s1_1d1_1t0"
 INFO - MIOpen(HIP): Info2 [AddKernelDumpKernelParams] runcl MIOpenIm2d2Col.cl -k Im2d2Col -dumpilisa -r 10 if#0: if#0: if#0: iv#0 12288,1,1/256,1,1  -DNUM_CH_PER_WG=1 -DNUM_IM_BLKS_X=1 -DNUM_IM_BLKS=2 -DLOCAL_MEM_SIZE=432 -DSTRIDE_GT_1=0 -DTILE_SZ_X=32 -DTILE_SZ_Y=8 -DUSE_IM_OFF_GUARD=1 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1
 INFO - MIOpen(HIP): Info2 [SQLiteBase] Initializing system database file
 INFO - MIOpen(HIP): Info [KernDb] database not present
 INFO - MIOpen(HIP): Info2 [SQLiteBase] Initializing user database file /home/miopenpdb/.cache/2.7.0.8186-ab68183b/gfx90878.ukdb
 INFO - MIOpen(HIP): Trace [Exec] 140589208588672:CREATE TABLE IF NOT EXISTS `kern_db` (`id` INTEGER PRIMARY KEY ASC,`kernel_name` TEXT NOT NULL,`kernel_args` TEXT NOT NULL,`kernel_blob` BLOB NOT NULL,`kernel_hash` TEXT NOT NULL,`uncompressed_size` INT NOT NULL);CREATE UNIQUE INDEX IF NOT EXISTS `idx_kern_db` ON kern_db(kernel_name, kernel_args, kernel_hash, uncompressed_size);
 INFO - MIOpen(HIP): Info2 [KernDb] Database created successfully
 INFO - MIOpen(HIP): Trace [Exec] 140589208588672:PRAGMA table_info(kern_db);
 INFO - MIOpen(HIP): Info2 [LoadBinary] Loading binary for: MIOpenIm2d2Col.cl ;args:  -DNUM_CH_PER_WG=1 -DNUM_IM_BLKS_X=1 -DNUM_IM_BLKS=2 -DLOCAL_MEM_SIZE=432 -DSTRIDE_GT_1=0 -DTILE_SZ_X=32 -DTILE_SZ_Y=8 -DUSE_IM_OFF_GUARD=1 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT kernel_blob, kernel_hash, uncompressed_size FROM kern_db WHERE (kernel_name = 'MIOpenIm2d2Col.cl.o') AND (kernel_args = ' -DNUM_CH_PER_WG=1 -DNUM_IM_BLKS_X=1 -DNUM_IM_BLKS=2 -DLOCAL_MEM_SIZE=432 -DSTRIDE_GT_1=0 -DTILE_SZ_X=32 -DTILE_SZ_Y=8 -DUSE_IM_OFF_GUARD=1 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908');
 INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 0.128953 ms
 INFO - MIOpen(HIP): Info2 [LoadBinary] Unable to load binary for: MIOpenIm2d2Col.cl ;args:  -DNUM_CH_PER_WG=1 -DNUM_IM_BLKS_X=1 -DNUM_IM_BLKS=2 -DLOCAL_MEM_SIZE=432 -DSTRIDE_GT_1=0 -DTILE_SZ_X=32 -DTILE_SZ_Y=8 -DUSE_IM_OFF_GUARD=1 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [SaveBinary] Saving binary for: MIOpenIm2d2Col.cl ;args:  -DNUM_CH_PER_WG=1 -DNUM_IM_BLKS_X=1 -DNUM_IM_BLKS=2 -DLOCAL_MEM_SIZE=432 -DSTRIDE_GT_1=0 -DTILE_SZ_X=32 -DTILE_SZ_Y=8 -DUSE_IM_OFF_GUARD=1 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [Prepare] INSERT OR REPLACE INTO kern_db(kernel_name, kernel_args, kernel_blob, kernel_hash, uncompressed_size) VALUES(?, ?, ?, ?, ?);
 INFO - MIOpen(HIP): Info2 [Measure] Db::StoreRecord time: 7.56533 ms
 INFO - MIOpen(HIP): Info2 [CallGemm] gemm_desc: {isColMajor 0, transA 0, transB 0, m 64, n 196, k 600, lda 600, ldb 196, ldc 196, batch_count 1, strideA 0, strideB 0, strideC 0, alpha 1, beta 0, dataType 0}
 INFO - MIOpen(HIP): Info2 [CallGemm] gemm_desc: {isColMajor 0, transA 0, transB 0, m 64, n 196, k 600, lda 600, ldb 196, ldc 196, batch_count 1, strideA 0, strideB 0, strideC 0, alpha 1, beta 0, dataType 0}
 INFO - MIOpen(HIP): Info2 [dummy_memset] dummy gpu memset
 INFO - MIOpen(HIP): Info [SetValues] 24-14-14-5x5-64-14-14-128-2x2-1x1-1x1-0-NCHW-FP16-F, content inserted: miopenConvolutionFwdAlgoGEMM:gemm,8.8064,235200,rocBlas,<unused>
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinograd3x3U: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxSf3x2: Not applicable
 INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxSf2x3
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
 INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP16,F,2,24,14,14,1,5,5,1,64,128,2,2,0,1,1,0,1,1,0,0,1]
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
 INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP16,F,2,24,14,14,1,5,5,1,64,128,2,2,0,1,1,0,1,1,0,0,1]
 INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvOclDirectFwd:8,16,16,16,2,1,8,4,2
 INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvOclDirectFwdFused:8,8,16,16,2,2,8,2,2
 INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvBinWinogradRxSf2x3:98
 INFO - MIOpen(HIP): Info [GetValues] =ConvBinWinogradRxSf2x3:98
 INFO - MIOpen(HIP): Info2 [Measure] Db::Load time: 138.273 ms
 INFO - MIOpen(HIP): Info2 [FindSolutionImpl] Perf Db: record loaded: ConvBinWinogradRxSf2x3
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxSf2x3: Success.
 INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxS (not searchable)
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxS: Success.
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<3-3>: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<4-3>: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<5-3>: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<6-3>: Not applicable
 INFO - MIOpen(HIP): Info2 [LoadBinary] Loading binary for: Conv_Winograd_v21_1_0_gfx9_fp16_dot2_edc_stride1.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT kernel_blob, kernel_hash, uncompressed_size FROM kern_db WHERE (kernel_name = 'Conv_Winograd_v21_1_0_gfx9_fp16_dot2_edc_stride1.s.o') AND (kernel_args = '-Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908');
 INFO - MIOpen(HIP): Info2 [LoadBinary] Loading binary for: Conv_Winograd_v14_3_3_fp16dot_stride1.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT kernel_blob, kernel_hash, uncompressed_size FROM kern_db WHERE (kernel_name = 'Conv_Winograd_v14_3_3_fp16dot_stride1.s.o') AND (kernel_args = '-Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908');
 INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 3.8644 ms
 INFO - MIOpen(HIP): Info2 [LoadBinary] Sucessfully loaded binary for: Conv_Winograd_v21_1_0_gfx9_fp16_dot2_edc_stride1.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 3.61705 ms
 INFO - MIOpen(HIP): Info2 [LoadBinary] Sucessfully loaded binary for: Conv_Winograd_v14_3_3_fp16dot_stride1.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [PrepareInvoker] Preparing kernel: miopenSp3AsmConv_v21_1_0_gfx9_fp16_dot2_edc_stride1
 INFO - MIOpen(HIP): Info2 [GetSolution]  N=128 G=1 C=24 H=14 W=14 K=64 n_groups=98 flags=1536 R=5 S=5 pad_H=2 pad_W=2 out_H=14 out_W=14 d_buf.byte_stride.nk=9408 d_buf.byte_stride.c=392 d_buf.byte_stride.h=28 d_buf.byte_stride.w=2 f_buf.byte_stride.nk=1200 f_buf.byte_stride.c=50 f_buf.byte_stride.h=10 f_buf.byte_stride.w=2 o_buf.byte_stride.nk=25088 o_buf.byte_stride.c=392 o_buf.byte_stride.h=28 o_buf.byte_stride.w=2 d_buf.byte_stride.g=9408 o_buf.byte_stride.g=25088 f_buf.byte_stride.g=76800
 INFO - MIOpen(HIP): Info [EvaluateInvokers] ConvBinWinogradRxSf2x3: miopenSp3AsmConv_v21_1_0_gfx9_fp16_dot2_edc_stride1: 0.111999 < 3.40282e+38
 INFO - MIOpen(HIP): Info2 [PrepareInvoker] Preparing kernel: miopenSp3AsmConvRxSU
 INFO - MIOpen(HIP): Info2 [GetSolution]  N=128 C=24 H=14 W=14 K=64 n_groups=120 flags=0 R=5 S=5 pad_H=2 pad_W=2 out_H=14 out_W=14
 INFO - MIOpen(HIP): Info [EvaluateInvokers] ConvBinWinogradRxS: miopenSp3AsmConvRxSU: 0.120479 >= 0.111999
 INFO - MIOpen(HIP): Info2 [Register] Invoker registered for algorithm 24x14x14x5x5x64x14x14x128xNCHWxFP16x2x2x1x1x1x1x1xF and solver ConvBinWinogradRxSf2x3
 INFO - MIOpen(HIP): Info2 [SetAsFound1_0] Solver ConvBinWinogradRxSf2x3 registered as find 1.0 best for miopenConvolutionFwdAlgoWinograd in 24x14x14x5x5x64x14x14x128xNCHWxFP16x2x2x1x1x1x1x1xF
 INFO - MIOpen(HIP): Info [EvaluateInvokers] Selected: ConvBinWinogradRxSf2x3: miopenSp3AsmConv_v21_1_0_gfx9_fp16_dot2_edc_stride1: 0.111999, workspce_sz = 0
 INFO - MIOpen(HIP): Info [SetValues] 24-14-14-5x5-64-14-14-128-2x2-1x1-1x1-0-NCHW-FP16-F, content inserted: miopenConvolutionFwdAlgoWinograd:ConvBinWinogradRxSf2x3,0.111999,0,miopenConvolutionFwdAlgoWinograd,<unused>
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm3x3U: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm1x1U: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm1x1UV2: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm5x10u2v2f1: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm7x7c3h224w224k64u2v2p3q3f1: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm5x10u2v2b1: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwd11x11: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwdGen: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwd3x3: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwd1x1: Not applicable
 INFO - MIOpen(HIP): Info2 [GetPerformanceConfig] Returns: 8,8,16,16,2,2,8,2,1
 INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvOclDirectFwd
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
 INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP16,F,2,24,14,14,1,5,5,1,64,128,2,2,0,1,1,0,1,1,0,0,1]
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
 INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP16,F,2,24,14,14,1,5,5,1,64,128,2,2,0,1,1,0,1,1,0,0,1]
 INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvOclDirectFwd:8,16,16,16,2,1,8,4,2
 INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvOclDirectFwdFused:8,8,16,16,2,2,8,2,2
 INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvBinWinogradRxSf2x3:98
 INFO - MIOpen(HIP): Info [GetValues] =ConvOclDirectFwd:8,16,16,16,2,1,8,4,2
 INFO - MIOpen(HIP): Info2 [Measure] Db::Load time: 133.11 ms
 INFO - MIOpen(HIP): Info2 [FindSolutionImpl] Perf Db: record loaded: ConvOclDirectFwd
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwd: Success.
 INFO - MIOpen(HIP): Info2 [LoadBinary] Loading binary for: MIOpenConvDirUni.cl ;args:  -DMLO_HW_WAVE_SZ=64 -DMLO_DIR_FORWARD=1 -DMLO_FILTER_SIZE0=5 -DMLO_FILTER_SIZE1=5 -DMLO_FILTER_PAD0=2 -DMLO_FILTER_PAD1=2 -DMLO_FILTER_STRIDE0=1 -DMLO_FILTER_STRIDE1=1 -DMLO_N_OUTPUTS=64 -DMLO_N_INPUTS=24 -DMLO_BATCH_SZ=128 -DMLO_OUT_WIDTH=14 -DMLO_OUT_HEIGHT=14 -DMLO_OUT_BATCH_STRIDE=12544 -DMLO_OUT_CHANNEL_STRIDE=196 -DMLO_OUT_STRIDE=14 -DMLO_IN_WIDTH=14 -DMLO_IN_HEIGHT=14 -DMLO_IN_BATCH_STRIDE=4704 -DMLO_IN_CHANNEL_STRIDE=196 -DMLO_IN_STRIDE=14 -DMLO_IN_TILE0=16 -DMLO_IN_TILE1=16 -DMLO_GRP_TILE0=16 -DMLO_GRP_TILE1=8 -DMLO_OUT_TILE0=1 -DMLO_OUT_TILE1=2 -DMLO_N_STACKS=1 -DMLO_N_OUT_TILES=8 -DMLO_N_OUT_TILES_PERSTACK=8 -DMLO_N_IN_TILES_PERSTACK=4 -DMLO_N_READ_PROCS=128 -DMLO_ALU_VTILE0=16 -DMLO_ALU_VTILE1=8 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -DMLO_CONV_BIAS=0 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT kernel_blob, kernel_hash, uncompressed_size FROM kern_db WHERE (kernel_name = 'MIOpenConvDirUni.cl.o') AND (kernel_args = ' -DMLO_HW_WAVE_SZ=64 -DMLO_DIR_FORWARD=1 -DMLO_FILTER_SIZE0=5 -DMLO_FILTER_SIZE1=5 -DMLO_FILTER_PAD0=2 -DMLO_FILTER_PAD1=2 -DMLO_FILTER_STRIDE0=1 -DMLO_FILTER_STRIDE1=1 -DMLO_N_OUTPUTS=64 -DMLO_N_INPUTS=24 -DMLO_BATCH_SZ=128 -DMLO_OUT_WIDTH=14 -DMLO_OUT_HEIGHT=14 -DMLO_OUT_BATCH_STRIDE=12544 -DMLO_OUT_CHANNEL_STRIDE=196 -DMLO_OUT_STRIDE=14 -DMLO_IN_WIDTH=14 -DMLO_IN_HEIGHT=14 -DMLO_IN_BATCH_STRIDE=4704 -DMLO_IN_CHANNEL_STRIDE=196 -DMLO_IN_STRIDE=14 -DMLO_IN_TILE0=16 -DMLO_IN_TILE1=16 -DMLO_GRP_TILE0=16 -DMLO_GRP_TILE1=8 -DMLO_OUT_TILE0=1 -DMLO_OUT_TILE1=2 -DMLO_N_STACKS=1 -DMLO_N_OUT_TILES=8 -DMLO_N_OUT_TILES_PERSTACK=8 -DMLO_N_IN_TILES_PERSTACK=4 -DMLO_N_READ_PROCS=128 -DMLO_ALU_VTILE0=16 -DMLO_ALU_VTILE1=8 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -DMLO_CONV_BIAS=0 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908');
 INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 0.085592 ms
 INFO - MIOpen(HIP): Info2 [LoadBinary] Unable to load binary for: MIOpenConvDirUni.cl ;args:  -DMLO_HW_WAVE_SZ=64 -DMLO_DIR_FORWARD=1 -DMLO_FILTER_SIZE0=5 -DMLO_FILTER_SIZE1=5 -DMLO_FILTER_PAD0=2 -DMLO_FILTER_PAD1=2 -DMLO_FILTER_STRIDE0=1 -DMLO_FILTER_STRIDE1=1 -DMLO_N_OUTPUTS=64 -DMLO_N_INPUTS=24 -DMLO_BATCH_SZ=128 -DMLO_OUT_WIDTH=14 -DMLO_OUT_HEIGHT=14 -DMLO_OUT_BATCH_STRIDE=12544 -DMLO_OUT_CHANNEL_STRIDE=196 -DMLO_OUT_STRIDE=14 -DMLO_IN_WIDTH=14 -DMLO_IN_HEIGHT=14 -DMLO_IN_BATCH_STRIDE=4704 -DMLO_IN_CHANNEL_STRIDE=196 -DMLO_IN_STRIDE=14 -DMLO_IN_TILE0=16 -DMLO_IN_TILE1=16 -DMLO_GRP_TILE0=16 -DMLO_GRP_TILE1=8 -DMLO_OUT_TILE0=1 -DMLO_OUT_TILE1=2 -DMLO_N_STACKS=1 -DMLO_N_OUT_TILES=8 -DMLO_N_OUT_TILES_PERSTACK=8 -DMLO_N_IN_TILES_PERSTACK=4 -DMLO_N_READ_PROCS=128 -DMLO_ALU_VTILE0=16 -DMLO_ALU_VTILE1=8 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -DMLO_CONV_BIAS=0 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [SaveBinary] Saving binary for: MIOpenConvDirUni.cl ;args:  -DMLO_HW_WAVE_SZ=64 -DMLO_DIR_FORWARD=1 -DMLO_FILTER_SIZE0=5 -DMLO_FILTER_SIZE1=5 -DMLO_FILTER_PAD0=2 -DMLO_FILTER_PAD1=2 -DMLO_FILTER_STRIDE0=1 -DMLO_FILTER_STRIDE1=1 -DMLO_N_OUTPUTS=64 -DMLO_N_INPUTS=24 -DMLO_BATCH_SZ=128 -DMLO_OUT_WIDTH=14 -DMLO_OUT_HEIGHT=14 -DMLO_OUT_BATCH_STRIDE=12544 -DMLO_OUT_CHANNEL_STRIDE=196 -DMLO_OUT_STRIDE=14 -DMLO_IN_WIDTH=14 -DMLO_IN_HEIGHT=14 -DMLO_IN_BATCH_STRIDE=4704 -DMLO_IN_CHANNEL_STRIDE=196 -DMLO_IN_STRIDE=14 -DMLO_IN_TILE0=16 -DMLO_IN_TILE1=16 -DMLO_GRP_TILE0=16 -DMLO_GRP_TILE1=8 -DMLO_OUT_TILE0=1 -DMLO_OUT_TILE1=2 -DMLO_N_STACKS=1 -DMLO_N_OUT_TILES=8 -DMLO_N_OUT_TILES_PERSTACK=8 -DMLO_N_IN_TILES_PERSTACK=4 -DMLO_N_READ_PROCS=128 -DMLO_ALU_VTILE0=16 -DMLO_ALU_VTILE1=8 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -DMLO_CONV_BIAS=0 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [Prepare] INSERT OR REPLACE INTO kern_db(kernel_name, kernel_args, kernel_blob, kernel_hash, uncompressed_size) VALUES(?, ?, ?, ?, ?);
 INFO - MIOpen(HIP): Info2 [Measure] Db::StoreRecord time: 9.46558 ms
 INFO - MIOpen(HIP): Info2 [PrepareInvoker] Preparing kernel: MIOpenConvUni
 INFO - MIOpen(HIP): Info [EvaluateInvokers] ConvOclDirectFwd: MIOpenConvUni: 0.215039 < 3.40282e+38
 INFO - MIOpen(HIP): Info2 [Register] Invoker registered for algorithm 24x14x14x5x5x64x14x14x128xNCHWxFP16x2x2x1x1x1x1x1xF and solver ConvOclDirectFwd
 INFO - MIOpen(HIP): Info2 [SetAsFound1_0] Solver ConvOclDirectFwd registered as find 1.0 best for miopenConvolutionFwdAlgoDirect in 24x14x14x5x5x64x14x14x128xNCHWxFP16x2x2x1x1x1x1x1xF
 INFO - MIOpen(HIP): Info [EvaluateInvokers] Selected: ConvOclDirectFwd: MIOpenConvUni: 0.215039, workspce_sz = 0
 INFO - MIOpen(HIP): Info [SetValues] 24-14-14-5x5-64-14-14-128-2x2-1x1-1x1-0-NCHW-FP16-F, content inserted: miopenConvolutionFwdAlgoDirect:ConvOclDirectFwd,0.215039,0,miopenConvolutionFwdAlgoDirect,<unused>
 INFO - MIOpen(HIP): Info [EuristicInit] 64,256,1,64,64,8,0,1,8
 INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvHipImplicitGemmForwardV4R4Xdlops
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
 INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP16,F,2,24,14,14,1,5,5,1,64,128,2,2,0,1,1,0,1,1,0,0,1]
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
 INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP16,F,2,24,14,14,1,5,5,1,64,128,2,2,0,1,1,0,1,1,0,0,1]
 INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvOclDirectFwd:8,16,16,16,2,1,8,4,2
 INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvOclDirectFwdFused:8,8,16,16,2,2,8,2,2
 INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvBinWinogradRxSf2x3:98
 INFO - MIOpen(HIP): Info [GetValues] =ConvHipImplicitGemmForwardV4R4Xdlops:<values not found>
 INFO - MIOpen(HIP): Info2 [Measure] Db::Load time: 133.105 ms
 INFO - MIOpen(HIP): Info [FindSolutionImpl] Perf Db: record not found for: ConvHipImplicitGemmForwardV4R4Xdlops
 INFO - MIOpen(HIP): Info [FindSolutionImpl] Starting search: ConvHipImplicitGemmForwardV4R4Xdlops, enforce: SEARCH(3), ALL(1)
 INFO - MIOpen(HIP): Info [EuristicInit] 64,256,1,64,64,8,0,1,8
 INFO - MIOpen(HIP): Info [GetPerformanceConfig] 64,256,1,64,64,8,0,1,8
 INFO - MIOpen(HIP): Warning [GenericSearch] ConvHipImplicitGemmForwardV4R4Xdlops: Searching the best solution among 0 (spare)...
 **INFO - MIOpen(HIP): Warning [GenericSearch] Done: 0/0/0, best #0 3.40282e+38 4,4,1,4,4,1,0,0,1
 INFO - MIOpen(HIP): Error [FindSolutionImpl] Search failed for: ConvHipImplicitGemmForwardV4R4Xdlops: /root/dMIOpen/src/include/miopen/generic_search.hpp:575: Search failed**
fpadmin@zigzag:/tmp/gfx908/120cu_10.216.64.100_30103p$

atamazov · 2020-09-11T15:37:16Z

Both failures relate to the ComputedContainer of PerformanceConfigs. In both cases the following occurs in the log:

MIOpen(HIP): Warning [GenericSearch] ConvHipImplicitGemmForwardV4R4Xdlops: Searching the best solution among 0 (spare)...

This means that the main ComputedContainer is empty. In other words, the Solver is unable to provide GenericSearch with a set of valid PerformanceConfigs for tuning.

Please note that #387 is not yet resolved.

daniellowell · 2020-10-19T14:43:34Z

@asroy Please update this issue. Who is working it?

asleepzzz · 2020-10-20T05:40:55Z

Hi Alex,
I reproduce this issue
with
./bin/MIOpenDriver conv -F 2 -n 128 -c 32 -H 7 -W 7 -k 128 -y 5 -x 5 -p 2 -q 2 -u 1 -v 1 -l 1 -j 1 -g 1 -t 1
&
./bin/MIOpenDriver convfp16 -F 1 -n 128 -c 24 -H 14 -W 14 -k 64 -y 5 -x 5 -p 2 -q 2 -u 1 -v 1 -l 1 -j 1 -g 1 -t 1

ConvHipImplicitGemmForwardV4R4Xdlops search failed
but ConvHipImplicitGemmBwdDataV1R1Xdlops can search if we don't disable it

I'll fix ConvHipImplicitGemmForwardV4R4Xdlops

asleepzzz · 2020-10-20T07:09:35Z

ConvHipImplicitGemmForwardV4R4Xdlops failed due to
if(GemmKPerBlock * GemmKPack < 16)
return false;
in IsFastToBeUsedForTuning

asleepzzz · 2020-10-21T08:15:46Z

will fix in PR #531

alexandraBara added the bug label Sep 11, 2020

alexandraBara assigned Kirpich30000, atamazov and shurale-nkn Sep 11, 2020

atamazov assigned daniellowell, asroy and jane-zxy and unassigned Kirpich30000 and shurale-nkn Sep 11, 2020

atamazov mentioned this issue Oct 21, 2020

[igemm]fix issue 427(don't drop small kpack*kperblock for all cases) #531

Merged

alexandraBara closed this as completed Nov 16, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Solver generic_search fail: ConvHipImplicitGemmBwdDataV1R1Xdlops and ConvHipImplicitGemmForwardV4R4Xdlops #427

Solver generic_search fail: ConvHipImplicitGemmBwdDataV1R1Xdlops and ConvHipImplicitGemmForwardV4R4Xdlops #427

alexandraBara commented Sep 11, 2020

alexandraBara commented Sep 11, 2020 •

edited by atamazov

Loading

alexandraBara commented Sep 11, 2020 •

edited by atamazov

Loading

atamazov commented Sep 11, 2020 •

edited

Loading

daniellowell commented Oct 19, 2020

asleepzzz commented Oct 20, 2020

asleepzzz commented Oct 20, 2020

asleepzzz commented Oct 21, 2020 •

edited by atamazov

Loading

Solver generic_search fail: ConvHipImplicitGemmBwdDataV1R1Xdlops and ConvHipImplicitGemmForwardV4R4Xdlops #427

Solver generic_search fail: ConvHipImplicitGemmBwdDataV1R1Xdlops and ConvHipImplicitGemmForwardV4R4Xdlops #427

Comments

alexandraBara commented Sep 11, 2020

alexandraBara commented Sep 11, 2020 • edited by atamazov Loading

alexandraBara commented Sep 11, 2020 • edited by atamazov Loading

atamazov commented Sep 11, 2020 • edited Loading

daniellowell commented Oct 19, 2020

asleepzzz commented Oct 20, 2020

asleepzzz commented Oct 20, 2020

asleepzzz commented Oct 21, 2020 • edited by atamazov Loading

alexandraBara commented Sep 11, 2020 •

edited by atamazov

Loading

alexandraBara commented Sep 11, 2020 •

edited by atamazov

Loading

atamazov commented Sep 11, 2020 •

edited

Loading

asleepzzz commented Oct 21, 2020 •

edited by atamazov

Loading