Code

Feat/perf opt#462

Code

Merged

byshiue merged 4 commits intomainNVIDIA/FasterTransformer:mainfrom feat/perf_optNVIDIA/FasterTransformer:feat/perf_optCopy head branch name to clipboardFeb 21, 2023

+111,489-221

Commits

Commits on Feb 21, 2023

• feat: optimize the string usage for many utils

  byshiue

  committed
  3c7d803

• feat: support streaming output for T5 Triton

  byshiue

  committed
  2c7773b

• feat: add deviceId to hashId to support multi-GPU cubin loading

  byshiue

  committed
  5fe46fb

• feat: support fp8 bert on ada

  byshiue

  committed
  e7d839e