add readme

TabbyML · Nov 28, 2023 · 6045826 · 6045826
1 parent c6c91a3
commit 6045826
Showing 1 changed file with 6 additions and 0 deletions.
diff --git a/python/tabby-loadtest/README.md b/python/tabby-loadtest/README.md
@@ -0,0 +1,6 @@
+# Run loadtest with tabby on modal GPUs
+
+Steps:
+1. Adjust `TABBY_API_HOST` in `run.sh` to match your modal deployment url.
+2. Add models you're interested in to benchmark at end of `run.sh`
+3. Run `run.sh`, output will be appended to `record.csv`