correct jitter pointed out by Yelong

xiaoxiawu-microsoft · xiaoxiawu-microsoft · commit 23be767d011b · 2024-05-13T03:18:23.000Z
diff --git a/vllm/model_executor/models/mixtral.py b/vllm/model_executor/models/mixtral.py
@@ -166,7 +166,7 @@ def backward(
         )
 
 
-def sparsemixer(scores, top_k, jitter_eps=0.1):
+def sparsemixer(scores, top_k, jitter_eps=0.01):
     assert top_k == 2
     
     ################ first expert ################

Original file line number	Diff line number	Diff line change
`@@ -166,7 +166,7 @@ def backward(`
`166`	`166`	`)`
`167`	`167`
`168`	`168`
`169`		`-def sparsemixer(scores, top_k, jitter_eps=0.1):`
	`169`	`+def sparsemixer(scores, top_k, jitter_eps=0.01):`
`170`	`170`	`assert top_k == 2`
`171`	`171`
`172`	`172`	`################ first expert ################`