You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: examples/megatron-lm/README.md
+9-1Lines changed: 9 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -110,6 +110,8 @@ Coming soon ...
110
110
111
111
### ⭐ Pruning
112
112
113
+
Checkout pruning [getting started section](../pruning/README.md#getting-started) and [guidelines](../pruning/README.md#pruning-guidelines) for configuring pruning parameters in the pruning README.
114
+
113
115
Pruning is supported for GPT and Mamba models in Pipeline Parallel mode. Available pruning options are:
114
116
115
117
-`TARGET_FFN_HIDDEN_SIZE`
@@ -121,14 +123,20 @@ Pruning is supported for GPT and Mamba models in Pipeline Parallel mode. Availab
121
123
-`TARGET_NUM_LAYERS`
122
124
-`LAYERS_TO_DROP` (comma separated, 1-indexed list of layer numbers to directly drop)
123
125
126
+
Example for depth pruning Qwen3-8B from 36 to 24 layers:
0 commit comments