File tree Expand file tree Collapse file tree 2 files changed +18
-3
lines changed Expand file tree Collapse file tree 2 files changed +18
-3
lines changed Original file line number Diff line number Diff line change @@ -16,7 +16,7 @@ export FP4_QUANT_PATH={path_to_save_quantization_results}
1616export HF_FP4_PATH={path_to_save_the_final_FP4_checkpoint}
1717```
1818
19- ### DeepSeek V3, R1 and V3.1
19+ ### DeepSeek V3 R1 V3.1
2020
2121``` bash
2222# download the FP8 checkpoint from Hugginface. This is an example of DeepSeek-R1
@@ -37,7 +37,7 @@ git clone https://github.com/deepseek-ai/DeepSeek-V3.2-Exp.git && cd DeepSeek-V3
3737
3838# Install requirements
3939pip install git+https://github.com/Dao-AILab/fast-hadamard-transform.git
40- pip install -r DeepSeek-V3.2-Exp/ inference/requirements.txt
40+ pip install -r inference/requirements.txt
4141```
4242
4343### Convert the Checkpoint
@@ -51,7 +51,7 @@ python inference/convert.py --hf-ckpt-path $HF_FP8_CKPT --save-path $DS_CKPT --n
5151
5252### Run the calibration scripts
5353
54- DeepSeek V3
54+ DeepSeek V3, R1, V3.1
5555
5656``` bash
5757torchrun --nproc-per-node 8 --master_port=12346 ptq.py --model_path $DS_CKPT --config DeepSeek-V3/inference/configs/config_671B.json --quant_cfg NVFP4_DEFAULT_CFG --output_path $FP4_QUANT_PATH
Original file line number Diff line number Diff line change 2020# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
2121# SOFTWARE.
2222
23+ # SPDX-FileCopyrightText: Copyright (c) 2023-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
24+ # SPDX-License-Identifier: Apache-2.0
25+ #
26+ # Licensed under the Apache License, Version 2.0 (the "License");
27+ # you may not use this file except in compliance with the License.
28+ # You may obtain a copy of the License at
29+ #
30+ # http://www.apache.org/licenses/LICENSE-2.0
31+ #
32+ # Unless required by applicable law or agreed to in writing, software
33+ # distributed under the License is distributed on an "AS IS" BASIS,
34+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
35+ # See the License for the specific language governing permissions and
36+ # limitations under the License.
37+
2338import torch
2439import triton
2540import triton .language as tl
You can’t perform that action at this time.
0 commit comments