Llama 3 faces errors of 'tuple' object has no attribute 'grad_fn'

[soonchangAI]

Command:

CUDA_VISIBLE_DEVICES=0 python llama3.py --base_model TinyLlama/TinyLlama_v1.1 \
  --pruning_ratio 0.25 \
  --device cuda --eval_device cuda \
  --block_wise --block_mlp_layer_start 4 --block_mlp_layer_end 18 \
  --block_attention_layer_start 4 --block_attention_layer_end 18 \
  --save_ckpt_log_name tinyllama_prune_log \
  --pruner_type taylor  --taylor param_first \
  --save_model  --max_seq_len 2048 \
  --test_before_train --test_after_train

GPU: Titan

Error:

{'wikitext2': 7.713474619002515, 'ptb': 24.609314266596865}
2025-01-22 22:55:54 - INFO :       PPL before pruning: {'wikitext2': 7.713474619002515, 'ptb': 24.609314266596865}
2025-01-22 22:55:54 - INFO :       Use taylor pruner...
2025-01-22 22:55:54 - INFO :       Pruning Attention Layer = [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]
2025-01-22 22:55:54 - INFO :       Pruning MLP Layer = [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]
Traceback (most recent call last):
  File "/home/cybertron/LLM-Pruner/llama3.py", line 321, in <module>
    main(args)
  File "/home/cybertron/LLM-Pruner/llama3.py", line 120, in main
    pruner = tp.pruner.MetaPruner(
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/cybertron/LLM-Pruner/LLMPruner/torch_pruning/pruner/algorithms/metapruner.py", line 80, in __init__
    self.DG = dependency.DependencyGraph().build_dependency(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cybertron/LLM-Pruner/LLMPruner/torch_pruning/dependency.py", line 374, in build_dependency
    self.module2node = self._trace(
                       ^^^^^^^^^^^^
  File "/home/cybertron/LLM-Pruner/LLMPruner/torch_pruning/dependency.py", line 696, in _trace
    module2node, o.grad_fn, gradfn2module, reused)
                 ^^^^^^^^^
AttributeError: 'tuple' object has no attribute 'grad_fn'

[wangziyannb]

Add model.config.use_cache = False before initialization of tp.pruner.MetaPruner might help.