Paper proposes lower-cost pruning for Transformer models during fine-tuning
REP-LIE estimates which Transformer weights to remove using gradients from LoRA low-rank matrices, aiming to reduce the resources required for model pruning and subsequent fine-tuning.