ClearCut.
Remove weights, preserve behaviour. ↘Structured pruning with activation-aware importance and low-rank refinement.
A post-training LLM compression toolkit implementing magnitude, Wanda, SparseGPT, RIA and a ClearCut importance criterion. It supports unstructured and N:M weight sparsity, channel reallocation and truncated SVD for attention projections, with perplexity and zero-shot evaluation.
The problem, and the approach.
What needed solving?
Large language models contain substantial weight redundancy, but naïve pruning can damage perplexity and downstream behavior. A pruning decision must consider both weight statistics and their interaction with calibration activations, while remaining comparable with strong baselines.
What was built?
Compute a ClearCut importance signal from local row/column weight relationships and activation statistics, prune to a specified sparsity pattern, optionally redistribute channels and factor attention projections using SVD. Compare perplexity at matched sparsity rather than claiming inference speedups without serving benchmarks.
Architecture, without the noise.
This is a simplified conceptual diagram of the components documented in the repository; it is not a claim that all operations use a single synchronous pipeline.
Load LLaMA / OPT weights with Transformers.
Collect activation statistics on C4 / WikiText-2 / PTB.
Magnitude · Wanda · RIA · SparseGPT · ClearCut.
Select masks for unstructured or N:M sparsity; optional channel reallocation.
Optional SVD factorization of q/k/v/o projections.
Perplexity and zero-shot datasets.
How it works, step by step.
- 01
Select model and calibration data
The main entry point loads a Transformer model, calibration samples and arguments defining sparsity, pruning method and target modules.
main.py - 02
Gather statistics
Layer wrappers accumulate weight/activation information needed by data-dependent pruning methods.
layerwrapper.py - 03
Compute importance and masks
prune.py implements baseline strategies and ClearCut with local and activation-aware weighting; users select unstructured or N:M sparsity.
prune.py - 04
Reallocate and factor
Optional channel reallocation and truncated SVD of selected attention projections reduce representation cost.
prune.py - 05
Evaluate at matched sparsity
Measure perplexity on language modelling data and optionally run zero-shot tasks such as BoolQ and HellaSwag.
eval.py
The choices behind the system.
Structured masks are explicit
The implementation supports 1:4, 2:4, 3:4 and 4:8 patterns as well as unstructured pruning, so different sparsity constraints can be studied.
Pair pruning with low rank
Optional SVD targets attention projections. Weight removal and rank reduction are separate compression levers with different trade-offs.
Compare against established methods
The README reports SparseGPT, Wanda and RIA at the same 50% sparsity condition; comparisons are meaningful only for the stated model/dataset configuration.
Inside the implementation.
Compression methods
ClearCut scoring
Combines row- and column-normalized absolute weights with an interaction term, then weights using activation statistics.
Channel reallocation
Optional heuristic allocation, with linear sum assignment as a configurable optimization method.
Truncated SVD
Factorizes chosen attention projections into low-rank components, controlled by rank_ratio.
Zero-shot checks
The repository includes BoolQ, RTE, HellaSwag, ARC-Challenge and MNLI evaluation paths.
Reproduce the documented pruning run
This command is shown in the project README and configures ClearCut with a 2:4 pattern; it is not evidence the command was run in this portfolio environment.
python main.py \
--model meta-llama/Llama-2-7b-hf \
--prune_method ClearCut \
--sparsity_type 2:4 \
--calib_dataset c4 \
--saveResults in context.
Perplexity at 50% sparsity
These figures are copied from the project README for WikiText-2 and are not verified on a separate machine. Lower perplexity is better. A sparsity pattern does not by itself demonstrate GPU latency improvements.
| Model | SparseGPT | Wanda | RIA | ClearCut |
|---|---|---|---|---|
| LLaMA-2-7B | 7.24 | 7.26 | 6.81 | 6.78 |
| LLaMA-3.1-8B | — | — | 9.43 | 9.39 |
| OPT-6.7B | 11.55 | 11.94 | 11.84 | 11.44 |
Numbers here are reported by the project author in the linked public repository or CV. They are not independent tests performed for this portfolio.
Trade-offs & open questions.
No demonstrated latency win
The README provides pruning/evaluation numbers but does not publish hardware-specific token throughput or end-to-end serving comparisons.
Model-/dataset-specific results
Perplexity findings cannot be generalized to other architectures, downstream tasks or sparse kernels without testing.
Quality vs compression
Rank ratio, mask layout and calibration data affect performance; pruning often requires careful evaluation beyond perplexity.
Source directory.
These source paths are identified in the project implementation or accompanying documentation. Links open the corresponding GitHub source location; availability can change as repositories evolve.