AWQ
More info
- Creator
- Ji Lin et al. (MIT)
- Released
- Type
- Post-training quantization algorithm
- License
- MIT
- Spec
- arXiv:2306.00978
- Source
- github.com/mit-han-lab/llm-awq
Activation-aware Weight Quantization — 4-bit post-training quantisation that protects the "important" weights based on what activations show is sensitive.
MIT, 2023. Observation: roughly 1 % of weights account for most of the output quality — protect them (scale up before quantising) and the rest can be compressed to INT4 without calibration data. Faster to apply than GPTQ and gives comparable or better perplexity.
Natively supported by vLLM, SGLang, TensorRT-LLM and llama.cpp. The default choice when deploying open models (Llama, Qwen, Mistral) on a single GPU.