Little LLM量化思考

[imoldpan]

小规模 LLM量化思考

参考

• Performance of Quantized Models - Qwen
• https://huggingface.co/neuralmagic/Qwen2-0.5B-Instruct-FP8