Little LLM量化思考

[imoldpan]

小规模 LLM量化思考

参考

• https://qwen.readthedocs.io/en/latest/benchmark/quantization_benchmark.html
• https://huggingface.co/neuralmagic/Qwen2-0.5B-Instruct-FP8