多模态大模型-TensorRT-LLM

[imoldpan]

遇到的问题

satisfyProfile Runtime dimension does not satisfy any optimization profile

参考

• https://github.com/NVIDIA/TensorRT-LLM/issues/444
• https://github.com/NVIDIA/TensorRT-LLM/issues/461
• How to handle variable length decoder_input_ids for batch prediction in the Nougat family model? · Issue #1166 · NVIDIA/TensorRT-LLM · GitHub