transformer结构简析

[imoldpan]

简化版教程，详细版可以看这篇。

self-attention

单个attention

MHA

decoder

Cross-Attention

stable diffusion中使用

参考

• https://sebastianraschka.com/blog/2023/self-attention-from-scratch.html
• https://towardsdatascience.com/illustrated-self-attention-2d627e33b20a
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.
• This post is all you need（上卷）——层层剥开Transformer - 知乎