SparK: the first successful BERT/MAE-style pretraining on any convolutional networks

AI大模型
,
1

Different masking strategies
Different masking strategies1056×483 125 KB

Sparse masked modeling with hierarchy
Sparse masked modeling with hierarchy1048×558 151 KB

Using sparse convolution to address “mask pattern vanishing” issue.
Using sparse convolution to address “mask pattern vanishing” issue.1039×567 113 KB

参考