Heedless Backbones

Backbone Families

namemodel typepretraining methodhierarchicalspikingpublication dategithubpaper
SwiftFormerAttn + ConvSupervisedTrueFalse2023-07-25linklink
InceptionNeXtConvolutionSupervisedTrueFalse2025-04-05linklink
LightViTAttentionSupervisedTrueFalse2022-07-12linklink
CETNetAttn + ConvSupervisedTrueFalse2022-08-01linklink
CE-CvTAttn + ConvSupervisedTrueFalse2022-08-01linklink
CE-PVTAttn + ConvSupervisedTrueFalse2022-08-01linklink
CE-SwinAttn + ConvSupervisedTrueFalse2022-08-01linklink
CE-CSWinAttn + ConvSupervisedTrueFalse2022-08-01linklink
InceptionNeXt (iso.)ConvolutionSupervisedFalseFalse2025-04-05linklink
CloFormerAttn + ConvSupervisedTrueFalse2023-06-01linklink
LaViTAttentionSupervisedTrueFalse2024-06-01linklink
RepNeXtConvolutionSupervisedTrueFalse2024-07-20linklink
WTConvNeXtConvolutionSupervisedTrueFalse2024-07-15linklink
L2ViTAttn + ConvSupervisedTrueFalse2025-01-27linklink
LSNetAttn + ConvSupervisedTrueFalse2025-03-29linklink
TransNeXtAttn + ConvSupervisedTrueFalse2023-11-28linklink
ConvNeXtConvolutionSupervisedTrueFalse2022-01-10linklink
SwinAttentionSupervisedTrueFalse2021-03-25linklink
DeiT IIIAttentionSupervisedFalseFalse2022-04-14linklink
ResNet (RSB)ConvolutionSupervisedTrueFalse2021-10-01linklink
ConvNeXt V2ConvolutionFCMAETrueFalse2023-01-02linklink
HieraAttentionMAETrueFalse2023-06-01linklink
InternImageConvolutionSupervisedTrueFalse2022-11-11linklink
CSWinAttentionSupervisedTrueFalse2021-07-01linklink
IdentityFormerIdentitySupervisedTrueFalse2022-10-24linklink
RandFormerRandomSupervisedTrueFalse2022-10-24linklink
CAFormerAttn + ConvSupervisedTrueFalse2022-10-24linklink
ConvFormerConvolutionSupervisedTrueFalse2022-10-24linklink
V2MState Space ModelSupervisedFalseFalse2024-10-14linklink
FocalNetConvolutionSupervisedTrueFalse2022-03-22linklink
V2M + local windowState Space ModelSupervisedFalseFalse2024-10-14linklink
V2M*State Space ModelSupervisedTrueFalse2024-10-14linklink
MaxViTAttn + ConvSupervisedTrueFalse2022-04-22linklink
MogaNetConvolutionSupervisedTrueFalse2024-02-16linklink
CoAtNetAttn + ConvSupervisedTrueFalse2021-06-09linklink
VMambaState Space ModelSupervisedTrueFalse2024-01-18linklink
UniRepLKNetConvolutionSupervisedTrueFalse2023-11-27linklink
SLaKConvolutionSupervisedTrueFalse2023-03-03linklink
RepLKNetConvolutionSupervisedTrueFalse2022-04-02linklink
BiFormerAttentionSupervisedTrueFalse2023-03-15linklink
MambaOutConvolutionSupervisedTrueFalse2024-05-20linklink
GroupMambaState Space ModelSupervisedTrueFalse2024-03-28linklink
VimState Space ModelSupervisedFalseFalse2024-02-29linklink
Hier-VimState Space ModelSupervisedTrueFalse2024-02-29linklink
PlainMambaState Space ModelSupervisedFalseFalse2024-08-15linklink
LocalVimState Space ModelSupervisedFalseFalse2024-03-14linklink
LocalVMambaState Space ModelSupervisedTrueFalse2024-03-14linklink
EfficientVMambaState Space ModelSupervisedTrueFalse2024-03-15linklink
DAMambaState Space ModelSupervisedTrueFalse2025-02-18linklink
VSSDState Space ModelSupervisedTrueFalse2024-08-04linklink
RMTAttn + ConvSupervisedTrueFalse2023-09-20linklink
DAT++Attn + ConvSupervisedTrueFalse2023-09-04linklink
FANAttentionSupervisedTrueFalse2022-07-17linklink
DATAttn + ConvSupervisedTrueFalse2022-05-24linklink
NATAttentionSupervisedTrueFalse2022-04-15linklink
SSViTAttentionSupervisedTrueFalse2024-05-22linklink
Iwin TransformerAttn + ConvSupervisedTrueFalse2025-07-24linklink
MA ViTAttentionSupervisedTrueFalse2025-08-02linklink
VCMambaConv + SSMSupervisedTrueFalse2025-09-04linklink
SpaRTANConvolutionSupervisedTrueFalse2025-07-15linklink
UniConvNetConvolutionSupervisedTrueFalse2025-08-12linklink
RecNeXtConvolutionSupervisedTrueFalse2024-12-27linklink
FractalMamba++State Space ModelSupervisedTrueFalse2025-08-13linklink
InceptionMambaConv + SSMSupervisedTrueFalse2025-07-24linklink
CoCA ViTAttn + ConvSupervisedTrueFalse2025-08-07linklink
HybridNetConv + SSMSupervisedTrueFalse2025-05-23linklink
S2AFormerAttn + ConvSupervisedTrueFalse2025-05-28linklink
VMINetAttentionSupervisedTrueFalse2025-01-03linklink
Mamba-AdaptorState Space ModelSupervisedTrueFalse2025-05-19linklink
AnchorFormerAttentionSupervisedTrueFalse2025-06-18linklink
UniNeXtAttentionSupervisedTrueFalse2023-04-26linklink
SDT-V1+SCAAttentionSupervisedFalseTrue2026-10-01linklink
QKFormer+SCAAttentionSupervisedTrueTrue2026-10-01linklink
A2MambaAttn + SSMSupervisedTrueFalse2025-07-22linklink
SDT-V3+SCAAttn + ConvSupervisedTrueTrue2026-10-01linklink
ResTAttentionSupervisedTrueFalse2021-10-14linklink
ConvNeXt-dclsConvolutionSupervisedTrueFalse2023-05-11linklink
PaleAttn + ConvSupervisedTrueFalse2021-12-28linklink
WinAttn + ConvSupervisedTrueFalse2022-11-25linklink
EViTAttn + ConvSupervisedTrueFalse2024-11-06linklink
TransXNetAttn + ConvSupervisedTrueFalse2025-03-31linklink
VBBAttn + ConvSupervisedTrueFalse2023-11-10linklink
FaViTAttentionSupervisedTrueFalse2023-12-14linklink
FViTConvolutionSupervisedTrueFalse2025-01-21linklink
DeBiFormerAttn + ConvSupervisedTrueFalse2024-10-11linklink
SKAFormerAttn + ConvSupervisedTrueFalse2024-12-09linklink
CSKAFormerAttn + ConvSupervisedTrueFalse2024-12-09linklink
SepViTAttentionSupervisedTrueFalse2023-06-15linklink
SWConvolutionSupervisedTrueFalse2025-03-13linklink
iFormerAttn + ConvSupervisedTrueFalse2022-05-26linklink
LinGloAttn + ConvSupervisedTrueFalse2022-07-04linklink
STViTAttn + ConvSupervisedTrueFalse2024-01-25linklink
FSTAttentionSupervisedFalseFalse2026-09-29linklink
E-SpikeFormer+Adaptive PT-SSAAttn + ConvSupervisedTrueTrue2026-10-02linklink
FlashSwinAttentionSupervisedTrueFalse2026-10-03linklink
FlyVisionConvolutionSupervisedTrueFalse2026-10-06linklink
FMViTAttn + ConvSupervisedTrueFalse2023-11-09linklink
P2TAttn + ConvSupervisedTrueFalse2022-08-31linklink
CMTAttn + ConvSupervisedTrueFalse2022-06-14linklink
TinyViMConv + SSMSupervisedTrueFalse2024-11-26linklink
MVFormerConvolutionSupervisedTrueFalse2024-11-28linklink
iFormer (Mobile)Attn + ConvSupervisedTrueFalse2025-02-17linklink
FATAttn + ConvSupervisedTrueFalse2025-02-27linklink
RepViTConvolutionSupervisedTrueFalse2024-03-14linklink
GGAttn + ConvSupervisedTrueFalse2021-06-04linklink
ShuffleAttn + ConvSupervisedTrueFalse2021-06-07linklink
LocalViTAttn + ConvSupervisedFalseFalse2025-02-12linklink
LocalViT-T2TAttn + ConvSupervisedFalseFalse2025-02-12linklink
LocalViT-TNTAttn + ConvSupervisedFalseFalse2025-02-12linklink
LocalViT-PVTAttn + ConvSupervisedTrueFalse2025-02-12linklink
LocalViT-SwinAttn + ConvSupervisedTrueFalse2025-02-12linklink
HAT-NetAttn + ConvSupervisedTrueFalse2024-03-26linklink
GSAPConvolutionSupervisedFalseFalse2026-09-22linklink
MSVMambaConv + SSMSupervisedTrueFalse2024-05-23linklink
Mamba-RState Space ModelSupervisedFalseFalse2025-05-30linklink
PPMAAttn + ConvSupervisedTrueFalse2025-06-19linklink
Spikformer+k-WTA RouterAttentionSupervisedFalseTrue2026-10-01linklink
SD-Transformer+k-WTA RouterAttentionSupervisedFalseTrue2026-10-01linklink
Spikingformer+k-WTA RouterAttentionSupervisedFalseTrue2026-10-01linklink
uViTAttentionSupervisedFalseFalse2026-10-07linklink