Mixture of Experts, MoR and Self-Attention: Architectures of Efficiency. A comparative analysis. MoE