|
Canada-0-Manicuring Azienda Directories
|
Azienda News:
- D2MoRA: Diversity-Regulated Asymmetric MoE-LoRA Decomposition for . . .
To address these issues, we propose a novel diversity-regulated asymmetric MoE-LoRA decomposition framework, which achieves flexible knowledge sharing through asymmetric expert decomposition and guarantees expert diversity with a dual orthogonality regularization
- Publications - Haokun Wen’s Homepage
2026 D2MoRA: Diversity-Regulated Asymmetric MoE-LoRA Decomposition for Efficient Multi-Task Adaptation
- D2MoRA: Diversity-Regulated Asymmetric MoE-LoRA Decomposition for . . .
While recent work integrates mixture-of-experts (MoE) mechanisms with multiple LoRA modules to handle multi-task or complex scenarios, existing approaches face two key limitations: restricted cross-expert knowledge sharing and subsequent expert homogenization
- MALoRA: Mixture of Asymmetric Low-Rank Adaptation for Enhanced Multi . . .
MALoRA introduces a novel PEFT approach for multi-task learning by leveraging a MoE structure with asymmetric low-rank adaptation Through a shared down-projection space and expanded up-projection ranks, MALoRA optimizes parameter efficiency while enhancing generalization across tasks
- D2MoRA: Diversity-Regulated Asymmetric MoE-LoRA Decomposition for . . .
In this work, we propose D2MoRA, a diversity-regulated, asymmetric MoE-LoRA decomposition framework for en-hancing parameter adaptation in multi-task scenarios
- ojs. aaai. org
D2MoRA: Diversity-Regulated Asymmetric MoE-LoRA Decomposition for Efficient Multi-Task Adaptation Proceedings of the AAAI Conference on Artificial Intelligence 40, 34 (Mar 2026), 29286-29294
- 每日论文速递 | 阿里推出Mixture-of-LoRAs，一个多任务高效 . . .
论文介绍了一种新颖的架构Mixture-of-LoRAs (MoA)，通过训练领域特定的LoRA模块和显式路由策略，有效解决大型语言模型在多任务学习中的问题，如防止任务干扰和灾难性遗忘，提升参数效率。 实验证明了MoA在各种任务上的优越性能和快速适应新领域的特性。
- 【番外篇】聊聊 MOE + LoRA 微调新方式 - 知乎
在 Transformer 层内，我们通过为 MLP 层创建一组专门用于 LoRA 的专家，扩展了 LoRA 方法，并基于路由函数将每个 token 路由到排名第一的专家，允许对来自不同领域的令牌进行自适应选择。
- MORE: A MIXTURE OF LOW-RANK EXPERTS FOR ADAPTIVE MULTI-TASK LEARNING
By jointly training low-rank experts, MoRE can enhance the adaptability and efficiency of LoRA in multi-task scenarios Finally, we conduct extensive experiments over multiple multi-task benchmarks along with different LLMs to verify model performance
- MALoRA: Mixture of Asymmetric Low-Rank Adaptation for Enhanced Multi . . .
MALoRA reduces the number of trainable parameters by 30% to 48%, increases training speed by 1 2x, and matches the computational efficiency of single-task LoRA models
|
|