[LoRA] Low Rank Adaptation. Beyond RAG Optimizing LLMs with MoE, LoRA and Advanced Preference Tuning