Crypto Flexs
  • DIRECTORY
  • CRYPTO
    • ETHEREUM
    • BITCOIN
    • ALTCOIN
  • BLOCKCHAIN
  • EXCHANGE
  • TRADING
  • HACKING
  • SLOT
  • CASINO
  • SUBMIT
Crypto Flexs
  • DIRECTORY
  • CRYPTO
    • ETHEREUM
    • BITCOIN
    • ALTCOIN
  • BLOCKCHAIN
  • EXCHANGE
  • TRADING
  • HACKING
  • SLOT
  • CASINO
  • SUBMIT
Crypto Flexs
Home»ADOPTION NEWS»NVIDIA unveils DoRA, a superior fine-tuning method for AI models
ADOPTION NEWS

NVIDIA unveils DoRA, a superior fine-tuning method for AI models

By Crypto FlexsJune 30, 20243 Mins Read
Facebook Twitter Pinterest LinkedIn Tumblr Email
NVIDIA unveils DoRA, a superior fine-tuning method for AI models
Share
Facebook Twitter LinkedIn Pinterest Email





NVIDIA announced that it has developed a new fine-tuning method called Weight-Decomposed Low-Rank Adaptation (DoRA) that provides a high-performance alternative to the widely used Low-Rank Adaptation (LoRA). According to the NVIDIA Technology Blog, DoRA improves both the learning ability and stability of LoRA without introducing additional inference overhead.

Advantages of DoRA

DoRA has demonstrated significant performance gains across a variety of large language models (LLMs) and vision language models (VLMs). For example, in the common sense reasoning task, DoRA outperformed LoRA with improvements of +3.7 points on Llama 7B and +4.4 points on Llama 3 8B. DoRA also showed better results in multi-turn benchmarks, image/video text understanding, and visual command coordination tasks.

This innovative method was accepted as an oral paper at ICML 2024, demonstrating its reliability and potential impact in the field of machine learning.

DoRA’s Mechanism

DoRA works by decomposing the pre-trained weights into magnitude and direction components and fine-tuning both. This method ensures efficient fine-tuning by leveraging LoRA for orientation adaptation. After the training process is over, DoRA merges the fine-tuned components back into the pre-trained weights, preventing additional latency during inference.

Visualizing the magnitude and directional differences between DoRA and the pretrained weights shows that DoRA makes significant directional adjustments with minimal change in magnitude, which is very similar to a full-fine-tuning (FT) learning pattern.

Performance across models

In a variety of performance benchmarks, DoRA consistently outperforms LoRA. For example, in large-scale language models, DoRA significantly improves common sense reasoning and conversation/following directions. In visual language models, DoRA shows excellent results in image-to-text and video-to-text understanding and visual instruction tuning tasks.

Large-scale language model

Comparative studies highlight that DoRA outperforms LoRA on common sense reasoning benchmarks and multi-turn benchmarks. In tests, DoRA demonstrates strong performance by achieving higher average scores across a variety of datasets.

Vision Language Model

DoRA also excels at vision language models, outperforming LoRA in tasks such as image-to-text understanding, video-to-text understanding, and visual command coordination. The effectiveness of this method is evident through higher average scores across multiple benchmarks.

Compression Awareness LLM

DoRA can be integrated into the QLoRA framework to improve the accuracy of low-bit pretrained models. Our joint efforts with Answer.AI on the QDoRA project have shown that QDoRA outperforms FT and QLoRA on Llama 2 and Llama 3 models.

Text-to-Image Generation

DoRA’s applications extend to text-to-image personalization via DreamBooth, delivering significantly better results than LoRA on challenging data sets such as 3D icons and Lego sets.

Meaning and future applications

DoRA is poised to become the default choice for fine-tuning AI models compatible with LoRA and its variants. Its efficiency and effectiveness make it an invaluable tool for applying foundational models to a variety of applications, including NVIDIA Metropolis, NVIDIA NeMo, NVIDIA NIM, and NVIDIA TensorRT.

For more information, visit the NVIDIA Technology Blog.

Image source: Shutterstock



Share. Facebook Twitter Pinterest LinkedIn Tumblr Email

Related Posts

Ether Lee (ETH) tests major support for $ 4,453 after the highest rejection.

August 31, 2025

Bitcoin analysts bet on $ 200K after hints of Fed.

August 23, 2025

‘Self -transactions, dressed in capital layout’: The cryptocurrency financial craze divides the industry.

August 15, 2025
Add A Comment

Comments are closed.

Recent Posts

RLUSD Stablecoin is extended to Africa to supply power to the border between the border.

September 5, 2025

Bybit Establishes New B2B Unit To Drive Institutional Adoption Of Digital Assets

September 5, 2025

Lowkick Studio Launches $SHARDS Token On Top Tier Exchanges For WorldShards MMORPG

September 5, 2025

The cryptocurrency is falling when the tokens and stocks connected to Trump are under pressure.

September 5, 2025

Cango Inc. Reports Second Quarter 2025 Unaudited Financial Results

September 5, 2025

Coindesk July 2025 Report: Stablecoins and CBDC

September 5, 2025

NOWPayments To Participate In SiGMA Europe Rome 2025

September 4, 2025

Web3 Enabler Announces Blockchain Payments V3.1 At Northeast Dreamin In Boston

September 4, 2025

Is XRP The Dark Horse Of The Cryptocurrency World? Earn 652 XRP Daily Using Invro Mining’s Smart Contract

September 4, 2025

TRX Was Early, ETH Set The Standard, BNB Built The Scale- Now SYC Brings The Next Evolution

September 4, 2025

Sign Up And Receive $500 Bonus, Ushering In A New Era Of Compliant And Secure Crypto Investment

September 4, 2025

Crypto Flexs is a Professional Cryptocurrency News Platform. Here we will provide you only interesting content, which you will like very much. We’re dedicated to providing you the best of Cryptocurrency. We hope you enjoy our Cryptocurrency News as much as we enjoy offering them to you.

Contact Us : Partner(@)Cryptoflexs.com

Top Insights

RLUSD Stablecoin is extended to Africa to supply power to the border between the border.

September 5, 2025

Bybit Establishes New B2B Unit To Drive Institutional Adoption Of Digital Assets

September 5, 2025

Lowkick Studio Launches $SHARDS Token On Top Tier Exchanges For WorldShards MMORPG

September 5, 2025
Most Popular

SEC Chairman Gary Gensler said disclosure is not enough to protect cryptocurrency exchanges.

June 6, 2024

Trump campaign raises $21 million at Bitcoin 2024 conference

July 31, 2024

Power naps may prevent brain shrinkage: Study

January 28, 2024
  • Home
  • About Us
  • Contact Us
  • Disclaimer
  • Privacy Policy
  • Terms and Conditions
© 2025 Crypto Flexs

Type above and press Enter to search. Press Esc to cancel.