Crypto Flexs
  • DIRECTORY
  • CRYPTO
    • ETHEREUM
    • BITCOIN
    • ALTCOIN
  • BLOCKCHAIN
  • EXCHANGE
  • TRADING
  • HACKING
  • SLOT
  • CASINO
  • SUBMIT
Crypto Flexs
  • DIRECTORY
  • CRYPTO
    • ETHEREUM
    • BITCOIN
    • ALTCOIN
  • BLOCKCHAIN
  • EXCHANGE
  • TRADING
  • HACKING
  • SLOT
  • CASINO
  • SUBMIT
Crypto Flexs
Home»ADOPTION NEWS»NVIDIA unveils DoRA, a superior fine-tuning method for AI models
ADOPTION NEWS

NVIDIA unveils DoRA, a superior fine-tuning method for AI models

By Crypto FlexsJune 30, 20243 Mins Read
Facebook Twitter Pinterest LinkedIn Tumblr Email
NVIDIA unveils DoRA, a superior fine-tuning method for AI models
Share
Facebook Twitter LinkedIn Pinterest Email





NVIDIA announced that it has developed a new fine-tuning method called Weight-Decomposed Low-Rank Adaptation (DoRA) that provides a high-performance alternative to the widely used Low-Rank Adaptation (LoRA). According to the NVIDIA Technology Blog, DoRA improves both the learning ability and stability of LoRA without introducing additional inference overhead.

Advantages of DoRA

DoRA has demonstrated significant performance gains across a variety of large language models (LLMs) and vision language models (VLMs). For example, in the common sense reasoning task, DoRA outperformed LoRA with improvements of +3.7 points on Llama 7B and +4.4 points on Llama 3 8B. DoRA also showed better results in multi-turn benchmarks, image/video text understanding, and visual command coordination tasks.

This innovative method was accepted as an oral paper at ICML 2024, demonstrating its reliability and potential impact in the field of machine learning.

DoRA’s Mechanism

DoRA works by decomposing the pre-trained weights into magnitude and direction components and fine-tuning both. This method ensures efficient fine-tuning by leveraging LoRA for orientation adaptation. After the training process is over, DoRA merges the fine-tuned components back into the pre-trained weights, preventing additional latency during inference.

Visualizing the magnitude and directional differences between DoRA and the pretrained weights shows that DoRA makes significant directional adjustments with minimal change in magnitude, which is very similar to a full-fine-tuning (FT) learning pattern.

Performance across models

In a variety of performance benchmarks, DoRA consistently outperforms LoRA. For example, in large-scale language models, DoRA significantly improves common sense reasoning and conversation/following directions. In visual language models, DoRA shows excellent results in image-to-text and video-to-text understanding and visual instruction tuning tasks.

Large-scale language model

Comparative studies highlight that DoRA outperforms LoRA on common sense reasoning benchmarks and multi-turn benchmarks. In tests, DoRA demonstrates strong performance by achieving higher average scores across a variety of datasets.

Vision Language Model

DoRA also excels at vision language models, outperforming LoRA in tasks such as image-to-text understanding, video-to-text understanding, and visual command coordination. The effectiveness of this method is evident through higher average scores across multiple benchmarks.

Compression Awareness LLM

DoRA can be integrated into the QLoRA framework to improve the accuracy of low-bit pretrained models. Our joint efforts with Answer.AI on the QDoRA project have shown that QDoRA outperforms FT and QLoRA on Llama 2 and Llama 3 models.

Text-to-Image Generation

DoRA’s applications extend to text-to-image personalization via DreamBooth, delivering significantly better results than LoRA on challenging data sets such as 3D icons and Lego sets.

Meaning and future applications

DoRA is poised to become the default choice for fine-tuning AI models compatible with LoRA and its variants. Its efficiency and effectiveness make it an invaluable tool for applying foundational models to a variety of applications, including NVIDIA Metropolis, NVIDIA NeMo, NVIDIA NIM, and NVIDIA TensorRT.

For more information, visit the NVIDIA Technology Blog.

Image source: Shutterstock



Share. Facebook Twitter Pinterest LinkedIn Tumblr Email

Related Posts

The best Solana depin project to form the future -Part 2

September 8, 2025

Ether Lee (ETH) tests major support for $ 4,453 after the highest rejection.

August 31, 2025

Bitcoin analysts bet on $ 200K after hints of Fed.

August 23, 2025
Add A Comment

Comments are closed.

Recent Posts

France can break the EU password market with ‘atomic weapons’.

September 15, 2025

Cardano (ADA) Signal Recovery -Is there a strong rise?

September 15, 2025

BitMine Immersion (BMNR) Announces Crypto And Cash Holdings Of $10.8 Billion, ETH Holdings Exceeding 2.151 Million

September 15, 2025

How SWLMiner Could Help You Get The IPhone 17 Air

September 15, 2025

Metabuses are increasing again -Records in August +13k NFT users

September 15, 2025

Rabby Wallet integrates XRPL EVM chain with peersyst

September 15, 2025

Stop Dreaming About The Lottery. Join H Mining And Start Earning!

September 14, 2025

Web3 EXEC warns that the US dollar Stablecoin end game is not priced.

September 14, 2025

Binance’s new Defi Initiative sparked Rollish Momentum, and BNB hit a new ATH of more than $ 900.

September 13, 2025

Top 5 Crypto PR Agencies to Scale Your Blockchain Project in Europe

September 13, 2025

The price of Etherrium surges beyond $ 4,500. -Main level for monitoring more profits

September 12, 2025

Crypto Flexs is a Professional Cryptocurrency News Platform. Here we will provide you only interesting content, which you will like very much. We’re dedicated to providing you the best of Cryptocurrency. We hope you enjoy our Cryptocurrency News as much as we enjoy offering them to you.

Contact Us : Partner(@)Cryptoflexs.com

Top Insights

France can break the EU password market with ‘atomic weapons’.

September 15, 2025

Cardano (ADA) Signal Recovery -Is there a strong rise?

September 15, 2025

BitMine Immersion (BMNR) Announces Crypto And Cash Holdings Of $10.8 Billion, ETH Holdings Exceeding 2.151 Million

September 15, 2025
Most Popular

The KIP protocol shows the involvement of Javier Milei-Endorsed Libra

February 16, 2025

Trader names one altcoin as his 2024 ‘AI Bet’ and updates outlook for two additional crypto assets.

May 7, 2024

HKMA issues a warning on fraudulent HSBC banking website.

May 27, 2025
  • Home
  • About Us
  • Contact Us
  • Disclaimer
  • Privacy Policy
  • Terms and Conditions
© 2025 Crypto Flexs

Type above and press Enter to search. Press Esc to cancel.