Connect with us

Uncategorized

NVIDIA Unveils DoRA: A Superior Fine-Tuning Method for AI Models | IDOs News

Avatar

Published

on

NVIDIA Unveils DoRA: A Superior Fine-Tuning Method for AI Models | IDOs News







NVIDIA has announced the development of a new fine-tuning method called DoRA (Weight-Decomposed Low-Rank Adaptation), which offers a high-performing alternative to the widely used Low-Rank Adaptation (LoRA). According to the NVIDIA Technical Blog, DoRA enhances both the learning capacity and stability of LoRA without introducing any additional inference overhead.

Advantages of DoRA

DoRA has demonstrated significant performance improvements across various large language models (LLMs) and vision language models (VLMs). For instance, in common-sense reasoning tasks, DoRA outperformed LoRA with improvements such as +3.7 points on Llama 7B and +4.4 points on Llama 3 8B. Additionally, DoRA showed better results in multi-turn benchmarks, image/video-text understanding, and visual instruction tuning tasks.

This innovative method has been accepted as an oral paper at ICML 2024, marking its credibility and potential impact in the field of machine learning.

Mechanics of DoRA

DoRA operates by decomposing the pretrained weight into its magnitude and directional components, fine-tuning both. The method leverages LoRA for directional adaptation, ensuring efficient fine-tuning. After the training process, DoRA merges the fine-tuned components back into the pretrained weight, avoiding any additional latency during inference.

Visualizations of the magnitude and directional differences between DoRA and pretrained weights reveal that DoRA makes substantial directional adjustments with minimal changes in magnitude, closely resembling full fine-tuning (FT) learning patterns.

Performance Across Models

In various performance benchmarks, DoRA consistently outperforms LoRA. For example, in large language models, DoRA significantly enhances commonsense reasoning abilities and conversation/instruction-following capabilities. In vision language models, DoRA shows superior results in image-text and video-text understanding, as well as visual instruction tuning tasks.

Large Language Models

Comparative studies highlight that DoRA surpasses LoRA in commonsense reasoning benchmarks and multi-turn benchmarks. In tests, DoRA achieved higher average scores across various datasets, indicating its robust performance.

Vision Language Models

DoRA also excels in vision language models, outperforming LoRA in tasks like image-text understanding, video-text understanding, and visual instruction tuning. The method’s efficacy is evident in higher average scores across multiple benchmarks.

Compression-Aware LLMs

DoRA can be integrated into the QLoRA framework, enhancing the accuracy of low-bit pretrained models. Collaborative efforts with Answer.AI on the QDoRA project showed that QDoRA outperforms both FT and QLoRA on Llama 2 and Llama 3 models.

Text-to-Image Generation

DoRA’s application extends to text-to-image personalization with DreamBooth, yielding significantly better results than LoRA in challenging datasets like 3D Icon and Lego sets.

Implications and Future Applications

DoRA is poised to become a default choice for fine-tuning AI models, compatible with LoRA and its variants. Its efficiency and effectiveness make it a valuable tool for adapting foundation models to various applications, including NVIDIA Metropolis, NVIDIA NeMo, NVIDIA NIM, and NVIDIA TensorRT.

For more detailed information, visit the NVIDIA Technical Blog.

Image source: Shutterstock




Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Uncategorized

Advantages of Mobile Apps in Gambling: The Example of Pin Up App | IDOs News

Avatar

Published

on

Advantages of Mobile Apps in Gambling: The Example of Pin Up App | IDOs News


By Terry Ashton, updated August 31, 2024

Online gambling is going mobile — over 50% of players are already playing casino games on their mobile devices, and their number is expected to grow further. But does a mobile app have actual advantages over browser-based play? We decided to do more profound research by accessing and trying gambling on a desktop browser, mobile browser, and the app. That allowed us to distinguish casino mobile applications’ key benefits and drawbacks. If you’re considering using one, just keep reading — we will share some helpful insights below. 

Benefits of Mobile Play at Pin Up Casino

The rise of online gambling happens for multiple reasons, including the following ones: 

  • Ultimate accessibility. You can access the app anywhere, even on the go. You don’t need to take additional actions — the casino opens with just one click. 
  • Lower Internet requirements, offline play. If you play for fun, you can do it even without an Internet connection. If you prefer to play real money, the requirements for an Internet connection will still be much lower because most data is already downloaded to your device. 
  • Push notifications. You can immediately learn about the new top promotions and the hottest games without checking your email. 
  • Special bonuses. Sometimes, special bonuses are granted to mobile players. Some casinos may add them occasionally to encourage players to play on apps. 
  • The same game selection. If a casino is modern and cooperates with top providers, all games will be compatible with mobile devices. For instance, if you play at Pin Up casino online, you can access the same collection of games. That goes not only for slots but also for live games, table games, etc. 
  • Higher security standards. The app is protected even better than the site. Data is encrypted, and the chance that anyone will access your account is close to zero. 

Registration also goes smoothly. Once you sign up on the browser or app, you can access the platform with just one click by entering your Pin Up login and password. 

Considering the Cons: Potential Drawbacks of Using a Pin-Up Mobile App 

Nothing is perfect, and neither are casino apps. Gamblers should also consider the drawbacks, and the most common ones are as follows: 

  • Installing software is a must. You need to install the software on your phone. It’s safe if it’s the official casino site and a good product. However, clicking on the wrong link and downloading the wrong APK file may result in problems. 
  • Battery drain and storage space. It’s no secret that charging the phone all the time is annoying, and innovative slots with top graphics may drain your battery quickly. Also, though most apps don’t take much space (in the case of Pin Up, it’s just about 100 Mb), they still require more effort to manage it. 
  • Compatibility requirements. Any app will have technical requirements, and most aren’t compatible with old mobile devices and tablets. Also, you’ll need to install updates quite regularly. 
  • Smaller screen. This is a disadvantage for those who prefer playing on larger screens, particularly those who prefer live dealer games. 

Do the pros outweigh the cons for you? If yes, the mobile app will boost your experience. If not, browser play may be a better option. 

Final Thoughts: The App vs. Browser Play at Pin-Up Casino

Technology is shaping the industry. Nowadays, there’s no such significant difference between playing on a mobile app and a mobile or desktop browser. You get the same game selection, the same bonuses, and the same smooth experience. So, it’s a matter of taste. Choose what will work best for you and enjoy your play.


Continue Reading

Uncategorized

NVIDIA Introduces Fast Inversion Technique for Real-Time Image Editing | IDOs News

Avatar

Published

on

NVIDIA Unveils DoRA: A Superior Fine-Tuning Method for AI Models | IDOs News




Terrill Dicki
Aug 31, 2024 01:25

NVIDIA’s new Regularized Newton-Raphson Inversion (RNRI) method offers rapid and accurate real-time image editing based on text prompts.





NVIDIA has unveiled an innovative method called Regularized Newton-Raphson Inversion (RNRI) aimed at enhancing real-time image editing capabilities based on text prompts. This breakthrough, highlighted on the NVIDIA Technical Blog, promises to balance speed and accuracy, making it a significant advancement in the field of text-to-image diffusion models.

Understanding Text-to-Image Diffusion Models

Text-to-image diffusion models generate high-fidelity images from user-provided text prompts by mapping random samples from a high-dimensional space. These models undergo a series of denoising steps to create a representation of the corresponding image. The technology has applications beyond simple image generation, including personalized concept depiction and semantic data augmentation.

The Role of Inversion in Image Editing

Inversion involves finding a noise seed that, when processed through the denoising steps, reconstructs the original image. This process is crucial for tasks like making local changes to an image based on a text prompt while keeping other parts unchanged. Traditional inversion methods often struggle with balancing computational efficiency and accuracy.

Introducing Regularized Newton-Raphson Inversion (RNRI)

RNRI is a novel inversion technique that outperforms existing methods by offering rapid convergence, superior accuracy, reduced execution time, and improved memory efficiency. It achieves this by solving an implicit equation using the Newton-Raphson iterative method, enhanced with a regularization term to ensure the solutions are well-distributed and accurate.

Comparative Performance

Figure 2 on the NVIDIA Technical Blog compares the quality of reconstructed images using different inversion methods. RNRI shows significant improvements in PSNR (Peak Signal-to-Noise Ratio) and run time over recent methods, tested on a single NVIDIA A100 GPU. The method excels in maintaining image fidelity while adhering closely to the text prompt.

Real-World Applications and Evaluation

RNRI has been evaluated on 100 MS-COCO images, showing superior performance in both CLIP-based scores (for text prompt compliance) and LPIPS scores (for structure preservation). Figure 3 demonstrates RNRI’s capability to edit images naturally while preserving their original structure, outperforming other state-of-the-art methods.

Conclusion

The introduction of RNRI marks a significant advancement in text-to-image diffusion models, enabling real-time image editing with unprecedented accuracy and efficiency. This method holds promise for a wide range of applications, from semantic data augmentation to generating rare-concept images.

For more detailed information, visit the NVIDIA Technical Blog.

Image source: Shutterstock



Continue Reading

Uncategorized

AMD Radeon PRO GPUs and ROCm Software Expand LLM Inference Capabilities | IDOs News

Avatar

Published

on

AMD Radeon PRO GPUs and ROCm Software Expand LLM Inference Capabilities | IDOs News




Felix Pinkston
Aug 31, 2024 01:52

AMD’s Radeon PRO GPUs and ROCm software enable small enterprises to leverage advanced AI tools, including Meta’s Llama models, for various business applications.





AMD has announced advancements in its Radeon PRO GPUs and ROCm software, enabling small enterprises to leverage Large Language Models (LLMs) like Meta’s Llama 2 and 3, including the newly released Llama 3.1, according to AMD.com.

New Capabilities for Small Enterprises

With dedicated AI accelerators and substantial on-board memory, AMD’s Radeon PRO W7900 Dual Slot GPU offers market-leading performance per dollar, making it feasible for small firms to run custom AI tools locally. This includes applications such as chatbots, technical documentation retrieval, and personalized sales pitches. The specialized Code Llama models further enable programmers to generate and optimize code for new digital products.

The latest release of AMD’s open software stack, ROCm 6.1.3, supports running AI tools on multiple Radeon PRO GPUs. This enhancement allows small and medium-sized enterprises (SMEs) to handle larger and more complex LLMs, supporting more users simultaneously.

Expanding Use Cases for LLMs

While AI techniques are already prevalent in data analysis, computer vision, and generative design, the potential use cases for AI extend far beyond these areas. Specialized LLMs like Meta’s Code Llama enable app developers and web designers to generate working code from simple text prompts or debug existing code bases. The parent model, Llama, offers extensive applications in customer service, information retrieval, and product personalization.

Small enterprises can utilize retrieval-augmented generation (RAG) to make AI models aware of their internal data, such as product documentation or customer records. This customization results in more accurate AI-generated outputs with less need for manual editing.

Local Hosting Benefits

Despite the availability of cloud-based AI services, local hosting of LLMs offers significant advantages:

  • Data Security: Running AI models locally eliminates the need to upload sensitive data to the cloud, addressing major concerns about data sharing.
  • Lower Latency: Local hosting reduces lag, providing instant feedback in applications like chatbots and real-time support.
  • Control Over Tasks: Local deployment allows technical staff to troubleshoot and update AI tools without relying on remote service providers.
  • Sandbox Environment: Local workstations can serve as sandbox environments for prototyping and testing new AI tools before full-scale deployment.

AMD’s AI Performance

For SMEs, hosting custom AI tools need not be complex or expensive. Applications like LM Studio facilitate running LLMs on standard Windows laptops and desktop systems. LM Studio is optimized to run on AMD GPUs via the HIP runtime API, leveraging the dedicated AI Accelerators in current AMD graphics cards to boost performance.

Professional GPUs like the 32GB Radeon PRO W7800 and 48GB Radeon PRO W7900 offer sufficient memory to run larger models, such as the 30-billion-parameter Llama-2-30B-Q8. ROCm 6.1.3 introduces support for multiple Radeon PRO GPUs, enabling enterprises to deploy systems with multiple GPUs to serve requests from numerous users simultaneously.

Performance tests with Llama 2 indicate that the Radeon PRO W7900 offers up to 38% higher performance-per-dollar compared to NVIDIA’s RTX 6000 Ada Generation, making it a cost-effective solution for SMEs.

With the evolving capabilities of AMD’s hardware and software, even small enterprises can now deploy and customize LLMs to enhance various business and coding tasks, avoiding the need to upload sensitive data to the cloud.

Image source: Shutterstock



Continue Reading

Trending