Crypto Flexs
  • DIRECTORY
  • CRYPTO
    • ETHEREUM
    • BITCOIN
    • ALTCOIN
  • BLOCKCHAIN
  • EXCHANGE
  • ADOPTION
  • TRADING
  • HACKING
  • SLOT
Crypto Flexs
  • DIRECTORY
  • CRYPTO
    • ETHEREUM
    • BITCOIN
    • ALTCOIN
  • BLOCKCHAIN
  • EXCHANGE
  • ADOPTION
  • TRADING
  • HACKING
  • SLOT
Crypto Flexs
Home»ADOPTION NEWS»Together AI, Kernel Collection Boosts NVIDIA H200 and H100 GPU Cluster Performance
ADOPTION NEWS

Together AI, Kernel Collection Boosts NVIDIA H200 and H100 GPU Cluster Performance

By Crypto FlexsSeptember 6, 20243 Mins Read
Facebook Twitter Pinterest LinkedIn Tumblr Email
Together AI, Kernel Collection Boosts NVIDIA H200 and H100 GPU Cluster Performance
Share
Facebook Twitter LinkedIn Pinterest Email

Jorg Hiller
Sep 6, 2024 07:14

Together AI enhances NVIDIA H200 and H100 GPU clusters with the Together Kernel Collection to dramatically improve AI training and inference performance.





According to together.ai, Together AI has announced a significant improvement to its GPU clusters by integrating NVIDIA H200 Tensor Core GPUs. The upgrade is accompanied by Together Kernel Collection (TKC), a custom kernel stack designed to optimize AI operations, delivering significant performance improvements for both training and inference tasks.

Improved performance with TKC

Together Kernel Collection (TKC) is designed to significantly accelerate common AI tasks. Compared to the standard PyTorch implementation, TKC delivers up to 24% speedup for commonly used training operators and up to 75% speedup for FP8 inference tasks. These improvements can reduce GPU time, leading to cost-effectiveness and faster time to market.

Training and inference optimization

TKC’s optimized kernels, such as multilayer perceptron (MLP) with SwiGLU activation, are essential for training large-scale language models (LLMs) such as Llama-3. These kernels are reported to be 22-24% faster than standard implementations, with potential improvements of up to 10% over the best existing baselines. Inference tasks benefit from a powerful FP8 kernel stack that Together AI has optimized to deliver over 75% speedup over the default PyTorch implementation.

Native PyTorch compatibility

TKC is fully integrated with PyTorch, allowing AI developers to seamlessly leverage optimizations within their existing frameworks. This integration simplifies the adoption of TKC, making it as easy as changing an import statement within PyTorch.

Production level testing

Together AI ensures that TKC undergoes rigorous testing to meet production-grade standards, ensuring high performance and stability for real-world applications. All Together GPU clusters (H200 or H100) are TKC ready out of the box.

NVIDIA H200: Faster performance and more memory

The NVIDIA H200 Tensor Core GPU, based on the Hopper architecture, is designed for high-performance AI and HPC workloads. According to NVIDIA, the H200 delivers 40 percent faster inference performance on the Llama 2 13B and 90 percent faster on the Llama 2 70B than its predecessor, the H100. The H200 features 141 GB of HBM3e memory and 4.8 TB/s of memory bandwidth, nearly doubling the capacity and 1.4x the bandwidth of the H100.

High-performance interconnectivity

Together GPU Clusters leverage the SXM form factor to deliver high bandwidth and fast data transfer, and support ultra-fast GPU-to-GPU communication via NVIDIA’s NVLink and NVSwitch technologies. Combined with NVIDIA Quantum-2 3200Gb/s InfiniBand Networking, this setup is ideal for large-scale AI training and HPC workloads.

Cost-effective infrastructure

Together AI offers significant cost savings with an infrastructure designed to be up to 75% more cost-effective than cloud providers like AWS. The company also offers flexible commitment options from one month to five years, ensuring adequate resources at every stage of the AI ​​development lifecycle.

Reliability and Support

Together AI’s GPU clusters come with a 99.9% uptime SLA and are backed by rigorous acceptance testing. The company’s White Glove Service provides end-to-end support from cluster setup to ongoing maintenance, ensuring peak performance of your AI models.

Flexible deployment options

Together AI offers multiple deployment options, including Slurm for high-performance workload management, Kubernetes for containerized AI workloads, and bare metal clusters running Ubuntu for direct access and ultimate flexibility. These options meet a variety of AI project needs, from large-scale training to production-level inference.

Together AI continues to support the entire AI lifecycle with high-performance NVIDIA H200 GPU clusters and the Together Kernel Collection. The platform is designed to optimize performance, reduce costs, and ensure stability, making it the ideal choice for accelerating AI development.

Image source: Shutterstock


Share. Facebook Twitter Pinterest LinkedIn Tumblr Email

Related Posts

Solana (SOL) introduces Alpenglow for faster blockchain agreement.

June 7, 2025

Vaneck launches GPZ ETF for alternative asset managers.

June 7, 2025

Thorchain integrates the director of the XRP to facilitate the distributed swap.

June 7, 2025
Add A Comment

Comments are closed.

Recent Posts

ETF publisher’s latest warning -SEC’s approval process ‘Innovation, AIDS GIANTS’

June 7, 2025

Solana (SOL) introduces Alpenglow for faster blockchain agreement.

June 7, 2025

The Foresight Ventures report shows a collection shift where more than 32,000 sellers around the world accept encryption.

June 7, 2025

$ AB is live on Binance, guiding the new era of new cross chain asset mobility.

June 7, 2025

Trump memoin is faced with a $ 520m lock in July and the price drops by 85%.

June 7, 2025

Vaneck launches GPZ ETF for alternative asset managers.

June 7, 2025

Apple, X, Airbnb Eye Stablecoin Integration

June 7, 2025

Strategy to raise almost $ 1B by STRD priority proposal for BTC accumulation

June 7, 2025

Hash research CEO is appointed chief policy officer at the presidential office

June 7, 2025

Thorchain integrates the director of the XRP to facilitate the distributed swap.

June 7, 2025

Thorchain integrates the director of the XRP to facilitate the distributed swap.

June 7, 2025

Crypto Flexs is a Professional Cryptocurrency News Platform. Here we will provide you only interesting content, which you will like very much. We’re dedicated to providing you the best of Cryptocurrency. We hope you enjoy our Cryptocurrency News as much as we enjoy offering them to you.

Contact Us : Partner(@)Cryptoflexs.com

Top Insights

ETF publisher’s latest warning -SEC’s approval process ‘Innovation, AIDS GIANTS’

June 7, 2025

Solana (SOL) introduces Alpenglow for faster blockchain agreement.

June 7, 2025

The Foresight Ventures report shows a collection shift where more than 32,000 sellers around the world accept encryption.

June 7, 2025
Most Popular

io.net partners with Synesis One to accelerate AI development

May 17, 2024

The CZ suggests a dark full dex to suppress the operation.

June 2, 2025

Binance ends BEP2 deposit and withdrawal support

October 29, 2024
  • Home
  • About Us
  • Contact Us
  • Disclaimer
  • Privacy Policy
  • Terms and Conditions
© 2025 Crypto Flexs

Type above and press Enter to search. Press Esc to cancel.