Crypto Flexs
  • DIRECTORY
  • CRYPTO
    • ETHEREUM
    • BITCOIN
    • ALTCOIN
  • BLOCKCHAIN
  • EXCHANGE
  • ADOPTION
  • TRADING
  • HACKING
  • SLOT
  • TRADE
Crypto Flexs
  • DIRECTORY
  • CRYPTO
    • ETHEREUM
    • BITCOIN
    • ALTCOIN
  • BLOCKCHAIN
  • EXCHANGE
  • ADOPTION
  • TRADING
  • HACKING
  • SLOT
  • TRADE
Crypto Flexs
Home»ADOPTION NEWS»Inference Engine 2.0 with Together AI, Turbo, and Lite Endpoints Announced
ADOPTION NEWS

Inference Engine 2.0 with Together AI, Turbo, and Lite Endpoints Announced

By Crypto FlexsJuly 21, 20242 Mins Read
Facebook Twitter Pinterest LinkedIn Tumblr Email
Inference Engine 2.0 with Together AI, Turbo, and Lite Endpoints Announced
Share
Facebook Twitter LinkedIn Pinterest Email

Terryl Dickey
18 July 2024 18:41

Together AI launches Inference Engine 2.0, offering Turbo and Lite endpoints for improved performance, quality, and cost efficiency.





Together AI announced the release of its new Inference Engine 2.0, which includes Turbo and Lite endpoints. This new inference stack is designed to deliver significantly faster decoding throughput and superior performance compared to existing solutions.

Performance Improvement

According to together.ai, Together Inference Engine 2.0 delivers 4x faster decoding throughput than open source vLLM and 1.3x to 2.5x faster than commercial solutions like Amazon Bedrock, Azure AI, Fireworks, and Octo AI. The engine achieves over 400 tokens per second on Meta Llama 3 8B thanks to advances in FlashAttention-3, faster GEMM and MHA kernels, quality-preserving quantization, and speculative decoding.

New Turbo and Lite endpoints

Together AI introduces new Turbo and Lite endpoints starting with Meta Llama 3. These endpoints balance performance, quality, and cost, allowing enterprises to avoid compromises. Together Turbo closely matches the quality of full-precision FP16 models, while Together Lite provides the most cost-effective and scalable Llama 3 models available.

Turbo endpoints provide fast FP8 performance while maintaining quality, are consistent with the FP16 reference model, and outperform other FP8 solutions on AlpacaEval 2.0. These Turbo endpoints are priced at $0.88 per million tokens for 70B and $0.18 per million for 8B, making them significantly cheaper than GPT-4o.

Together Lite endpoints provide high-quality AI models at a low cost using INT4 quantization, and for Llama 3 8B Lite, it is 6x cheaper than GPT-4o-mini at $0.10 per million tokens.

Adoption and Approval

More than 100,000 developers and companies, including Zomato, DuckDuckGo, and The Washington Post, are already leveraging the Together Inference Engine for their generative AI applications. Rinshul Chandra, COO of Food Delivery at Zomato, praised the engine for its high quality, speed, and accuracy.

Technological innovation

Together Inference Engine 2.0 incorporates several technological advancements, including FlashAttention-3, custom speculators, and quality-preserving quantization techniques. These innovations contribute to the engine’s superior performance and cost-effectiveness.

Future outlook

Together AI plans to continue pushing the boundaries of AI acceleration. The company aims to ensure that the Together Inference Engine remains at the forefront of AI technology by expanding support for new models, technologies, and kernels.

Turbo and Lite endpoints for the Llama 3 model are available starting today, with plans to expand to other models soon. Visit the Together AI pricing page for more information.

Image source: Shutterstock


Share. Facebook Twitter Pinterest LinkedIn Tumblr Email

Related Posts

Stablecoin startups surpass 2021 venture capital peaks as institutional money spills.

June 28, 2025

Gala Games improves leader board rewards and introduces preference systems.

June 20, 2025

Ether Leeum Whale starts a $ 11 million leverage betting in the 30% increase in ETH prices.

June 12, 2025
Add A Comment

Comments are closed.

Recent Posts

Checkpoint #4: Berlinterop | Ether Leeum Foundation Blog

June 28, 2025

TRON Price Propects USDT supply exceeded $ 80 billion

June 28, 2025

Stablecoin startups surpass 2021 venture capital peaks as institutional money spills.

June 28, 2025

No Altcoin Season 2025 ? Why Bitcoin Dominance Is Holding Strong In The Crypto Market

June 28, 2025

Why It Matters For Every Crypto Investor

June 27, 2025

Why It Matters For Every Crypto Investor

June 27, 2025

Safe smart account audit summary

June 27, 2025

CARV’s New Roadmap Signals Next Wave Of Web3 AI

June 27, 2025

CARV’s New Roadmap Signals Next Wave Of Web3 AI

June 27, 2025

Bybit Expands Global Reach With Credit Card Crypto Purchases In 25+ Currencies And Cashback Rewards

June 27, 2025

BYDFi Joins Seoul Meta Week 2025, Advancing Web3 Vision And South Korea Strategy

June 27, 2025

Crypto Flexs is a Professional Cryptocurrency News Platform. Here we will provide you only interesting content, which you will like very much. We’re dedicated to providing you the best of Cryptocurrency. We hope you enjoy our Cryptocurrency News as much as we enjoy offering them to you.

Contact Us : Partner(@)Cryptoflexs.com

Top Insights

Checkpoint #4: Berlinterop | Ether Leeum Foundation Blog

June 28, 2025

TRON Price Propects USDT supply exceeded $ 80 billion

June 28, 2025

Stablecoin startups surpass 2021 venture capital peaks as institutional money spills.

June 28, 2025
Most Popular

Fantom (FTM) Andre Cronje Warns About Risks of Ethena’s USDe Stablecoin

April 4, 2024

Fuzzy Penguins to release mobile game in 2025

May 31, 2024

Tether, East Asian crypto exchange revived after Taiwan’s treason: Asia Express

September 6, 2024
  • Home
  • About Us
  • Contact Us
  • Disclaimer
  • Privacy Policy
  • Terms and Conditions
© 2025 Crypto Flexs

Type above and press Enter to search. Press Esc to cancel.