Crypto Flexs
  • DIRECTORY
  • CRYPTO
    • ETHEREUM
    • BITCOIN
    • ALTCOIN
  • BLOCKCHAIN
  • EXCHANGE
  • TRADING
  • HACKING
  • SLOT
  • CASINO
  • SUBMIT
Crypto Flexs
  • DIRECTORY
  • CRYPTO
    • ETHEREUM
    • BITCOIN
    • ALTCOIN
  • BLOCKCHAIN
  • EXCHANGE
  • TRADING
  • HACKING
  • SLOT
  • CASINO
  • SUBMIT
Crypto Flexs
Home»BLOCKCHAIN NEWS»StripedHyena-7B: Next-generation AI architecture for improved performance and efficiency
BLOCKCHAIN NEWS

StripedHyena-7B: Next-generation AI architecture for improved performance and efficiency

By Crypto FlexsJanuary 4, 20242 Mins Read
Facebook Twitter Pinterest LinkedIn Tumblr Email
StripedHyena-7B: Next-generation AI architecture for improved performance and efficiency
Share
Facebook Twitter LinkedIn Pinterest Email

Recent advances in AI have been greatly influenced by the Transformer architecture, a key component of large models across fields as diverse as language, vision, audio, and biology. However, the complexity of Transformer’s attention mechanism limits its application in processing long sequences. Even sophisticated models such as GPT-4 suffer from this limitation.

Breakthrough Advances with StripedHyena

To address these issues, Together Research recently open sourced StripedHyena, a language model that boasts a new architecture optimized for long contexts. StripedHyena can handle up to 128,000 tokens and has demonstrated improved performance over the Transformer architecture in both training and inference performance.​​ It is the first model to have the performance of the best open source Transformer model for both short and long contexts. .

StripedHyena’s Hybrid Architecture

StripedHyena incorporates a hybrid architecture that combines multi-head, grouped query attention with gate convolution within hyena blocks. This design differs from traditional decoder-only Transformer models. Represent the convolution with a state-space model or truncated filter to decode it into a persistent memory of Hyena blocks. This architecture has lower latency, faster decoding, and higher throughput compared to Transformers.

Improve training and efficiency

StripedHyena improves performance by more than 30%, 50%, and 100% over existing Transformer in end-to-end training on 32k, 64k, and 128k token sequences, respectively. In terms of memory efficiency, it reduces memory usage during autoregressive generation by over 50% compared to Transformers.

Comparative performance using attention mechanisms

StripedHyena significantly reduces the quality gap through large-scale attention, reducing computational cost and providing similar disruption and downstream performance without the need for mixed attention.​​

Applications beyond language processing

StripedHyena’s versatility extends to image recognition. The researchers tested the applicability of Visual Transformers (ViT) to attention substitution and showed similar accuracy in an image classification task on the ImageNet-1k dataset.

StripedHyena represents an important advancement in AI architecture, providing a more efficient alternative to Transformer models, especially when processing long sequences. Its hybrid structure of training and inference and improved performance make it a promising tool for a wide range of applications in language and vision processing.

Image source: Shutterstock

Share. Facebook Twitter Pinterest LinkedIn Tumblr Email

Related Posts

Rabby Wallet integrates XRPL EVM chain with peersyst

September 15, 2025

Skale Labs fairly publishes a technical white paper and introduces the industry’s first “encryption proof” consensus protocol.

September 10, 2025

Coindesk July 2025 Report: Stablecoins and CBDC

September 5, 2025
Add A Comment

Comments are closed.

Recent Posts

What is the next after the Fed’s 25bps is cut? Everything you need to know

September 18, 2025

The XRP market value surpasses Shopify, Verizon, and Citigroup. Whales sell 40m coins.

September 18, 2025

Green Hood Contracts Thanksgiving Summary -Ackee Blockchain

September 17, 2025

BetFury Is At SBC Summit Lisbon 2025: Affiliate Growth In Focus

September 17, 2025

FED Mining’s Cloud Mining Platform Is Helping Users Earn $8,800 Per Day, And XRP’s Growth Is Driving Market Enthusiasm.

September 17, 2025

Stablecoin Holdings Drop As Investors Pivot To SOL, XRP, And Altcoins

September 17, 2025

Flipster Partners With WLFI To Advance Global Stablecoin Adoption Through USD1 Integration

September 17, 2025

Zircuit Launches $495K Grants Program To Accelerate Web3 Super Apps

September 16, 2025

Kintsu Launches SHYPE On Hyperliquid

September 16, 2025

New Cryptocurrency Mutuum Finance (MUTM) Raises $15.8M As Phase 6 Reaches 40%

September 16, 2025

How XRP Enthusiasts Can Earn $15k/Day

September 16, 2025

Crypto Flexs is a Professional Cryptocurrency News Platform. Here we will provide you only interesting content, which you will like very much. We’re dedicated to providing you the best of Cryptocurrency. We hope you enjoy our Cryptocurrency News as much as we enjoy offering them to you.

Contact Us : Partner(@)Cryptoflexs.com

Top Insights

What is the next after the Fed’s 25bps is cut? Everything you need to know

September 18, 2025

The XRP market value surpasses Shopify, Verizon, and Citigroup. Whales sell 40m coins.

September 18, 2025

Green Hood Contracts Thanksgiving Summary -Ackee Blockchain

September 17, 2025
Most Popular

Ethereum Spot ETF: When will trading begin upon SEC approval?

May 25, 2024

Bitcoin decline leads to overall cryptocurrency market correction

December 20, 2024

Morgan Stanley discloses US physical Bitcoin ETF holdings worth $270 million

May 16, 2024
  • Home
  • About Us
  • Contact Us
  • Disclaimer
  • Privacy Policy
  • Terms and Conditions
© 2025 Crypto Flexs

Type above and press Enter to search. Press Esc to cancel.