Crypto Flexs
  • DIRECTORY
  • CRYPTO
    • ETHEREUM
    • BITCOIN
    • ALTCOIN
  • BLOCKCHAIN
  • EXCHANGE
  • TRADING
  • SUBMIT
Crypto Flexs
  • DIRECTORY
  • CRYPTO
    • ETHEREUM
    • BITCOIN
    • ALTCOIN
  • BLOCKCHAIN
  • EXCHANGE
  • TRADING
  • SUBMIT
Crypto Flexs
Home»BLOCKCHAIN NEWS»StripedHyena-7B: Next-generation AI architecture for improved performance and efficiency
BLOCKCHAIN NEWS

StripedHyena-7B: Next-generation AI architecture for improved performance and efficiency

By Crypto FlexsJanuary 4, 20242 Mins Read
Facebook Twitter Pinterest LinkedIn Tumblr Email
StripedHyena-7B: Next-generation AI architecture for improved performance and efficiency
Share
Facebook Twitter LinkedIn Pinterest Email

Recent advances in AI have been greatly influenced by the Transformer architecture, a key component of large models across fields as diverse as language, vision, audio, and biology. However, the complexity of Transformer’s attention mechanism limits its application in processing long sequences. Even sophisticated models such as GPT-4 suffer from this limitation.

Breakthrough Advances with StripedHyena

To address these issues, Together Research recently open sourced StripedHyena, a language model that boasts a new architecture optimized for long contexts. StripedHyena can handle up to 128,000 tokens and has demonstrated improved performance over the Transformer architecture in both training and inference performance.​​ It is the first model to have the performance of the best open source Transformer model for both short and long contexts. .

StripedHyena’s Hybrid Architecture

StripedHyena incorporates a hybrid architecture that combines multi-head, grouped query attention with gate convolution within hyena blocks. This design differs from traditional decoder-only Transformer models. Represent the convolution with a state-space model or truncated filter to decode it into a persistent memory of Hyena blocks. This architecture has lower latency, faster decoding, and higher throughput compared to Transformers.

Improve training and efficiency

StripedHyena improves performance by more than 30%, 50%, and 100% over existing Transformer in end-to-end training on 32k, 64k, and 128k token sequences, respectively. In terms of memory efficiency, it reduces memory usage during autoregressive generation by over 50% compared to Transformers.

Comparative performance using attention mechanisms

StripedHyena significantly reduces the quality gap through large-scale attention, reducing computational cost and providing similar disruption and downstream performance without the need for mixed attention.​​

Applications beyond language processing

StripedHyena’s versatility extends to image recognition. The researchers tested the applicability of Visual Transformers (ViT) to attention substitution and showed similar accuracy in an image classification task on the ImageNet-1k dataset.

StripedHyena represents an important advancement in AI architecture, providing a more efficient alternative to Transformer models, especially when processing long sequences. Its hybrid structure of training and inference and improved performance make it a promising tool for a wide range of applications in language and vision processing.

Image source: Shutterstock

Share. Facebook Twitter Pinterest LinkedIn Tumblr Email

Related Posts

Korea’s largest bank provides cross-border payment services to Kinexys

July 27, 2026

L Bank celebrates Argentina’s World Cup journey with a $100,000 global campaign

July 22, 2026

Nvidia’s RoboLab addresses key challenges in robot policy evaluation.

July 12, 2026
Add A Comment

Comments are closed.

Recent Posts

BC Game Launches 100 Million $BC Token Bonus Campaign Through Its Live-Market Crashx Game

November 20, 2026

How endpoint monitoring protects the people behind a crypto project

November 20, 2026

Bybit’s Global User Base Surpasses 90 Million as Company Expands Into Traditional Finance

November 20, 2026

Bybit Joins Proof of Talk in Abu Dhabi 2026 as Title Partner and Official Exchange Partner

November 19, 2026

Hashi Mainnet is Live With 20-Plus Partners, Bringing Bitcoin Finance to Sui at Scale

November 19, 2026

MEXC’s Hacken-Audited November 2026 Proof of Reserves Shows Reserve Ratios Above 100% Across Major Assets

November 19, 2026

My Wallet Introduces 1:1 Cross-Chain Stablecoin Swaps After Being Listed First on WalletConnect

November 19, 2026

The Promo Gaming -Discover Great Sportsbook & Casino Bonuses from Reliable Online Gaming Platforms

November 18, 2026

How to Read Online Gambling Statistics Without Being Misled

November 18, 2026

Tria and Noah Bring Bank Transfers Directly to Self-Custodial Wallets With Automatic EUR-to-USDC Conversion

November 18, 2026

Ondo Stocks to Launch ANTon with 24/7 Mint and Redeem, Live on IPO Day

November 18, 2026

Crypto Flexs is a Professional Cryptocurrency News Platform. Here we will provide you only interesting content, which you will like very much. We’re dedicated to providing you the best of Cryptocurrency. We hope you enjoy our Cryptocurrency News as much as we enjoy offering them to you.

Contact Us : Partner(@)Cryptoflexs.com

Top Insights

BC Game Launches 100 Million $BC Token Bonus Campaign Through Its Live-Market Crashx Game

November 20, 2026

How endpoint monitoring protects the people behind a crypto project

November 20, 2026

Bybit’s Global User Base Surpasses 90 Million as Company Expands Into Traditional Finance

November 20, 2026
Most Popular

Lucky Crypto Trader sees a whopping 86,000% increase in the value of its Ethereum Memecoin investment: on-chain data

May 10, 2024

Bitcoin vs. Ethereum: Expert Predicts Doom for ETH Price

June 7, 2024

How blockchain is changing Christmas gifts

December 9, 2023
  • Home
  • About Us
  • Contact Us
  • Disclaimer
  • Privacy Policy
  • Terms and Conditions
© 2026 Crypto Flexs

Type above and press Enter to search. Press Esc to cancel.