Google Labs has announced a major upgrade to its Gemini Pro AI tool, a mid-sized AI model that powers chatbots in the free version, introducing the ability to process up to 1 million tokens in preview. This provides unprecedented “context size” while still maintaining the best tools available today and 128K capacity.
The upgrade to Gemini Pro v1.5 is theoretically 700% more powerful than OpenAI’s paid GPT-4 model and sets a new benchmark for computational linguistics and machine learning among large language models (LLMs).
This figure is “the longest context window of any large fundamental model.” According to Google.
“Before today, the world’s largest publicly available context window for a large-scale language model was 200,000 tokens. We were able to significantly increase this by running up to 1 million tokens continuously,” the Google Labs team shared.
This feature will allow Gemini Pro to outperform the most powerful version of the current Gemini lineup and any other LLM currently available. However, this context has been brought online for testing purposes, while the upcoming stable version of Gemini Pro will handle up to 128K tokens.
The release will be a big upgrade over the 32,000 tokens that Gemini 1.0 can handle, but users will have to wait and see what 1 million tokens can do.
This move is Google’s latest offensive in its race to dominate the AI industry. Last week, Gemini Advanced became the first credible competitor to ChatGPT Plus. Unlike Anthropic’s Claude, Google’s chatbot is multimodal, performs well in a variety of tests, and offers a set of features that OpenAI doesn’t.
However, Gemini Advanced will catch up with GPT-4.5 Turbo, which already handles 128,000 tokens.
The versatility of Gemini 1.5 was vividly demonstrated through several demonstrations. Google says it can “process massive amounts of information at once, including one hour of video, 11 hours of audio, more than 30,000 lines of code, or a codebase of more than 700,000 words.”
“We have successfully tested up to 10 million tokens in our research.” Team added.
One drawback: the Gemini model cannot analyze PDF files. decryption This was pointed out in the comparison between Gemini and ChatGPT.
A ‘mix of experts’ is here
Another difference between Gemini 1.5 and previous versions is that it uses Mixture of Experts, the same technology that Mistral AI used to create lighter models. Mistral’s entrant was strong enough to beat GPT 3.5 and leap into the upper tier of the best open source LLMs.
“(Mix of Experts) routes your request to a smaller group of ‘expert’ neural networks, resulting in faster and higher quality responses,” Google said in the announcement, noting that this not only results in faster responses, but also higher quality. .
Like Mistral, Google has also been able to make its model shine. Gemini 1.5 Pro performed well in several benchmarks compared to Gemini Ultra 1.0, suggesting a promising future for Google LLM.
“Showing dramatic improvements across multiple dimensions, 1.5 Pro achieves similar quality to 1.0 Ultra while using less compute,” Google CEO Sundar Pichai said in a blog post today.
The announcement did not provide a release date for Gemini Advanced 1.5. Meanwhile, OpenAI is actively developing GPT-5. Gemini’s improved token handling capabilities will help strengthen Google’s position in the AI arms race.
editor Ryan Ozawa.