Exclusive: Snowflake and Reka add multimodal LLMs to data cloud

Snowflake Data Cloud is getting multimodal large language models (LLMs). Today, the Sridhar Ramaswamy-led company announced it is partnering with Reka, an AI startup founded by researchers from DeepMind, Google and Meta, to bring its proprietary models into its data platform. 

The move comes mere weeks after the company’s partnership with Mistral and will enable enterprises using its data cloud to build generative AI applications that will work with not only text inputs but also images and videos. This will open a whole new way for teams to drive value from datasets. 

Snowflake participated in Reka’s $60 million round of funding last year. However, it did not confirm if it is increasing its investment with this partnership.

Baris Gultekin, the head of product management at Snowflake AI, told VentureBeat the company is “always exploring ways” to support partners and drive innovation for customers but has nothing more to share on the investment front at this stage.

Reka Flash and Core coming to Snowflake Cortex

Since its inception, Snowflake has focused on building the data infrastructure of choice for its customers.

However, the vision has not been easy to achieve. The company initially provided users with a simple data warehouse but as the ecosystem evolved it added support for more data formats and capabilities to build a full-fledged data cloud — enabling a wide range of downstream use cases, including AI and analytics.

Last year, as the generative AI wave took the world by storm, it took another step to continue this evolution with the launch of Snowflake Cortex, a fully managed service tailored for building LLM apps.

Cortex provides enterprises using data cloud with a suite of AI building blocks, including open-source LLMs, to analyze data they have on the platform – with the same security, privacy and governance of Snowflake – and build applications targeting different business-specific use cases. The company started with LLMs for specialized tasks such as sentiment analysis and is now going multimodal with the inclusion of two LLMs trained by Reka: Flash and Core.

Reka Flash is a state-of-the-art 21-billion parameter model that serves as an optimized “turbo class” offering and matches the performance of many larger models across various language and vision benchmarks. The Core model, on the other hand, is said to be Reka’s largest and most performant model that approaches GPT-4 and Gemini Ultra in terms of performance. However, unlike the Flash model, the Core version is not available publicly at this stage. 

Snowflake said it is adding the Flash model to Cortex right away and developing support for the bigger one to make sure it becomes available right after launch.

Gultekin did not share a timeline but said it would drop soon. He also noted that the company may also add other models from Reka, including the smaller Edge model, based on the demand. 

How will multimodal AI benefit Snowflake users?

With Cortex and Reka’s AI models, users will be able to build generative AI applications that will work with not only text inputs but also images and video files.

This can come in handy across various use cases, including solutions for captioning videos, labeling images, generating e-commerce product descriptions, and answering analytical questions based on charts.

“Some example applications our customers are excited about are chatbots that can understand charts, or entertainment businesses generating marketing or advertising content given their video or image assets,” Gultekin told VentureBeat. 

While the AI product head did not share how many companies are looking to use or using Reka models, he did share that over 400 enterprises are leveraging Cortex and the models it hosts to build gen AI apps in the data cloud.

This includes applications targeting domains, right from analyzing security vulnerabilities from service tickets to chatbots that help healthcare providers better interface with insurer data. 

The addition of Reka’s models will take the total number of LLMs on the service to a dozen, he said. This includes models from Mistral and Google, which were introduced just a few weeks ago.

“Our AI innovation pipeline is in overdrive. At its core, Snowflake’s vision is to get AI into the hands of every user, regardless of their technical expertise, to deliver recognizable business value faster. By furthering our partnership with Reka, and through integrations with Gemma and Mistral, and with what we have coming in the next weeks and months, we’re doing just that,” Gultekin said while hinting the company will announce more AI innovations in the run-up to its annual summit in June.

However, he did not share specifics of what is in the pipeline.

“Our roadmap stems from the foundational principle that there is no AI strategy without a data strategy – Data is the fuel for AI. We’re leaning into that and enhancing our ability to boost productivity, increase collaboration, and ultimately accelerate end-to-end AI and ML workflows for our customers, all with Snowflake’s secure and trusted data foundation,” Gultekin added. 

Notably, Databricks, which competes with Snowflake in the data ecosystem, has also had the same play. The company has been introducing open models since acquiring MosaicML last year while also bolstering its tooling to build gen AI apps at the same time. In fact, just a few days back, it acquired Lilac, an AI startup that helps teams analyze and fix unstructured data for AI training.