The Imperative of AI Sovereignty: Insights from Cohere at VB Transform 2026

As enterprises increasingly rely on artificial intelligence (AI) to drive their operations, the concept of AI sovereignty has emerged as a critical consideration for organizations. At VB Transform 2026, held at the opulent Hotel Nia in Menlo Park, attendees gathered to explore how generative AI agents can enhance business outcomes while maintaining robust control over sensitive data. Rachad Alao, Vice President of Product Engineering at Cohere, highlighted the imperative of AI sovereignty during a fireside chat with Matt Marshall, CEO of VentureBeat. Alao's insights underscore a growing recognition that true control over AI systems extends well beyond merely deploying models behind corporate firewalls.

Alao's perspective on AI sovereignty revolves around the need for organizations to have comprehensive oversight of their entire AI stack, particularly for mission-critical operations such as those in banking, healthcare, and government. The conversation brought to light essential considerations for enterprise leaders navigating the complexities of AI implementation.

Defining AI Sovereignty in the Enterprise Context

At its core, AI sovereignty refers to an organization’s ability to control the AI systems it employs, including where data is stored, how AI models are utilized, and the governance structures that dictate these processes. According to Alao, the control over AI should encompass everything from the underlying hardware, such as GPUs and private-cloud infrastructure, to the governance systems that manage data requests and model interactions.

This comprehensive approach to AI sovereignty is particularly vital for organizations that handle sensitive information. Alao pointed to banks and hospitals as examples of industries where data privacy and security are non-negotiable. “It is important to have very tight control on where the data resides, and have tight control on the AI,” he emphasized. This control is not merely about compliance with regulations but about ensuring the integrity and confidentiality of the data itself.

The Role of Infrastructure in AI Sovereignty

To achieve AI sovereignty, organizations must consider several infrastructural elements:

• Data Residency: Ensuring that sensitive data remains within specific jurisdictions that comply with legal requirements.
• Model Operations: Maintaining operational control over AI models to prevent unauthorized access and ensure that usage aligns with organizational policies.
• Vendor Flexibility: Avoiding vendor lock-in by implementing governance systems that allow for the easy switching of providers without losing control over data and processes.

Economic Considerations: Balancing Costs and Token Utilization

During the discussion, Matt Marshall raised an important point regarding the economic implications of AI deployment. As AI models evolve, the costs associated with inference—essentially the processing required to generate outputs from models—are decreasing. This trend could potentially undermine the rationale for optimizing every token used in AI operations. However, Alao countered this argument by highlighting a significant trend: as enterprises adopt more complex AI applications, their total token consumption is skyrocketing.

“Your token utilization is going exponentially up because you’re dealing with more and more complex agentic use cases,” Alao explained. This complexity arises from the need for AI systems to not only respond to queries but also to reason through issues, interact with various tools, and perform multiple steps to deliver comprehensive answers. The challenge for enterprises will be to manage these increasing demands while keeping costs in check.

Token Utilization and Cost Structures

Alao further elaborated on Cohere’s unique pricing model, which diverges from the traditional token consumption-based approaches common among competitors. While many providers incentivize maximizing token usage, Cohere focuses on helping enterprises address their most challenging problems without unnecessary resource consumption. “Use the right model for the task at hand,” he advised, advocating for a strategic routing of tasks based on their complexity and sensitivity. This not only optimizes resource use but also enhances security.

Leveraging Smaller Models for Efficiency

Alao also addressed the competitive landscape surrounding AI models, particularly in reference to Cohere’s recent release of North Mini Code, an open-source model designed for agentic software engineering. While larger frontier models may excel at tackling complex tasks, Alao noted that they may not always justify their use. “For 80% of the use cases that they needed, this was a lot more effective, a lot cheaper,” he remarked, highlighting the practicality of smaller models in everyday applications.

Cohere’s North Mini Code operates efficiently on a single Nvidia H100 GPU and is tailored for tasks such as terminal work, code reviews, and tool usage. Additionally, the company has introduced Command A+, a mixture-of-experts model that optimizes performance by activating only a portion of its parameters during each generation step. This innovative design reduces the computational resources required for private deployments, making it an attractive option for enterprises seeking to balance performance with cost.

Integrating Search and AI Workflows

AI is not just limited to processing data; it increasingly includes sophisticated search capabilities. Alao emphasized the transition from traditional text-based search to multimodal search, which integrates various data forms—text, images, and more—into a cohesive workflow. “Today, the state of the art is around multimodal search,” he stated, underscoring the importance of embedding search functionality into agentic workflows.

This integration allows AI systems to determine when and how to utilize search tools effectively, making them an essential component of the overall agentic process. As enterprises continue to evolve their AI capabilities, the ability to perform dynamic searches across diverse data types will become increasingly crucial.

Overcoming Vendor Lock-In with Governance Layers

One of the pressing concerns voiced by many enterprises is the risk of vendor lock-in when relying on bundled AI services from established cloud providers. Alao highlighted how Cohere is addressing this concern through its governance layer, which enables customers to route traffic to the most appropriate models based on their specific needs. This capability is instrumental in enhancing AI sovereignty, as it allows organizations to maintain control over their data and the models they use.

“If you’re interested in sovereignty, you want to have more control on your data,” Alao stated, reinforcing the idea that organizations must prioritize data governance to navigate the complexities of AI deployment effectively. By breaking free from vendor lock-in, enterprises can leverage the best tools available while safeguarding their operational integrity.

Key Takeaways

• AI sovereignty is critical for organizations handling sensitive data, requiring control over data residency and model operations.
• As enterprises adopt more complex AI applications, the demand for token utilization is increasing, necessitating strategic resource management.
• Cohere’s unique pricing model emphasizes solving problems efficiently rather than maximizing token consumption.
• Smaller models can be more effective and cost-efficient for the majority of enterprise use cases.
• Integrating search capabilities into AI workflows enhances the efficiency and effectiveness of AI systems.

Frequently Asked Questions

What is AI sovereignty, and why is it important?

AI sovereignty refers to an organization’s ability to control its AI systems, including data management, model operations, and governance. It is crucial because it ensures that sensitive data remains secure and compliant with regulations, especially for industries like finance and healthcare.

How can enterprises balance the costs associated with AI deployment?

To balance costs, enterprises should strategically manage their AI resource utilization. This includes using smaller models for less complex tasks, optimizing the routing of requests based on sensitivity, and employing governance structures to prevent unnecessary token consumption.

What role does search play in AI workflows?

Search is becoming an integral part of AI workflows, moving beyond simple text retrieval to multimodal search capabilities that integrate various data types. This evolution allows AI systems to enhance their decision-making processes and deliver more comprehensive outputs.

How can companies avoid vendor lock-in with AI solutions?

To avoid vendor lock-in, companies should implement governance layers that allow them to route traffic to different AI models based on their specific needs. This flexibility enables organizations to maintain control over their data and choose the best tools for their operations without being tied to a single provider.