The $100B AI Infrastructure Opportunity No One Is Talking About

The conversation about AI investment has been dominated by foundation models — the massive neural networks trained on vast datasets that underpin ChatGPT, Claude, Gemini, and their peers. But as foundation model training costs plateau and inference becomes the dominant workload, the next wave of value creation in AI will come not from building bigger models but from the infrastructure that makes AI economically viable at scale. This is the $100 billion opportunity that most investors are still not talking about, and it is disproportionately a European one.

Why the Foundation Model Race Is Not the Investment Thesis

The market cap gains captured by OpenAI, Anthropic, Google DeepMind, and their investors are real and extraordinary. But for venture investors deploying capital today, the foundation model layer presents a challenging investment thesis for several reasons. Training costs for frontier models now run into the hundreds of millions of dollars, placing them beyond the reach of seed and early-stage venture capital. The competitive dynamics are brutal — foundation model providers are racing to offer increasingly capable models at decreasing prices, compressing margins throughout the stack. And the intellectual property position of any foundation model company is perpetually threatened by new entrants and by open-source alternatives that are closing the capability gap faster than most observers expected.

The infrastructure layer tells a different story. As AI deployment scales from tens of millions of users to billions, as enterprise AI adoption moves from proof-of-concept to production, and as AI capabilities expand into industrial, scientific, and specialised professional domains, the demand for the infrastructure that enables cost-effective, reliable, and compliant AI deployment will grow by orders of magnitude. The companies that solve the hard engineering problems in this layer — specialised silicon, efficient inference engines, reliable orchestration platforms, and observability and compliance tooling — will capture enormous value while facing more sustainable competitive dynamics than the model providers above them in the stack.

The Inference Infrastructure Gap

The most immediate and arguably largest opportunity within AI infrastructure is inference infrastructure — the hardware, software, and systems that power AI models once they are trained and deployed. Training is a one-time cost per model version. Inference is the continuous, ongoing workload that scales with every user query, every API call, every autonomous agent action. As AI adoption grows, the ratio of inference compute to training compute will expand dramatically. Some estimates project that inference will account for 80–90% of all AI compute by 2030, up from perhaps 50–60% today.

The economics of inference are distinct from the economics of training. Training workloads are highly parallelisable and run on large clusters of identical hardware, making them amenable to the general-purpose GPU clusters that NVIDIA has come to dominate. Inference workloads are more heterogeneous — they range from latency-sensitive real-time applications requiring sub-100ms responses to batch processing workloads that prioritise throughput over latency — and they are deployed in environments ranging from hyperscale data centres to enterprise on-premise servers to edge devices with severe power and thermal constraints.

This heterogeneity creates a large surface area for specialised inference hardware companies. European companies are well positioned in this space. Graphcore, founded in Bristol, pioneered the intelligence processing unit (IPU) architecture specifically designed for machine learning inference. SiPearl, a French company, is developing high-performance AI processors for European data centre deployments. Untether AI and Hailo, while not European, illustrate the broader opportunity for specialised inference chips that is attracting significant investment globally. European semiconductor expertise, concentrated in countries like the Netherlands (ASML, NXP), Germany (Infineon, Bosch), and Switzerland (STMicroelectronics), creates the engineering talent pool and supply chain relationships needed to compete in this space.

The Efficiency Engineering Opportunity

Beyond specialised hardware, there is an enormous software opportunity in making AI models more efficient. The current generation of foundation models is extraordinarily powerful but also extraordinarily expensive to run. A single query to a large language model can cost orders of magnitude more in compute than a traditional database query. At consumer scale, this cost is manageable. At enterprise scale, deployed across thousands of internal users and hundreds of use cases, AI inference costs become a major business expense — and in some cases, a barrier to adoption.

The field of model efficiency is addressing this through techniques including quantisation (reducing the precision of model weights to decrease memory footprint and increase inference speed), pruning (removing unnecessary neural network parameters), distillation (training smaller models to mimic the behaviour of larger ones), and mixture-of-experts architectures (routing queries to specialised model components rather than activating the full parameter count for every query). Each of these techniques, applied correctly, can reduce inference costs by 2–10x while maintaining most of the quality of the original model.

European companies are making important contributions in this space. Several spinouts from ETH Zurich, EPFL, and Oxford have commercialised novel efficiency techniques. The tooling ecosystem around model compression and optimisation — libraries, deployment frameworks, benchmarking platforms — remains fragmented and is ripe for consolidation around well-engineered, enterprise-grade solutions. We view this as a particularly compelling area for seed investment because the technical barriers are high (requiring genuine expertise in numerical methods, hardware architecture, and machine learning theory) while the commercial barriers are lower than in hardware.

Orchestration, Observability, and the Enterprise Stack

As enterprises move from experimenting with AI to deploying it in production, a new category of infrastructure has emerged: the orchestration and observability tools that make AI deployments reliable, explainable, and compliant with enterprise requirements. This category encompasses model monitoring platforms, prompt engineering tools, retrieval-augmented generation (RAG) infrastructure, AI testing and evaluation frameworks, and the governance tooling that enterprise legal and compliance teams need to approve AI deployments.

The enterprise AI tooling market is growing rapidly, but it is also fragmented and, in many cases, immature. Many of the tools enterprises are using today were built by open-source communities or small startups without enterprise-grade reliability, security, or support. The opportunity for well-capitalised, professionally built companies to offer enterprise-grade versions of these tools — with proper SLAs, security certifications, audit trails, and integration with enterprise identity and access management systems — is substantial.

European companies have a structural advantage in this market because of the regulatory environment. The EU AI Act, which began partial enforcement in 2024 and will be fully in effect by 2026, creates specific compliance requirements for AI systems deployed in certain high-risk contexts. European AI tooling companies that build compliance features into their products from day one are better positioned to serve European enterprises — and, increasingly, global enterprises that operate in Europe and must comply with the EU AI Act — than US-based tools that treat compliance as an afterthought.

Edge AI and the IoT Infrastructure Layer

One of the most significant structural shifts in AI deployment over the next decade will be the migration of AI workloads from centralised cloud data centres to distributed edge devices. The drivers of this migration are compelling: latency requirements for real-time applications like autonomous vehicles, industrial robotics, and medical devices cannot be met by cloud-based inference; data sovereignty regulations in Europe and elsewhere increasingly restrict the transmission of sensitive data to remote servers; and the sheer volume of data generated by IoT devices makes centralised processing economically impractical.

Edge AI requires an entirely different infrastructure stack than cloud AI. At the edge, power consumption is constrained, connectivity is intermittent, hardware is heterogeneous, and software update mechanisms are limited. Building reliable AI systems for edge deployment is a hard engineering problem that requires expertise in embedded systems, hardware-software co-design, and low-power neural network architectures that most cloud AI infrastructure companies do not possess.

European companies have genuine competitive advantages in edge AI infrastructure, rooted in the continent's strength in industrial automation, automotive technology, and embedded systems engineering. Companies like Bosch, Siemens, and Continental have been deploying intelligent systems at the edge for decades, and the spinouts, suppliers, and talent ecosystems that have grown around these industrial anchors are producing a new generation of edge AI infrastructure companies. Germany's automotive and industrial AI ecosystem is particularly compelling — the combination of world-class embedded systems expertise, proximity to major enterprise customers, and access to European deep tech venture capital creates an environment where edge AI infrastructure companies can progress from seed stage to commercial traction faster than almost anywhere else in the world.

Why Europe Will Capture Disproportionate Value

The AI infrastructure opportunity is global, but we believe Europe will capture disproportionate value from it for three reasons. First, European semiconductor and hardware engineering expertise is world-class and undervalued as a source of competitive advantage in AI hardware. The engineers and designers who have built European leadership in automotive chips, industrial processors, and communications hardware are exactly the people needed to design the specialised AI inference chips and edge AI processors of the next generation.

Second, European regulatory frameworks are creating a compliance infrastructure market that is effectively European-first. Companies building AI compliance tooling, auditability platforms, and privacy-preserving AI infrastructure have a built-in go-to-market advantage in the EU market — the largest regulated economy in the world — before they even begin to address global markets.

Third, the enterprise AI opportunity plays to European strengths in serving large, complex, regulated businesses. European industrial companies, financial institutions, and healthcare systems are among the most sophisticated enterprise AI buyers in the world, and they have strong preferences for suppliers who understand their regulatory environments, speak their languages, and can deliver the enterprise-grade reliability they require. European AI infrastructure companies serving these customers can build durable, high-retention business models that generate returns regardless of which foundation models emerge as long-term winners.

For founders building in AI infrastructure, this is an extraordinarily fertile moment. The problems are real and urgent, the customers are ready to pay, and the technological challenges are hard enough that genuine engineering excellence creates durable competitive advantage. At Hilberts AI Capital, we are actively investing in AI infrastructure at the seed stage. Get in touch if you are building in this space.