AI Infrastructure Moving to the Edge to Transform User Experience

The shift from building to proving value

Analysts expect hyperscalers to spend US$300 billion to US$600 billion on AI infrastructure in 2026. But as Purpose Investments’ Nicholas Mersch notes, the focus is turning “from who can build fastest to who can drive the highest revenue and margin per dollar of AI infrastructure.”

Power limits, with some data centers pushing past 1 gigawatt, and supply shortages for key components like high‑bandwidth memory, are biting. Centralized architectures also force user requests to travel long distances to distant servers, adding three to 10 times more lag than traditional web traffic.

That design breaks the experience for voice assistants or video agents, where even a one‑second pause feels wrong.

As models and chips have improved, on‑chip inference times for leading voice agents have dropped into the hundreds‑of‑milliseconds range, close to human reaction time, shifting the main source of delay to the network path between user and data center.

As PolarGrid CEO Rade Kovacevic puts it, “inference latency is the bottleneck for real-time AI at scale—whether it’s real-time voice or video solutions.”

The company is an edge‑focused player trying to attack that bottleneck; its prototype cuts network latency by more than 70 percent versus centralized hyperscalers and brings total response times toward 300 milliseconds, making it feel more like a human reply.

Why latency matters

Kovacevic compares today’s AI moment to the early commercial internet, when waiting 30 seconds for an image to load or 12 minutes to download a song on dial‑up still felt magical compared to mailing photos or driving to the mall for a CD.

As people got used to that technology, their tolerance for delay collapsed to near‑instant loads, and he expects the same pattern to play out with AI.

“Initially we’ve all been enamored with the new features and capabilities,” he explained, “but as we’ve gotten used to it, our expectations have continued to increase.”

For voice agents, that means anything more than a brief, human‑like pause starts to feel jarring and breaks trust.

In practice, that gap shows up in everyday workflows. Kovacevic points to talent‑recruitment platforms that rely on voice agents for first‑round interviews: if latency causes the bot and the candidate to talk over each other, top applicants drop off, and the whole funnel underperforms.

The same thing happens in customer service, where consumers might accept an AI agent to avoid an hour on hold, but not if responses feel slow, misheard or robotic.

Edge is the ‘neighborhood vending machine’

Sending data to a central cloud in, for example, Virginia or California and back to Canada creates a speed ceiling for real-time applications like autonomous driving, remote surgery and instant financial fraud detection.

The core idea behind edge AI is simple: instead of sending every request to a handful of giant campuses, inference runs on regional or local nodes closer to where users actually are.

Latency comparison visuals

Image via PolarGrid

Kovacevic describes it as swapping a warehouse in another state for a neighborhood vending machine, shortening the trip so results arrive fast enough to feel instant. That approach doesn’t remove the need for large, centralized training clusters, but it does change where the latency‑sensitive part of the workload runs.

For policymakers, that architectural shift intersects with a parallel push for sovereign AI. Canada’s federal government has signaled plans for large, domestically owned data solutions, while global enterprises explore regional and bare‑metal platforms to gain more control over security‑sensitive workloads. Edge networks that can keep data local while reducing latency stand to benefit from both trends.

Startups like PolarGrid are positioning themselves as the networking “plumbing” for that world: infrastructure that other AI builders plug into so their voice, video and agentic applications behave in real time without rebuilding their own global networks.

PolarGrid’s prototype: a real-world test

That gain doesn’t come from hardware so much as where it is placed: PolarGrid distributes GPUs across major population centers in North America, so requests travel shorter physical distances before being processed.

Strategically, this approach fits the broader verticalization trend in AI infrastructure, where the winners are expected to control more of the stack and squeeze more utility out of each dollar of capex.

Instead of pouring money into new data centers, PolarGrid is trying to wring better user experience and utilization from existing capacity, potentially easing power constraints and overbuild risk. Its early pilots are focused on latency‑sensitive verticals like voice agents and interactive entertainment, where any improvement in responsiveness can translate directly into higher engagement and revenue.

What investors should watch

In a year of capex digestion, plays like this could deliver the ROI hyperscalers chase: higher revenue from usable AI without endless spending.

As Mersch put it, success goes to those capturing “revenue per dollar of infrastructure.” PolarGrid shows edge might be that path, turning AI from novelty to an everyday tool. Investors eyeing efficient bets may want to take note.