Alex Medick

Explore key takeaways from AI4 2025, where TensorWave CEO Darrick Horton and industry leaders shared how to build scalable, efficient AI infrastructure for the future.

AI4 2025 Replay: Building Scalable AI Infrastructure for Long-Term Success

AI is scaling fast, and infrastructure is the make-or-break variable. At AI4 2025, a panel led by Lu Zhang brought together leaders from Amazon, PayPal, Aligned, and TensorWave to unpack what it actually takes to build and operate AI systems at scale - across hardware, software, and the increasingly loud constraint of energy. The conversation cut through hype and landed on four realities: scale determines where you run, abstractions drive adoption, hybrid architectures are the default, and the r

AI is scaling fast, and infrastructure is the make-or-break variable. At AI4 2025, a panel led by Lu Zhang brought together leaders from Amazon, PayPal, Aligned, and TensorWave to unpack what it actually takes to build and operate AI systems at scale - across hardware, software, and the increasingly loud constraint of energy. The conversation cut through hype and landed on four realities: scale determines where you run, abstractions drive adoption, hybrid architectures are the default, and the rise of agents reshapes everything from identity to observability.

TensorWave CEO Darrick Horton made a straightforward case: enterprises need real optionality, not lock-in. That means AMD-powered clusters at massive scale and a developer experience that hides the plumbing and delivers outcomes.









The Core Question: What Really Scales?

Lu Zhang (Fusion Fund) framed the discussion around cost-efficient, sustainable infrastructure that can evolve as quickly as the models do. 

Darrick opened with TensorWave’s thesis: go all-in on AMD, build large GPU clusters for training, distributed inference, and fine-tuning, and present a clean interface so customers don’t have to care about drivers or vendor quirks. If the abstraction is right, teams can ship value faster without rewriting their stack every quarter.


Cloud, On-Prem, and the Rise of Neocloud

Much of the debate focused on where workloads should live. Rajesh Sura (Amazon) advised starting from the use case, not the model. Build for reliability first, then scale. In practice, that points to hybrid architectures: keep sensitive, regulated workloads close to the data while bursting to cloud or Neocloud for GPU-hungry jobs. Interoperability becomes the unlock.

Prakhar Mehrotra (PayPal) added a future-leaning lens: as the internet fills with agentic systems, infrastructure must extend beyond inference into identity, trust, and protocol design. Highly regulated organizations may still choose on-prem for sovereignty, but the pace of R&D favors elastic capacity.

David Stanfill (Aligned) argued that data gravity and team expertise are decisive. Neoclouds can meet customers where their data lives, offer bespoke configurations, and lend engineers to get from prototype to production. Darrick drew the practical line: tens to low hundreds of GPUs can be operated on-prem by a competent team; once you cross into thousands or tens of thousands, reliable operation and TCO optimization become a specialization. That’s precisely where Neoclouds earn their keep.


Developer Experience Is the Equalizer

A recurring theme was the “buy AI” mandate many enterprises receive without a plan for production. The fix is not another dashboard; it is a disciplined developer experience. Clean, high-level interfaces, strong orchestration, and end-to-end observability let teams move from proof of concept to durable production. When abstractions are right, hardware differences fade into the background and the conversation shifts to throughput, latency, and cost per outcome.


SLMs, LLMs, and Composable Agents

The panel pushed back on the instinct to default to a single giant model. Rajesh argued for task-tuned small language models (SLMs) composed into agentic pipelines. This approach is cheaper, faster to iterate, and easier to govern. Prakhar went further: the web will soon host more agents than humans. That reality demands a new “agency layer” with identity, trust, tool-use standards, and communication protocols for agent-to-agent work. Designing this layer is as important as adding more GPUs.


Energy: Constraint and Catalyst

Energy dominated the second half of the conversation. David framed demand growth as a forcing function: sustained AI load finally justifies grid upgrades that were hard to pencil out for volatile consumer demand. In that sense, AI is a catalyst for national infrastructure investment.

Darrick agreed on the opportunity but underscored timing. Today there is stranded power to tap; tomorrow, scarcity will bite. When “add more power” stops working, necessity will drive foundational changes in cluster design and model efficiency. The gap between the human brain’s efficiency and today’s best models makes one thing obvious: there is a massive optimization frontier ahead, and new architectures will arrive in waves.


The TensorWave POV

Darrick’s position throughout was consistent:

First, freedom of choice matters. A single-vendor AI world is fragile. 

Second, the hard part is operating at scale: thousands to tens of thousands of GPUs, tuned for throughput, reliability, and TCO, is not a weekend project. 

Third, experience beats specs. If customers can interact with a simple, portable interface, they will not care which driver or card family is underneath. Finally, the energy curve will reward builders who prioritize efficiency-per-token and latency-aware design. We are early in a long S-curve.


Practical Guidance

For enterprise leaders: Start with one sharp use case and the decisions you want AI to make. Assume a hybrid end state. Invest in interoperability and observability from day one so you can shift workloads as policy, cost, and performance change.

For platform and ML teams: Prefer composable SLMs and agents over one mega-model. Treat identity and trust as first-class features of agent systems. If your footprint approaches a four-digit GPU count, talk to a Neocloud before DIY operations burn your roadmap.

For founders: Optimize for abstraction and outcomes. If customers need an internal enablement tour to deploy you, your TAM shrinks. As energy constraints tighten, buyers will reward efficiency and predictability - not just benchmarks.


Memorable Lines

“Scale decides venue. Tens of GPUs can live on-prem; thousands require specialists.” — Darrick Horton, TensorWave

“Hybrid is the default. Interoperability is the unlock.” — Rajesh Sura, AWS

“Agency infrastructure - identity, trust, protocols - will matter as much as GPUs.” — Prakhar Mehrotra, PayPal

“AI demand is a grid upgrade opportunity, not just a cost.” — David Stanfill, Aligned


Closing Thoughts

The panel landed on a pragmatic truth: AI doesn’t scale on vibes. It scales on optional silicon, clean interfaces, hybrid architectures, and an energy strategy that keeps up. Optionality beats lock-in, abstraction beats vendor trivia, and efficiency beats peak FPS. The winners will be teams that can move quickly on stable foundations, then keep moving as constraints change.

Want to test on AMD Instinct? Run training, distributed inference, and fine-tuning on TensorWave’s AMD-only cloud all without rewrites and with real scale. 

If you’re staring down a four-figure GPU footprint and want clean economics plus zero-drama ops, we should talk.


About TensorWave

TensorWave is the AI neocloud purpose-built for performance. Powered exclusively by AMD Instinct™ Series GPUs, we deliver high-bandwidth, memory-optimized infrastructure that scales with your most demanding models—training or inference.

Ready to get started? Connect with a Sales Engineer.

AMD Instinct™ MI355X GPUs Now Available

AI4 2025 Replay: Building Scalable AI Infrastructure for Long-Term Success

Subscribe to our Blog

Stay ahead of the curve with the latest in AI, AMD accelerators, and all things TensorWave.

Product

Resources

Company

© 2025 TensorWave Inc. - All rights reserved.