Google's TPU 8tx Brings Confidential Computing to TPU Inference

Enterprise AI deployments typically protect data in two places: at rest in storage, and in transit over the network. The third place is weaker. In many deployments, data and model weights are decrypted while in use — while a model is running, and while an agent built on it reads records, calls tools and produces output. On November 9, Google announced two new systems aimed at that gap and at a second bottleneck, memory for very long context, per Introducing TPU 8tx and TPU 8i Max. TPU 8tx is built for secure and confidential AI inference. TPU 8i Max is built for massive context windows. Neither is a new chip: TPU 8tx reuses TPU 8t silicon, and TPU 8i Max packages two TPU 8i chips together — a distinction worth keeping, because it shapes what is proven and what is still a claim.

What shipped on November 9

Google expanded its eighth-generation TPU portfolio to four systems in a single year, which it says is a first, per the Google announcement linked above. The first two, introduced in April, were TPU 8t for large-scale training and TPU 8i for real-time reasoning. Google says those offer up to 2.7 times better training performance and 80% better inference price performance respectively versus the previous generation — figures from the April launch, not new measurements of the systems announced this week.

The two additions complete the set. Coverage summarises the split plainly: TPU 8t for scale-up training, TPU 8i for low-latency reasoning, TPU 8tx for confidential computing, and TPU 8i Max for massive context windows and the highest compute density per rack, per Google advances agentic AI custom silicon expanding TPU family with 8tx and 8i Max - SiliconANGLE. Independent reporting makes the same point about what these are: the variants are not new chips, but reconfigured versions designed for workloads customers have been asking for, per Google to introduce two variants of in-house chips, Bloomberg News reports. Google frames that as a feature, not a limitation — systems can be repackaged around existing silicon in months, by changing packaging, networking, memory and security, rather than waiting for a new chip design.

Google also says TPUs now serve nearly 100 customers, including frontier labs, trading firms, retailers and universities — none named, so treat that as a scale signal, not a checkable reference list.

TPU 8tx: protection while the model is running

TPU 8tx is optimised for secure and confidential inference and is also suited to small-scale training. Its stated audience is regulated enterprises and governments running frontier or proprietary models and autonomous agents alongside sensitive data, per the Google announcement linked above. It is built on the same silicon as TPU 8t and paired with 5th Generation AMD EPYC CPU hosts, with security built into the hardware layer so data and model weights stay protected and isolated even while in active use.

Google claims TPU 8tx delivers up to 2x lower latency compared to TPU 8t, with a near-zero performance penalty from the security layer, helped by adopting the low-latency Boardfly network created for real-time reasoning. No absolute latency or throughput figures were published, so that claim cannot yet be converted into a capacity plan. The second claim matters just as much operationally: because the same system pairs training-grade compute with fast inference, teams can fine-tune models on private data, distill large models into smaller ones, and train agents through reinforcement learning without migrating sensitive data between separate clusters, per Google adds TPU 8tx for confidential AI and a two-chip TPU 8i Max, its third and fourth TPU systems this year. Every migration you remove is one less copy of sensitive data to govern, log and eventually delete.

What Google has not published is equally specific: no attestation design a security architect can review, no independent audit, and no availability date or pricing in the announcement itself. Confidential inference is verifiable only when the verification mechanism is documented. Until it is, this is a direction of travel, not a control you can put in an audit file. It is also not a category Google is creating: confidential computing is already offered on other accelerator platforms, a point the MLQ coverage linked below makes in comparing this launch with existing GPU options — so the right question is comparative, not whether the idea is new.

TPU 8i Max: attacking the memory wall, not the model

TPU 8i Max takes the opposite problem. When many agents work together across a large codebase or hours of video, memory bandwidth becomes the bottleneck before raw compute does. Google combined two TPU 8i chips onto a single package, delivering double the compute, double the SRAM and double the HBM bandwidth of TPU 8i, while fitting twice the silicon within the existing rack footprint, per the Google announcement linked above. That raises compute density per rack and, Google says, per watt.

The practical target is long-context work: agents that hold a large working set in memory instead of repeatedly fetching it. Doubling memory bandwidth alongside compute is the right ratio for that job — adding compute alone leaves processors waiting on data. Whether it changes your cost per task depends on pricing Google has not disclosed, and on whether your workloads are actually memory-bound. Many are not.

The wider software announcements point the same way, and are usable sooner. Google open-sourced TorchTPU so PyTorch code and Hugging Face models run natively on TPUs, and open-sourced TPU Sync, a model-state layer that tiers large KV caches across chip memory, host memory, storage and the network. Its Agent Compute Engine automatically suspends and resumes idle agent sessions, cutting wasted compute while agents wait.

What this means for your business

1. Add "data in use" to your AI security review, explicitly — regulated teams and multi-tenant service providers first. If your threat model only covers encryption at rest and in transit, agents have widened the gap: they hold decrypted context, tool outputs and model weights in memory for long, multi-step runs. Ask every inference provider how data in use is isolated, what attestation is available, and what evidence an auditor can inspect. Accept a documented mechanism, not the word "confidential" on a product page.

2. Do not plan capacity on undisclosed pricing or dates. Google gave no availability date, pricing or deployment details for either new system in its announcement. Put TPU 8tx and TPU 8i Max on a watch list, not in a committed architecture, a client proposal or a cost forecast. If a vendor quotes you savings based on either system today, ask which published figure the quote rests on.

3. Check whether you are memory-bound before chasing long-context hardware. Long-context capacity only pays when your agents genuinely reason over large working sets — large codebases, long document sets, extended video. Measure where time goes first: memory bandwidth, model latency, tool calls, or waiting on people. Buy for the bottleneck you measured.

4. Use the software layer now, regardless of hardware. Native PyTorch support, standard serving stacks, and suspend-and-resume for idle agent sessions address costs most teams are already paying. Idle agents waiting on approvals or external systems are a pure waste line in many deployments. Auditing that waste does not need new silicon and should not wait for it.

5. Price the lock-in before the performance. These are provider-specific systems, not portable hardware. If confidential inference becomes a requirement in your stack, ask how workloads, fine-tuned models and agent tooling move if pricing or availability changes — before the workload depends on them, not after.

6. Keep the reconfiguration lesson for your own stack. Google shipped two new products by repackaging proven silicon with different networking, memory and security. The same discipline applies at application scale: most automation wins come from recombining proven models, tools and controls for a specific workflow, not from waiting for a new model generation to arrive.

The clear signal in this launch is where the constraints have moved. The hard problems in production AI are now protection while data is in use, memory for long-context agents, and compute wasted while agents wait — not raw model capability alone. Google has named hardware for the first two and software for the third. The claims worth acting on today are the ones you can test on your own workloads; the rest belongs on a dated watch list until pricing, availability and attestation detail are published.

Ready to find where automation would cut cost or cycle time in your operation? Get a free AI audit — we'll map your highest-leverage automation opportunities in one session.