architecture
Smaller Numbers, Faster Tokens
Quantisation is usually framed as a quality tradeoff. The more useful framing is bandwidth, because decode re-reads every weight for every token. Tags: architecture, operations
operations
architecture
Quantisation is usually framed as a quality tradeoff. The more useful framing is bandwidth, because decode re-reads every weight for every token. Tags: architecture, operations
architecture
Passthrough, time-sliced vGPU and MIG do different things to the thing that actually limits you. And fractional slices don't divide capacity the way the spreadsheet says.
architecture
Inference behaves like two workloads with opposite requirements, sharing the same silicon. Once you see that, the strange utilisation numbers start making sense.
architecture
Of all the decisions in a VCF build, tenancy is the one made worst. Not because it's hard, but because of when it gets made.
operations
You upgraded and nothing changed. Most estates land on the platform and keep operating exactly as before. Here's the sequence I'd work through instead.
architecture
VCF on EC2 bare metal inside your own VPC. Not the managed service you may be thinking of, and the networking constraints will shape your design whether you like them or not.
architecture
Run AI, containers and virtual machines on one platform, on hardware you already own. What that adds up to, and what to plan around.
architecture
A single tenant on four GPUs used 37% of the streaming multiprocessors. Four tenants used 71%, with four times the throughput. Same box, no extra hardware.
architecture
The data plane barely changes across all four stages. The management plane triples. That is the real cost curve of adoption, and it belongs in front of an operations lead early.
lifecycle
Rushing a full adoption to hit a compliance date produces a worse VCF outcome than staging it. Five stages, three stopping points, a fourth option worth naming, and why going to 9.0 first may cost you a year.
architecture
Most material on this question is written by people who already know what they want you to choose. This is the method instead — requirements first, weighted before you see a demo.
architecture
One Argo CD instance per hosting namespace, scoped to one region. It's Tech Preview, but the namespace design constraint applies now.