Overview · AMD Instinct MI300X

Large models need memory first. MI300X starts with 192 GB.

Most LLM projects are limited by GPU memory before they are limited by compute. When the weights and the KV cache fit on fewer GPUs, you split the model less, move less data between cards and serve more users per node.

The AMD Instinct MI300X is AMD's data-centre accelerator for generative AI and HPC. Per AMD, each GPU carries 192 GB of HBM3 at up to 5.3 TB/s peak bandwidth, with 304 compute units on the CDNA 3 architecture. Eight of them sit on one baseboard, linked by AMD Infinity Fabric, for 1.5 TB of GPU memory in a single node.

That much memory changes the maths for large models. A 70-billion-parameter model at FP16 needs about 140 GB for its weights. That fits on one MI300X, where an 80 GB card needs two. Fewer GPUs per model means simpler serving and more headroom for long prompts.

XcellHost offers MI300X capacity as a managed rental. We confirm availability, size the node with you, install ROCm and your frameworks, and run it 24×7 from our Mumbai NOC and SOC. You pay one monthly invoice in rupees with GST. MI300X is sourced on request, so availability and price are confirmed in your quotation.

At a glance per AMD
0GB HBM3 per GPU
0TB/s peak bandwidth
0compute units
0MB Infinity Cache
0W max board power
0TB HBM3 per 8-GPU node
Instance options

One GPU to a full 8-GPU node. Every option priced on quote.

Pick the GPU count your model needs, then choose a term. Longer reservations usually lower the monthly rate. Your quotation states the exact rate, CPU, RAM, storage and network for the configuration we can confirm.

Term
1× MI300X

Single GPU

GPU memory
192 GB HBM3
FP16 / BF16 dense
1,307 TFLOPS
GPU-to-GPU
n/a (one GPU)
CPU · RAM · NVMe
sized in quote

Good for 7–70B inference at FP16, LoRA on mid-size models, development.

Price on quoteOn-demand · monthlyReserve →
2× MI300X

Dual GPU

GPU memory
384 GB HBM3
FP16 / BF16 dense
2,615 TFLOPS
GPU-to-GPU
Infinity Fabric
CPU · RAM · NVMe
sized in quote

Good for 70B at FP16 with long context, 100B+ models at FP8.

Price on quoteOn-demand · monthlyReserve →
4× MI300X

Half node

GPU memory
768 GB HBM3
FP16 / BF16 dense
5,230 TFLOPS
GPU-to-GPU
Infinity Fabric
CPU · RAM · NVMe
sized in quote

Good for 180B-class serving, full fine-tuning of 13–30B models.

Price on quoteOn-demand · monthlyReserve →
Full node 8× MI300X

8-GPU node

GPU memory
1.5 TB HBM3
FP16 / BF16 dense
10,459 TFLOPS
GPU-to-GPU
full Infinity Fabric mesh
CPU · RAM · NVMe
sized in quote

Good for 405B-class inference, distributed training, HPC jobs.

Price on quoteOn-demand · monthlyReserve →

GPU figures per AMD (peak theoretical, dense; totals are per-GPU figures × GPU count). Real performance depends on model, precision, batch size and software. Multi-node clusters (16+ GPUs) are designed and quoted separately. *MI300X availability, delivery date and all prices are confirmed at quote; GST at 18% is extra.

Ready to run large-memory AI workloads?Tell us the model and the users. We confirm MI300X availability and send a rupee quotation.
Get started →
Features & benefits

Five reasons teams choose MI300X. All from AMD's published design.

Memory, bandwidth, chiplets, open software and multi-GPU scaling. Each card shows what the feature does for your workload.

01 / 05
01

192 GB HBM3 per GPU

About 2.4 times the 80 GB of an NVIDIA H100. Larger models and longer prompts fit on one card, with less splitting across GPUs.

  • 70B at FP16 fits on one GPU
  • Room left for KV cache
02

5.3 TB/s memory bandwidth

Token generation is mostly a memory read. Higher bandwidth keeps 304 compute units and 1,216 matrix cores fed during inference.

  • Faster token generation
  • 304 CUs · 1,216 matrix cores
03

Chiplet architecture

Eight compute dies are stacked on four I/O dies and share 256 MB of AMD Infinity Cache. The design packs more compute and memory into one package.

  • 8 compute dies · 4 I/O dies
  • 256 MB Infinity Cache
04

Open software with ROCm

AMD's open-source ROCm platform runs PyTorch, TensorFlow, JAX and ONNX Runtime, plus serving engines such as vLLM. Many models run with little or no code change.

  • Open-source drivers & libraries
  • HIP for porting CUDA code
05

Multi-GPU scaling

Each GPU has seven Infinity Fabric links at 128 GB/s, one to each other GPU on the baseboard. An 8-GPU node pools 1.5 TB of HBM3 for one large job.

  • 7 × 128 GB/s links per GPU
  • 1.5 TB per 8-GPU node

Specifications per AMD. H100 memory per NVIDIA. Software compatibility varies by model and version; we test your stack before go-live.

Technical specifications

The MI300X datasheet, grouped the way engineers read it.

AMD's published peak theoretical figures. Sparsity figures are shown where AMD lists them. Pick a group to see its rows.

Architecture
AMD CDNA 3
Process
5 nm and 6 nm FinFET
Form factor
OAM module
Compute units
304
Matrix cores
1,216
Stream processors
19,456
Peak engine clock
2,100 MHz
Virtualisation
SR-IOV, up to 8 partitions
Memory
192 GB HBM3
Peak bandwidth
5.3 TB/s
Memory interface
8,192-bit
Memory clock
up to 5.2 GT/s
Last-level cache
256 MB AMD Infinity Cache
ECC
full-chip ECC, page retirement
8-GPU node
1.5 TB HBM3 in total
PrecisionDenseWith sparsity
FP64 vector81.7 TFLOPS–
FP64 matrix163.4 TFLOPS–
FP32 vector / matrix163.4 TFLOPS–
TF32653.7 TFLOPS1,307.4 TFLOPS
FP16 / BF161,307.4 TFLOPS2,614.9 TFLOPS
FP82,614.9 TFLOPS5,229.8 TFLOPS
INT82,614.9 TOPS5,229.8 TOPS
Scale-up links
7 × AMD Infinity Fabric, 128 GB/s each
Host interface
PCIe Gen 5 x16 (128 GB/s)
Scale-out
via PCIe Gen 5 x16 to network cards
Max board power
750 W TBP
Platform
8 GPUs on an AMD Universal Base Board
Cooling
per server platform (quoted)
Platform
AMD ROCm (open source)
Programming
HIP, OpenMP, OpenCL
Frameworks
PyTorch, TensorFlow, JAX, ONNX Runtime
Serving
vLLM and other ROCm-enabled engines
Libraries
rocBLAS, hipBLASLt, MIOpen, RCCL
Media
HEVC, AVC, AV1 decoders

Per AMD's MI300X data sheet. Peak theoretical figures, not measured performance. XcellHost confirms the server platform, CPU, RAM, storage and network in your quotation.

Open software

From GPU to your app, every layer is open.

ROCm is AMD's open-source software platform. XcellHost installs and pins a tested ROCm version, the libraries your model uses and the framework you work in, so you start from a working node. Tap a layer.

Real-world use cases

Where 192 GB per GPU makes the biggest difference.

Six workload types that benefit most from large GPU memory and an open stack. Each is sized against your own model and traffic in the quotation.

LLM inference

Serve 70B-class models on a single GPU and 400B-class models on one node, with room for long context and many users.

Sample: 1–8 GPUs · vLLM

Generative AI & fine-tuning

LoRA or full fine-tuning on your own data, plus image, speech and video generation with large batch sizes.

Sample: 2–8 GPUs · PyTorch

High-performance computing

81.7 TFLOPS of FP64 vector compute per GPU (per AMD) suits simulation, CFD, genomics and scientific code.

Sample: 8 GPUs · HIP / OpenMP

Multi-GPU distributed training

Data, tensor and pipeline parallelism across eight GPUs over Infinity Fabric, with RCCL for collective operations.

Sample: 8 GPUs · RCCL

Open-source AI development

Build and test on open models and open tooling, and port CUDA code with HIP where you need to.

Sample: 1–2 GPUs · Jupyter

Enterprise AI & analytics

RAG over company documents, embedding large corpora and analytics copilots, with data kept in India.

Sample: 1–4 GPUs · RAG stack
Tool · model memory fit calculator

Will your model fit in 192 GB? Do the maths here.

Choose model size, precision and task. The calculator multiplies parameters by bytes per parameter, adds an overhead for the task and divides by usable GPU memory. The answer is illustrative. We confirm it with a test run.

Raise this for long prompts or many users at once.
Estimated GPU memory0 GBillustrative
Weights / states 0Overhead 0Free
AMD MI300X · 192 GB1×
NVIDIA H200 · 141 GB1×
80 GB-class GPU2×

—

Formula
    Use in quotation →
    Tool · MI300X vs H200 spec comparer

    AMD MI300X or NVIDIA H200? Compare the published numbers.

    Vendor specifications only, no prices and no benchmarks. Pick the NVIDIA GPU and whether to compare dense or sparsity figures. Bars scale to the larger value in each row.

    AMD Instinct MI300XNVIDIA H200 SXM
    Why choose MI300X
      When NVIDIA may fit better

        MI300X per AMD's data sheet (dense figures; sparsity doubles TF32 to INT8). H200 and H100 per NVIDIA for the SXM form factor; NVIDIA publishes tensor figures with sparsity, and dense figures are half. Peak theoretical numbers do not predict real throughput, which depends on model, precision, software and batch size. H200 is part of XcellHost's line-up; H100 and MI300X are sourced on request.

        Reserve MI300X · quotation

        Configure your MI300X node. We confirm availability and price.

        Choose GPUs, term, workload, software and storage. The summary updates as you go. XcellHost replies with availability, a delivery date and a rupee quotation with GST shown separately, usually within one business day.

        Quotation summaryXH-MX-Q · draftPrice on quote
        PricingOn quotation · INR · excl. 18% GST
        Please enter your name
        Please enter your company
        Enter a valid email
        Enter a valid phone number
        Run larger AI models on fewer GPUs.Check the fit, then send us the configuration. We do the rest.
        Check my model →
        How XcellHost delivers

        We design, deploy and operate. You run your models.

        A rented GPU is only useful once drivers, libraries, storage, network and monitoring all work together. XcellHost handles that work before and after you go live.

        01 · Design

        Size it with you

        We start from your model, precision, context length and users, and work out GPU count, storage and network. Where MI300X is the right fit, we confirm availability; where it is not, we say so.

        • Memory and throughput sizing
        • MI300X or H200 recommendation
        02 · Deploy

        Hand over a working node

        Server racked, ROCm installed and pinned, frameworks and drivers tested, GPU burn-in and RCCL bandwidth checks run, access set up the way you asked.

        • ROCm, PyTorch, vLLM ready
        • Burn-in and fabric tests
        03 · Operate 24×7

        Keep it healthy

        Our Mumbai NOC watches GPU health, temperature, memory errors and network around the clock. The SOC watches access. Patches are scheduled with you.

        • GPU and node monitoring
        • SOC-watched access logs
        Monthly INR billing + GST

        One rupee invoice, with no dollar exchange-rate swings.

        01
        Flexible terms

        On-demand monthly or reserved for 1, 6 or 12 months.

        02
        99.95% uptime SLA

        XcellHost's standard infrastructure SLA, stated in your contract.

        03
        Set up for you

        ROCm, frameworks, storage and access configured and tested.

        04
        24×7 NOC & SOC

        Phone, WhatsApp and email support from Mumbai.

        05
        Certified since 1999

        ISO 27001 & 20000-1, Microsoft Gold Partner, MeitY empanelled.

        06
        India first

        Your models and data stay in India.

        MI300X capacity from XcellHost is operated from Mumbai. Training data, prompts, outputs, model weights and logs stay on infrastructure we run in India, and the people with admin access are our own NOC and SOC staff.

        stays in Indiablocked offshoreillustrative
        0 hCERT-In incident reportingCERT-In's April 2022 directions ask for cyber incidents to be reported within 6 hours. XcellHost's SOC keeps the evidence trail.
        0 dLogs kept in IndiaThe same directions ask for ICT logs to be kept for 180 days within Indian jurisdiction. Node and access logs are retained in Mumbai.
        2023DPDP ActThe Digital Personal Data Protection Act 2023 and its Rules govern personal data used to train or prompt models. In-India processing and access logs help you meet your obligations.
        IndiaAI Mission contextThe IndiaAI Mission, approved by the Union Cabinet in March 2024, aims to widen access to AI compute inside India. Domestic GPU capacity fits that direction.
        MeitY AI governanceMeitY's guidance favours accountable and transparent AI. Access control and audit trails on your node support that approach.
        Energy and efficiencyMore memory per GPU can mean fewer GPUs per model. We size to avoid idle cards, and treat power efficiency as a design goal.
        Certified operationsISO 27001, 27017, 27018 and 20000-1, SOC 2 Type I & II, MeitY empanelled. These cover XcellHost's own operations.

        Running AI in India supports your compliance evidence; on its own it does not make you compliant. Notices, consent, retention and model-risk processes remain your responsibility. This is not legal advice.

        Compare

        Rent from XcellHost, use a hyperscaler, or buy your own. Where they differ.

        All three can give you MI300X-class compute. They differ in who runs it, where it sits and how you pay.

        XcellHost MI300X rentalHyperscaler GPU instanceBuy your own 8-GPU server
        BillingMonthly INR invoice + GSTUsage-based, often USD-linkedLarge upfront capex + AMC
        LocationRun from Mumbai, IndiaRegion by choice; MI300X region options varyYour premises or co-location
        SetupROCm and frameworks installed and testedYou build the imageYou install and maintain everything
        Support24×7 NOC & SOC, named teamTiered support plansYour staff plus vendor warranty
        CommitmentMonthly or 1/6/12-month reservedOn-demand or multi-year savings plansOwned for the hardware's life
        Power & coolingIncluded in the serviceIncludedUp to 750 W per GPU to plan for
        FallbackH200 offered if MI300X is not availableDepends on quotaNone until delivery
        How to get started

        From request to running node. Five steps.

        Timelines depend on MI300X availability and how much of the stack you want managed. Your quotation states the delivery date.

        1. 01Tell us the workload

          Model, precision, context, users and data rules.

        2. 02Availability check

          We confirm MI300X supply and a delivery date, or suggest H200.

        3. 03Quotation

          Rupee quotation with GST shown separately, by term.

        4. 04Provision & test

          ROCm, frameworks, burn-in and fabric tests, access.

        5. 05Run 24×7

          Monitoring, patching, monthly reports and reviews.

        Related XcellHost services

        More AI infrastructure from the same team.

        AMD, the AMD Arrow logo, AMD Instinct, AMD CDNA, Infinity Fabric, Infinity Cache, ROCm and combinations thereof are trademarks of Advanced Micro Devices, Inc. NVIDIA, H100, H200, NVLink and CUDA are trademarks and/or registered trademarks of NVIDIA Corporation in the U.S. and other countries. PyTorch, TensorFlow, JAX, ONNX Runtime and vLLM are trademarks of their respective owners. Their use here describes compatibility and does not imply endorsement. *Capacity figures on related pages are confirmed by XcellHost at proposal.