Technical Note

Nemotron-3 Super-120B Hybrid Topology

Local governed execution and a canonical mixed-edge topology for an 88-layer hybrid model.

Local Execution and Hybrid Topology Checkpoint for Nemotron-3 Super-120B

Download PDF Back to research Verify a receipt

Simple version

A careful map of how a much bigger, more complex AI model routes its work internally.

In plain terms

Nemotron-3 Super has 120 billion parameters and a mixture-of-experts design that sends different parts of an answer to different sub-networks. We captured both the expert-routing decisions and the attention edges, and showed the map reproduces exactly across independent runs on the same machine.

Technical artifact

Canonical mixed-edge topology for a 120B hybrid model, 896 bounded edges across expert routing and attention, emitted under a determinism-gated admission rule.

Evidence path

  1. Step 1

    Run a governed streamed pass

    Nemotron-3 Super-120B executes locally under governed streamed execution with a frozen anchor.

  2. Step 2

    Capture routing and attention edges

    880 expert-routing edges and 16 attention edges across 88 layers (40 Mamba2, 40 MoE, 8 full-attention).

  3. Step 3

    Emit a canonical topology artifact

    Mixed-edge topology.v2 with 896 bounded edges under a determinism-gated admission rule.

  4. Step 4

    Confirm reproducibility

    All topology phases reproduce across independent runs and pass the determinism gate.

Artifact details

Surface
Model Topology
Artifact
Technical note · PDF
Model
Nemotron-3 Super — 120B parameters · 88 layers
Layer mix
40 Mamba2 · 40 MoE · 8 full-attention
Evidence collected
896 bounded edges · expert-routing + attention
Verification result
Reproduces across independent runs

Scope of this note

This page summarizes one bounded topology capture over the expert-routing and attention edges of an 88-layer hybrid model. The PDF contains the full admission rule, edge schema, and observation methodology.

Open the technical artifact

The PDF contains the full admission rule, edge schema, layer breakdown, and observation methodology.

Download PDF Back to research

Have an artifact?

Verify a receipt, digest, or bundle.

Verify a receipt