VMware NSX-T Knowledge Base

Configure BGP route reflector on physical router

# NSX Edge nodes typically peer with physical ToR switches
# For iBGP between multiple Edge clusters, use route reflector

# Physical router (Cisco) — configure as RR:
# router bgp 65000
#   neighbor <edge-1-ip> route-reflector-client
#   neighbor <edge-2-ip> route-reflector-client

# NSX side — configure iBGP neighbor:
# Networking → Tier-0 → BGP → Add Neighbor
# Remote AS = Same as local (iBGP)
# Source Address = Edge uplink IP

# Verify routes reflected:
get bgp neighbor <rr-ip> received-routesCopy

Verify ECMP configuration and path selection

# Check T0 HA mode:
# Active-Standby = no ECMP (single path)
# Active-Active = ECMP enabled

# Verify ECMP paths:
get route-table
# Multiple next-hops for same prefix = ECMP active

# For stateful services — use Active-Standby mode:
# NSX UI → Tier-0 → Edit → HA Mode → Active-Standby
# This ensures all traffic uses same Edge for state tracking

# If ECMP is required — ensure physical routers also hash symmetrically:
# Match 5-tuple hash on both directionsCopy

Audit route redistribution configuration

# Check what's being redistributed on T0:
GET https://<nsx-manager>/policy/api/v1/infra/tier-0s/<t0-id>/locale-services/<ls-id>/bgp

# Check route redistribution rules:
# NSX UI → Networking → Tier-0 → Route Redistribution
# Verify only intended sources are redistributed:
#   T0 Connected Subnets → BGP (typically yes)
#   T1 Connected → BGP (yes, for VM subnets)
#   Static → BGP (be careful — can create loops)
#   BGP → Connected (NEVER do this)

# Apply route filters to prevent loops:
# Use prefix lists to only advertise known-good prefixes
# NSX UI → Tier-0 → BGP → Neighbors → Route FiltersCopy

Verify SR-DR transit link

# The SR and DR communicate via internal transit segments
# Check transit link status:
get logical-routers
# Look for T0 SR and T0 DR entries

# On Edge — check transit interface:
get interfaces
# Look for "transit" type interface — should show UP

# Verify routing between SR and DR:
# In SR VRF:
vrf <sr-vrf-id>
get route-table
# Should have routes to DR subnets via transit link

# If transit link is down — check overlay connectivity to Edge:
vmkping ++netstack=vxlan -d -s 1572 <edge-tep-ip>Copy

Identify and fix blackhole static routes

# List all static routes:
GET https://<nsx-manager>/policy/api/v1/infra/tier-0s/<t0-id>/static-routes

# On Edge — verify next-hop reachability:
vrf <sr-vrf-id>
ping <next-hop-ip>

# If next-hop unreachable — the static route is a blackhole
# Options:
# 1. Fix next-hop connectivity (VLAN, interface, etc.)
# 2. Remove the static route
# 3. Use BFD for static route health monitoring

# Configure BFD for static route:
# NSX UI → Tier-0 → Static Routes → BFD Peer
# If BFD fails, route is withdrawn from FIBCopy

Create prefix list for route filtering

# NSX API — Create prefix list:
PUT https://<nsx-manager>/policy/api/v1/infra/tier-0s/<t0-id>/prefix-lists/Allow-Internal
{
  "display_name": "Allow-Internal",
  "prefixes": [
    {"network": "10.0.0.0/8", "action": "PERMIT", "le": 24},
    {"network": "172.16.0.0/12", "action": "PERMIT", "le": 24},
    {"network": "0.0.0.0/0", "action": "DENY"}
  ]
}

# Apply to BGP neighbor as outbound filter:
# NSX UI → Tier-0 → BGP → Neighbors → Route Filter
# Direction: OUT
# Prefix List: Allow-Internal
# This prevents advertising unwanted routes to physical network

# Verify on Edge:
get bgp neighbor <peer-ip> advertised-routes
# Should only show permitted prefixesCopy

Configure optimal BFD timers

# BFD Detection Time = Interval × Multiplier
# Aggressive: 300ms × 3 = 900ms detection (risk of false positives)
# Balanced:   500ms × 3 = 1.5s detection (recommended)
# Conservative: 1000ms × 4 = 4s detection (stable, slower failover)

# Set BFD configuration:
PUT https://<nsx-manager>/policy/api/v1/infra/tier-0s/<t0-id>
{
  "internal_transit_subnets": ["169.254.0.0/28"],
  "ha_mode": "ACTIVE_STANDBY",
  "bfd_peers": [{
    "bfd_profile_path": "/infra/bfd-profiles/default",
    "peer_address": "<peer-ip>"
  }]
}

# Custom BFD profile:
PUT https://<nsx-manager>/policy/api/v1/infra/bfd-profiles/Fast-BFD
{
  "display_name": "Fast-BFD",
  "interval": 500,
  "multiple": 3
}

# Monitor BFD stability after change:
get bfd-sessionsCopy

Configure stretched T0 gateway across sites

# On Global Manager:
# Networking → Tier-0 Gateways → Add Gateway
# Span: All Locations (or specific sites)

# Each site gets its own SR on local Edge cluster
# DR is distributed across all hosts at all sites

# Configure per-site BGP peering:
# Each site's Edge peers with local physical routers
# Routes learned at Site-A are advertised to Site-B via RTEP

# Verify cross-site routing:
# On Edge at Site-A:
get route-table
# Should see routes for Site-B subnets

# Check RTEP (Remote Tunnel Endpoint) connectivity:
# RTEP enables cross-site overlay communication
ping <remote-site-rtep-ip>Copy

Audit NAT rules and priority

# List all NAT rules on T0/T1:
GET https://<nsx-manager>/policy/api/v1/infra/tier-0s/<t0-id>/nat/rules

# NAT processing order:
# 1. DNAT processed first (inbound)
# 2. Then routing decision
# 3. Then SNAT (outbound)
# Priority field determines order within same type (lower = first)

# Check for overlapping source/destination:
# Two SNAT rules matching same source → first match wins
# Two DNAT rules matching same destination → first match wins

# Fix: Set explicit priority values:
PATCH https://<nsx-manager>/policy/api/v1/infra/tier-0s/<t0-id>/nat/rules/<rule-id>
{
  "priority": 100
}

# Use "No SNAT" rules to exclude traffic from NAT:
# Action: NO_SNAT, Source: internal-subnet, Dest: other-internal-subnet
# This prevents internal traffic from being unnecessarily NATedCopy

Check current SR allocation and rebalance

# View current SR placement:
GET https://<nsx-manager>/api/v1/edge-clusters/<cluster-id>/allocation-status

# Check which Edge has which SRs:
get logical-routers
# Note the edge-node assignment for each SR

# If imbalanced after node recovery — trigger rebalance:
POST https://<nsx-manager>/api/v1/edge-clusters/<cluster-id>?action=rebalance

# Verify redistribution:
GET https://<nsx-manager>/api/v1/edge-clusters/<cluster-id>/allocation-status
# SRs should be evenly distributed

# Note: Rebalancing causes brief (sub-second) failover for moved SRs
# Schedule during maintenance window for stateful servicesCopy