VMware NSX-T KB — Phase 6: Segments & Overlays

Port Blocking — MAC Table Full

Logical switch MAC address table overflow causing port blocking and traffic loss

CriticalMAC TablePort Block

⚠Impact: Ports placed in blocking state — VMs lose all network connectivity. Traffic flooding occurs when MAC table is full, causing performance degradation for all VMs on the segment.

Resolution

Check MAC table utilization and clear stale entries

# On host — check MAC table for segment:
nsxcli -c "get logical-switch <ls-uuid> mac-table"

# Check MAC table size:
nsxcli -c "get logical-switch <ls-uuid> mac-table-size"
# Default limit: 2048 MACs per logical switch

# Identify source of MAC flooding:
# Look for VMs with multiple MACs (nested hypervisors, containers)
# Check for MAC spoofing or looping

# Clear MAC table:
nsxcli -c "clear logical-switch <ls-uuid> mac-table"

# Increase MAC table limit if legitimate:
# NSX UI → Networking → Segments → Segment Profile → Spoofguard
# Adjust MAC address limit per port

# Long-term: Identify and fix MAC source
# Nested VMs: Enable MAC Learning on the port
# Loops: Check for bridging loops in physical network

GENEVE Overlay MTU Issues

MTU mismatch causing packet drops for overlay-encapsulated traffic — silent data loss

CriticalGENEVEMTUOverlay

⚠Impact: Large packets (SCP, SMB, database) fail while small packets (ping, DNS) work. Classic "ping works but file transfer fails" scenario. GENEVE adds 54-byte overhead to each frame.

Resolution

Verify end-to-end MTU path supports 1600+ bytes

# NSX-T GENEVE overhead: 54 bytes
# VM sends 1500-byte frame → becomes 1554 bytes on wire
# Physical infrastructure must support at least 1600 MTU (1700 recommended)

# Test MTU between TEPs:
vmkping ++netstack=vxlan -d -s 1572 <remote-tep-ip>
# -s 1572 + 28 (ICMP header) = 1600 bytes
# If this fails — physical MTU is too low

# Check physical switch MTU:
# Cisco: show interface <int> | include MTU
# Must be 9000 (jumbo) or at least 1700

# Set ESXi TEP vmk MTU:
esxcli network ip interface set -i vmk10 -m 1600

# Best practice: Set physical to 9000, ESXi TEP to 1700
# This covers GENEVE (54) + any future overhead

MAC Learning & Unknown Unicast Flooding

Excessive BUM traffic due to unknown unicast flooding — performance impact on segments

HighMAC LearningBUMFlooding

Resolution

Enable MAC learning for ports with multiple MACs

# Create MAC Discovery segment profile:
PUT https://<nsx-manager>/policy/api/v1/infra/segments/<segment-id>/segment-discovery-profile-binding-maps/mac-learn
{
  "mac_discovery_profile_path": "/infra/mac-discovery-profiles/mac-learning-enabled"
}

# Create MAC learning profile:
PUT https://<nsx-manager>/policy/api/v1/infra/mac-discovery-profiles/mac-learning-enabled
{
  "display_name": "MAC-Learning-Enabled",
  "mac_learning_enabled": true,
  "unknown_unicast_flooding_enabled": true,
  "mac_limit": 4096,
  "mac_limit_policy": "ALLOW"
}

# Apply to specific ports (nested VMs, containers):
# NSX UI → Networking → Segments → Ports → Edit Port
# Attach MAC Discovery Profile

# Monitor BUM traffic:
nsxcli -c "get logical-switch <ls-uuid> stats" | grep "flood\|unknown"

Post-Migration Segment Issues

VM loses connectivity after vMotion or migration between transport nodes

HighMigrationvMotionConnectivity

Resolution

Diagnose post-vMotion connectivity loss

# After vMotion — VM should automatically reconnect to same segment
# If connectivity lost:

# Check VM port attachment on destination host:
nsxcli -c "get logical-switch <ls-uuid> ports"
# VM should appear on the new host

# Verify transport zone membership:
# Source and destination hosts must be in same transport zone
GET https://<nsx-manager>/api/v1/transport-zones/<tz-id>/transport-node-status

# Check VTEP connectivity from new host:
vmkping ++netstack=vxlan -d -s 1572 <remote-tep-ip>

# If VM was on VLAN-backed segment — verify VLAN is trunked on new host:
esxcli network vswitch dvs vmware list | grep -A5 "uplink"

# Force re-learn on destination:
# Power cycle VM vNIC (disconnect/reconnect in vCenter)
# Or: nsxcli -c "clear logical-switch <ls-uuid> mac-table"

ARP Suppression Configuration

Reduce ARP broadcast traffic in overlay segments using ARP suppression tables

MediumARPSuppressionBroadcast

Resolution

Configure ARP suppression on segments

# ARP suppression is enabled by default on NSX-T overlay segments
# Each host maintains a local ARP table for known VMs
# When an ARP request arrives, host replies locally if it knows the answer

# Check ARP suppression table on host:
nsxcli -c "get logical-switch <ls-uuid> arp-table"

# If ARP suppression not working:
# 1. Verify IP Discovery profile is attached to segment
# 2. Check that ARP snooping is enabled

# Create/verify IP Discovery profile:
PUT https://<nsx-manager>/policy/api/v1/infra/ip-discovery-profiles/default-ip-discovery
{
  "arp_snooping_enabled": true,
  "dhcp_snooping_enabled": true,
  "vmtools_enabled": true,
  "arp_binding_limit": 128
}

# Monitor ARP suppression effectiveness:
nsxcli -c "get logical-switch <ls-uuid> stats"
# Check suppressed ARP count vs forwarded ARP count

Cross-vCenter Segment Extension

Extend L2 segments across multiple vCenter/NSX domains for workload mobility

HighCross-vCenterL2 Extension

Resolution

Configure stretched segment in NSX Federation

# NSX Federation — stretched segments via Global Manager:
# On GM: Networking → Segments → Add Segment
# Transport Zone: Select stretched TZ (spans multiple locations)
# The same VNI is used at both sites

# Verify stretched segment:
# On GM:
GET https://<gm>/global-manager/api/v1/global-infra/segments/<stretched-seg-id>

# Traffic flow for stretched segment:
# Same-site: Normal overlay (host-to-host via TEP)
# Cross-site: Via RTEP (Remote TEP) on Edge nodes

# Verify RTEP connectivity between sites:
# On Edge at Site-A:
ping <site-b-rtep-ip>

# Ensure inter-site link has sufficient bandwidth
# L2 extension increases cross-site traffic significantly

VLAN Trunk Port Configuration

Configure VLAN trunk ports on segments for VMs that need to receive tagged traffic

HighVLANTrunkTagged

Resolution

Configure VLAN trunk on segment port

# Use case: VM needs to receive VLAN-tagged frames
# (nested ESXi, firewall VM with sub-interfaces, router VM)

# Create segment with trunk VLAN range:
PUT https://<nsx-manager>/policy/api/v1/infra/segments/Trunk-Segment
{
  "display_name": "Trunk-Segment",
  "transport_zone_path": "/infra/sites/default/enforcement-points/default/transport-zones/<tz-id>",
  "vlan_ids": ["100-200", "300"]
}

# Or configure per-port trunk (overlay segment):
# NSX UI → Segments → Select Segment → Ports → VM Port → Edit
# Address Binding: None
# VLAN Trunk: Enable → Specify VLAN ranges

# Verify trunk works:
# Inside the VM — create sub-interface:
# ip link add link eth0 name eth0.100 type vlan id 100
# Verify tagged traffic received

Segment Security Profile Configuration

Configure SpoofGuard, BPDU filter, and storm control profiles for segment protection

MediumSpoofGuardBPDUSecurity

Resolution

Configure segment security profiles

# Create SpoofGuard profile:
PUT https://<nsx-manager>/policy/api/v1/infra/spoofguard-profiles/Strict-SG
{
  "display_name": "Strict-SpoofGuard",
  "address_binding_allowlist": true,
  "address_binding_whitelist": true
}

# Create Segment Security profile (BPDU + Storm Control):
PUT https://<nsx-manager>/policy/api/v1/infra/segment-security-profiles/Protected
{
  "display_name": "Protected-Profile",
  "bpdu_filter_enabled": true,
  "bpdu_filter_allow": [],
  "rate_limits_enabled": true,
  "rx_broadcast": 1000,
  "rx_multicast": 1000,
  "tx_broadcast": 1000,
  "tx_multicast": 1000
}

# Attach profiles to segment:
PUT https://<nsx-manager>/policy/api/v1/infra/segments/<segment-id>/segment-security-profile-binding-maps/default
{
  "segment_security_profile_path": "/infra/segment-security-profiles/Protected",
  "spoofguard_profile_path": "/infra/spoofguard-profiles/Strict-SG"
}

TEP IP Pool Exhaustion

Tunnel Endpoint IP pool depleted — new transport nodes cannot get TEP IPs, overlay connectivity fails

CriticalTEPIP PoolExhaustion

⚠Impact: New hosts cannot join NSX fabric. Cluster expansion blocked. Existing hosts work fine but new deployments fail with "IP address allocation failed" errors.

Resolution

Check TEP pool utilization and expand

# Check IP pool allocation:
GET https://<nsx-manager>/api/v1/pools/ip-pools/<tep-pool-id>/allocations

# Check pool capacity:
GET https://<nsx-manager>/api/v1/pools/ip-pools/<tep-pool-id>
# Compare total IPs in range vs allocated count

# Look for leaked allocations (hosts removed but IP not freed):
# Cross-reference allocated IPs with active transport nodes
GET https://<nsx-manager>/api/v1/transport-nodes

# Release orphaned IPs:
POST https://<nsx-manager>/api/v1/pools/ip-pools/<pool-id>?action=RELEASE
{
  "allocation_id": "<leaked-allocation-id>"
}

# Expand pool range:
# Add additional IP subnet to existing TEP pool
PATCH https://<nsx-manager>/api/v1/pools/ip-pools/<pool-id>
# Add new range in "subnets" array

Segment Port Disconnect / Orphan Ports

VM logical ports detached from segments — network connectivity lost due to orphan port state

HighSegmentPortOrphan

Resolution

Identify and reconnect orphan ports

# Find orphan ports (ports not attached to any VM):
GET https://<nsx-manager>/api/v1/logical-ports?attachment_type=VIF&attachment_status=UNATTACHED

# Check specific VM's port:
# Get VM external-id from vCenter (vm-xxx)
GET https://<nsx-manager>/api/v1/fabric/virtual-machines?external_id=<vm-id>

# Check if VM has a logical port:
GET https://<nsx-manager>/api/v1/logical-ports?attachment_id=<vm-vnic-uuid>

# If port exists but disconnected — reconnect VM's vNIC:
# In vCenter: Edit VM → Network adapter → Disconnect → Save → Connect → Save

# If port doesn't exist — VM may need to be reconnected to NSX segment:
# Change VM's network to the correct NSX segment port group
# The logical port will be auto-created

# Clean up orphan ports:
DELETE https://<nsx-manager>/api/v1/logical-ports/<orphan-port-id>?detach=true