Design and Automate

VXLAN BGP EVPN Fabric

with NDFC
Parth Patel, Technical Leader,
Technical Marketing Engineer
Data Center and Provider Connectivity BU
BRKDCN-2918

#CiscoLive
Cisco Webex App
https://ciscolive.ciscoevents.com/
ciscolivebot/#BRKDCN-2918

Questions?
Use Cisco Webex App to chat
with the speaker after the session

How
1 Find this session in the Cisco Live Mobile App

2 Click “Join the Discussion”

3 Install the Webex App or go directly to the Webex space

4 Enter messages/questions in the Webex space

Webex spaces will be moderated Enter your personal notes here

by the speaker until June 7, 2024.

BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 2
 • Flexible Design options for
VXLAN EVPN
• External Handoff options for
VXLAN EVPN
Agenda • Introduction to NDFC
• Automate VXLAN EVPN
Single-Site with NDFC
• Conclusion

BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Data Center
Network
Requirements
 What are our basic network requirements?
1) Provide paths for endpoints to communicate at 4) Communication to external L2 networks (DCI)
Layer2(MAC) and Layer3(IP) 5) Communication to external L3 networks (WAN)
2) Provide separation of endpoint into Layer2
forwarding domains (VLAN or BD)
3) Routing between IPv4/IPv6 subnets and allow
separation of these into multiple VRFs

EP1 VLAN EP2 EP3 VLAN EP4

L2 L3
1 2 External External
EP3
VRF-1

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
What are our basic network requirements?

6) Allow security policies in order to limit communication to between endpoints to allowed protocols.

ip access-list web-in
VLAN 1 VRF1 VLAN 2 permit tcp Subnet1 Subnet2 eq 80
Subnet1 Subnet2 ip access-list web-out
permit tcp Subnet2 eq 80 Subnet1

EP1 EP3
80
ip access-group web1 in

ip access-group web2 out

EP2
22 EP4

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
What Physical Topology is required?
• Physical topology must support our endpoint communication
(layer-2 / layer-3), and the location of endpoints within the physical
network will affect the supporting design/configuration.

VLAN VLAN L2 L3
EP1 EP2 EP3 EP4
1 2 External External

VRF-1

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
Data Center Network Evolution
Well-Known but Legacy Methods

Core

Layer-2 STP Forwarding

STP Root STP 2nd Root Layer-2 STP Blocked

FHRP Active FHRP Standby
(Aggregation) (Aggregation) Layer-3 ECMP

Access

Classic Spanning-Tree
#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
Data Center Network Evolution
Well-Known but Legacy Methods

Core

Layer-2 STP Forwarding

STP Root STP Root

Layer-3 ECMP
FHRP Active FHRP Active
(Aggregation) (Aggregation)

Access

VPC and Spanning-Tree

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
Data Center Network Evolution
Well-Known but Legacy Methods

Core

Layer-2 FabricPath

Anycast HSRP Anycast HSRP

(Spine) Layer-3 ECMP
(Spine)

Leaf

FabricPath (Mac-in-Mac)
#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
Data Center Network Challenges
Of Legacy Methods

Hierarchical Topology Hair-Pining Flood & Learn

• Scale-Up with Big • Suboptimal • Convergence
Centralized Chassis performance, traffic dependent on Single
(Aggregation) forwarding constrained Tree and MAC Flush
by spanning-tree rules (TCN)
• STP limits full bandwidth
utilization • Rigid Network Service • Exposed to Large
Placement (L4-L7) Broadcast Domains (All
Access and
• Limited Endpoint
Aggregation)
Mobility

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
Next Generation– VXLAN BGP EVPN Fabrics
Becoming Industry De-Facto Standard
• An IGP is recommended for the underlay
(OSPF or IS-IS) R R
• BGP can also be used if needed

• BGP EVPN must be used in the overlay to

exchange endpoints information V V V V
• Spines act as route-reflectors

• VXLAN is used to transport endpoint traffic Underlay

in the fabric BGP EVPN Overlay
• Leafs are considered VTEP as they R BGP Route-Reflector
encapsulate and decapsulate VXLAN
traffic V VXLAN Tunnel Endpoint (VTEP)

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
Data Center Network Challenges
Solving it with VXLAN EVPN

Hierarchical Topology No More Hair-Pining Control-Plane Learned

• Scale-Out • Default Gateway at every • Active Learning and
• Add more Spine for Leaf Distribution with BGP
bandwidth and EVPN
redundancy • Distributed Anycast
Gateway • Reduces the Broadcast
• Add more Leaf for port
capacity
Domain by configuring
• Flexible Network Service
VLANs where needed
• All Links are used (IP
Placement (L4-L7)
ECMP) • Pervasive Subnet and
Endpoint Mobility

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
A Leaf and Spine
Paradigm
 Kick Start your VXLAN Fabric!
A Leaf and Spine Paradigm

Spine
• IP Transport
• BGP EVPN
Control Plane Tier-2
(RR)
Stage-2

Stage-1 Stage-3
• Connect Endpoints,
Service Nodes, and
External Networks.
• BGP EVPN Control Tier-1
Plane
• VXLAN
Leaf

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 15
VXLAN Single-Site
Flexible Design
Leaf Node Placement Option
Spine EVPN Route
Attachment Next-hop
Type
Leaf as Standalone VTEP
vPC N/A
- Seen as an individual VTEP (PIP) Type 2
(Host Routes) Orphan Port Advertised by PIP
- No HA, Orphan hosts only PIP- 10.1.1.1 PIP- 10.1.1.2
Type 5 vPC N/A
Leaf-1 Leaf-2 (IP Prefix
Routes) Orphan Advertised by PIP
Spine
Leaf as vPC VTEP EVPN Route
Attachment Next-hop
- Seen as a single logical VTEP (VIP) Type
- Legacy vPC Peer-Link required Type 2 vPC Advertised by VIP
VIP- 10.1.1.3
- HA with Dual-Attached and Orphan hosts PIP- 10.1.1.1 PIP- 10.1.1.2
(Host Routes) Orphan Port Advertised by VIP
- Possibility for L4-7 peering Type 5 vPC Advertised by VIP
Leaf-1 Leaf-2 (IP Prefix
Routes) Orphan Advertised by VIP

Spine
Leaf as vPC Fabric-Peering VTEP EVPN Route
Attachment Next-hop
- Seen as a 3 “three” VTEP (PIP + VIP) Type
- No need for Physical Peer-Link Type 2 vPC Advertised by VIP
VIP- 10.1.1.3
- More ports available for EPs (Host Routes) Orphan Port Advertised by PIP
PIP- 10.1.1.1 PIP- 10.1.1.2
- HA with Dual-Attached and Orphan hosts Type 5 vPC Advertised by PIP
- Possibility for L4-7 peering Leaf-1 Leaf-2 (IP Prefix
Routes) Orphan Advertised by PIP

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
Super-Spine Node Placement Option
Scale-out Multi-Clos Fabric to Interconnect the PODs using
Super-Spine R R
Super-Spine
Architecture beyond a single server room. Simpler capacity
planning

Clean role separation and uniform reachability from the entire

fabric are the major advantages

Spine
Leaf hosts:
- East-West VXLAN (VTEP)

Spine hosts: V V V V
- Layer-3 IP Transit Leaf

Super-Spine hosts:
- Route Reflector (RR) (iBGP EVPN) POD-1 POD-2
- Rendezvous Point (RP) (Multicast Underlay BUM)

VXLAN EVPN Site

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
External Handoff
Border Placement
 Border Node Placement Option
Border as Leaf (Flexible Design option-1)

• Leaf VTEP (East-West traffic) External Network

• Connectivity to endpoints, the first-hop routing

• Server-to-server traffic Edge Router

• Border Leaf VTEP (North-South traffic)

IPv4/IPv6/L3VPN
Capacity planning only for North-South traffic flows
V V Handoff
Border Leaf

North-South
Clean role separation and uniform reachability from
the entire fabric are the major advantages L2VPN EVPN
R R
Support for Inter-AS option A (VRF-LITE) and Spine
seamless VXLAN-MPLS gateway (Border-PE)
V V V V
Optionally it can have directly attached endpoints Leaf

VXLAN EVPN Site

Border Leaf hosts:
- North <> South VXLAN (VTEP) <> IP handoff East-West

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
Border Node Placement Option
Border Spine (Flexible Design option-2)

Flexible option for Small deployments External Network

Support for Inter-AS option A (VRF-LITE) and seamless

VXLAN-MPLS gateway (Border-PE) Edge Router

Extra functional dependency (Border + Spine) IPv4/IPv6/L3VPN

Capacity planning needs to accommodate all flows Handoff

North-South
V V L2VPN EVPN
Border Spine hosts: R
R
- North <> South VXLAN (VTEP) <> IP handoff Border Spine
- Route Reflector (RR) (iBGP EVPN)
- Optionally Rendezvous Point (RP) (Multicast Underlay BUM)
V V V V
Leaf

V V = VTEP VXLAN EVPN Site

East-West
R = RR/RP

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
Border Node Placement Option
Border on top of Super-Spine (Flexible Design option-3)

External Network
Capacity planning only for North-South traffic flows
Edge Router
Clean role separation and uniform reachability from the
entire fabric are the major advantages IPv4/IPv6/L3VPN
V V
Support for Inter-AS option A (VRF-LITE) and seamless Border Leaf Handoff
VXLAN-MPLS gateway (Border-PE)
L2VPN EVPN
R R

North-South
Optionally it can have directly attached endpoints Super-Spine

Border Leaf hosts:

Spine
- North <> South VXLAN (VTEP) <> IP handoff

V V V V
Leaf
V V = VTEP
POD-1 POD-2
R = RR/RP VXLAN EVPN Site
East-West
#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
Border Node Placement Option
Border Super-Spine (Flexible Design option-4)

Support for Inter-AS option A (VRF-LITE) and seamless

VXLAN-MPLS gateway (Border-PE) External Network

Extra functional dependency (Border + Spine). Not Edge Router

recommended due to Multi-POD failure dependency
V V IPv4/IPv6/L3VPN
Capacity planning needs to accommodate all flows Border Super- R
R

North-South
Spine Handoff

Border Super-Spine hosts: L2VPN EVPN

- North <> South VXLAN (VTEP) <> IP handoff Spine
- Route Reflector (RR) (iBGP EVPN)
- Optionally Rendezvous Point (RP) (Multicast Underlay BUM)
V V V V
Leaf
POD-1 POD-2
V V = VTEP VXLAN EVPN Site
East-West
R = RR/RP

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
External Handoff
Connectivity
External Layer-3 connectivity options
Inter-AS Option A
Clear separation of Autonomous Systems Separated Border + PE
Simple, Straight forward, and Commonly used (Inter-AS Option A)
No need for redistribution VPNv4/VPNv6 External Network
MPLS-LDP or SR-MPLS
Easy and Flexible BGP route-filtering mechanisms
Edge Router BGP 65099
BGP natural loop avoidance

Structured handoff between the VXLAN BGP EVPN fabric and the IPv4/IPv6
external routing domain (Backbone, WAN, Campus, etc.)
V V
Not ideal for High scale VRF handoff deployment Border Leaf BGP 65001

Peering Type = Sub-interfaces on physical routed (or L3 Port-

channel) interfaces. L3 SVI also supported. R R
Spine
• Sub-interface with dot1q tag to mark the traffic to a specific VRF EVPN
V V V V
• Sub-interface used for eBGP peering and as next-hop Leaf

• Per VRF, Per Sub-interface eBGP session

VXLAN EVPN
#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
External Layer-3 connectivity options
Inter-AS Option A
External Network
An EP is connected to Leaf and records an ARP
Edge Router BGP 65099
EVPN Type-2 (/32) and Type-5 (/24) are created with
Leaf VTEP IP as Next-Hop
V V
Leaf advertises EVPN route towards Spine with Route-Target Border Leaf BGP 65001
attachment with MAC-VRF (ASN:L2VNI) and IP-VRF (ASN:L3VNI)

Border imports EVPN route based on matching Route-Target and R R

Spine
downloads the route in BGP-VRF.

Border advertises route as IPv4/IPv6 towards External Edge with V V V V

Leaf
itself as Next-Hop

EP1- 192.168.10.10/32
Network – 192.168.10.0/24

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
External Layer-3 connectivity options
Inter-AS Option A Network – 10.0.0.0/8

External Network
Edge Router uses per-VRF eBGP session to advertise External network
with itself as Next-Hop Edge Router BGP 65099

Border receives External route in BGP-VRF and export it as

EVPN Type-5 alongside IP-VRF Route-Target (ASN:L3VNI)
V V
All VTEPs (Leaf) learns EVPN Type-5 and install them in BGP-VRF Border Leaf BGP 65001
based on matching IP-VRF Route-Target (ASN:L3VNI). From
BGP-VRF it downloads in RIB-VRF and Forwarding table
R R
Spine

V V V V
Leaf

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
External Layer-3 connectivity options
Seamless Protocol Gateway Model (Border-PE)
Combines two different encapsulations and
Address Family, using a “single-box (Border- Seamless Stitching between
PE)” instead of a “two-box (CE-PE)” model
VXLAN, MPLS, and SR
VXLAN VTEP Border nodes also becomes a
External Network
MPLS L3VPN Provider Edge (PE), resulting in
a role called Border-PE
Remote PE BGP 65099
Best suited for high scale VRF deployment
VPNv4/VPNv6
Saves CAPEX and OPEX MPLS-LDP or SR-MPLS
Seamless stitching between L2VPN EVPN
and VPNv4/v6 Address Family Reoriginate EVPN Prefix in L3VPN V V
Border-PE
Reoriginate L3VPN Prefix in EVPN BGP 65001
BGP route-filtering mechanisms available

Specific Hardware support R R

Spine
• MPLS LDP: Nexus 3600-R, Nexus 9500-R EVPN
V V V V
• SR MPLS: Nexus 9300 FX2/FX3/GX/GX2, Leaf
Nexus 9500-R

VXLAN EVPN
#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
External Layer-3 connectivity options
Seamless Protocol Gateway Model (Border-PE)
EVPN routes are imported into the local VRF instance according to
the BGP Route-Target filtering

Imported routes are re-exported to the VPN address family, and a MPLSP VPN AF
local VPNv4/v6 MPLS label is allocated External Network

Re-exported routes are Re-originated and advertised to eBGP BGP Update BGP 65099
peers with the allocated VPNv4/v6 MPLS label alongside local RD [Prefix/RD2/RT2/LB1]
and RT of Border-PE. The per-VRF VXLAN fabric relevant BGP RT is Remote PE
stripped off VRF Export MPLS-LDP or SR-MPLS
[Prefix/RD2/RT2]
Key Description

Prefix IPv4/IPv6 route in VXLAN going towards MPLS

Routing Table V V
VRF CORP BGP 65001
RD1 VPN Route-Distinguisher for VRF CORP on Source Leaf Border-PE
VRF Import
RT1 BGP Route-Target for VRF CORP [Prefix/RT1] R R
RD2 VPN Route-Distinguisher for VRF CORP on Border-PE Spine
BGP Update
[Prefix/RD1/RT1]
RT2 BGP Route-Target for VRF CORP facing MPLS V V V V
L2VPN EVPN AF
LB1 MPLS Label Leaf

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
External Layer-3 connectivity options
Seamless Protocol Gateway Model (Border-PE)
MPLS routes are imported into local VRF according to Route-
Target filtering. Routes are imported into local RIB and FIB with
MPLS label.
MPLSP VPN AF
Imported routes are re-exported to the EVPN address family External Network

BGP Update BGP 65099

Re-exported routes are Re-originated and advertised to EVPN [Prefix/RD3/RT2/LB2]
Spine peers with fabric specific encapsulation information such Remote PE
as VXLAN VNI and local RD/RT. The per-VRF MPLS network VRF Import
relevant BGP RT is stripped off MPLS-LDP or SR-MPLS
[Prefix/RT2]

Key Description Routing Table V V

Prefix IPv4/IPv6 route in MPLS going towards VXLAN VRF CORP BGP 65001
Border-PE
RD3 VPN Route-Distinguisher for VRF CORP on Remote MPLS PE VRF Export
RT2 BGP Route-Target for VRF CORP facing MPLS
[Prefix/RT1] R R
Spine
LB2 MPLS Label BGP Update
[Prefix/RD1/RT1]
RD1 VPN Route-Distinguisher for VRF CORP on Border-PE V V V V
L2VPN EVPN AF
RT1 BGP Route-Target for VRF CORP Leaf

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 30
Advertise VTEP Primary IP Address
What exactly is Advertise-PIP and Why you need it?

External routes are injected into VXLAN fabric

Backbone
Border advertises External routes as EVPN Type-5 with
the BGP Next-Hop of vPC VIP (Anycast)
vPC VIP
From Leaf perspective the Next-Hop to reach Backbone is 10.13.0.3
Border Anycast vPC VIP PIP-1 PIP-2
10.13.0.1 10.13.0.2

Problem Statement: While ARP/MAC/IPv6 ND entries are

synced between the peers of a vPC pair, prefix routes
belonging to an individual peer as well as external routes
received by a peer are not synced between vPC peer
switches. Using the VIP as the BGP next-hop for these
MP-BGP EVPN Table
routes can cause traffic to be forwarded to the wrong
Backbone →
vPC peer and hence be black-holed. 10.13.0.3

VXLAN EVPN Site

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
Advertise VTEP Primary IP Address
What exactly is Advertise-PIP and Why you need it?

Border-2 losses the link towards Backbone and Border-1 is

the only available path towards the Fabric Backbone
Border-1 continues to advertise External routes as EVPN
Type-5 with the BGP Next-Hop of vPC VIP (Anycast)
vPC VIP
X
10.13.0.3
From Leaf perspective the Next-Hop to reach Backbone is PIP-1 PIP-2
Border Anycast vPC VIP. Hence, traffic can hash to either 10.13.0.1 10.13.0.2

Border-1 or Border-2. If packet hits Border-2, it will drop the X

traffic!

Note: Border-1 still has an active link towards the

Backbone and advertises the routes towards the Spine
(RR). Later, the Spine will reflect the route to Border-2, but
MP-BGP EVPN Table
it will reject it due to Next-Hop being its own IP (VIP
Backbone →
10.13.0.3)
10.13.0.3

VXLAN EVPN Site

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 32
Advertise VTEP Primary IP Address
What exactly is Advertise-PIP and Why you need it?
router bgp 65001
address-family l2vpn evpn
advertise-pip
On both VPC peer Border Backbone
Interface nve1 Devices
advertise virtual-rmac
vPC VIP
10.13.0.3
The advertise-pip command lets BGP use the PIP as next-hop PIP-1 PIP-2
when advertising prefix routes or leaf-generated routes if vPC is 10.13.0.1 10.13.0.2
enabled. With the advertise-pip and advertise virtual-rmac
commands, EVPN Type-5 routes are advertised with PIP, and
EVPN Type-2 routes are still advertised with VIP. In addition, a
virtual MAC will be used with the VIP that is shared by both vPC
peers, and individual peer specific system Router MAC will be
used with PIP when the advertise-pip feature is enabled. In this
way, the traffic will always be destined to the right vPC peer. MP-BGP EVPN Table
Backbone →
10.13.0.1, 10.13.0.2
VXLAN EVPN Site

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
Who can be the
External Edge Router?
 Edge Router Placement Option
Nexus and Non-Nexus support
External Network External Network External Network

Edge Router Edge Router Edge Router

Border Leaf Border Leaf Border Leaf

Data Center VXLAN EVPN Data Center VXLAN EVPN Data Center VXLAN EVPN

Managed NX-OS as Edge Router Managed IOS-XR as Edge Router Managed IOS-XE as Edge Router
- Nexus 9000 - Cisco 8000 - Catalyst 9000
- Nexus 7000 - ASR 9000 - Catalyst 8000
- NCS 5500 - ASR 1000
- CSR 1000

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
Cool! But What’s the Catch?
There is always a catch ☺

1 This seems really cool! Is it easy to configure?

There are lots of moving parts: OSPF/BGP/VXLAN. Manual
configuration can be challenging.

2 How easy is it to make changes?

You still rely on traditional SSH based management to each device,
which can be cumbersome and error prone.

3 How much Visibility do I have into the network?

Visibility and Troubleshooting is still performed on a “switch-by-
switch” basis.

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
What is Nexus
Dashboard
Fabric
Controller?
Cisco Nexus Dashboard Powering automation
Unified agile platform
Simple to automate, simple to consume

Insights

Orchestrator Fabric Controller

Data Broker SAN Controller

Consume all services in one place

Private cloud Third-party Apps Public cloud

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
Cisco Nexus Dashboard Fabric Controller
App accessed through Cisco Nexus Dashboard

Cisco Nexus Dashboard Access NDFC

Benefits Automation Management and Compliance Visibility and Monitoring

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 39
Cisco Nexus Dashboard Fabric Controller

Automation Management Visibility

Accelerate provisioning In depth Management Get Centralized Visibility
and simplify deployments and control for all and Monitoring views
network deployments

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
Automation
Accelerate provisioning from days to minutes

Easy to understand approach to

auto-bootstrapping of entire fabric

Rapid Deployment with Fabric Builder

best practice templates for VXLAN-EVPN

Optimized for both large deployments

and traditional deployment models

Service Insertion and Layer-3 handoff

DevOps friendly

Benefits

Simplify fabric deployments Developer agility VXLAN EVPN Multi-Site

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 41
Management Single point for management
for data center operations
Optimized for both large deployments
and traditional deployment models

Granular RBAC

Image management

RMA

Change Control

Management for non-Nexus platforms

Benefits

Reliability Compliance Secure

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
Visibility & Monitoring
Get comprehensive monitoring

Enhanced topology views

Compute and endpoint visibility

VXLAN OAM support with NDFC

Obtain detailed inventory, health, resource

consumption information on devices

End-to-end visibility, monitoring

and troubleshooting

Integrate with NDI for Day 2 operations

Benefits

Intuitive Deep visibility Enhanced monitoring

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 43
Cisco NDFC Modes
Make decision at run-time!

Runtime Feature Installer Easy switch between modes

Fabric discovery for SAN controller for MDS Fibre

LAN Deployments Channel Deployments

Fabric controller for LAN

and IPFM Deployments

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 44
Fabric Discovery

Run fabric discovery for LAN deployments:

Enable inventory, discovery, monitoring only

Enable Cisco Nexus Dashboard’s Day 2 operations

capabilities without deploying fabric controller

You can switch anytime from Fabric Discovery to

Fabric Controller Mode

Benefit

Deep visibility into deployments

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 45
Fabric Controller

Provides fabric management for multiple types of

LAN solutions, including VXLAN-EVPN, and
traditional 3-tier LAN deployments

Compliance management ensures that network is in

sync with intended deployments and allows users to
deploy any corrections

Benefit

Most configurations are automatically done following Cisco Best Practices

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 46
SAN Controller

Completely redesigned web-based zoning interface

to drastically reduce the cycle time for common
administration tasks. Provides IVR zoning function as
well, all on the same page.

SAN Insights provides useful data to the

administrators so they can be fully aware about the
fabric status

Benefit

Transition to a web-based configuration method is made easy

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 47
Cisco Nexus Dashboard

Physical ND Cluster

Orchestrator Insights Fabric Controller

VM VM VM

VM VM
Data Broker SAN Controller

Virtual ND Cluster

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
Cisco Nexus Dashboard Formats - NDFC

Physical ND Cluster Virtual ND Cluster

Each node is a UCS Server with: For NDFC each vND VM must
satisfy the following requirements:
2.8GHz AMD CPU Specs APP
256G RAM
4x2.4TB HDD vCPU 16
960 GB SSD
1.6 TB NVMe drive RAM (GB) 64

For the latest information check the specific SSD (GB) 550
scalability guide. 12.1.3b Verified Scalability

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 49
Cisco Nexus Dashboard Scaling - NDFC Cisco NDFC 12.1(3)

Physical ND Virtual ND VXLAN EVPN VXLAN EVPN

Cluster Cluster (Greenfield) (Brownfield)

Full scale for NDFC can Full scale for NDFC can Switches per Fabric: Switches per Fabric:
be achieved with 3 be achieved with 5 200 200
nodes nodes
Overlays: 500 VRF and Overlays: 400 VRF and
Managed mode (VXLAN Managed mode (VXLAN 2000 Layer-3 Networks 1050 Layer-3/Layer-2
and BGP fabrics): 500 and BGP fabrics): 400 OR 2500 Layer-2 Networks
switches switches Networks
Multi-Site Domain: 30
Managed/Monitor mode Managed/Monitor mode Multi-Site Domain: 30 fabrics
(External fabrics): 1000 (External fabrics): 1000 fabrics
switches switches
ToR/Leaf: 40 Leaf
Overall fabric count: 50 Overall fabric count: 50 (VTEP) and 320 ToRs in
DC VXLAN EVPN fabric
Recommended 3x vND can support 100
switches in managed
mode

In any case at least 3 nodes

must be deployed for proper
redundancy. 1x vND also
supported for Production
#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 50
Nexus Dashboard
Interface Types

• Each ND node has two interface types:

ND Mgmt. Interface

• Management Interface: should be dedicated to the management of

the ND cluster → connectivity to NTP and DC Proxy servers,
Intersight, DNS, ND (and ND Apps) UI access and to perform
ND Fabric Interface firmware upgrade (for ND or Apps)

• Fabric Interface: used for the bring up of the ND cluster (node to

node communication) and application to application (NDO, NDI,
NDFC, etc.) communication

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 51
Cisco Nexus Dashboard Connectivity - NDFC
Data/Fabric Network The two interfaces cannot share the
L2
bond0br same subnet

MAX RTT 50msec Intra/Inter APP PTP

ND Clustering
10.1.1.101/24 10.1.1.102/24 10.1.1.103/24 Switch Access*
DNS SNMP TRAPS
POAP DHCP
10.2.2.101/24 NTP
GUI Access
10.2.2.102/24 10.2.2.103/24
10.2.2.150/24 10.2.2.151/24

MAX RTT 50msec CLI via SSH

DC App Center
Management Network Intersight
bond1br * by default

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 52
NDFC Persistent IPs
Persistent IPs are tied to a
service, like the SNMP trap
receiver

If the SNMP trap POD gets

re-spawned into a different
ND host the sticky IP will be
moved there

L2 adjacency uses ARP, L3

adjacency BGP
announcements

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 53
NDFC Persistent IPs – Normal conditions
Data/Fabric Network
bond0br

10.1.1.101/24 10.1.1.102/24 10.1.1.103/24

10.2.2.101/24 10.2.2.102/24 10.2.2.103/24

10.2.2.150/24 10.2.2.151/24

Management Network
bond1br
#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 54
NDFC Persistent IPs - Failover
Data/Fabric Network
bond0br

10.1.1.101/24 10.1.1.102/24 10.1.1.103/24

10.2.2.101/24 10.2.2.102/24 10.2.2.103/24

10.2.2.150/24 10.2.2.151/24

Management Network
bond1br
#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 55
 Cisco Nexus Dashboard Connectivity - NDFC
Data/Fabric Network bond0br
L3
This L3 options is valid since
MAX RTT 50msec
12.1.1e

172.17.20.127/24 172.17.20.128/24 Each ND node on a different

10.1.1.101/24 10.1.2.101/24 10.1.3.101/24
Subnet

BGP Sessions are

10.2.1.101/24 10.2.2.102/24 10.2.3.103/24 established for Persistent IP
advertisement (no multi-hop)
MAX RTT 50msec
Persistent IPs must not
overlap with ND subnets
Management Network bond1br
#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 56
Cisco Nexus Dashboard Connectivity - NDFC
Data/Fabric Network bond0br
L3

MAX RTT 50msec

172.17.20.127/24 172.17.20.128/24
10.1.1.101/24 10.1.2.101/24 10.1.3.101/24

10.2.1.101/24 10.2.2.102/24 10.2.3.103/24

MAX RTT 50msec

Management Network bond1br

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 57
Cisco Nexus Dashboard Connectivity - NDFC
Data/Fabric Network bond0br
L3

MAX RTT 50msec

172.17.20.127/24

172.17.20.128/24
10.1.1.101/24 10.1.2.101/24 10.1.3.101/24

10.2.1.101/24 10.2.2.102/24 10.2.3.103/24

MAX RTT 50msec

Management Network bond1br

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 58
Cisco NDFC Connectivity to the Switches Use case #1

Discovery and Deployment happen

via ND Management Interface as that
subnet is directly connected
Fabric/Inband
bond0br ND Data Interface eventually used for
ND Cluster Endpoint Locator Feature (BGP
bond1br
SW1 SW2 SW3 towards Spine RR)
OOB mgmt0 mgmt0 mgmt0

Persistent IPs are allocated on the

MAX RTT
Management Subnet
50msec

ND Mgmt ND Data Subnet Fabric Mgmt 0 Fabric Inband

Subnet Subnet Subnet Works by default!
10.2.2.0/24 10.1.1.0/24 10.2.2.0/24 10.3.3.0/24

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 59
Cisco NDFC Connectivity to the Switches Use case #2

Everything is done over the ND Data

Interface as that subnet is directly
connected
bond0br
Fabric/Inband
Persistent IPs are allocated on the
ND Cluster Data Subnet
bond1br
SW1 SW2 SW3
OOB mgmt0 mgmt0 mgmt0

MAX RTT
50msec

ND Mgmt ND Data Subnet Fabric Mgmt 0 Fabric Inband LAN Device Management
Subnet Subnet Subnet Connectivity must be set to
Data
10.2.2.0/24 10.1.1.0/24 10.1.1.0/24 10.3.3.0/24
(see next slide)

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 60
Cisco NDFC Connectivity to the Switches Use case #2
continues

The change is global for the

NDFC Instance

Persistent IPs will be

provisioned over ND Data
Interface

Settings --> Server Settings

--> LAN Device
Management Connectivity

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 61
Cisco NDFC Connectivity to the Switches Use case #3

Discovery and Deployment happen

via ND Management Interface

ND Data Interface eventually used for

Fabric/Inband Endpoint Locator Feature (BGP
bond0br
towards Spine RR)
ND Cluster
bond1br
SW1 SW2 SW3 Persistent IPs are allocated on the
OOB mgmt0 mgmt0 mgmt0
Management Subnet

MAX RTT
50msec

ND Mgmt ND Data Subnet Fabric Mgmt 0 Fabric Inband A static route to 10.4.4.0/24
Subnet Subnet Subnet must be added in ND
Management Interface
10.2.2.0/24 10.1.1.0/24 10.4.4.0/24 10.3.3.0/24
(see next slide)

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 62
Cisco NDFC Connectivity to the Switches Use case #3
continues

The static route needs to be

added in the Nexus
Dashboard Control Panel.

Infrastructure --> Cluster

Configuration --> Routes

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 63
Cisco NDFC Connectivity to the Switches Use case #4

Everything is done over the ND Data

Interface as that subnet is directly
connected
Fabric/Inband
bond0br Persistent IPs are allocated on the
ND Cluster Data Subnet
bond1br
SW1 SW2 SW3
OOB mgmt0 mgmt0 mgmt0

LAN Device Management

MAX RTT Connectivity must be set to
50msec Data
ND Mgmt ND Inband Fabric Mgmt 0 Fabric Inband
Subnet Subnet Subnet Subnet A static route to 10.4.4.0/24
must be added in ND Data
10.2.2.0/24 10.1.1.0/24 10.4.4.0/24 10.3.3.0/24
Interface, not for routing but
for POAP
#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 64
Why do YOU
need NDFC?
Why NDFC?
Multi-Architecture
Layer-3 MSDC Fabric
Campus VXLAN EVPN

Data Center VXLAN EVPN

VXLAN EVPN Multisite
Traditional 3-Tier LAN

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 66
Why NDFC?
Multi-Topology, Multi-Protocol

Rich set of control plane and data plane possibilities available

VXLAN BGP EVPN

vPC BGP Routed
MPLS FHRP
STP
VLAN IS-IS
FabricPath OSPF
IP
IPFM

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 67
Why NDFC?
Multi-Domain, Multi-Platform

NX-OS Nexus 9000 and 3000 IOS-XE Catalyst 9000 IOS-XR ASR 9000

Supported Hardware and Software

might vary depending on NDFC version
Check compatibility matrix 12.1.3b
NX-OS Nexus 7000 IOS-XE ASR 1000

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 68
Why NDFC?
In a nutshell…

Step into SDN via VXLAN BGP EVPN Multi-OS management and support

Config and Compliance across Cisco Products Simplify Complex Network Operations

Single Source of Truth Automate, Manage, and Interconnect

Multi-Fabric topologies

End to End Automation Layer-3 Boundary across Zones, L2/L3

across IOS-XE, NXOS, and Multicast Overlay

Single Pane of Glass for Day-0/Day-1 Provisioning Programmability and Orchestration

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 69
 Automate VXLAN EVPN deployments
Provision a new fabric in minutes

Un-provisioned switches Cisco best practice implemented

VXLAN Fabric

Support for both brownfield and Fast, automated process

greenfield deployments
Benefit

Accelerate fabric deployments Automated consistency Minimize risk Support for both Greenfield and Brownfield deployment

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 70
Manage and Deploy
VXLAN BGP EVPN
with NDFC
VXLAN BGP EVPN Greenfield
Not on VXLAN EVPN Today?

NDFC Fabric Controller Build VXLAN fabric Templates already IP addresses, overlay pool,
Mode in few minutes embed best practices routing profiles, replication
attributes –all taken
care by NDFC

Cisco best practice

Step 2 implemented
Discover
Import switches with POAP or Day-0 config
Define switch Roles (Border, Leaf, Spine, etc)
[Optional] Create vPC pairs VXLAN Fabric

Step 1 Step 3
Create Recalculate and Deploy Fast, automated process
Define fabric settings (Underlay, Overlay) - Generates config based on intent
AS#, Replication Mode, IGP, IP Pools, etc. Preview side by side diffs

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 72
NDFC Day-0:
VXLAN EVPN
Underlay
Step1 -> Create a Fabric

Create Fabric

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 74
Step1 -> Create a Fabric
(continued)

BGP ASN

VXLANv4
or
VXLANv6

Underlay IP > /30 or /31

IGP > OSPF or ISIS

RR > 2 or 4

Distributed Anycast Gateway

PM Metrics
#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 75
 Step1 -> Create a Fabric
(continued)
BUM >
Multicast or
Ingress
Replication

L2VNI
Multicast
Group

RP > 2 or 4

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 76
Step1 -> Create a Fabric
(continued)

Cisco’s Best
Practice
Configuration
Templates

VXLAN Overlay
Mode > CLI or
Config-Profile

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 77
Step1 -> Create a Fabric
(continued)
L2VNI Label

L3VNI Label

Router ID

VRF_LITE
VTEP IP Handoff

RP IP

P2P Underlay IP
VRF_LITE IP
Range

L4-L7 Service
Network

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 78
Step1 -> Create a Fabric
(continued)

NDFC Built-In
Bootstrap POAP
Services. Supports
OOB and In band
POAP

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 79
Step2 -> Add Switches

Add Switches

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 80
Step2 -> Add Switches
(continued)

Switch Mgmt0 IP

Switch Discovery
Credentials

Switch Hops
based on CDP
Discover
Switches
VXLAN Greenfield or Brownfield

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 81
Step2 -> Add Switches
(continued)

Switch Inventory
Management

Add Switches

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 82
Step3 -> Set Role

Topology View

Switch Roles

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 83
 Step4 -> VPC Pairing
(optional)

Hierarchical Topology

Leaf VPC Pairing

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 84
Step4 -> VPC Pairing
(optional)

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 85
Step5 -> Recalculate and Deploy

Fabric
Recalculate and
Deploy

Pending
Configuration

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 86
Step5 -> Recalculate and Deploy
(continued)

Fabric
Configuration
status

Pending
Configuration

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 87
Step5 -> Recalculate and Deploy
(continued)

Spine related
features

CDP Link &

Spine RR/RP IGP configs
function

Spine EVPN
RR Client

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 88
Step5 -> Recalculate and Deploy
(continued)

VPC Best Practice

Configs

Switch running config

v NDFC Intent

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 89
NDFC VXLAN EVPN Topology View
Link Status

Fabric Fabric
Operational Configuration
View View

Fabric
Configuration
Status
Fabric Minor
Alarms

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 90
NDFC Day-1:
VXLAN EVPN
Overlay
 Deploy Network and VRF
in Data Center VXLAN EVPN
Have VXLAN EVPN fabric ready

Create Network and VRF from VXLAN EVPN

topology or fabric overview

Attach Network and VRF to switches

(optional) Preview the attached configuration

Deploy the configuration

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 92
Step1 -> Create Network and VRF

Auto
Generated
Name
L2 or L3
Network
VXLAN
Overlay VRF
Auto
Create VXLAN Generated
Overlay L2VNI Label
Network

CSV Option

Network SVI

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 93
Step2 -> Attach Network and VRF

Attach
Network to
Switches

Select
Switches

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 94
 Step2 -> Attach Network and VRF
(continued)

Port
Attachment

Switches

Switch Port Type

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 95
Network and VRF Configs

VXLAN Tenant
L3VNI VRF
Configs

L2VNI
Network
Configs
NVE Tunnel
Configs

L3 SVI
Configs

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 96
Network and VRF Deployment Status
In VXLAN EVPN Fabric

VXLAN Overlay
Deployment Status

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 97
Manage and Deploy
External IP Handoff
with NDFC
VRF-LITE: Border to Nexus Edge
Topology and IFC considerations
External Network

ASN 65100
VRF
“CORP”
• IFC Automated

• Advertise Host (disabled) Edge Router

”Nexus 9000”
• Advertise Default-Route (enabled) E1/3.2 – 10.33.0.1/30

• Config Static 0/0 Route (enabled) “Nexus 9000”

E1/3.2 – 10.33.0.2/30

Border Leaf

VRF
“CORP”

ASN 65001
VXLAN EVPN

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 99
Managing Edge Devices
part of “External Connectivity Network” Fabric
• Create Fabric, Discover Switches, Set Role, and Recalculate & Deploy

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 100
 VRF-LITE: Border to Nexus Edge
Defining IFC deployment type
• Review Fabric Settings for VRF-Lite IFC deployment type:

LAN > Fabrics > Select (your DC VXLAN Fabric) > Actions > Edit Fabric > Resources Tab
Per VRF Per Dotq1
association

Select
Deployment Type

Deploy VRF-LITE if
Edge device is
Nexus and
managed by NDFC

eBGP peering
subnet details

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 101
VRF-LITE: Border to Nexus Edge
Defining IFC Link on physical interface
• IFC link has been defined
• Policy should be ext_fabric_setup
o IPs auto selected from VRF-Lite Subnet IP Range

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 102
VRF-LITE: Border to Nexus Edge
Defining VRF extensions on Border

• Verify DC VXLAN Fabric VRFs were created and customize if needed

• LAN > Fabrics > Double click (your DC VXLAN Fabric) > VRFs

Edit VRF if you want to modify

route advertisement or other
VRF specifics

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 103
VRF-LITE: Border to Nexus Edge
Defining VRF extensions on Border

Define according to your

use case

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 104
VRF-LITE: Border to Nexus Edge
Defining VRF extensions on Border

• Attach VRF to Border and extend through VRF-Lite

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 105
VRF-LITE: Border to Nexus Edge
Defining VRF extensions on Border

• Attach VRF to Border and extend through VRF-Lite

Extend Options for Border

Extend Options for Border Gateway

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 106
VRF-LITE: Border to Nexus Edge
Defining VRF extensions on Border

Physical Interface
MTU

Provide VRF name for

External Edge. By default,
NDFC uses the same name
Extension Destination Edge
as Border VRF
Type router details

Provide Route-MAP
if different from
NDFC default

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 107
VRF-LITE: Border to Nexus Edge
Preview and Deploy VRF extensions on Border

L3 Tenant
CORP Advertisement of Default route
towards site internal VTEP

L3VNI VRF
Configs

eBGP External
Edge neighbor
Default route static
towards External Edge

Border Physical
Interface Config

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 108
VRF-LITE: Border to Nexus Edge
Preview and Deploy VRF extensions on External Edge
• Once configurations are deployed on Border Leaf (DC VXLAN EVPN), navigate to
External Network Fabric and perform Recalculate and Deploy

VRF Configs

eBGP Border
Leaf neighbor

Edge Physical
Interface Config

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 109
Enabling VRF-Lite: Manual
Border Leaf to Non-Nexus Edge Router

Step 2 Step 4
IFC prototypes
External Network Fabric VRF-Lite Extension
Define IFC Type: “Manual” Define sub-interfaces, invoke BGP policies
Define IFC link on physical interface NDFC will generate Sub-int and eBGP peering on Edge router
Recalculate config and deploy Deploy on External Network Fabric

Step 1 Step 3
Create and Import DC VXLAN EVPN Fabric VRF-Lite Extension
Define VXLAN EVPN and External Fabric. Set Define Individual VRF extension on the Border leaf
respective roles (e.g. Border, BGW, Edge)
NDFC will generate Sub-int and eBGP peering on Border leaf
Uncheck Fabric monitor mode in External
Deploy on Data Center VXLAN EVPN Fabric
Fabric if the Edge router is in managed mode
For Non-Nexus ensure SNMP configs for
discovery

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 110
VRF-LITE: Border to Non-Nexus Edge
Topology and IFC considerations
External Network

ASN 65099
VRF
“CORP”
• IFC Manual

• Advertise Host (disabled) Edge Router

”ASR 9000”
• Advertise Default-Route (enabled) Gig0/0/0/0.2 – 10.10.10.1/31

• Config Static 0/0 Route (enabled) “Nexus 9000”

E1/1.2 – 10.10.10.0/31

Border Leaf

VRF
“CORP”

ASN 10
VXLAN EVPN

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 111
VRF-LITE: Border to Non-Nexus Edge
Defining IFC deployment type
• Review Fabric Settings for VRF-Lite IFC deployment type:

LAN > Fabrics > Select (your DC VXLAN Fabric) > Actions > Edit Fabric > Resources Tab

Per VRF Per Dotq1

association

Select
Deployment Type

eBGP peering
subnet details

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 112
VRF-LITE: Border to Non-Nexus Edge
Defining IFC Link on physical interface
• Verify link from DC VXLAN Fabric to External Fabric was discovered

LAN > Fabrics > Double Click (your DC VXLAN Fabric) > Links

Use filters when having multiple fabrics

If no policy exists, or
neighbor not found. A link
can be created manually

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 113
VRF-LITE: Border to Non-Nexus Edge
Defining IFC Link on physical interface

• If Link discovered, values are pre-filled

• When not, make sure:

• Link Type is Inter-Fabric

• Link Sub-Type is VRF_Lite

• Link Template: ext_fabric_setup

External Fabric
“WAN-EXT”
ASN 65099
Edge Router

Gig0/0/0/0.2 – 10.10.10.1/31

E1/1.2 – 10.10.10.0/31
Border Leaf

ASN 10
DC Fabric
“VXLAN”

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 114
VRF-LITE: Border to Non-Nexus Edge
Defining VRF extensions on Border

• Verify DC VXLAN Fabric VRFs were created and customize if needed

• LAN > Fabrics > Double click (your DC VXLAN Fabric) > VRFs

Edit VRF if you want to modify

route advertisement or other
VRF specifics

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 115
VRF-LITE: Border to Non-Nexus Edge
Defining VRF extensions on Border

Define according to your

use case

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 116
VRF-LITE: Border to Non-Nexus Edge
Defining VRF extensions on Border

• Attach VRF to Border and extend through VRF-Lite

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 117
VRF-LITE: Border to Non-Nexus Edge
Defining VRF extensions on Border

• Attach VRF to Border and extend through VRF-Lite

Extend Options for Border

Extend Options for Border Gateway

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 118
VRF-LITE: Border to Non-Nexus Edge
Preview and Deploy VRF extensions on Border

Default route static

towards External Edge

Advertisement Default
route towards site internal

eBGP External
Edge neighbor

Border Physical
Interface Config

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 119
VRF-LITE: Border to Non-Nexus Edge
Defining the policies on the Edge Router

• After completing the configurations on the VXLAN Fabric (Border Leaf), Navigate to
External Fabric (Edge Router) and apply the following policies
• ios_xr_base_bgp

Policy = Ios_xr_base_bgp

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 120
VRF-LITE: Border to Non-Nexus Edge
Defining the policies on the Edge Router
• After completing the configurations on the VXLAN Fabric (Border
Leaf), Navigate to External Fabric (Edge Router) and apply the
following policies
• ios_xr_base_bgp
• ios_xr_Ext_VRF_Lite_Jython

Policy =
External Fabric Ios_xr_Ext_VRF_Lite_Jython
“WAN-EXT”

Edge Router

Gig0/0/0/0.2 – 10.10.10.1/31

E1/1.2 – 10.10.10.0/31
Border Leaf

ASN 10
DC Fabric
“VXLAN”

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 121
VRF-LITE: Border to Non-Nexus Edge
Deploy configs on the Edge Router

IOS-XR eBGP Policy

for allowing routes

IOS-XR VRF Definition

eBGP Border
Leaf neighbor
IOS-XR Physical
Interface Config

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 122
Verification and
Validation with NDFC
Verification through NDFC
Keeping you away from CLI

Step 2
Deployment History
Configuration Execution Status:
Verify Deployment History Status Success for
Underlay, Overlay, Interfaces, and more

Step 1 Step 3
Verify Network and VRF attachments Show commands
Job Execution Status: Service / features status
Network Status Deployed (CLI through NDFC)
VRF Status Deployed

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 124
 Attachment deployment status
Job execution perspective

Success or Failure deployment details

Multi-Stage Preview and Deployment

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 125
Deployment History Tool
Commands execution perspective

CLI response messages for

easier troubleshooting

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 126
Show Commands Tool
Switch Config perspective

NDFC pre-built
commands or user
commands

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 127
Conclusion
Key points to remember
NDFC simplifies automation and
management of VXLAN EVPN fabrics
using Cisco’s best practices

NDFC provides flexible design options

alongside automation, consistency,
compliance, and management for
VXLAN EVPN and Multi-Site

NDFC provides a single plane of glass

solution to automate and manage
Nexus and Non-Nexus devices

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 128
Complete Your Session Evaluations

Complete a minimum of 4 session surveys and the Overall Event Survey to be

entered in a drawing to win 1 of 5 full conference passes to Cisco Live 2025.

Earn 100 points per survey completed and compete on the Cisco Live
Challenge leaderboard.

Level up and earn exclusive prizes!

Complete your surveys in the Cisco Live mobile app.

#CiscoLive BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 129
 • Visit the Cisco Showcase
for related demos

• Book your one-on-one

Meet the Engineer meeting

Continue • Attend the interactive education

with DevNet, Capture the Flag,
your education and Walk-in Labs

• Visit the On-Demand Library

for more sessions at
www.CiscoLive.com/on-demand

Contact me at: Cisco Webex

BRKDCN-2918 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 130
Thank you

#CiscoLive