networks - 04

MPLS

Latest 10 Articles

EVPN design - EVPN eBGP over IPv4 eBGP. [ipSpace]
Per-prefix and per-VRF MPLS/VPN and EVPN labels/VNIs. [ipSpace]
EVPN design - eBGP everywhere. [ipSpace]
One-arm hub-and-spoke VPN with MPLS/VPN. [ipSpace]
EVPN hub-and-spoke Layer 3 VPN. [ipSpace]
Hub-and-spoke VPN topology. [ipSpace]
Hub-and-spoke on a single PE router. [ipSpace]
PMTUD in MPLS networks. [Lost In Transit]
EVPN design - scaling iBGP with RRs. [ipSpace]
BGP, EVPN, VXLAN, or SRv6? [ipSpace] [LinkedIn]

Older Articles

What exactly is MPLS? [ipSpace]
Can we skip the network layer? [ipSpace]
EVPN designs - iBGP full mesh between leaf switches. [ipSpace]
Cisco VPC in VXLAN/EVPN network. [Lost In Transit: part 1, part 2, part 3]
LISP vs EVPN - mobility in campus networks. [ipSpace]
SR/MPLS security framework. [ipSpace]
EVPN designs: spine and leaf fabric. [ipSpace]
Multi-vendor EVPN fabrics. [ipSpace]
Routed packet walk in VXLAN/EVPN network. [Lost In Transit]
EVPN - asymmetric vs symmetric IRB. [Lost In Transit]
EVPN terminology. [Lost In Transit]
Bridging packet walk in VXLAN/EVPN network. [Lost In Transit]
EVPN route type 5. [Lost In Transit]
Advertising IPs in EVPN route type 2. [Lost In Transit]
EVPN deep dive route types 2 and 3. [Lost In Transit]
VRF without route target - will the route be exported? [Lost In Transit]
Configuring EVPN on NX-OS. [Lost In Transit]
SD-WAN vs MPLS: what's the difference & which is better? [Network Computing]
Introduction of EVPN at DE-CIX. [ipSpace] [APNIC]
Dataplane MAC learning with EVPN. [ipSpace]
Layer 3 handoff (L3Out) in VXLAN/EVPN fabrics. [ipSpace]
Silent hosts in EVPN fabrics. [ipSpace]
Pros and cons of EVPN-based multihoming. [ipSpace]
Is dynamic MAC learning better than EVPN? [ipSpace]
Multi-vendor EVPN fabrics. [ipSpace]
Was MPLS TE worth the effort? [ipSpace] [Systems Approach]
DHCP relaying in EVPN VRFs. [ipSpace]
Studying EVPN to prepare for a job interview. [ipSpace]
vPC Fabric Peering with EVPN Multihoming. [ipSpace]
CE-to-CE iBGP session in a multihomed site. [ipSpace]
MLAG vith EVPN deep dive. [ipSpace]
Troubleshooting EVPN control plane. [ipSpace] [DC Overlords]
Scalable aspects of SR-MPLS. [ipSpace]
On the applicability of MPLS Segment Routing (MPLS-SR). [ipSpace]
EVPN VLAN-aware bundle service. [ipSpace]
EVPN/MPLS bridging forwarding model. [ipSpace]
SR-MPLS or SRv6 for greenfield networks. [ipSpace]
Do you care about MPLS in 2022? [ipSpace]
Netsim tools - MPLS edition. [ipSpace]
Is MPLS/VPN too complex? [ipSpace]
MPLS/LDP creation myths. [ipSpace]
Hub-and-Spoke VPLS: revenge of LDP. [ipSpace]
Segment routing vs LDP in hub-and-spoke networks. [ipSpace]
RSVP TE on JUNOS sample lab. [ipSpace] [Github]
LDP and Anycast routing. [Routing Craft]
- Anycast works just fine with MPLS/LDP. [ipSpace]
- EVPN/VXLAN complexity. [ipSpace]
- ARP problems in EVPN. [Routing Craft] [ipSpace]
- WISP cheat sheet for MPLS/VPLS MTU. [StubArea51]
- MPLS VPLS:
  - Part 1 - Use cases. [StubArea51]
- Comparing EVPN with flood-and-learn fabrics. [ipSpace]
- Intro to EVPN with Tony Bourke: [YouTube]
  - Part 01 - Introduction. [Packet Pushers]
  - Part 02 - Why EVPN? [Packet Pushers]
  - Part 03 - Building EVPN topologies, underlays and overlays. [Packet Pushers]
  - Part 04 - Understanding EVPN VRFs - part 1. [Packet Pushers]
  - Part 05 - Understanding EVPN VRFs - part 2. [Packet Pushers]
  - Part 06 - Stretched VLANs and STP. [Packet Pushers]
  - Part 07 - VRF L3VNI with Symmetric IRB. [Packet Pushers]
  - Part 08 - Hardware requirements. [Packet Pushers]
  - Part 09 - Populating L3VNI routes + type 2 sequence numbers. [Packet Pushers]
  - Part 10 - Self learning & labbing recommendations. [Packet Pushers]
- BGP-free MPLS core with segment routing. [ipSpace]
- Segment routing segment IDs and MPLS labels. [ipSpace]
- MPLS label distribution. [ipCisco]
- EVPN – the great unifying theory of VPN control planes? [ipSpace]
- Next-hop and VTEP reachability in EVPN networks. [ipSpace]
- Pragmatic EVPN design. [ipSpace]
- The EVPN/BGP saga continues. [ipSpace]
- EVPN Auto-RD and duplicate MAC addresses. [ipSpace]
- EVPN route targets, route distinguishers, and VXLAN network IDs. [ipSpace]
- The EVPN dilemma. [ipSpace]
- MPLS in DC fabrics. [NANOG YouTube]
- VMware NSX killed my EVPN fabric. [ipSpace]
- MPLS is dead - long live MPLS. [Network Collective]
- Demystifying IPv6 over MPLS -- connecting IPv6 islands. [NANOG YouTube]
- EVPN use cases. [Russ White] [SDX Central]
- Is MPLS dead? [Network Chuck YouTube]
- History of networking - pseudowires. [Network Collective] [Russ White]
- Why is MPLS segment routing better than LDP? [ipSpace]
  - Principles of MPLS-based transport core. [ipSpace]
  - Drawbacks of MPLS label distribution with LDP. [ipSpace]
  - MPLS label distribution in Segment Routing. [ipSpace]
- History of networking: MPLS-TE. [Network Collective] [Russ White]
- Building blocks in EVPN for multi-service fabrics. [NANOG 75 YouTube]
- MPLS and ExaBGP. [ipSpace] [Jon Langemak]
- Q-in-Q support in multi-site EVPN. [ipSpace]
- History of EVPN. [Network Collective]
- The Internet is the future of the WAN, but MPLS will persist. [Network Computing]
- Recent changes in LSR protocols. [Russ White]
- MPLS fast reroute. [Russ White]
- Using MPLS+EVPN in the data center fabrics. [ipSpace]
- EVPN behind the curtain. [Russ White] [Cumulus Networks]
- History of networking - MPLS. [Network Collective] [Russ White]

- MPLS intro series:
  - Destination routing. [PacketU]
  - Understanding a simple LSP. [PacketU]
  - Introduction to VPNv4. [PacketU]
  - VPNv4 packet walk. [PacketU]
  - Customer connection with BGP. [PacketU]
  - Route reflectors. [PacketU]
  - MPLS and VRFs. [PacketU]
- Evolved campus core -- EVPN for everyone else. [NANOG YouTube]
- When to use a VRF. [PacketU]
- Packet size - it matters. [PacketU]
- MPLS label switching. [ipCisco]
- Typical EVPN BGP routing designs. [ipSpace]
- What is EVPN? [ipSpace]
- EVPN route target considerations. [ipSpace]
- MPLS:
  - Part 1. [Network Collective] [Russ White]
  - Part 2: VPNs. [Network Collective] [Russ White]
  - Part 3: Traffic Engineering. [Network Collective] [Russ White]
  - Part 4: Fast Reroute. [Network Collective]
- Who's pushing L2 VPN services? [ipSpace]
- EVPN with MPLS data plane in data centers. [ipSpace]
- EVPN is more than VPLS on steroids. [ipSpace]
- Using EVPN in very small data centre fabrics. [ipSpace]
- BGP in EVPN-based data centre fabrics. [ipSpace, ipSpace]
  - BGP in EVPN-based data centre fabrics - part 2. [ipSpace]
- VPWS - Virtual Private Wire Service. [ipCisco: part 1]
- Distributed Epipe service configuration. [ipCisco]
- How does VPRN work? [ipCisco]
  - VPRN labels. [ipCisco]
  - VPRN control plane. [ipCisco]
  - VPRN data plane. [ipCisco]
- VPLS overivew. [ipCisco]
  - VPLS topologies. [ipCisco]
  - VPLS MAC learning. [ipCisco]
  - VPLS labelling. [ipCisco]
- The end of MPLS? [Jason Wells]
  - Reaction. [Russ White]
- Cisco buying SD-WAN startup Viptela may herald a sunset for MPLS. [DC Knowledge]
- MPLS tutorial. [NANOG YouTube]
- DMEVPN: DMVPN-based alternative to EVPN. [Packet Pushers]
- RSVP-TE deep dive. [Packet Pushers: part 1, part 2]
- MPLS-TE design. [Packet Pushers: part 1, part 2, part 3]
- Back to basics: label distribution and assignment modes. [Packet Pushers]
- MPLS lives on in the hybrid WAN era. [Network Computing]
- Do enterprises need MPLS? [ipSpace]
- Creating a PCE prototype. [Russ White] [PacketPushers]
- Label Switched Multicast (LSM) -- an introduction. [Packet Mischief]
  - LSM configuration. [Packet Mischief]
- CCDE -- MPLS-TE auto tunnels. [Lost In Transit]
- The Importance of BGP NEXT_HOP in L3VPNs. [Packet Mischief]
  - The correct mask for a PE's loopback. [Packet Mischief]
- LDP label allocation revisited. [ipSpace]
- MPLS "no label" vs "pop label". [Packet Mischief]
- Walking with packets: traceroute through MPLS cloud. [Packet Mischief]
- EVPN -- the essential parts. [Aldrin Isaac, presentation]
- Combining MPLS/VPN, MPLS-TE, and QoS on MPLS talks. [ipSpace, video]
- Unique RD per PE in MPLS VPN for load sharing and faster convergence. [LostInTransit]
- How does MPLS-TE interact with QoS? [ipSpace, video]
- MPLS tech talks: MPLS TE 101. [ipSpace]
- FECs, LDP and BGP in an MPLS world. [ipSpace]
- Handling the bottom of MPLS stack. [ipSpace]
- MPLS-TE basics. [ipSpace]
- The essence of MPLS. [ipSpace]
- A quick look at MPLS-TE. [LostInTransit]
- Load sharing in MPLS core. [ipSpace]
  - MPLS load sharing - data plane considerations. [ipSpace]
- Replacing an MPLS WAN with an Internet VPN overlay. [PacketLife]
  - Could you replace MPLS/VPN with IPSec-over-Internet? [ipSpace]
- eBGP vpnv4 MPLS VPN over multiple BGP autonomous systems - getting correct next hop? [Reddit]
- Cisco: Is it possible to see an EFP path within an EVC? [StackExchange]
- EoMPLS, MSTP, and RPVST. [StackExchange]
- BGP MPLS vpn design question. [Reddit]
- Deploying a datacenter MPLS/VPN on JUNOS. [PacketLife]
- Inter-VRF routing IOS XR. [Reddit]
- Network design question on MPLS and Layer 2 domains. [Reddit]
- MPLS requires custom silicon -- really? [ipSpace]
- MPLS can reroute, but can it route? [StackExchange]
- Using VPLS as an underlay to DIY an MPLS core - why not? [Reddit]
- Dual stack OSPF redistribution in MP-BGP. [StackExchange]
- Simple guide to MPLS/L3VPN with Junos. [Reddit] [JuniperLabs] [JuniperLabs]
- MPLS lab examples for Juniper routers. [Reddit]
- Interfacing overlay virtual networks with MPLS/VPN WAN. [ipSpace]
- How are the LERs determined in an MPLS LSP using LDP? [StackExchange]
- Overlay networking & VXLAN means MPLS in the data center is dead. [EtherealMind]
- There is an MPLS for every occasion. [EtherealMind]
- Internet traffic gets MPLS labels when you deploy MPLS/VPN. [ipSpace]
- Junos vs IOS: MPLS and LDP. [ipSpace]
- Label to route mapping, label generation scalability. [StackExchange]
- Can two peering LSR generate same label to same or different prefix and distribute among them? [StackExchange]
- What is a FEC in MPLS domain? [StackExchange]
- Is it possible to run an IPv6-only MPLS core? [StackExchange]
- Eliminating VLANs and fragility in underlay with network. [NetworkStatic]
- Using a GRE tunnel VRF to separate the physical interface. [PacketU]
- VRF export maps. [PacketLife]
- VPLS multihoming on Junos -- FEC confusion. [StackExchange]
- LDP in MPLS. [StackExchange]
- MPLS/VPN carrier's carrier -- myth or reality? [ipSpace]
- Introduction to VRF Lite. [PacketLife]
  - Inter-VRF routing with VRF Lite. [PacketLife]
  - Route distinguishers and route targets. [PacketLife]
- Load balancing across multiple MPLS/VPN providers. [ipSpace]
- Could you run an MPLS-TE-only MPLS/VPN network without LDP? [ipSpace]
- Edge protocol independence: another benefit of edge-and-core layering. [ipSpace]
- Extending MPLS/VPN to customer sites. [ipSpace]
- Secondary MPLS-TE tunnels and FRR. [ipSpace]
- Is it safe to run Internet in a VRF? [ipSpace]
  - Internet-in-a-VRF and LFIB explosion. [ipSpace]
- BGP route reflection in MPLS/VPN PE-routers. [ipSpace]
- LDP-IGP synchronisation in MPLS networks. [IOS Hints]
- MPLS is not tunneling. [IOS Hints]
- PHP demystified. [IOS Hints]
- Asymmetric MPLS MTU problem. [IOS Hints]
- The MPLS MTU challenges. [IOS Hints]
- Random MPLS/VPN Q&A. [IOS Hints]
- Building CsC-enabled MPLS backbone. [IOS Hints]
- MPLS/VPN transport options. [IOS Hints]
- MPLS VPN with common services. [PacketLife]
- MPLS/VPN common services design. [IOS Hints]
  - Scalability of common services MPLS/VPN topology. [IOS Hints]
- Creating an MPLS VPN. [PacketLife]
- MPLS/VPN-over-GRE-over-IPsec: does it really work? [IOS Hints]
- Load sharing in MPLS/VPN networks with route reflectors. [IOS Hints]
- Building MPLS/VPN services across an enterprise WAN. [IOS Hints]
- Campfire: the true story of MPLS. [IOS Hints]
- MPLS VPNs in enterprise networks. [IOS Hints]
- Scaling VPLS. [CCIE Blog]
- VPLS is a technology, not just a service provider offering. [IOS Hints]
- Solving the MPLS/VPN QoS challenge. [IOS Hints]
- What is MPLS-TP and is it relevant? [IOS Hints]
- QoS over MPLS/VPN networks. [IOS Hints]
- MPLS tunnels explained. [CCIE Blog]
- Scaling MPLS network. [CCIE Blog]
- Troubleshooting tips for MPLS. [CCIE Blog]
- MPLS and EIGRP: going the (admin) distance. [CCIE Blog]
- What's the difference between MPLS and IP? [Search Telecom] [IOS Hints]
- MPLS components. [CCIE Blog - part 1] [CCIE Blog - part 2]
- Is LDP required for VPNv4 labels? [MPLS VPN]
- How does LDP initialise? [MPLS VPN]
- What are downstream and upstream routers in MPLS? [MPLS VPN]
- MPLS and QoS DiffServ. [CCIE Blog]
- Types of pseudowire. [MPLS VPN]
- The advantages of MPLS. [MPLS VPN]
- MPLS TE autoroute basics. [IOS Hints] [NIL Wiki]
- Inter-VRF routing with VRF Lite. [PacketLife]
- Do you use MPLS to transport Internet traffic? [IOS Hints]
- Using MPLS and M-LDP signaling for multicast VPNs. [CCIE Blog]
- MPLS control plane and forwarding plane interaction. [CCIE Blog]
- The MPLS forwarding plane - LDP. [CCIE Blog]
- The MPLS forwarding plane. [CCIE Blog]
- Why do we need MPLS? [CCIE Blog]
- Understanding modern VPN service offerings. [IOS Hints] [NIL Wiki]
- The long road to M-LSPs. [CCIE Blog]
- Intro to VRF Lite. [PacketLife] [CCIE Journey]

Overview

Originally created to enable faster routing/switching of packets, but today's hardware/wire-speed routers make this benefit redundant.
MPLS header -- 4 bytes:
- 20 bits: label
- 3 bits: experimental (used for QoS)
- 1 bit: bottom of label stack
- 8 bits: TTL

Router Types

P = LSR = label switch router
PE = LER = label edge router

Operation

When an unlabelled packet arrives, the PE router determines the FEC and assigns a label before forwarding.
When a labelled packet arrives, either:
- swap
- push (add label, i.e. encapsulate/stack)
- pop
FEC = forwarding equivalence class:
- typically based on the destination IP + QoS.
- usually corresponds to an LSP.
PHP = penultimate hop popping: "implicit null" label = 3

MPLS VPNs

RFC 2547bis → RFC 4364
Define VRF and add interfaces to the VRF.
Each VRF has its own FIB (in addition to the main FIB).
Each prefix is tagged with the RD for that VRF (to maintain address space separation):
- uses extended communities (RT = route target).
Layer 2 VPNs
Layer 3 VPNs

Comparison

There are advantages and disadvantages for each approach, so one needs to consider those in addition to:

- The current/future requirements of the service.
- The existing provider infrastructure.
- Cost.

Type of Traffic

- L3: IP only.
- L2: multiple protocols (IPv4, IPv6, IPX, DECnet, OSI, etc.).

Connectivity Scenarios

Point-to-point
Hub and spoke
Partial mesh
Full mesh
Overlapping VPNs

- L3: does 1, 4, 5 well (2 and 3 are more complex).
- L2: does 1, 2, 3, 4 well (5 more complex).

Scalability

Both: maximum number of LSPs and/or VCs supported on a PE router.
Both: maximum configuration file size:
- L3: VRF, RD, extended communities, filtering policies.
- L2: VPN peer PEs, ports associated with VPNs.
L3: maximum number of routs (use summarisation).
L2: maximum number of layer 2 forwarding table entries (require that CE be a router to limit the number of MACs per VPN).

Deployment

L3:
- usually require large PE routers.
- BGP knowledge/expertise.
- possible Route Reflector change to avoid overload.
- confederations require inter-AS VPNs.
L2:
- simpler PE routers.
- no BGP.

Provisioning

L3:
- design routing for VPN topology.
- assign RDs and RT communities (and configure).
- CE → PE peering configuration.
L2:
- PE → PE in a VPN to establish VCs.
- assign CE interfaces to VPN.

Management / Maintenance

L3:
- BGP peering sessions.
- BGP routes with different extended communities (multiple tables/VRF).
- BGP route propagation and selection.
- CE peering.
- potentially large configuration files.
L2:
- no BGP peering (unless used for VPN signalling).
- no customer routes.
- VCs that make up the VPN.
- ports assigned to a VPN.
- VFI MAC tables.

Cost

L3:
- probably a bit more expensive for deployment (higher hardware demands).
- higher management/maintenance costs.

IOS Configuration Example

ip vrf <vrf-name>

rd xxx:yyy

route-target export aaa:bbb

route-target import aaa:bbb

mpls label protocol ldp

mpls ldp neighbor x.x.x.x password <...>

interface <...>

description <...>

mtu 1530

ip address x.x.x.x

ip ospf message-digest-key <...>

ip ospf network point-to-point

ip ospf cost <...>

tag-switching ip

router bgp

<...> ! all the usual stuff here

address-family ipv4 vrf <vrf-name>

redistribute connected

max-paths 2

no auto-summary

no synchronization

JUNOS Configuration Example

protocols

ldp

import|export

keepalive-[interval|timeout]

preference <...>

interface <...>

enable|disable

hello-interval

hold-time

transport-address

protocols

mpls

interface <...>

unit <...>

family mpls

routing-instances

<vrf-name>

instance-type vrf

interface <...>

route-distinguisher xxx:yyy

protocols

bgp

<...> ! all the usual stuff here

vrf-import

vrf-export