Enabling a Converged World™

Carrier Ethernet
 Carrier Ethernet

Carrier Ethernet
Introduction
Ethernet is—far and away—the dominant LAN technology used in networks today. Due to its
speed, simplicity, plug-and-play capability, multipoint connectivity, and low cost it has been widely
adopted and deployed.

In its first stage of adoption, advanced features such as full-duplex interfaces, and advanced
management capabilities, increasing speed (from 10 Mbps to 100 Mbps, and virtual LANs, or
VLANs), enabled Ethernet to overcome other technologies such as token ring, FDDI, and ATM in the
enterprise.

In the second stage, Ethernet switches became more reliable and feature rich and Ethernet became
the technology of choice for network POP interconnections and point-to-point WAN links. Service
providers began using Ethernet instead of the more expensive packet over SONET (PoS) to
interconnect routers, and chose Ethernet over token ring and FDDI to interconnect servers and
storage in major data centers. As a result, Ethernet became widely used and understood in
enterprise networks, while service providers chose Ethernet for inexpensive, reliable Layer 2
transport whenever they could.

Then customers began asking for native Ethernet connectivity in metro and access networks over
legacy technologies such as lower-speed time division multiplexed (TDM) circuits, highly complex
ATM, or high-cost PoS. This lead to the third stage—the adoption of Ethernet in access and metro
networks and the development of “carrier” Ethernet services such as Ethernet Internet access, IP-
based virtual private networks, and multicast networks. With Ethernet transport speeds from 1 Gbps
to 10 Gbps now available, and customer demand continuing to grow, service providers face the
challenge of offering scalable services that make full use of their converged legacy and Ethernet
networks.

26601 W. Agoura Road · Calabasas, CA 91302 USA · Tel + 1-818-871-1800 · Fax + 1-818-871-1805 · www.ixiacom.com
25 September 2007 – Page 1 of 11
 Carrier Ethernet

Benefits of Carrier Ethernet

For Service Providers
• Technology convergence provides CAPEX and OPEX reductions
• Access network technologies leverage Ethernet to provide backhaul—the ability to move
volumes of traffic from subscribers to broadband aggregation devices adjacent to the core
network—for IP DSLAMs, PON, WiMAX, and direct Ethernet over fiber/copper
• Flexible Layer 2 VPN services, including private line, virtual private line, or emulated LAN
offer new revenue streams

For the Enterprise

• A converged network for VoIP, data, video conferencing, and other services reduces overall
costs
• Standardizing on Ethernet reduces complexity and benefits IT staff support and training
budgets
• High-speed, low-latency service is easily upgraded by changing the service policy

The Goal: ‘Carrier-Grade’ Ethernet

The first native Ethernet services to emerge were point to point-based, followed by emulated LAN
(multipoint to multipoint-based). Services were first defined and limited to metro area networks.
They have now been extended across wide area networks and are available worldwide from many
service providers. The term “carrier Ethernet” implies that Ethernet services are “carrier grade.”
The benchmark for carrier grade has been set by the legacy TDM telephony networks to describe
services that achieve “five nines (9.9999%)” uptime. Although it is debatable whether carrier
Ethernet will reach that level of reliability, the goal of one particular standards organization is to
accelerate the development and deployment of services that live up to the name.

Carrier Ethernet looks to become the major component of next-generation metro area networks,
which serve as the aggregation layer between customers and core carrier networks. A metro
Ethernet network, which uses IP Layer 3 MPLS forwarding, is currently the hotbed of carrier Ethernet
activity.

The Metro Ethernet Forum (MEF) is a global industry alliance started in 2001. In 2005, the MEF
formally defined carrier Ethernet services. Members of the MEF include the majority of the top
network equipment manufactures and service providers in the world. Members serve on technical
committees that author technical specifications defining services, architecture, management, and test
and measurement. The MEF 6 specification defines carrier Ethernet services.

26601 W. Agoura Road · Calabasas, CA 91302 USA · Tel + 1-818-871-1800 · Fax + 1-818-871-1805 · www.ixiacom.com
25 September 2007 – Page 2 of 11
 Carrier Ethernet

MEF 6-defined basic connection point terminology:

• UNI: user network interface

• UNI-C: UNI-customer side
• UNI-N: UNI-network side
• NNI: network-to-network interface
• E-NNI: external NNI
• I-NNI: internal NNI
• EVC: Ethernet virtual circuit
• MEN: metro Ethernet Network
• TLS: transparent LAN services
• SLA: service-level agreement
• SLS: service-level specification
• MEF 6 also defines service types: EPL: Ethernet private line
• EVPL: Ethernet virtual private line
• E-Line: Ethernet line
• ELAN: Ethernet LAN

E-Line defines a service type that uses a point-to-point EVC, which can be delivered with an EPL or
EVPL. An EPL uses a unique physical UNI port at each end. An EVPL uses a virtual port (VLAN) at
each end of the service. ELAN defines a service type that uses a multipoint-to-multipoint EVC. It is
important to note that, while the standard does specify Ethernet as the technology connecting
customer equipment to a service provider network, it does not define what underlying technology is
used to deliver the service.

Typically, service providers provide customers service-level agreements or service-level specifications

that guarantee the details of a service’s performance. Ethernet service attributes are found in MEF
specification 10.1. Service attributes include:

• Physical layer service attributes

o 10 Mbps full duplex
o 100 Mbps full duplex
o 10/199 Mbps Auto-negotiation full duplex
o 1 Gbps full duplex
o 10 Gbps Full duplex
• Bandwidth Profile Service Attribute
o CIR: Committed Information Rate
o CBS: Committed Burst Size
o EIR: Excess Information Rate

26601 W. Agoura Road · Calabasas, CA 91302 USA · Tel + 1-818-871-1800 · Fax + 1-818-871-1805 · www.ixiacom.com
25 September 2007 – Page 3 of 11
 Carrier Ethernet

o EBS: Excess Burst Size

o CM: Color Mode

EVC1 Bandwidth Profileper EVC1

UNI EVC2 Bandwidth Profile per EVC2

EVC3 Bandwidth Profile per EVC3

Figure 1: An Ethernet virtual connection is defined between UNI(s). Service attributes are first
defined and then applied to the UNI and EVC.

Technology That Enables Carrier Ethernet

Carrier Ethernet Architecture

Figure 2: In typical carrier Ethernet architecture, there are three main components: access,
aggregation/metro, and core. By definition, access must be Ethernet. Technologies used in
aggregation networks and the core can vary.

Aggregation/Metro Ethernet Technologies

Legacy metro transport networks are built primarily of time-division multiplexing technology. TDM
“circuit-switched” services are optimized for delivering voice. The underlying technology of a TDM
network is a SONET/SDH ring. A TDM network consists of digital multiplexers, digital access cross-

26601 W. Agoura Road · Calabasas, CA 91302 USA · Tel + 1-818-871-1800 · Fax + 1-818-871-1805 · www.ixiacom.com
25 September 2007 – Page 4 of 11
 Carrier Ethernet

connects (DACs), SONET/SDH add/drop multiplexers (ADMs), and SONET/SDH cross-connects.

The benefit of this type of service is that subscribers receive the specific bandwidths they have
ordered. A major challenge is that TDM interfaces do not offer much flexibility in bandwidth
options. Significant and expensive steps are required, such as fractional T1 (64 Kbps), full T1 (1.5
Mbps), DS3 (45 Mbps), OC3 (155 Mbps), OC12 (622 Mbps), OC 192 (10 Mbps). An increase
in bandwidth requires new circuit provisioning plus, typically, a truck roll to install the service. In
many markets, particularly in the United States, there is a significant installed base and investment
in the TDM infrastructure. As service providers look to enable new packet-based services, they
would like to leverage legacy investments where possible. Several new protocols are designed to
accomplish that.

Generic Framing Procedure (GFP)

Generic framing procedure (GFP) is defined by ITU-T G.7041. It allows mapping of variable-length,
higher-layer client signals over SDH/SONET. GFP can be used to encapsulate native Ethernet
frames into a SONET payload. There are two modes of GFP: generic framing procedure-framed
(GFP-F) and generic framing procedure-transparent (GFP-T). GFP-F maps each client frame into a
single GFP frame. GFP-F is used where the client signal is framed, or “packetized,” by the client
protocol. GFP-T allows mapping of multiple 8b/10-b block-coded client data streams into an
efficient 64b/65b block code for transport within a GFP frame. GFP-F is used when optimal
bandwidth efficiency is desired at the expense of latency. GFP-T is used when low latency transport
is required. When GFP is enabled it is commonly used in conjunction with virtual concatenation
(VCAT). VCAT is an inverse multiplexing technique used to split SONET/SDH bandwidth into
logical groups, which may be transported or routed independently. VCAT — specified in ITU-T
G.7043 — provides the ability to multiplex many services onto the same transport. However, VCAT
inherently induces a varying propagation delay known as differential delay, which can affect
service.

Another protocol, link capacity adjustment scheme (LCAS), is typically used with VCAT. LCAS is a
method to dynamically increase or decrease the bandwidth of virtual concatenated containers in a
hitless manner, which enables service providers to add bandwidth on demand to a data service.
The LCAS protocol is specified in ITU-T G.7042.

With the addition of GFP, a new breed of transport products emerged, known collectively as the
multi-service provisioning platform (MSPP). MSPP supports SONET/SDH and Ethernet over SONET
services. With it, Ethernet services can be easily added with the addition of new line cards.

26601 W. Agoura Road · Calabasas, CA 91302 USA · Tel + 1-818-871-1800 · Fax + 1-818-871-1805 · www.ixiacom.com
25 September 2007 – Page 5 of 11
 Carrier Ethernet

Benefits of Ethernet over SONET

• Enables service providers to provide SLAs similar to TDM private lines. This provides sub-50
ms SONET protection/restoration of transport circuits.
• Provides line-rate and/or sub-line-rate Ethernet private lines over OC-N or STM-N
SONET/SDH rings for flexible GbE service delivery over SONET/SDH ring.
• Permits “right-sizing” of transport bandwidth for gigabit Ethernet services through standard
VCAT for transport bandwidth efficiency.
• Provides in-service capacity upgrades of EPL services through VCAT and LCAS (SW-LCAS).
This enables rapid, on-demand changes in EPL service capacity.)
• Provides advanced QoS for sub-rate services

Resilient Packet Ring (RPR) IEEE 802.17

Resilient Packet Ring (RPR), defined by IEEE 802.17, is a Layer 2 transport architecture based on a
dual, counter-rotating ring topology. RPR runs over SONET and provides packet-switching
technology. With RPR, service providers can leverage their existing SONET/SDH infrastructures
without significant upgrades. RPR provides high-bandwidth utilization by means of spatial reuse
within the ring. A fairness algorithm provides efficient data transport under congested conditions
without “starving” a single station (255-station limit). RPR has a protection switching mechanism
that provides sub-50 ms service restoration with no reserved bandwidth requirements. There is
integrated, three-level traffic class support built into the MAC with additional rate-limiting features.
RPR rings also provide optimization for multicast traffic delivery. Existing SONET/SDH MSPPs can
add RPR capability by adding RPR-capable modules.

Provider Ethernet
Provider Ethernet is a new term that defines Ethernet technologies used in aggregation/metro
service networks. It refers primarily to an IEEE standards-based Ethernet switched network. There
are several IEEE protocols that can be used to provide carrier Ethernet services in
aggregation/metro networks. Many new “greenfield” deployments are using or considering pure
Ethernet switched networks. Advantages over legacy technologies include bandwidth flexibility,
flexibility in service provisioning, and lower cost.

IEEE 802.1Q (Virtual LANs)

The virtual LAN (VLAN) standard provides a method for “tagging” Ethernet frames with VLAN
information. There are various ways the VLAN can be assigned, including port based, MAC

26601 W. Agoura Road · Calabasas, CA 91302 USA · Tel + 1-818-871-1800 · Fax + 1-818-871-1805 · www.ixiacom.com
25 September 2007 – Page 6 of 11
 Carrier Ethernet

based, and protocol based. A port-based implementation is the most common. With it, traffic from
a specific port can be added to a virtual LAN. VLAN technology has been widely used in
enterprise networks. Service providers are also using VLAN tagging to logically separate customer
traffic in aggregation/metro networks.

IEEE 802.1ad Provider Bridges (aka Q-in-Q)

A challenge service providers face when using VLANs arises when dealing with traffic that has
already been VLAN-tagged. That is where provider bridging — also known as Q-in-Q or VLAN
stacking — comes into play. Q-in-Q provides customer VLAN transparency by adding an additional
tag called the service tag, or “S-tag.” The original customer VLAN tag is then known as the “C-tag.”
One of the limitations of Q-in-Q is that the 12-bit VLAN tag is limited to 4094 unique customer
addresses that are not highly scalable for service provider networks.

IEEE 802.1ah Provider Backbone Bridges (PBB, aka MAC-in-MAC)

Provider Backbone Bridges (PBB) were designed to address the limitations of Q-in-Q and provide
additional capabilities. PBB is similar to Q-in-Q in the fact that it encapsulates customer data.
However, it uses a second MAC header and tag. The new tag, called the “I-tag,” or service
instance identifier, is 24 bits long and provides highly scalable services. PBB also provides
significant scale improvements by using a MAC address called the B-SA/B-DA (backbone
source/destination) for forwarding traffic. There is inherent backwards compatibility for 802.1Q
and 802.1ad networks because the address is encapsulated. PBB even be used in the core to
provide hierarchy for Q-in-Q networks. However, in most cases, it is used in the aggregation/metro
networks that connect to an IP/MPLS core.

IEEE 802.1Qay Provider Backbone Bridge Traffic Engineering (PBB-TE, aka PBT)
Before PBB-TE became an official project within the IEEE, it was known as provider backbone
transport. This emerging standard, championed by Nortel, is now moving forward within the IEEE.
PBB is used as the data plane that provides a mechanism to “tunnel” traffic over a service provider
network, yet a critical requirement for service providers is to support traffic-engineered paths. To
accomplish this, the typical source-address MAC “learning capability and broadcasting unknown
traffic” function is disabled. Instead, traffic is explicitly mapped to a precisely configured network
“tunnel.” The provisioning of these paths is typically supported by SNMP MIB commands set to each
switch. Another critical component of PBB-TE is resiliency. Using PBB-TE 1:1, tunnel path protection
is configured and Ethernet CFM (802.1ag) continuity check messages (CCMs) are used over each
path for fault-detection-triggering notification. PBB-TE provides theoretical high scalability with a 58-

26601 W. Agoura Road · Calabasas, CA 91302 USA · Tel + 1-818-871-1800 · Fax + 1-818-871-1805 · www.ixiacom.com
25 September 2007 – Page 7 of 11
 Carrier Ethernet

bit space for tunnel ID and a 24-bit space for service ID. Service providers are interested in PBB-TE
as an alternative to extending IP/MPLS with expensive routers.

Multiprotocol Label Switching (MPLS)

Multiprotocol label switching has established itself as a proven technology in service provider core
networks. Most service providers have launched successful IP/MPLS Layer 3 VPN services.
Additional development by the IETF has lead to two additional technologies that can be used to
enable new Layer 2-based carrier Ethernet services.

Pseudowire Emulation Edge- to-Edge (PWE3)

The Internet Engineering Task Force (IETF) standard pseudowire emulation edge-to-edge (PWE3)
provides a tunneling service over a MPLS core. The label switched path (LSP) can be manually or
dynamically built. Specific traffic is then mapped on to the LSP. PWE3 can carry legacy
technologies like frame relay and ATM and can also carry Ethernet traffic. PWE3 provides the
inherent benefits of an MPLS network, including the ability to define backup paths with fast failover
and QoS support. PWE3 can be used to build carrier Ethernet E-Line services.

Virtual Private LAN Switching (VPLS)

Virtual private LAN switching provides a virtual LAN service over an MPLS network. There are two
standards within the IETF that define VPLS: VPLS-LDP (RFC 4762) and VPLS-BGP (RFC 4761). Similar
to IP/MPLS L3 VPNs, VPLS is enabled at a provider’s edge router, where a virtual switching instance
(VSI) is defined. The virtual switch performs MAC address learning and floods broadcast, multicast,
and unknown unicast traffic. MPLS uses split horizon and a full mesh of pseudowires (PW) for loop
avoidance in the core. Spanning tree is not run in the core. Two standards, VPLS-LDP and VPLS-BGP,
primarily differ in the control plane used to establish the mesh of pseudowires. VPLS is attractive to
service providers since they can leverage their existing IP/MPLS networks and overlay a new L2
VPN service. Challenges include a requirement for full mesh PWs and the handling of flooding
traffic, which is a scalability concern. Because of these issues, new developments are underway to
implement hierarchical VPLS (H-VPLS).

26601 W. Agoura Road · Calabasas, CA 91302 USA · Tel + 1-818-871-1800 · Fax + 1-818-871-1805 · www.ixiacom.com
25 September 2007 – Page 8 of 11
 Carrier Ethernet

Service Management
A major shortcoming in native Ethernet is the requirement to provide carrier-grade services. It has no
management capability that would allow it to detect and report failures at Layer 2 over a virtual
link. Other transport technologies in this space like frame relay, ATM, and SONET have a suite of
services that provide the ability to monitor and troubleshoot the network. These services are called
operations, administration, and maintenance (OAM). To replace these legacy technologies, Ethernet
needs similar capabilities. To address this, work has begun in IEEE and ITU-T. Within the IEEE there
are two standards: IEEE 802.3ah E-OAM, “Ethernet in the first mile (used in conjunction with access
technologies),” and IEEE 802.1ag “connectivity fault management” (CFM). Similar to the IEEE
802.1ag specification, the ITU-T has authored the Y.1731 “OAM functions and mechanisms for
Ethernet-based networks” standard. These standards provide the tools to manage a network in the
data plane and the control plane over an Ethernet service (E-Line or E-LAN) and to verify continuity,
connectivity, and performance. The alternative, a higher-layer management solution like SNMP, is
slow and does not provide easy correlation of information to verify a specific path an Ethernet
virtual connection may take.

IEEE 802.3ah Ethernet Operations, Administration, and Maintenance (OAM)

(Clause 57)
802.3ah Ethernet OAM is known as “link OAM” since it runs over a point-to-point 802.3 link. E-
OAM provides mechanisms such as link monitoring, remote failure indication, remote loopback
control, and OAM discovery. E-OAM uses a simple OAM PDU frame that runs directly over a
standard Ethernet MAC header. OAM provides useful fault management capability, including fault
isolation, fault detection, and link performance testing. E-OAM is considered most useful in “the last
mile” to monitor Ethernet access services, but can also be used on any Ethernet link in the network.

IEEE 802.1ag Ethernet Connectivity Fault Management (E-CFM)

802.1ag Ethernet CFM runs end-to-end over an Ethernet virtual circuit and is viewed as a critical
component for delivering carrier-grade Ethernet services. E-CFM includes protocols that provide the
ability to detect, verify, and isolate connectivity faults. It also runs directly over standard Ethernet
frames that travel in-band with customer traffic. Devices that do not support E-CFM will forward the
frames and not participate in the messaging. The three E-CFM protocols are continuity check,
loopback, and link trace. Continuity check provides a keep-alive type of message over an Ethernet
service and is used to detect and notify that a fault has occurred. The loopback protocol is like an IP
ping at Layer 2. A loopback message (LBM) and a loopback reply (LBR) verify any faults. The link
trace protocol is like an IP trace route at Layer 2. There is a link trace message (LTM) and a link
trace reply (LTR) that isolate faults. E-CFM can define hierarchy within a domain, which gives an

26601 W. Agoura Road · Calabasas, CA 91302 USA · Tel + 1-818-871-1800 · Fax + 1-818-871-1805 · www.ixiacom.com
25 September 2007 – Page 9 of 11
 Carrier Ethernet

administrator visibility into specific nodes along the path. For example, a customer may only have
visibility from end to end, but a service provider can “see” every hop.

ITU-T Y.1731
The Y.1731 specification is a superset of 802.1ag. It provides all the features of E-CFM, plus the
following signals: Ethernet locked, Ethernet test, multicast loopback, and alarm indication.
Performance management capabilities include frame loss measurement, frame loss delay, and
throughput measurement. At this point, Y.1731 is not widely implemented, but with its enhanced
capabilities, it is of great interest to service providers.

Testing Challenges
Each stage of carrier Ethernet deployment has its own testing requirements. It is expected that
deployments will start with Q-in-Q and/or MAC-in-MAC. PBT or other transport protocols will be
implemented as networks increase in size. Whatever the means, interfaces to the core networks,
usually via IP/MPLS, need to be thoroughly exercised—and at full line rates. New and legacy
devices must be tested for interoperability.

Performance testing of network subsystems and entire end-to-end networks, both at and beyond
expected capacity, is required to ensure proper forwarding and handling of over capacity.

Security over exclusively Ethernet transport is another concern because everyone’s traffic travels side
by side, separated only as VLAN, MAC-in-MAC, and VPN traffic. Tests must ensure no leakage
between those methods.

To do all this, the MEF has defined set of conformance and performance tests. And, since metro
Ethernet protocols are used in densely layered combinations—tunnels transport other protocols and
services, maintenance protocols enable resiliency, billing, and other services—each network
configuration requires a different testing scenario.

Ixia’s testing tools not only combine multiple protocols, but enable network emulations that
simultaneously include Q-in-Q, MAC-in-MAC, and PBB-TE/PBT — at line speeds up to 10 Gbps.
Ixia’s applications test conformance and interoperability, function and performance, and service
scalability.

For more information about Ixia’s suite of test applications, go to http://www.ixiacom.com/

26601 W. Agoura Road · Calabasas, CA 91302 USA · Tel + 1-818-871-1800 · Fax + 1-818-871-1805 · www.ixiacom.com
25 September 2007 – Page 10 of 11
 Carrier Ethernet

Summary
Carrier Ethernet is made up of three main components: the definition of services, network
implementation, and service management. While the services defined by the MEF have been
widely accepted, the technologies to enable and manage them can vary greatly from one service
provider to another because of all the possible combinations available. Because of customer
demand, service providers are highly motivated to roll out and enhance their carrier Ethernet
networks. According to Infonetics, service providers continue to shift CAPEX spending from legacy
TDM to IP/MPLS and Ethernet-based services. And network equipment manufacturers report that
carrier Ethernet routers and switches comprise their fastest-growing routing and switching market
segment. For the future, look for the deployment of more carrier Ethernet services to a fast-growing
customer base, which will require continued innovation in Carrier Ethernet technologies.

References
Metro Ethernet Forum - http://www.metroeternetforum.org/

IEEE - http://www.ieee802.org/

IETF - http://www.ietf.org/

ITU-T - http://www.itu.int/ITU-T/

Ixia - http://www.ixiacom.com/

26601 W. Agoura Road · Calabasas, CA 91302 USA · Tel + 1-818-871-1800 · Fax + 1-818-871-1805 · www.ixiacom.com
25 September 2007 – Page 11 of 11