BRKSPG 2904 PDF
BRKSPG 2904 PDF
BRKSPG 2904 PDF
Architecture
Aleksandar Vidakovic; Technical Leader, CCIE 5697 R&S
Agenda
Introduction
Troubleshooting Packet Forwarding Update and
Recap
Understanding TCAM operations and ACLs
ARP
Fragmentation
Load-balancing review
QOS
Tomahawk
Usability
3
Acknowledgements
With contributions from
Xander Thuijs
Sadanande Phadke
Eddie Chami
YOU
Our team really REALLY appreciates the interaction online via the forums and the direct feedback
that you have given.
We hope to have shown you the last few years that your feedback didnt get unnoticed with a large
emphasis on usability and ease of use.
Your ratings on this session the last few years has been overwhelming which gives us so much pride
and joy to continue to improve whatever we need to or can.
So PLEASE keep that input coming and YOUR participation is necessary to build a better product
TOGETHER!
4
Introduction
What is new and what has
changed recently
5
New hardware
ASR9910
Beefed up backplane for higher speed cards (8x100)
Wildchild SFP
P
CPU
SFP
SFP
P Switch 0
Tomahawk linecards
SFP
SFP
SFP
SFP
H Fabric Swi
Y tch
SFP
SFP
4x100G/8x100G SFP
SFP
Fab
P
ric
SFP
SFP
RSP880 H
SFP
SFP
SFP
SFP
SFP
Y
6
Organizational and Software
No more BU. Routing segment taking care of all routing platforms
The goods from every group and architecture put together, eliminating the bad from
before
Dedicated dev/test teams focusing on usability
Extreme focus on usability and ease of use
Installation ASR 9000 Product Survey What can be improved?
Knobs, tweaks and tunes
7
Troubleshooting Packet
Forwarding
Update and recap
8
NPU Packet Processing - Ingress
5 Stages:
Queueing
Parse Search Resolve Modify Scheduling
CPU
CPAK 0 iFIB LC1 Switch Fabric Switch Fabric
Control CPAK 1
PHY NP
LPTS FIA
packet CPAK 2
PHY NP FIA
CPAK 3
Switch RP CPU: Routing, MPLS, IGMP, PIM,
Fabric
CPAK 4
PHY NP FIA
(SM15) HSRP/VRRP, etc
CPAK 5
Up to
14x120G
CPAK 6 LC CPU: ARP, ICMP, BFD, NetFlow,
PHY NP FIA OAM, etc
CPAK 7
10
Input Drops Troubleshooting
Troubleshooting this?
Piece of cake starting with
GigabitEthernet0/0/1/6.1 is up, line protocol is up
<..output omitted..>
IOS XR 5.3.3!!!
307793 packets input, 313561308 bytes, 227987 total input drops
11
Monitor NP Interface
RP/0/RSP0/CPU0:our9001#monitor np interface g0/0/1/6.1 count 2 time 1 location 0/0/CPU0
Monitor NP counters of GigabitEthernet0_0_1_6.1 for 2 sec
<..output omitted..>
**** Sun Jan 31 22:14:32 2016 ****
(Count 2 of 2)
RP/0/RSP0/CPU0:our9001#
12
Monitor NP Interface
RP/0/RSP0/CPU0:our9001#monitor np interface g0/0/1/6.1 count 2 time 1 location 0/0/CPU0
Monitor NP counters of GigabitEthernet0_0_1_6.1 for 2 sec
<..output omitted..> Temporarily uses a separate memory region for drop counters on selected
**** Sun Jan 31 22:14:32 2016 ****
interface
Monitor 2 non-zero NP1 counters: GigabitEthernet0_0_1_6.1
Counters reported with _MONITOR appendix
Offset Counter FrameValue Rate (pps)
These counters are not added to global NP counters
-------------------------------------------------------------------------------
262 RSV_DROP_MPLS_LEAF_NO_MATCH_MONITOR 101 49
By default runs
1307 PARSE_DROP_IPV4_CHECKSUM_ERROR_MONITOR one capture during
101 5 seconds50(configurable count and time)
(Count 2 of 2) One session at the time per LC
RP/0/RSP0/CPU0:our9001#
Supports physical and BE (sub)interfaces
Physical (sub)int: monitoring runs on NP that hosts the interface
BE(sub)int: monitoring runs on all NP that hosts the member
13
Show Controllers NP Capture
RP/0/RSP0/CPU0:our9001#sh controllers np capture np1 location 0/0/CPU0
14
Show Controllers NP Capture
RP/0/RSP0/CPU0:our9001#sh controllers np capture np1 location 0/0/CPU0
15
Show Controllers NP Capture Next Steps
Ethernet II, Src: 30:f7:0d:f8:af:81, Dst: 84:78:ac:78:ca:3e
interface GigabitEthernet0/0/1/6.1
Type: 802.1Q Virtual LAN (0x8100)
ipv4 address 172.18.0.1 255.255.255.0
802.1Q Virtual LAN, PRI: 0, CFI: 0, ID: 901
encapsulation dot1q 901
Type: MPLS label switched packet (0x8847) !
MultiProtocol Label Switching Header, Label: 24001, Exp: 0, S: 1, TTL: 255
MPLS Label: 24001
MPLS Experimental Bits: 0
MPLS Bottom Of Label Stack: 1
MPLS TTL: 255
Internet Protocol, Src: 172.18.0.2 (172.18.0.2), Dst: 172.16.255.2 (172.16.255.2)
Internet Control Message Protocol
Type: 0 (Echo (ping) reply)
Code: 0 ()
16
Troubleshooting Input Drops Next Steps
Ethernet II, Src: 30:f7:0d:f8:af:81, Dst: 84:78:ac:78:ca:3e interface GigabitEthernet0/0/1/6.1
Type: 802.1Q Virtual LAN (0x8100) ipv4 address 172.18.0.1 255.255.255.0
802.1Q Virtual LAN, PRI: 0, CFI: 0, ID: 901 encapsulation dot1q 901
Type: MPLS label switched packet (0x8847) !
MultiProtocol Label Switching Header, Label: 24001, Exp: 0, S: 1, TTL: 255
MPLS Label: 24001 RP/0/RSP0/CPU0:our9001#sh mpls forwarding labels 24001
MPLS Experimental Bits: 0
MPLS Bottom Of Label Stack: 1
MPLS TTL: 255 RP/0/RSP0/CPU0:our9001#sh mpls ldp bindings local-label 24001
Internet Protocol, Src: 172.18.0.2 (172.18.0.2), Dst: 172.16.255.2 (172.16.255.2)
Internet Control Message Protocol
RP/0/RSP0/CPU0:our9001#sh mpls ldp bindings 172.16.255.2/32
Type: 0 (Echo (ping) reply)
Code: 0 () 172.16.255.2/32, rev 48
Local binding: label: 24010
Remote bindings: (1 peers)
Peer Label
----------------- ---------
172.16.255.3:0 23 Drop reason: Upstream peer is
RP/0/RSP0/CPU0:our9001# sending packets with a wrong
MPLS label because it has a wrong
prefix to MPLS label binding!
17
Monitor NP Counter
18
Monitor NP Counter
RP/0/RSP0/CPU0:our9001#monitor np counter ACL_CAPTURE_NO_SPAN.1 np1 location 0/0/CPU0
Warning: Every packet captured will be dropped! If you use the 'count'
option to capture multiple protocol packets, this could disrupt
protocol sessions (eg, OSPF session flap). So if capturing protocol
packets, capture only 1 at a time.
show drops all commands shows the constituent commands that will be called
for parsing the final output
20
Show drops all sample output (1)
RP/0/RP0/CPU0:ios#sh drops all commands
Wed Feb 4 05:27:40.915 UTC
Module CLI
[arp] show arp traffic
[cef] show cef drops
[fabric] show controllers fabric fia drops egress
[fabric] show controllers fabric fia drops ingress
[lpts] show lpts pifib hardware entry statistics
[lpts] show lpts pifib hardware police
[lpts] show lpts pifib hardware static-police
[netio] show netio drops
[netio] fwd_netio_debug
[niantic-driver] show controllers dmac client punt statistics
[np] show controller np counters
[np] show controllers np tm counters all
[spp] show spp node-counters
[spp] show spp client detail
[spp] show spp ioctrl
21
Show drops all sample output (2)
RP/0/RP0/CPU0:ios#sh drops all location 0/5/CPU0
Wed Feb 4 05:26:30.192 UTC
=====================================
Checking for drops on 0/5/CPU0
=====================================
22
Troubleshooting NP Forwarding
1. Identify interface in question.
2. Identify the mapping from interface to NPU.
3. Examine NP counters.
4. Look for rate counters that match lost traffic rate.
If none of the counters match the expect traffic, check for drops at interface controller
23
Understanding TCAM and
ACL
24
What is a TCAM
It is reversed memory:
Normal memory receives an address and provides the data on that location
TCAM receives a data pattern (aka key) and searches where that content is found
That is GREAT for matching!
Deterministic performance!
When A9K receives a packet a key is built and passed to tcam this returns
both ACL match results and qos class-map results. The key has a particular
width (e.g.: 160 or 640 bits)
25
TCAM Capacity
ACLs are programmed in TCAM memory in LC
TCAM entries are shared between QOS, netflow, LPTS and ACL
TCAM allocation happens when ACL is applied to an interface
26
How Much Space My ACL Requires? Object groups simplify
long ACL configuration
object-group network ipv4 CLIENTS object-group port NFS_UDP object-group network ipv4 HOST
10.10.10.0/24 eq sunrpc 192.168.162.140/32
10.40.40.0/24 eq 635
! range 38465 38467
27
TCAM Partitioning If you have large
RP/0/RSP0/CPU0:nv-cluster-escalation#show prm server tcam partition all location 0/1/CPU0 ACLs and running
Node: 0/1/CPU0:
----------------------------------------------------------------
out of TCAM space
TCAM partition information: 1 ods2 blk = 2048 entries, 1 ods8 blk = 512 entries
NP0 : tot-ods2-blks 47 [60% of ods2+ods8 blks], used-ods2-blks 4 [ 5% of ods2+ods8 blks]
NP0 : tot-ods8-blks 31 [40% of ods2+ods8 blks], used-ods8-blks 2 [ 3% of ods2+ods8 blks]
NP1 : tot-ods2-blks 47 [60% of ods2+ods8 blks], used-ods2-blks 4
NP1 : tot-ods8-blks 31 [40% of ods2+ods8 blks], used-ods8-blks 2
[
[
5% of ods2+ods8
3% of ods2+ods8
blks]
blks]
Confirm current partitioning
(ods2 == 160 bit key)
RP/0/RSP0/CPU0:nv-cluster-escalation(admin-config)#hw-module profile tcam ?
(ods8 == 640 bit key)
default Default tcam partitions ods2:ods8 to 60:40
tcam-part-30-70 Set tcam partitions ods2:ods8 to 30:70
tcam-part-40-60 Set tcam partitions ods2:ods8 to 40:60
tcam-part-50-50 Set tcam partitions ods2:ods8 to 50:50 1. Configure new
tcam-part-70-30 Set tcam partitions ods2:ods8 to 70:30
RP/0/RSP0/CPU0:nv-cluster-escalation(admin-config)#hw-module profile tcam tcam-part-30-70 location 0/1/CPU0
partitioning
Tue Oct 21 15:03:35.493 UTC 2. reload the line card
In order to activate this new tcam partition profile, you must manually reload the line card.
RP/0/RSP0/CPU0:nv-cluster-escalation#show prm server tcam partition all location 0/1/CPU0 Confirm new partitioning
Node: 0/1/CPU0:
----------------------------------------------------------------
TCAM partition information: 1 ods2 blk = 2048 entries, 1 ods8 blk = 512 entries
NP0 : tot-ods2-blks 23 [29% of ods2+ods8 blks], used-ods2-blks 4 [ 5% of ods2+ods8 blks]
NP0 : tot-ods8-blks 55 [71% of ods2+ods8 blks], used-ods8-blks 2 [ 3% of ods2+ods8 blks]
NP1 : tot-ods2-blks 23 [29% of ods2+ods8 blks], used-ods2-blks 4 [ 5% of ods2+ods8 blks]
NP1 : tot-ods8-blks 55 [71% of ods2+ods8 blks], used-ods8-blks 2 [ 3% of ods2+ods8 blks]
28
ARP
29
ARP Architecture LDP RSVP-TE BGP OSPF
IOS XR: Static
ISIS EIGRP
fully distributed
two-stage forwarding operation (clear LSD RIB
separation of ingress and egress feature RP
processing)
Internal EOBC
Layer 2 header imposition is an egress
operation
only the line card that hosts the FIB Adjacency
egress interface needs to know the ARP LC NPU
Layer 2 encapsulation for packets that SW FIB
have to be forwarded out of that
AIB AIB: Adjacency Information Base
interface. LC CPU
RIB: Routing Information Base
As a consequence, ARP and FIB: Forwarding Information Base
adjacency tables are local to a line card. LSD: Label Switch Database
Exceptions:
Bundle-Ethernet interfaces: ARP synced b/w all LCs that host the bundle members.
Bridged Virtual Interfaces (BVI): ARP synced b/w all LCs. (Ingress+Egress L3
processing on ingress LC). 30
High-Scale ARP Deployments
Challenges:
synchronisation of large number of ARP entries across line cards during large ARP
churn:
ARP storm in the attached subnet
Router starts forwarding to end devices in a large attached subnet, triggering ARP resolution
requests
Running the show arp command or poll ARP via SNMP in very high scale can further
slow down the ARP process
Supported scale:
128k entries per LC tested thoroughly at Cisco
ARP table can grow up to the dynamic memory limit imposed on the ARP process
You can go higher, but make sure you test your deployment scenario
31
ARP Data Plane Line card ARP LC
CPU
Ingress NP classifies the packet to an
interface and detects that it's an ARP packet. spio netio
34
ARP Configuration Commands
!
Configure the LPTS ARP policer.
lpts punt police location 0/1/CPU0
protocol arp rate 400 This rate is applied to all NPs on the LC. If you have 8 NPs on the LC the max
! total ARP rate hitting the LC CPU would be: 400 x 8 = 3200 pps
!
Recommended in BNG deployments
subscriber arp scale-mode-enable
! Available starting from 5.3.3
35
How Do I Tell If ARP Process Is Overloaded?
LC/0/1/CPU0:Jan 27 06:38:12.445 : netio[270]: %PKT_INFRA-PQMON-6-QUEUE_DROP : Taildrop on XIPC queue 2 owned
by arp (jid=115)
NetIO queue is full, packets that require ARP resolution are dropped
36
High-Scale ARP Deployments Tips
Validate your solution if youre going beyond 128k/LC
Tighten the LPTS policer rate.
A safe ball-park number is 1000 per LC (if the LC has 4xNPs, configure 250).
Monitor ARP counters at NP
Monitor NetIO ARP resolution queue
Monitor adjacencies summary info
If using BVI, look for GSP failures in ARP traces
In BNG deployment on IOS XR release 5.3.3 or later configure "subscriber arp
scale-mode-enable
Read more on ARP operation and troubleshooting in:
https://supportforums.cisco.com/document/12766486/troubleshooting-arp-asr9000-routers
37
Fragmentation
38
Fragmentation and Reassembly at a glance
All fragmentation is handled on the (LC) CPU on the egress side (ingress LC does not do MTU check)
Fragmentation is outsourced to NETIO (process switching)
No features applied on fragment injection by netio. Directly from LC CPU to NPU xmit.
BNG has SPP based fragmentation, at 10kpps, feature applied, packets are injected through the fabric
Memory
Verify with show spp node-counters CPU
0
BGP 0
1 1
CDP ICMP 2
5
HDLC
application receives complete packets 6
Each grey arrow is an IPC inter process call. raw UDP TCP 0
1
8
10
13
Unless shared mem is used which can be seen by all, this is
14
limited in size however
15
SPP
39
Verifying
Looking at fragmentation at the SPP (the interupt switching path) level.
RP/0/RSP0/CPU0:A9K-BNG#show spp node-counters | i frag
ipv4_frag
Drop: Dont-fragment set: 3125 <<<< packets that have DF bit set
ipv4-frag: 3854 <<<< packets fragmented
Verifying the NP counters with show controller np counters NP<x> location 0/<y>/CPU0
416 RSV_PUNT_IP_MTU_EXCEEDED 9615 99 <<<< 100pps requiring frag
1048 PPPOE_FRAG_NEEDED_PUNT 9615 99 <<<< on pppoe sessions
842 IPV4_FRAG_NEEDED_PUNT 677 100 <<<< packets punted for FRAG reasons
Note that in both cases the DF bit is NOT assessed in the hardware, this is handled
by the controlling agent (NETIO/SPP)
40
Preventing fragmentation 1: MSS adjust
MSS adjust https://supportforums.cisco.com/document/12417241/tcp-mss-adjust-asr9000
Segment size in TCP is the maximum frame size (as opposed to window size which is
the number of packets that can be sent to be ackd together.
ASR9000 allows for TCP adjusting in hardware (typhoon and up).
The MSS value needs to be the first option in the TCP header.
The MSS is part of the initial syn from client to server
MSS rewrite requires a checksum update.
Needs to be enabled per NP
Value possible 1280-1535 (only 8 MSB adjusted)
MSS value designated
BNG:
Normal L3 routing:
subscriber
pta tcp mss-adjust 1410 hw-module location 0/0/CPU0 tcp-mss-adjust np 1 value 1300
interface Bundle-Ether48.10
ipv4 address 4.8.10.4 255.255.255.0
ipv4 tcp-mss-adjust enable 41
Preventing fragmentation 2: MTU tuning
Gotchas:
OSPF
DBD Exchange can get stuck
ISIS
Hello padding
42
Loadbalancing revisted
43
Access L2VPN PE Core
Loadbalancing FAQ
If packets are fragmented, L4 is omitted from the hash
calculation
V6 flow label addition (includes the 5 tupple/needs config) to the
hash coming (532)
Show cef exact route or bundle hash BE<x> can be used to
feed info and determine actual path/member, but this is a
shadow calculation that is *supposed* to be the same as the
HW.
Mixing Bundle members between trident/typhoon/tomahawk
can be done, not recommended (though hash calc same).
44
How does ECMP path or LAG member selection work?
Path (ECMP)
Member (LAG)
46
MPLS vs IP Based loadbalancing
When a labeled packet arrives on the interface.
If the number of labels <=4 and the next nibble seen right after that label is
4: default to IPv4 based balancing
6: default to IPv6 based balancing
This means that if you have a P router that has no knowledge about the MPLS service of the packet, that nibble can
either mean the IP version (in MPLS/IP) or it can be the DMAC (in EoMPLS).
RULE: If you have EoMPLS services AND macs are starting with a 4 or 6. You HAVE to use Control-Word
45 (ipv4)
L2 MPLS MPLS 0000 (CW) 4111.0000.
41-22-33 (mac)
Control Word inserts additional zeros after the inner label showing the P nodes to go for label based balancing.
In EoMPLS, the inner label is VC label. So LB per VC then. More granular spread for EoMPLS can be achieved with FAT PW
(label based on FLOW inserted by the PE device who owns the service).
Take note of the knob to change the code: PW label code 0x11 (17 dec, as per draft specification). (IANA assignment is 0x17)
47
Loadbalancing ECMP vs UCMP and polarization
Support for Equal cost and Unequal cost
32 ways for IGP paths
32 ways (Typhoon) for BGP (recursive paths) 8-way Trident
64 members per LAG
Make sure you reduce recursiveness of routes as much as possible (static route
misconfigurations)
All loadbalancing uses the same hash computation but looks at different bits from that hash.
Use the hash shift knob to prevent polarization.
Adj nodes compute the same hash, with little variety if the RID is close
This can result in north bound or south bound routing.
Hash shift makes the nodes look at complete different bits and provide more spread.
Trial and error (4 way shift trident, 32 way typhoon, values of >5 on trident result in modulo)
48
Hash shift, what does it do?
L2 L3 L4 payload
8 bits selected
Hash shift 8
100 = 8 1 1 HASH
1 HASH 8 bits selected
010 = 4
(3 drawn) 256 buckets
1 2 3 4 7 8
Path (ECMP)
Member (LAG)
On bundle (sub)interfaces:
Loadbalance on srcIP, dest IP, src/dest or fixed hash value (tie vlan to hash result)
Used to be on L2transport only, now also on L3.
GRE (no knob needed anymore)
Encap: based on incoming ip
Decap: based on inner ip
Transit: based on inner payload if incoming is v4 or v6 otherwise based on GRE header
So running mpls over GRE will result in LB skews!
50
ASR9000 L2VPN Load-Balancing (cont.)
ASR9000 PE Imposition load-balancing
behaviors PE Per-Flow load-balance configuration based
Per-PW based on MPLS VC label (default) on L2 payload information
Per-Flow based on L2 payload information; i.e. !
L2 DMAC / L2 SMAC, RTR ID l2vpn
load-balancing flow src-dst-mac
Per-Flow based on L3/L4 payload information; !
i.e. L3 D_IP / L3 S_IP / L4 D_port / L4 S_port1,
RTR ID
L2VPN (PW) traffic treated as Non-IP For L2VPN, bottom of stack 802.1q Tag (0x8100)
label could be: C-VID
PW Control-Word strongly recommended to avoid
PW VC label or E-Type (0x0800)
erroneous behavior on P router when DMAC starts
with 4/6 Flow label (when using
FAT PWs) IPv4 Payload
(1) MPLS encap IP packets with up to four (4) MPLS labels Typical EoMPLS frame
52
Flow Aware Transport PWs
54
PW2 (Service Y)
P router with ECMP and P router without ECMP
Bundle interfaces and Bundle interfaces
55
L2VPN Load-balancing (Per-VC LB)
Default - ASR9000 P with PW2 (Service Y)
Default - ASR9000 PE with
Core-facing Bundle AC Bundle
P rtr load-balances traffic PE load-balances traffic across
across Bundle members based PW1 (Service X) Bundle members based on
P3
on VC label; i.e. all traffic from P1
DA/SA MAC
F1x F2x F3x
a PW assigned to one member F4x
Svc X Flow 1
Svc X Flow 2
Svc X Flow 3
Svc X Flow 4
Svc Y Flow 1
Svc Y Flow 2 F1y F2y F3y
Svc Y Flow 3
PE1 F4y PE2
Svc Y Flow 4
Default - ASR9000 PE with P4
P2
ECMP
Default - ASR9000 PE with Default - ASR9000 P with
PE load-balances PW traffic
Core-facing Bundle ECMP
across ECMPs based on VC
PE load-balances traffic across P rtr load-balances traffic
label; i.e. all traffic from a PW
Bundle members based on VC across ECMPs based on VC
assign to one ECMP
label; i.e. all traffic from a PW label; i.e. all traffic from a PW
assigned to one member assigned to one ECMP
56
L2VPN Load-balancing (L2/L3 LB)
Default - ASR9000 P
PW loadbalancing based on
VC label; only one ECMP and PE L2VPN load-balancing knob:
one bundle member used for l2vpn
PE L2VPN load-balancing knob: load-balancing flow {src-dst-mac
l2vpn all PW traffic PW1 (Service X) | src-dst-ip}
P1 P3
load-balancing flow {src-dst-mac
| src-dst-ip} F1x F2x
Svc X Flow 1
Svc X Flow 2
Svc X Flow 3
Svc X Flow 4
Svc X Flow 1
Svc X Flow 2
Svc X Flow 3 F2x
Svc X Flow 4
ASR9000 PE F3x
PE1 PE2
PE now adds Flow
labels based on L2 or F4x
L3 payload info P2 P4 ASR9000 PE with AC Bundle
ASR9000 PE with ECMP ASR9000 PE with Core- PE load-balances now traffic
PE now load-balances PW facing Bundle ASR9000 P with ECMP across Bundle members based
traffic across ECMPs based PE now load-balances traffic P rtr now load-balances traffic on L2 or L3 payload info
on L2 or L3 payload info; i.e. across Bundle members based across ECMPs based on Flow
flows from a PW distributed on L2 or L3 payload info; i.e. label; i.e. flows from a PW
over ECMPs flows from a PW distributed distributed over ECMPs
over members
58
L2VPN LB Summary
ASR9000 as L2VPN PE router performs multi-stage hash algorithm to select
ECMPs / Bundle members
User-configurable hash keys allows for the use of L2 fields or L3/L4 fields in PW
payload in order to perform load-balancing at egress imposition
ASR9000 (as PE) complies with RFC6391 (FAT PW) to POP/PUSH Flow labels
and aid load-balancing in the Core
PE load-balancing is performed irrespective of Flow PW label presence
FAT PW allows for load-balancing of PW traffic in the Core WITHOUT requiring any
HW/SW upgrades in the LSR
Cisco has prepared a draft to address current gap of FAT PW for BGP-signaled PWs
Solution:
Problem:
Add PW Control Word in
DANGER for LSR
front of PW payload. This RTR DA
LSR will confuse payload as
guarantees that a zero will
IPv4 (or IPv6) and attempt to RTR DA RTR SA
always be present and thus
load-balance based off RTR SA MPLS E-Type (0x8847)
no risk of confusion for LSR
incorrect fields
MPLS E-Type (0x8847) PSN MPLS Label
PSN MPLS Label PW MPLS Label
PW MPLS Label 0 PW CW
4 DA 4 DA 4 DA
SA SA SA
802.1q Tag (0x8100) 802.1q Tag (0x8100) 802.1q Tag (0x8100)
C-VID C-VID C-VID
Payload E-Type Payload E-Type Payload E-Type
60
QOS:
Queuing and Scheduler
Hierarchy
61
Queuing Hierarchy Of Default Interface Queues
P1 P2 P3 L
Level 4 Queues
Level 3 Schedulers
Level 2 Schedulers
Level 1 Schedulers
62
Old format was:
show qoshal default-queue subslot 1 port 0 location 0/0/CPU0
Default Interface Queues
RP/0/RSP0/CPU0:A9K#show qoshal default-queue interface Gig0/0/1/0
Thu Apr 3 06:00:08.931 UTC
Interface Default Queues : Subslot 1, Port 0 current number of packets
===============================================================
Port 64 NP 1 TM Port 16 in the queue
Ingress: QID 0x20000 Entity: 1/0/2/4/0/0 Priority: Priority 1 Qdepth: 0
StatIDs: commit/fast_commit/drop: 0x690000/0x660/0x690001
Statistics(Pkts/Bytes):
Tx_To_TM 0/0 Fast TX: 425087/91780021 TX statistics
Total Xmt 425087/91780021 Dropped 0/0
L2
Inactive entity
L1 Active entity
64
policy-map child
MQC Hierarchy in Queuing ASIC class c1
priority level 1
police rate 640 kbps
Port default queues G0/0/0/0.1 G0/0/0/0.2 class c2
P1 P2 P3 L c1 c2 cd-c c1 c2 cd-c bandwidth 20 mbps
class class-default cd-c
bandwidth 1 mbps
!
policy-map parent
class class-default cd-p
shape average 35 mbps
service-policy child
L4 !
interface GigabitEthernet0/0/0/0.1
service-policy output parent
!
L3 cd-p cd-p interface GigabitEthernet0/0/0/0.2
service-policy output parent
L2
Inactive entity
L1 Active entity
65
MQC Hierarchy in Queuing ASIC
policy-map child
Port default queues G0/0/0/0.1 class c1
priority level 1
P1 P2 P3 L c1 c2 cd-c cd-p-vc police rate 640 kbps
cd-c
class c2
bandwidth 20 mbps
class class-default
bandwidth 1 mbps
!
policy-map parent
class c3
cd-p
L4
shape average 35 mbps
service-policy child
class class-default
bandwidth 1 mbps
cd-gp
L3 c3 cd-p !
policy-map grand-parent
class class-default
cd-gp shape average 35 mbps
L2 service-policy parent
!
interface GigabitEthernet0/0/0/0.1
service-policy output grand-parent
L1
66
Priority Propagation
All ASR9000 cards support priority propagation
Traffic belonging to high priority will trump all other traffic of lower priority throughout the
Queuing ASIC hierarchy up to the port
Priority propagation is enabled on all queues by default; it cannot be turned off.
priority keyword can only be specified at the leaf level and not in the middle node of
a MQC policy-map hierarchy.
67
Priority Propagation policy-map child
class Pr1
Pr1 Pr2 Cl3 Cl4 L4 police rate 64 kbps
priority level 1
class Pr2
police rate 10 mbps
priority level 2
Parent min BW class Cl3
bandwidth 3 mbps
L3 class Cl4
Parent shaper bandwidth 1 mbps
!
policy-map parent
class parent1
shape average 25 mbps
Commit Excess service-policy child
P1 P2 class parent2
scheduler scheduler
To other L3 shape average 25 mbps
service-policy child
!
policy-map grand-parent
class class-default
Grand-Parent shaper shape average 500 mbps
service-policy parent
Commit Excess
P1 P2 L2
scheduler scheduler
68
QoS:
Queue Limits/Occupancy
Buffer Limits/Occupancy
69
Queue Accounting Vs. Port Accounting
Queue occupancy is accounted in bytes (units of 64 bytes)
Inst-queue-len (Packets) field in show policy-map interface command output.
If there are 2 packets of 80 bytes each in the queue, the Inst-queue-len shows 3 (3 x 64 = 192 bytes).
This value is then used to compare against the queue-limit and random-detect configured on the
policy-map
Staring with 5.1.1 (5.3.0 for TH) queue occupancy can be calculated in numbers of 256-byte buffers
hw-module all qos-mode wred-buffer-mode
71
MQC Queue Occupancy
RP/0/RSP0/CPU0:av-asr9001#sh policy-map interface g0/0/1/0 output
72
MQC - What Is Programmed In Hardware
RP/0/RSP0/CPU0:av-asr9001#sh qos interface Gig0/0/1/0 output
Interface: GigabitEthernet0_0_1_0 output
Bandwidth configured: 1000000 kbps Bandwidth programed: 1000000 kbps policy-map core-child
ANCP user configured: 0 kbps ANCP programed in HW: 0 kbps class TGN
Port Shaper programed in HW: 0 kbps bandwidth percent 10
Policy: core-parent Total number of classes: 4 !
---------------------------------------------------------------------- class class-default
Level: 0 Policy: core-parent Class: af21 bandwidth percent 1
QueueID: N/A
!
Shape CIR : NONE
Shape PIR Profile : 2/3(S) Scale: 156 PIR: 9984 kbps PBS: 124800 bytes
policy-map core-parent
WFQ Profile: 2/9 Committed Weight: 10 Excess Weight: 10
class af21
Bandwidth: 0 kbps, BW sum for Level 0: 0 kbps, Excess Ratio: 1 service-policy core-child
---------------------------------------------------------------------- shape average 10 mbps
Level: 1 Policy: core-child Class: TGN !
Parent Policy: core-parent Class: af21 class class-default
QueueID: 131746 (Priority Normal) shape average 100 mbps
Queue Limit: 114 kbytes Abs-Index: 27 Template: 0 Curve: 0
Shape CIR Profile: INVALID
WFQ Profile: 2/76 Committed Weight: 91 Excess Weight: 91
Bandwidth: 1000 kbps, BW sum for Level 1: 1100 kbps, Excess Ratio: 1
----------------------------------------------------------------------
Level: 1 Policy: core-child Class: class-default
Parent Policy: core-parent Class: af21
QueueID: 131747 (Priority Normal)
Queue Limit: 12 kbytes Abs-Index: 7 Template: 0 Curve: 0
Shape CIR Profile: INVALID
WFQ Profile: 2/8 Committed Weight: 9 Excess Weight: 9
Bandwidth: 100 kbps, BW sum for Level 1: 1100 kbps, Excess Ratio: 1
73
policy-map core-child
class TGN
bandwidth percent 10
Excess BW for class- !
Service Rate
Excess BW for class TGN
(calculated from Excess
default (child level) class class-default
bandwidth percent 1
(calculated from Excess !
Weight)
Weight) policy-map core-parent
class af21
service-policy core-child
shape average 10 mbps
!
class class-default
Total excess BW shape average 100 mbps
Parent
Shaper Service rate of class TGN
Total guaranteed BW
74
Queue Limit and Service Rate
Queue Limit is by default 100ms worth of Service Rate
Service Rate is the sum of minimum guaranteed bandwidth and bandwidth
remaining assigned to a given class
Bandwidth remaining = Parent Max Rate Sum of Guaranteed BW at child level
Parent Max Rate is typically the parent shaper rate; if no parent, its the physical
interface BW
Bandwidth remaining assigned to a given class = BR * Excess Weight / Sum of
Excess Weights at child level
( Parent Max Rate Guaranteed BW ) * Excess Weight
Service Rate = Guaranteed BW +
Excess Weights
75
Committed vs Excess Weight
Configured BW allocation is always translated into a WFQ weight
Bandwidth Remaining Ratio is directly translated into WFQ weight
Bandwidth Percentage is translated as percentage of 1024
76
policy-map core-child
Queue Limit Calculation class TGN
bandwidth percent 10
Level: 0 Policy: core-parent Class: af21 !
Shape PIR Profile : 2/3(S) Scale: 156 PIR: 9984 kbps PBS: 124800 bytes class class-default
---------------------------------------------------------------------- bandwidth percent 1
Level: 1 Policy: core-child Class: TGN !
Queue Limit: 114 kbytes Abs-Index: 27 Template: 0 Curve: 0 policy-map core-parent
WFQ Profile: 2/76 Committed Weight: 91 Excess Weight: 91 class af21
Bandwidth: 1000 kbps, BW sum for Level 1: 1100 kbps, Excess Ratio: 1 service-policy core-child
---------------------------------------------------------------------- shape average 10 mbps
Level: 1 Policy: core-child Class: class-default !
class class-default
WFQ Profile: 2/8 Committed Weight: 9 Excess Weight: 9
shape average 100 mbps
Queue Limit is 100ms worth of Service Rate
Queue Limit = Service Rate * 100ms
Service Rate is the sum of minimum guaranteed bandwidth and bandwidth remaining assigned to a given
class
Parent BW = 9984 kbps
Guaranteed BW of class TGN = 1000 kbps
Sum of guaranteed BW at Level1 = 1100 kbps
Total remaining BW at Level1 = 9984 1100 = 8884 kpbs
Remaining BW of class TGN = ( 91 / ( 91 + 9 ) ) * 8884 kpbs = 8084 kbps
Service Rate of class TGN = 1000 + 8084 = 9084 kbps
Queue Limit of class TGN = 9084 / 10 = 908400 [bits] = 113550 [Bytes] ~= 114 kB
77
Configuring The Queue Limit
Queue Limit can be configured manually
Supported units:
Bytes
Kilobytes
Megabytes
Milliseconds
Packets (default)
For conversion into 64-Byte units of queue occupancy, packet size of 256 bytes is presumed
Microseconds
Queue Limit sizes are pre-fit
The one closest to the calculated queue limit is picked for HW programming
78
Configuring The Queue Limit
RP/0/RSP0/CPU0:av-asr9001#sh qos interface Gig0/0/1/0 output location 0/0/CPU0
<. . .>
Level: 1 Policy: core-child Class: TGN <..>
Parent Policy: core-parent Class: af21 policy-map core-child
Queue Limit: 2624 kbytes (10000 packets) Abs-Index: 90 Template: 0 Curve: 0 class TGN
WFQ Profile: 2/76 Committed Weight: 91 Excess Weight: 91 bandwidth percent 10
Bandwidth: 1000 kbps, BW sum for Level 1: 1100 kbps, Excess Ratio: 1 queue-limit 10000 packets
!
<..>
Presuming 256 Byte packet size
Closest pre-fit value picked
Fields - IPV4 source address - Outer VLAN/COS/DEI - Outer VLAN/COS/DEI - Outer VLAN/COS/DEI - IPV6 source address
supported (Specific/Range)[1] - Inner VLAN/COS - Inner VLAN/COS - Inner VLAN/COS (Specific/Range)
- IPV4 Destination - IPV4 Source address - IPV4 Destination - MAC Destination - IPV6 Destination
address (Specific/Range) address address address
(Specific/Range) - IP DSCP / TOS / (Specific/Range) - MAC source address (Specific/Range)
- IPV4 protocol Precedence - IP DSCP / TOS / - QOS-group (output - IPV6 protocol
- IPV4 TTL - QOS-group (output Precedence policy only) - IPV6 TOS /EXP
- IPV4 Source port policy only) - QOS-group (output - Discard-class (output - IPV6 TTL
(Specific/Range) - Discard-class (output policy only) policy only) - IPV6 Source port
- IPV4 Destination port policy only) - Discard-class (output (Specific/Range)
(Specific/Range) - EXP policy only) - IPV6 Destination port
- TCP Flags - EXP (Specific/Range)
- IP DSCP / TOS / - TCP Flags
Precedence - Outer VLAN/COS/DEI
- QOS-group (output - Inner VLAN/COS
policy only) - IPV6 header flags
- Discard-class (output- - QOS-group (output
policy only) policy only)
- EXP - Discard-class (output-
policy only)
[1] All fields marked in green are defined using an ACL used for QOS classification.
80
Tomahawk
NP5
81
Line card Generations & Silicon Evolution
240Gbps & 150Mpps
High
Performance
Ultra-fast 4Tbps on-chip mem.
Internal TCAM for ACL/QoS
1st Gen
Trident
Class Trident Octopus Santa Cruz PowerPC
90nm 130nm 130nm Dual Core 1M policers & 1M queues
120G Rich QoS
15 Gbps 60 Gbps 90 Gbps 1.2 Ghz 64k subscribers/NPU
83
Tomahawk Scale in Phase 1
Metric -TR Scale -SE Scale
MPLS Labels 1M
MAC Addresses 2M (6M Future)
FIB Routes (v4/v6) Search Memory 4M(v4) / 2M(v6) XR (10M/5M Future)
Mroute/MFIB (v4/v6) 128k/32k (256k/64k Future)
VRF 8K (16k Future)
Bridge Domains 64K
TCAM (acl space v4/v6) 20Mbit 80Mbit
Packet Buffer 100ms (6G/NPU) 200ms (12G/NPU)
EFPs 16K/LC 128K/LC (64K/NP)
L3 Subif (incl. BNG) 8k 128k (64k/NP)
IP/PPP/LAC subscriber sessions per
16k 256k (64k/NP)
LC
Egress Queues 8 Queues / port + nV Sat Qs 1M/NPU (4M for 8x100GE!)
Policers 32K/NPU 512K/NPU
ACL (v4/v6) 16k v4 or 4k v6 ACEs/LC 98k v4 / 16k v6 ACEs
84
V3 Power Supplies
85
Power System Version 3 for ASR9k
Introduction
Power System Version 3 is a next-gen Power Supply and Power Tray for ASR9k
More powerful Power Supplies allow to increase the System Power density
Available for ASR9010, ASR9912 & ASR9922
Supported from IOS-XR version 5.3.0 onwards
Two new Power Supplies: 6kW AC, 4.4kW DC
Two new Power Trays: 3xAC V3 (18 kW/Tray), 4xDC V3 (17.6 kW/Tray)
Routers are field-upgradeable to V3 power
Very important: Although the new power supplies are more powerful, all the feed
specifications stay the same as with V2. Therefore no need to change the electrical
installation / cords when using V3 power!
86
Version 3 AC Power Supply
New Power Tray and Power Supply
Front
PWR-6KW-AC-
V3
New AC Tray
Rear
A9K-AC-PEM-V3
Three 6kW AC PMs per tray
Two logically ANDed single phase 3kW AC power inputs per PM
If both inputs are active, output is max 6kW
If only one input is active, Power Supply is still working, output will be max 3kW
Front
PWR-4.4KW-DC-V3
New DC Tray
Rear
AC V2 AC V3 DC V2 DC V3
Max Power 3 kW 6 kW 2.1 kW 4.4 kW
# of Feeds 1 2 2 2
Feed redundancy in PEM n/a No Yes No
# of PSs per power tray 4 3 4 4
Redundancy scheme1) N+N N+N N+1 N+N
1) number of modules required to protect from feed failure (e.g. power grid outage)
89
Tomahawk Power Budgets
Card Budget
A99-SFC2 165 W
A9K-8X100GE 1150 W
A99-RP2-SE 410 W
ASR-9922-FAN 1100 W
90
Tomahawk LC
Architecture
91
Tomahawk LC: 8x100GE Architecture L2/L3/L4 lookups, all
VPN types, all feature VoQ buffering, Fabric
Slice Based Architecture processing, mcast FPOE, Auto- Ivy Bridge CPU
credits, mcast hashing,
CPAK: Macsec Suite B+, replication, QoS, ACL, Spread, DWRR, Complex
scheduler for fabric and
100G, 40G, 10G G.709, OTN, etc RBH, replication
egress port
Clocking
240G 240G
Tomahawk Tigershark
PHY
NP FIA
240G 240G
Tomahawk Tigershark
PHY
NP FIA
Central XBAR
(SM15)
XBAR
240G 240G
Tomahawk Tigershark
PHY
NP FIA
240G 240G
Tomahawk Tigershark
PHY
NP FIA
92
Tomahawk Flexible MAC PHY
Configurable MAC PHY for MLG between optical interface to NPU:
10GE/40GE/100GE LANPHY
100G LR4/ER4 (4x25.78G) 1xCAUI (100GE, 802.3ba)
100G SR10 (10x 10G) 2x CAUI or 2x10x 10GE XFIs
10x 10G LR 10x 10GE XFIs
2x 40GE LR4/SR4 2x 40GE XLAUIs (2x40GE, 802.3ba)
10GE WANPHY
10x 10GE WANPY (9.954Gbps WIS) 10x 10GE XFIs
CPU
Fabric
CPAK 4
(SM15)
PHY NP FIA
More features: CPAK 5
Up to
14x120G
802.1qbb Priority based Ethernet flow control CPAK 6
TM0
Line port Input bypass Line port Output
WRED
I/F x12 ICU & Pre-parse I/F x12
TOP Resolve
TOP Search
Line port Input
TOP Modify
TOP Parse
I/F x12
SPri WRR
I/F x12 ICFDQ per flow queuing
TM1 FIA Output I/F
FIA Input I/F
bypass x16
WRED
x16
95
3rd Generation Fabric
96
ASR9912/9922 Switch Fabric Card (FC2) Overview
6+1 All Active 3-Stage Fabric Planes, Scale to 1.6Tbps LCs
Fabric frame format:
5x 2x115G (120G raw) Super-frame
bi-directional Fabric load
= 1.15Tbps balancing:
Unicast is per-packet
Multicast is per-flow
FIA
FIA
FIA Fabric Fabric FIA
SM15 SM15
FIA
FIA
Tomahawk Line Tomahawk Line
Card Card
FIA
FIA
FIA Fabric FIA
fabric
SM15 FIA
FIA
Typhoon Linecard Tomahawk Line
Card
(5-1)x2x55G bi-directional
= 440Gbps (protected) Fabric Lanes 6 & 7 are only used towards
Tomahawk 7-fabric plane linecards
SFC v2
98
Fabric Deployment Modes
Each 8x100 LC slot is assigned 96 fabric destinations (VQIs) to operate any CPAK
breakout. 10 LC slots need 960 VQIs and 20 LC slots need 1920 VQIs.
ASR9912 chassis can operate with mix of Typhoon and Tomahawk LCs or Tomahawk
only LCs without requiring explicit fabric mode configuration.
ASR9922 requires explicit fabric mode configuration (and chassis reload) to operate in
Tomahawk only mode in order to enable 2K VQI scale.
Configuration command to enable high-VQI mode:
RP/0/RP0/CPU0:A9K(admin-config)#fabric enable mode highbandwidth
For ASR9922 in mix mode, upon exhausting 1K VQIs, new LCs inserted in the chassis are
kept in IN-RESET state.
Mix mode is default (chassis is limited to 1K VQIs).
All FIAs in the chassis must support high VQI scale (including the RP)
99
Each 8x100 LC slot is assigned 96 fabric destinations (VQIs) to operate any CPAK port in
1x100, 2x40, or 10x10 mode in a flexible manner. So 10 LC slots need 960 VQIs and 20 LC
slots need 1920 VQIs.
ASR9912 chassis can operate with mix of Typhoon and Tomahawk LCs or Tomahawk only
LCs without requiring explicit fabric mode config.
ASR9922 requires explicit fabric mode config (and chassis reload) to operate in Tomahawk
only mode in order to enable 2K VQI scale.
For ASR9922 in mix mode, upon exhausting 1K VQIs, new LCs inserted in the chassis are kept in IN-
RESET state.
Mix mode is default (chassis is limited to 1K VQIs).
All FIAs in the chassis must support high VQI scale (note that RP1 uses Skytrain FIA)
100
CPAK
101
ASR 9000 Optical Interface Support
Cisco CPAK 100G Pluggable Transceiver
Standards Compliant
Meets IEEE 802.3ba 100GBASE-SR10 / LR4 / ER4 requirements
OTN OTU4 compliant, Electrical interface OIF compliant
Highest Density
1/3 the size of CFP
20% smaller than CFP2
Enables 10+ Pluggable 100G Ports (1Tb/s) per card
103
CPAK Breakout What Does It Mean For
Operations?
104
CPAK Breakout
RP/0/RP0/CPU0:ios(config)#hw-module location 0/5/CPU0 port 6 breakout 10xTenGigE
RP/0/RP0/CPU0:ios(config)#commit
Sun Feb 1 12:55:58.344 UTC
RP/0/RP0/CPU0:Feb 1 12:55:59.029 : config[65752]: %MGBL-CONFIG-6-DB_COMMIT : Configuration committed by user 'root'. Use 'show configuration
commit changes 1000000243' to view the changes.
LC/0/5/CPU0:Feb 1 12:56:00.585 : ifmgr[205]: %PKT_INFRA-LINK-3-UPDOWN : Interface HundredGigE0/5/0/6, changed state to Down
LC/0/5/CPU0:Feb 1 12:56:00.585 : ifmgr[205]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface HundredGigE0/5/0/6, changed state to Down
RP/0/RP0/CPU0:Feb 1 12:56:00.965 : config[65752]: %MGBL-SYS-5-CONFIG_I : Configured from console by root
LC/0/5/CPU0:Feb 1 12:56:07.778 : ifmgr[205]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/5/0/6/0, changed state to Administratively Down
LC/0/5/CPU0:Feb 1 12:56:07.778 : ifmgr[205]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/5/0/6/1, changed state to Administratively Down
LC/0/5/CPU0:Feb 1 12:56:07.778 : ifmgr[205]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/5/0/6/2, changed state to Administratively Down
LC/0/5/CPU0:Feb 1 12:56:07.778 : ifmgr[205]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/5/0/6/3, changed state to Administratively Down
LC/0/5/CPU0:Feb 1 12:56:07.778 : ifmgr[205]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/5/0/6/4, changed state to Administratively Down
LC/0/5/CPU0:Feb 1 12:56:07.779 : ifmgr[205]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/5/0/6/5, changed state to Administratively Down
LC/0/5/CPU0:Feb 1 12:56:07.779 : ifmgr[205]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/5/0/6/6, changed state to Administratively Down
LC/0/5/CPU0:Feb 1 12:56:07.779 : ifmgr[205]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/5/0/6/7, changed state to Administratively Down
LC/0/5/CPU0:Feb 1 12:56:07.779 : ifmgr[205]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/5/0/6/8, changed state to Administratively Down
LC/0/5/CPU0:Feb 1 12:56:07.779 : ifmgr[205]: %PKT_INFRA-LINK-5-CHANGED : Interface TenGigE0/5/0/6/9, changed state to Administratively Down
LC/0/5/CPU0:Feb 1 12:56:07.779 : cfgmgr-lc[136]: %MGBL-CONFIG-6-OIR_RESTORE : Configuration for node '0/5/0' has been restored.
LC/0/5/CPU0:Feb 1 12:56:19.129 : inv_agent[213]: %PLATFORM-INV_AGENT-6-CPAK_OIRIN : CPAK OIR: 0/5/CPU0/6 Sn: FBN17462015 is inserted, state: 1
106
CPAK Breakout Interface Numbering
RP/0/RP0/CPU0:ios#sh int brief location 0/5/CPU0
Sun Feb 1 13:27:18.699 UTC
107
Interface Down Troubleshooting
Look for RX_LOS alarm in:
show controller <interface> <location> internal
Sample output:
RP/0/RSP0/CPU0:td02#show controllers HundredGigE 0/0/0/2 internal | include Defects
H/W Link Defects : (0x00001328) HW_LINK LASI RFI TX_FAULT MOD_NR
H/W Raw Link Defects : (0x0400132a) RX_LOS HW_LINK LASI RFI TX_FAULT MOD_NR ADMIN_DOWN
If RX_LOS is reported:
1. Check if CPAK is inserted correctly
2. Make sure PID is read/detected correctly
RP/0/RSP0/CPU0:td02#show controllers HundredGigE 0/0/0/3 internal | include PID
Pluggable PID : CPAK-100G-SR10
Pluggable PID Supp. : (Service Un) Supported
108
Interface Down Troubleshooting
For further troubleshooting collect:
show tech ethernet interface
show controller <interface> internal
show controller <interface> phy
show controller <interface> regs
show controller <interface> xgxs
109
Tomahawk Advanced
Power Management
(APM)
110
Tomahawk APM Overview
Tomahawk line card supports Advanced Power Management (APM) to enable
users to power up and down a slice without affecting functioning of other slices.
A slice includes ports on Sacramento/SM15, one Tigershark, one Tomahawk
NP, two PHYs, Gear Boxes, and two optical modules.
SM15 Interface
Slice Tigershark NP PHY Gear Box Optics
Ports (100G)
0 0-1 0 0 0 0 0 0-1
1 2-3 1 1 1 1 1 2-3
2 4-5 2 2 2 2 2 4-5
3 6-7 3 3 3 3 3 6-7
111
Tomahawk APM Overview
When a slice is powered down:
All interfaces on the slice are deleted; hence no traffic can pass through this slice;
Optics are powered off; DOM stops monitoring the port sensors;
Envmon stops polling the power and temperature sequencers and on this slice;
Online diag is stopped for this slice.
Slice 0 is not allowed to be powered off because certain applications require NP0 be up all
the time.
Slice power-on may take ~40 seconds
112
Slice Power Off
RP/0/RP0/CPU0:ios(admin-config)#hw-module power saving location 0/5/CPU0 slice 3
LC Status
---------
Line Card Slice Config Status ENVMON DIAG0 DIAG1 INVMGR PSA
0/5/CPU0 0 On Completed Completed Completed Not registered Completed Completed
1 On Completed Completed Completed Not registered Completed Completed
2 On Completed Completed Completed Not registered Completed Completed
3 Saving Completed Completed Completed Not registered Completed Completed
113
Slice Power Off
RP/0/RP0/CPU0:ios(admin-config)#hw-module power saving location 0/5/CPU0 slice 3
0/5/CPU0
114
Tomahawk APM Troubleshooting
Commands:
show platform slices
admin show apm psm status
admin show apm psa status
show tech-support apm
show tech ethernet interface
115
Tomahawk Life of a
transient packet
116
Tomahawk Intelligent HW Based EFD
- for guaranteed protection of high-priority traffic
Tomahawk implements HW (ICFDQ) based priority EFD
Existing Typhoon SW based EFD for priority classification and discard logic
hw location 0/x/cpu0 early-fast-discard <ip, mpls, vlan cos, i/o encp> val op
& Pre-parse
120
Tomahawk Diagnostics
No change in diagnostic
architecture and
troubleshooting compared
to Typhoon(!)
121
NP Diagnostic - Health Monitoring
A single health monitor packet is injected directly into each NP after LC boot.
If this packet is dropped or lost, then a new health monitor packet re-inject is
attempted three times.
Failure detection time is ~12-13 seconds (polling interval + original health drop +
3 re-inject health drops) ( 1 sec + 3 sec + 9 sec).
Error recovery with NP fast reset
Error message:
LC/0/0/CPU0: Jan 20: 11:09:22:122 : prm_server_to: NP-DIAG health monitoring failure on NP1
122
NPU Packet Processing
PRM
LC CPU
Health Mon packet PRS_HEALTH_MON Health Mon
counter packet
loopback
NPU
123
RSP loopback Diagnostic
CP
U RPS loopback
packets
LC
CPAK
RSP Tsec driver
0
PHY NP FIA
CPAK
1
CPAK
2
Punt
PHY NP FIA Switch
CPAK Fabric FPGA
3
Switch
CPAK Fabric
4 FIA (SM15)
PHY NP
CPAK
5
CPAK
6
Switch
PHY NP FIA Fabric
CPAK
7
124
Online Diagnostic - Linecard NP loopback
To test linecard CPU and NP punt path integrity.
Every minute the LC CPU sends a diagnostic packet over the local host punt
path to each NP
Failure detection time is approximately 3mins.
PFM alarm will be raised.
Error message:
LC/0/5/CPU0:Mar 23 20:09:34 : pfm_node_lc: %PLATFORM-DIAGS-0-
LC_NP_LOOPBACK_FAILED_TX_PATH : Set|online_diag_lc[151635]|Line card NPU
loopback Test(0x2000006)|NP loopback
125
Line Card Loopback Diagnostic
CPU LC loopback
packets
Tsec driver
LC
Punt Switch
CPAK
Fabric ASIC
Switch
0 PH
CPAK NP FIA
Y
1
CPAK
2 PH
CPAK NP FIA
Y
3
CPAK Switch
4 PH Fabric
CPAK NP FIA (SM15
Y
5 )
CPAK
6 PH
CPAK NP FIA
Y
7
126
Show controllers np ports all loc <LC>
Show controllers np counters <> location <LC>
Show spp sid stats Show controllers np fabric-counters <rx | tx>
loc <> <np> loc <LC> show controllers fabric fia
Show spp node- sh controllers np tm counters <np> loc<> link-status loc <LC>
counters loc <> Show lpts pifib hardware entry type <ipv4 | ipv6> Show controllers fabric fia
Show spp interface statis loc <LC> <drops | errors> <ing | egr
loc <> > loc <LC>
show controllers
dpaa tsec port 9
Show controllers fabric
location 0/3/CPU0 Show controllers epm-switch
Show controllers fabric Crossbar instance <>
port-status loc <LC> statistics location <LC>
fia stats location <LC>
show controllers epm-switch
mac-stats <> location <LC>
Show controllers
TM
Show qoshal punt-queue np <> loc <LC> MAC <interface> stats
show qoshal default-queue port <> loc <LC>
127
Usability
128
IOS-XR Usability Initiative Progress
Number of usability related features / featurettes delivered and
planned
5.1.x / 5.2.x 5.3.0 5.3.1 5.3.2
183 18 43 20
Delivered Planned
129
Few usability highlights
To that extent, IOS XR 5.3.1 delivers twenty six (26) new usability related features / featurettes / fixes. Some of the highlights
include:
Global Configuration Replace Ever wanted to quickly move interface configuration from one port to another? This new feature
allows for quick customization of router configuration by match and replace based on interface names and / or regular expressions
(see presentation below for details)
Non-interactive EXEC commands Ever wanted to initiate a router reload without being asked for confirmation? A new global
knob has been introduced to remove user interaction with the parser
BGP advertised prefix count statistics A new knob provides access to advertised count stats (something you could do easily in
IOS but not in IOS XR)
OSPF post-reload protocol shutdown A new knob that would keep OSPF in shutdown state after a node reload
Interactive Rollback operations Ever issued the wrong rollback ID by mistake? a new knob would ask for user confirmation
before committing
CLI / XML serviceability enhancements to several platform dependent commands such as show controllers and show hw-
module fpd commands
130
Some more cool ones
CSCua10564 A 6 1034 Featurette: request auto fpd on newly inserted linecards
CSCuj15553 R 6 532 need provide xml support for "show hw-module fpd location all"
CSCue46774 R 3 757 add xml support for "show controllers tengige/gige/hundredGigE phy"
CSCuo15664 R 6 329 add support "Monitor interface Bundle-Ether * to include error counters
CSCue33274 R 6 759 exec commands must not require interaction over a session
CSCut21292 R 3 2 Need to suppress FRR-Ready syslog
CSCup78779 R 3 248 sh mpls traffic-eng topology srlg output missing local/remote address
CSCuo01750 R 6 256 "sh controllers TenGigE * phy" needs interface name in the output
CSCus28994 R 3 51 Need ability to throttle FRR Ready Message
CSCul81622 R 3 457 show qos for bundle not reporting info for all members
CSCun39879 R 4 326 no support for wildcards for scp
CSCun08218 R 6 310 ASR9K Fabric VOQ Serviceability CLI improvements
CSCup73636 R 3 138 BGP advertised prefixes statistics in IOS XR
CSCup99228 R 6 170 Add CLI to display configs for all the commits
131
Atomic Configuration
Replace Feature
132
Operational and Automation
Enhancements
Atomic Configuration Replace Problem Statement
Example:
Consider an interface with a Problem Statement:
target config where all config It is operationally challenging to
lines are new expect prior knowledge of existing
config in order to manually remove
unwanted items
133
What about
3 Committed Configuration
Example:
RP/0/RSP0/CPU0:pE2#show configuration commit changes last 1
Mon Feb 16 13:33:25.972 UTC Consider an interface with a
Building configuration... new target config where
!! IOS XR Configuration 5.1.2 some config lines are
interface GigabitEthernet0/0/0/19 untouched and the rest are
no description ***To 7604-2 2/12***
description ***TEST-after-change***
either deleted , changed or
! added
no interface GigabitEthernet0/0/0/19
interface GigabitEthernet0/0/0/19 CURRENT Behavior:
ipv4 address 13.3.6.6 255.255.255.0 When issuing the no interface config
ipv6 address 2603:10b0:100:10::31/126
submode, the entire interface config is
no negotiation auto
negotiation auto
destroyed to later be re-created
no load-interval 30 This causes unnecessary interface flaps
load-interval 60
! 134
end
Operational and Automation
Enhancements
Atomic Configuration Replace NEW Behavior
1 Original Configuration 2 Target Configuration
RP/0/RSP0/CPU0:PE1#sh run int gigabitEthernet 0/0/0/19 RP/0/RSP0/CPU0:PE1(config)#no interface GigabitEthernet0/0/0/19
Mon Feb 16 13:00:32.153 UTC RP/0/RSP0/CPU0:PE1(config)#
interface GigabitEthernet0/0/0/19 RP/0/RSP0/CPU0:PE1(config)#interface GigabitEthernet0/0/0/19
description ***AAABBBCCC*** RP/0/RSP0/CPU0:PE1(config-if)# ipv6 address 2603:10b0:100:10::21/126
cdp RP/0/RSP0/CPU0:pE1(config-if)# commit
ipv4 address 13.3.5.5 255.255.255.0
negotiation auto
shutdown
load-interval 30
!
3 Committed Configuration Example:
RP/0/RSP0/CPU0:PE1#show configuration commit changes last 1 Consider an interface with a
Mon Feb 16 13:15:36.655 UTC target config where all config
Building configuration...
lines are new
!! IOS XR Configuration 5.1.2
interface GigabitEthernet0/0/0/19
no description ***AAABBBCCC***
no cdp
no ipv4 address 13.3.5.5 255.255.255.0 NEW Behavior:
ipv6 address 2603:10b0:100:10::21/126 When issuing the no interface config,
no negotiation auto the system does not destroy the subtree
no shutdown
but instead performs a SET of new config
no load-interval 30
!
and REMOVE of unwanted config lines
end
135
Operational and Automation
Enhancements
Atomic Configuration Replace NEW Behavior
1 Original Configuration 2 Target Configuration
RP/0/RSP0/CPU0:PE1#sh run int gigabitEthernet 0/0/0/19 RP/0/RSP0/CPU0:PE1(config)#no interface GigabitEthernet0/0/0/19
Mon Feb 16 13:00:32.153 UTC RP/0/RSP0/CPU0:PE1(config)#interface GigabitEthernet0/0/0/19
interface GigabitEthernet0/0/0/19 RP/0/RSP0/CPU0:PE1(config-if)# description ***TEST-after-change***
description ***AAABBBCCC*** RP/0/RSP0/CPU0:PE1(config-if)# ipv4 address 13.3.5.5 255.255.255.0
cdp RP/0/RSP0/CPU0:PE1(config-if)# ipv6 address 2603:10b0:100:10::21/126
ipv4 address 13.3.5.5 255.255.255.0 RP/0/RSP0/CPU0:PE1(config-if)# negotiation auto
negotiation auto RP/0/RSP0/CPU0:PE1(config-if)# load-interval 60
load-interval 30 RP/0/RSP0/CPU0:pE1(config-if)# commit
!
3 Committed Configuration
Example:
RP/0/RSP0/CPU0:PE1#show configuration commit changes last 1
Mon Feb 16 13:15:36.655 UTC Consider an interface with a
Building configuration... new target config where
!! IOS XR Configuration 5.1.2 some config lines are
interface GigabitEthernet0/0/0/19 untouched and the rest are
description ***TEST-after-change***
no cdp
either deleted , changed or
ipv6 address 2603:10b0:100:10::21/126 NEW Behavior: added
load-interval 60 When issuing the no interface config,
! the system does not destroy the subtree
End but instead performs a SET of new config
and REMOVE of unwanted config lines
Only the diffs (changes, removals,
additions) are applied
No interface flaps
136
What about
other config
Atomic Configuration Replace submodes?
3 Committed Configuration
Example:
RP/0/RSP0/CPU0:PE1#show configuration commit changes last 1
Tue Mar 3 09:05:19.008 UTC
Consider an BGP neighbor-
Building configuration... group with a new target
!! IOS XR Configuration 5.1.2 config where some config
router bgp 100 EXISTING Behavior: lines are untouched and the
neighbor-group NG-test When issuing the no neigh-group config rest are either changes or
description *** NEW NEW *** , the system does not destroy the subtree additions
address-family vpnv4 unicast
but instead performs a SET of new config
!
! and REMOVE of unwanted config lines
! Only the diffs (changes, removals,
end additions) are applied
137
Global Configuration
Replace Feature
138
Operational and Automation
Enhancements
Global Configuration Replace
Description / Use Case
Easy manipulation of router
configuration; e.g. moving around Want to change all
configuration blocks Want to move repetitions of a given
interface config pattern?
Release around?
Target IOS-XR 5.3.2 (08/2015) Interface Y
CSCte81345 description my_uplink
ipv4 address x.x.x.x
load-interval 30
Configuration / Example
RP/0/0/CPU0:router(config)#replace interface <int_id> with <int_id>
Interface X
RP/0/0/CPU0:router(config)#replace pattern <regex_1> with <regex_2> description my_uplink
ipv4 address x.x.x.x
RP/0/0/CPU0:ios(config)#replace interface gigabitEthernet 0/0/0/0 with loopback 450
RP/0/0/CPU0:ios(config)#replace pattern '10\.20\.30\.40' with '100.200.250.225 load-interval 30
RP/0/0/CPU0:ios(config)#replace pattern 'GigabitEthernet0/1/0/([0-4])' with 'TenGigE0/3/0/\1'
139
Operational and Automation
Global Configuration Replace Example 1 Enhancements
hostname fella
group test Original
interface 'GigabitEthernet*'
description grouped Running Configuration RP/0/0/CPU0:fella(config)#replace interface gigabitEthernet 0/0/0/0
mtu 500
! with loopback 450
end-group Building configuration...
ipv4 access-list mylist
10 permit tcp 10.20.30.40/16 host 1.2.4.5 Loading.
20 deny ipv4 any 1.2.3.6/16 232 bytes parsed in 1 sec (230)bytes/sec
!
interface GigabitEthernet0/0/0/0
description first RP/0/0/CPU0:fella(config-ospf-ar-if)#show configuration
ipv4 address 10.20.30.40 255.255.0.0
shutdown Wed Feb 25 18:27:16.110 PST
! Building configuration...
interface GigabitEthernet0/0/0/2
description 10.20.30.40 !! IOS XR Configuration 0.0.0
shutdown interface Loopback450
!
interface GigabitEthernet0/0/0/3 description first
description 1020304050607080 ipv4 address 10.20.30.40 255.255.0.0
shutdown
! shutdown
interface GigabitEthernet0/0/0/4 !
description 1.2.3.4.5.6.7.8
shutdown no interface GigabitEthernet0/0/0/0
! router ospf 10
route-policy temp
if ospf-area is 10.20.30.40 or source in (2.3.4.5/20) then area 200
pass interface Loopback450
endif
end-policy transmit-delay 5
! !
router ospf 10
cost 100 no interface GigabitEthernet0/0/0/0
area 200 !
cost 200
interface GigabitEthernet0/0/0/0 !
transmit-delay 5 End
!
!
!
router bgp 1 140
!
end
Operational and Automation
Global Configuration Replace Example 2 Enhancements
hostname fella
group test Original RP/0/0/CPU0:fella(config)#replace pattern '10\.20\.30\.40' with
interface 'GigabitEthernet*' '100.200.250.225'
description grouped Running Configuration Building configuration...
mtu 500
! Loading.
end-group
ipv4 access-list mylist 434 bytes parsed in 1 sec (430)bytes/sec
10 permit tcp 10.20.30.40/16 host 1.2.4.5
20 deny ipv4 any 1.2.3.6/16
! RP/0/0/CPU0:fella(config)#show configuration
interface GigabitEthernet0/0/0/0 Thu Feb 26 09:00:11.180 PST
description first
ipv4 address 10.20.30.40 255.255.0.0 Building configuration...
shutdown !! IOS XR Configuration 0.0.0
!
interface GigabitEthernet0/0/0/2 ipv4 access-list mylist
description 10.20.30.40 no 10
shutdown
! 10 permit tcp 100.200.250.225/16 host 1.2.4.5
interface GigabitEthernet0/0/0/3 !
description 1020304050607080
shutdown interface GigabitEthernet0/0/0/0
! no ipv4 address 10.20.30.40 255.255.0.0
interface GigabitEthernet0/0/0/4
description 1.2.3.4.5.6.7.8 ipv4 address 100.200.250.225 255.255.0.0
shutdown !
!
route-policy temp interface GigabitEthernet0/0/0/2
if ospf-area is 10.20.30.40 or source in (2.3.4.5/20) then no description
pass
endif description 100.200.250.225
end-policy !
!
router ospf 10 !
cost 100 route-policy temp
area 200
cost 200 if ospf-area is 100.200.250.225 or source in (2.3.4.5/20) then
interface GigabitEthernet0/0/0/0 pass
transmit-delay 5
! endif
! end-policy
!
router bgp 1 ! 141
! end
end
Operational and Automation
Global Configuration Replace Example 3 Enhancements
interface GigabitEthernet0/1/0/0
ipv4 address 20.0.0.10 255.255.0.0 Original RP/0/0/CPU0:ios(config)#replace pattern 'GigabitEthernet0/1/0/([0-
! 4])' with 'TenGigE0/3/0/\1'
interface GigabitEthernet0/1/0/1 Running Configuration Building configuration...
ipv4 address 21.0.0.11 255.255.0.0
! Loading.
interface GigabitEthernet0/1/0/2
ipv4 address 22.0.0.12 255.255.0.0 485 bytes parsed in 1 sec (482)bytes/sec
!
interface GigabitEthernet0/1/0/3
ipv4 address 23.0.0.13 255.255.0.0 RP/0/0/CPU0:ios(config-if)#show configuration
! Fri Feb 27 16:52:56.549 PST
interface GigabitEthernet0/1/0/4
ipv4 address 24.0.0.14 255.255.0.0 Building configuration...
! !! IOS XR Configuration 0.0.0
interface TenGigE0/3/0/0
shutdown no interface GigabitEthernet0/1/0/0
! no interface GigabitEthernet0/1/0/1
interface TenGigE0/3/0/1
shutdown no interface GigabitEthernet0/1/0/2
! no interface GigabitEthernet0/1/0/3
interface TenGigE0/3/0/2
shutdown no interface GigabitEthernet0/1/0/4
! interface TenGigE0/3/0/0
interface TenGigE0/3/0/3
shutdown ipv4 address 20.0.0.10 255.255.0.0
! !
interface TenGigE0/3/0/4
shutdown interface TenGigE0/3/0/1
! ipv4 address 21.0.0.11 255.255.0.0
interface TenGigE0/3/0/5
shutdown !
! interface TenGigE0/3/0/2
interface TenGigE0/3/0/6
shutdown ipv4 address 22.0.0.12 255.255.0.0
! !
end
interface TenGigE0/3/0/3
ipv4 address 23.0.0.13 255.255.0.0
!
interface TenGigE0/3/0/4
ipv4 address 24.0.0.14 255.255.0.0 142
!
end
MPLS Traffic Engineering
143
TE Tunnel Naming
Requirements:
be able to create named TE tunnels
be able to query state on head/mid/tail via the creation name
be able to run services over named TE tunnel
144
TE Tunnel Naming
Configuration:
mpls traffic-eng RP/0/0/CPU0:rtrA(config)#show run group
auto-tunnel named group TUNS
tunnel-id min 1000 max 1500 mpls traffic-eng
! tunnel-te FROM_NY_TO_.*
tunnel-te FOO destination 192.168.0.3
apply-group TUNS path-option 10 dynamic
! autoroute-announce
tunnel-te WEST !
destination 192.168.0.3 !
autoroute-announce end-group
path-option 10 dynamic
path-option 20 explicit name west-path
!
145
TE Tunnel Naming
Operational State
can be queried by head/mid/tail with the creation name using existing CLIs
Also displays (tunnel ID) used in signaling and internally
RP/0/0/CPU0:rtrA#show mpls tr tun name FOO RP/0/0/CPU0:rtrB#show mpls tr tunnels name FOO
Name: tunnel-te1000 Destination: 192.168.0.3 Ifhandle:0x2480 LSP Tunnel 192.168.0.1 1000 [2] is signalled, Signaling State: up
Signalled-Name: FOO Tunnel Name: FOO Tunnel Role: Mid
Status: InLabel: GigabitEthernet0/0/0/0, 24007
Admin: up Oper: up Path: valid Signalling: connected OutLabel: GigabitEthernet0/0/0/2, 24004
Signalling Info:
path option 0, type (Basis for Setup, path weight 3) Src 192.168.0.1 Dst 192.168.0.3, Tun ID 1000, Tun Inst 2, Ext ID
path option 10, type dynamic 192.168.0.1
G-PID: 0x0800 (derived from egress interface properties) Router-IDs: upstream 192.168.0.1
Bandwidth Requested: 0 kbps CT0 local 192.168.0.2
Creation Time: Tue Mar 31 10:41:40 2015 (04:13:44 ago) downstream 192.168.0.4
Config Parameters: Bandwidth: 0 kbps (CT0) Priority: 7 7 DSTE-class: 0
Bandwidth: 0 kbps (CT0) Priority: 7 7 Affinity: 0x0/0xffff Soft Preemption: None
Metric Type: TE (default) SRLGs: not collected
Path Selection Tiebreaker: Min-fill (default) Path Info:
Hop-limit: disabled Incoming Address: 10.10.10.2
Cost-limit: disabled Incoming:
AutoRoute: disabled LockDown: disabled Policy class: Displayed 1 Explicit Route:
up, 0 down, 0 recovering, 0 recovered heads Strict, 10.10.10.2
Strict, 14.14.14.2
146
TE Tunnel Naming
Services over Tunnel
Configured under named tunnel work seamlessly, e.g.
autoroute announce, forwarding-adjacency, forward-class, load-share
autoroute destination installs static route over the tunnel
Configured another protocol (e.g. LDP, static, etc.)
require extending other application to accept named tunnel
require extending other application to listen to tunnel name as an IM
attribute
Existing New
mpls ldp mpls ldp
interface tunnel-te10 tunnel-te FOO
! !
! !
router static router static
address-family ipv4 unicast address-family ipv4 unicast
3.3.3.3/32 tunnel-te10 3.3.3.3/32 tunnel-te FOO
147
Per LSP event history
Do we need per process syslog buffer (so other processes syslog do not
wrap events) - CSCuo49107
How much more valuable is per LSP event history enhancement?
148
Logging Discriminator
CSCuo49107
149
Operational & Automation
Enhancements1
150
Feature Status Notes1
Provide commit failure details as part of commit execution output CSCue31515 5.1.1
Grep utility switch "-A" (after context) & "-B" (before context) CSCsy52070 5.2.2
"show configuration failed load" should point out where the syntax
CSCup55705 5.3.0
error is
commit show-error should remain in config submode after failure CSCur00689 5.3.0
show commit changes diff should remain in config submode CSCur07806 5.3.0
(1) DDTS associated with enhancement (if applicable) and first shipping release Feature shipping / code committed
152
Feature Status Notes1
CSCue33274 5.3.1
Non-interactive EXEC commands CSCut27602 5.3.1
(1) DDTS associated with enhancement (if applicable) and first shipping release Feature shipping / code committed
153
Feature Status Notes1
Show command to display all interfaces configured for Netflow CSCtc82728 5.3.1
(1) DDTS associated with enhancement (if applicable) and first shipping release
154
Feature Status Notes1
(1) DDTS associated with enhancement (if applicable) and first shipping release Development in progress
155
Feature Status Notes1
(1) DDTS associated with enhancement (if applicable) and first shipping release
156
Manageability &
Instrumentation
Enhancements
157
Feature Status Notes1
FTP: should close socket after keep expiry or hard idle timeout CSCug34777 5.1.1
SNMP: MPLS TE needs to populate auto-bw data for tunnel index CSCuq12567 5.1.3
(1) DDTS associated with enhancement (if applicable) and first shipping release Feature shipping / code committed
158
Feature Status Notes1
"show inventory" command truncates pluggable optic serial number CSCum12533 5.1.3
Slow TCP transfer rates affecting ability upgrade and collect traces CSCuo25887 5.1.3
"show controllers" needs to include interfaces name in the output CSCuo01750 5.3.1
XML: schema for "show hw-module fpd location all" CSCuj15553 5.3.1
(1) DDTS associated with enhancement (if applicable) and first shipping release
Feature shipping / code committed
Development in progress
160
Feature Status Notes1
(1) DDTS associated with enhancement (if applicable) and first shipping release
Development in progress
161
Troubleshooting,
Debugability &
Diagnostics
Enhancements
162
Feature Status Notes1
MPLS-TE: Syslog with the exact link id in LSP Failure scenarios CSCtl12202 5.1.1
Add support for TE tunnel interfaces to "monitor interface" command CSCuj66947 5.1.3
Lower the severity from critical to warning in PFM for non-cisco SFP CSCuo71034 5.1.3
(1) DDTS associated with enhancement (if applicable) and first shipping release Feature shipping / code committed
163
Feature Status Notes1
Support for error counters in "monitor interface Bundle-Ethernet * " CSCuo15664 5.3.1
CSCut21292 5.3.1
MPLS-TE FRR protection logging enhancements CSCus28994 5.3.1
(1) DDTS associated with enhancement (if applicable) and first shipping release
Feature shipping / code committed
Development in progress
164
Feature Status Notes1
(1) DDTS associated with enhancement (if applicable) and first shipping release
165
Call to Action
Visit the World of Solutions for
Cisco Campus
Walk in Labs
Technical Solution Clinics
166
Complete Your Online Session Evaluation
Please complete your online session
evaluations after each session.
Complete 4 session evaluations
& the Overall Conference Evaluation
(available from Thursday)
to receive your Cisco Live T-shirt.
167
Thank you
168