ISP Edge design

Josef Ungerman
CCIE #6167

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 1

 Agenda

ƒ The Internet
ƒ IXP Intro
ƒ Euro-IX
ƒ Technical Details
ƒ Live Examples
ƒ OTT, Video and IXP
ƒ Summary
y & Resources

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 2

 Categorising ISPs

Global ISP Global ISP

$
$
$ Global ISP Global ISP
$
$
$
$ Regional ISP Regional ISP
$ Regional ISP Regional ISP
$
$
$ IXP IXP
$
$ Access ISP Access ISP Access ISP Access ISP
$
$ Access ISP Access ISP

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 3

Peering and Transit

ƒ Transit
Carrying traffic across a network
y for a fee
Usually
Example: Access provider connects to a regional provider

ƒ Peering
Exchanging routing information and traffic
Usually for no fee
Sometimes called settlement free peering
Example: Regional provider connects to another regional
provider

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 4

Private Interconnect

ƒ Two ISPs connect their networks over a private link

Can be peering arrangement
g for traffic
No charge
Share cost of the link
Can be transit arrangement
O ISP charges
One h the
th other
th for
f traffic
t ffi
One ISP (the customer) pays for the link

ISP 1 ISP 2

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 5

Public Interconnect

ƒ Several ISPs meeting in a common neutral location and

interconnect their networks
Usually is a peering arrangement between their networks

ISP 1 ISP 2

ISP 6 ISP 3
IXP

ISP 5 ISP
S 4

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 6

IXP
(Internet Exchange
Points)

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 7

 IXP (Internet eXchange Point)
A physical network infrastructure operated by a single
entity with the purpose to facilitate the exchange of
Internet traffic between Autonomous Systems. The
y
number of Autonomous Systems connected should at
least be three and there must be a clear and open
policy for others to join.

ƒ High-speed/Low-cost Internet Traffic Exchange

ƒ A.k.a.
A k Public
P bli P
Peering
i or S
Settlement-Free
ttl tF Peering
P i
ƒ Non-Profit Associations or Commercial Datacenters
ƒ Around 300 big IXPs in the world

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 8

 IXP (Internet eXchange Point)

ISP 1 eBGP
ISP 4

IXP
ISP 2
ISP 5

ISP 3

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 9

 IXP (Internet eXchange Point)

ISP 1 eBGP
ISP 4

ISP 2
ISP 5

ISP 3

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 10

 IXP (Internet eXchange Point)

ISP 1 eBGP
ISP 4

ISP 2
ISP 5

ISP 3
single
VLAN

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 11

 Euro IX
Euro-IX
Euro-IX (European Internet Exchange Association)
was formed in May 2001 with the intention to further
develop, strengthen and improve the Internet Exchange
Point ((IXP)) community
y

ƒ 105 IXPs in 102 cities

in 31 countries
ƒ 9 non-european
non european members
ƒ www.euro-ix.net

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 12

Euro IX Report 2008
Euro-IX

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 13

Euro IX Report 2008
Euro-IX

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 14

Euro IX Report 2008
Euro-IX

LV PL,
LV, PL UA –
• highly fragmented ISP market
• maybe a lot of Hosting DC’s

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 15

 Example: GoogleNet…
A PortalNet
PortalNet… Dedicated CDN
CDN… Parallel Internet BackBone

ƒ Google has been buying Fiber on

GoogleNet (Faster, Cheaper, More Reliable) a Worldwide basis
DataCenters can be colocated at Peering Points ƒ Google builds it’s own worldwide
IP Backbone.
10G N*10G 40G 100G N*100G ƒ Google peers locally, often on a
Settlement Free Basis, with
E b ll C
Eyeball Carriers.
i
ƒ Google can send any amount of
Some 300 Exchanges Worldwide traffic into the Internet without
paying anyone, they are Nobody’s
Customer.
IX IX IX IX IX ƒ Google distributes it’s DataCenters
to be virtually ONnet to Eyeball
networks. Google is now only a
few Hops away from Any User on
the Internet.
IPTV Local Loop ƒ Tier2 ISP’s
S ’ invest in massive Local
Loop upgrades to support IPTV.
Upgrades ƒ Google drives Net Neutrality so
that whatever Traffic they send,
Google-WIFI Mobile can’t be impaired.
/ Users
ƒ Google can now addresses
Service Substitution (Google TV,
Voice…)

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 16

Internet Edge

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 17

ISP design
g –p
peering
g layer
y

P
MPLS Core

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 18

ISP design
g –p
peering
g layer
y
INTERNET
Upstream ISP’s

eBGP

International
IGW

P
MPLS Core

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 19

 ISP design
g –p
peering
g layer
y
INTERNET
Upstream ISP’s

eBGP

International
IGW

P iBGP
IPv4 Route
Reflectors MPLS Core

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 20

 ISP design
g –p
peering
g layer
y
INTERNET
Upstream ISP’s
IXP
eBGP

National International
IGW IGW

P iBGP
IPv4 Route
Reflectors MPLS Core

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 21

 ISP design
g –p
peering
g layer
y
INTERNET
Upstream ISP’s
IXP
eBGP

National International
IGW IGW

P iBGP
IPv4 Route
Reflectors MPLS Core
ISP Transit
Routers
eBGP
ISP Customers
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 22
 ISP design
g –p
peering
g layer
y
INTERNET
Other ISP’s
IXP
eBGP

IGW

P iBGP
IPv4 Route
Reflectors MPLS Core
ISP Transit
Routers
eBGP
ISP Customers
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 23
ISP design
g –p
peering
g layer
y
INTERNET
Other ISP’s
IXP
eBGP

Internet GW
+ ISP Transit

N-PE
EoMPLS
MPLS pseudowire eBGP
U-PE

ISP Customers
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 24
Internet Gateway

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 25

 Cisco Internet Gateway Routers

ASR 1000 CRS-1/4 CRS-1/8 CRS-1/16 CRS-1 MC

Throughput 20 Gbps 320 Gbps 640 Gbps 1.28 Tbps 10 Tbps

Scalability 40 Gbps 960 Gbps 1.92 Tbps 3.84 Tbps 100+ Tbps
FIB entries 2 Million 2 Million 2 Million 2 Million 2 Million
Netflow entries 2 Million 4 Million 8 Million 16 Million 100+ Million

Existing
E i ti deployments
d l t (~60%
( 60% marketshare)
k t h )
• The most used ISP GW is Cisco 12000 (GSR)
• Many deployments are based on Cisco 7600
• Many small IGW’s are still Cisco 7200

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 26

IGW – Essential Feature set
Broad LAN and WAN interfaces support
• international links – POS STM-1/4/16/64
• national links – GE, 10GE, future full-rate 100GE

IPv4 and IPv6 Routing and Forwarding

• 2M hardware entries (IPv4 + IPv6) – no compression tricks!
• BGP, OSPF/ISIS, BFD – fast, prefix
prefix-independent
independent convergence

IPv4 and IPv6 filters (access-lists)

• thousands of L3/L4 entries (IPv4 + IPv6) – no impact on forwarding rate!
• loose uRPF (Unicast Reverse Path Forwarding)

IPv4 and IPv6 netflow monotoring

• at least 1:1000 sampling rate, V9 export

DDoS attack protection and Control Plane protection

• in-hardware protection of router’s brain
• anti
anti-hacking
hacking tools – management plane protection

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 27

IGW – some optional features
MPLS support
• rarely used on IGW, but sometimes yes
• MPLS Netflow is required
q too

Traffic Shaping with RED – per-interface or per-VLAN

• if the circuit runs over MAN or ISP subrate service
• shaping prevents unnecessary drops and improves TCP goodput
Accounting
• BGP Policy Accounting – per-AS accounting for large networks
• BGP Policy Propagation – packet marking based on BGP Communities
• MAC accounting – for peering/transit via IXP
Secure Virtualization of the router
• Logical Routers with secure resources allocation

Carrier Grade NAT

• IPv4 exhaustion is close!
• large scale IPv4 NAT and IPv6 AFT with V6 Tunneling is desirable
LI (Lawful Intercept)
• if used as a ISP Transit, LI may be mandatory
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 28
ISP Security

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 29

 Anti-spoofing
RFC2827/BCP38 Ingress Packet Filtering
Anti-spoofing filter (ingress filter on source IP)
allow only source addresses from the customer’s 96.0.X.X/24
RFC2827 and RFC3704 (BCP 38 and 84)

Bogon filter (ingress filter on destination IP)

Drops packets
D k t with
ith “insane”
“i ”ddestination
ti ti IP address
dd
RFC1918, own block, internal IP core, NMS
96.0.20.0/24

96.0.21.0/24
Internet ISP ISP’s Customer
Allocation
oca o Block:
oc
96.0.19.0/24
96.0.0.0/19
96.0.18.0/24

Anti-spoofing Filter Applied

ingress on Downstream
Aggregation or NAS Routers
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 30
uRPF (Unicast Reverse Path Forwarding)
“Strict
Strict Mode”
Mode (v1) and “Loose
Loose Mode”
Mode (v2)

router(config-if)# ip verify unicast source reachable-via rx

i/f 2 i/f 2
i/f 1 i/f 3 i/f 1 i/f 3 “Strict Mode”
S D i/f 1
data S D i/f 1
data (aka “v1”)
FIB: FIB:
... ...
S -> i/f 1 S -> i/f 2
D -> i/f 3 D -> i/f 3
... ...
Same i/f: Other i/f:
FORWARD DROP

router(config-if)# ip verify unicast source reachable-via any

i/f 2 i/f 2
i/f 1 i/f 3 i/f 1 i/f 3 “Loose Mode”
S D i/f 1
data S D i/f 1
data (aka “v2”)
v2 )
FIB: FIB:
... ...
S -> i/f x ... ?
D -> i/f 3 D -> i/f 3
... ...

Any i/f: Src not in FIB

FORWARD or route = null0:
DROP

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 31

Bogons
ƒ A Bogon prefix is a route that should never appear in the Internet
routing table
ƒ Different from DSUA.
Bogons are defined as Martians (private and reserved addresses
defined by RFC 1918 and RFC 3330) and netblocks that have not
been allocated to a (RIR) by IANA
ƒ CYMRU maintains list of Bogons, works with IANA and RIR etc.
ƒ http://www.cymru.com/Bogons/index.html
ƒ BOGON List Keeps on Changing as IANA allocates routes.
BE AWARE!
The bogon prefixes are announced unaggregated by the bogon route-
servers is 65333:888; as of 14 JUL 2008 this includes 45 prefixes
ƒ BOGON Router Server.
Peer with CYMRU Route Server keep BOGON list upto date.

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 32

 Hardware protection against DOS attacks
CRS-1
CRS 1 Control Plane Protection
CPU
Input processes
RP

CoPP CSAR queue

Ingress LC

CPU
To RP 4: Multiple queues to
raw queues queue
LC and RP CPU

ASIC 3 LPTS in
3: i iFIB police
li traffic
t ffi

2b: Skip LC CPU!

2a: LPTS iFIB lookup (Match, BTSH/GTSM)
1: Ingress iACL, uRPF
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 33
 IOS XR – Dynamic Control Plane Protection

Router bgp
neighbor 202.4.48.99
…ttl_security
!

mpls ldp
…
!
LC 1 PreIFIB TCAM HW Entries
Local port Remote port Rate Priority

Any ICMP ANY ANY 1000 low

any
y 179 any
y any
y 100 medium

any 179 202.4.48.99 any 1000 medium ttl

Sockett
255

LPTS
202.4.48.1 179 202.4.48.99 2223 10000 medium
bgp
200.200.0.2 13232 200.200.0.1 646 100 medium

ldp

LC 2 PreIFIB TCAM HW Entries …

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal TCP Handshake
34
Detecting
g an attack:

Netflow

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 35

 Netflow is a Security tool #1 today!
7 Keys define a flow
Source Address
Address, Destination Address
Address, Source
Port, Destination Port, Layer 3 Protocol Type,
TOS byte (DSCP), Input Logical Interface
(
(ifIndex)
)

A flow is unidirectional

Turning it on (generic):
interface GigabitEthernet 1/1/1
ip route-cache flow [sampled]
Export (optional):
ip flow-export destination 172.17.246.225 9995

Sampled Netflow (mostly used for Security):

ip flow-sampling-mode packet-interval x
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 36
 Flow Is Defined By Seven Unique Keys

Traffic
• Source IP address
• Destination IP address Enable NetFlow
New
• Source port SNMP MIB
Interface
• Destination port
• Layer 3 protocol type
NetFlow
N tFl
• TOS byte (DSCP) Export Packets

• Input logical interface (ifIndex) Traditional Export &

SNMP Poller
Collector

GUI
© 2004
© 2006 Cisco Systems, Cisco
Inc. Systems,
All rights Inc. All rights Cisco
reserved. reserved.
Internal 37
37
 NetFlow Cache Example
1. Create and update flows in NetFlow cache
Src Src Src Dst Dst Dst Bytes/
Srclf SrclPadd Dstlf DstlPadd Protocol TOS Flgs Pkts NextHop Active Idle
Port Msk AS Port Msk AS Pkt
00A
Fa1/0 173.100.21.2 Fa0/0 10.0.227.12 11 80 10 11000 00A2 /24 5 /24 15 10.0.23.2 1528 1745 4
2
Fa1/0 173.100.3.2 Fa0/0 10.0.227.12 6 40 0 2491 15 /26 196 15 /24 15 10.0.23.2 740 41.5 1
00A
Fa1/0 173.100.20.2 Fa0/0 10.0.227.12 11 80 10 10000 00A1 /24 180 /24 15 10.0.23.2 1428 1145.5 3
1
Fa1/0 173.100.6.2 Fa0/0 10.0.227.12 6 40 0 2210 19 /30 180 19 /24 15 10.0.23.2 1040 24.5 14

• Inactive timer expired (15 sec is default)

• Active timer expired (30 min (1800 sec) is default)
2. Expiration • NetFlow cache is full (oldest flows are expired)
• RST or FIN TCP Flag

Src Src Src Dst Dst Dst Bytes/

Srclf SrclPadd Dstlf DstlPadd Protocol TOS Flgs Pkts NextHop Active Idle
Port Msk AS Port Msk AS Pkt
Fa1/0 173.100.21.2 Fa0/0 10.0.227.12 11 80 10 11000 00A2 /24 5 00A2 /24 15 10.0.23.2 1528 1800 4

3. Aggregation
e.g. Protocol-Port Aggregation
4. Export version Scheme Becomes
Non-Aggregated
gg g Flows—Export
p Version 5 or 9 Payl Protocol Pkts SrcPort DstPort Bytes/Pkt

Export
Header

11 11000 00A2 00A2 1528

5. Transport protocol oa
Aggregated Flows—Export Version 8 or 9
Packet d
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal (Flows) 38
 Netlow Export – V5 fixed format

Usage • Packet
P k t Count
C t •• Source
S
Source
S IPIP
Address
Add
Address
Add From/To
• Byte Count •• Destination
Destination IPIPAddress
Address

Time • Start sysUpTime • Source TCP/UDP Port

Application
of Day • End sysUpTime • Destination TCP/UDP Port

Port • Input ifIndex

• Next
N t Hop
H Add
Address
Utilization • Output ifIndex Routing
• Source AS Number
and
• Dest. AS Number
Peering
yp of Service
• Type • Source Prefix Mask
QoS • TCP Flags • Dest. Prefix Mask
• Protocol

Version 5 used extensively today

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 39
 NetFlow Export – V9 flexible format
Example of Export Packet right after router boot or NetFlow configuration

Option Option Data

Header Template FlowSet
Template FlowSet
Template Template Template Template FlowSet FlowSet ID
(version, Record Record Record Record Template ID
# packets, Template ID Template ID Template ID Template ID Option Option
p
seq ence #
sequence #, ((specific
ifi Data Data
Source ID) (specific Field (specific Field (specific Field (specific Field Field types Record Record
and (Field (Field
types and types and types and types and values)
lengths) values)
lengths) lengths) lengths) lengths)

Example of Export Packets containing mostly flow information

Header Data
a a FlowSet
o Se Data
a a FlowSet
o Se
FlowSet ID FlowSet ID

(version, Data Data Data Data Data Data Data

# packets, Record Record Record Record Record Record Record
sequence #,
(
(Field (Field (
(Field (Field ((Field ((Field ((Field
Source ID) values) values) values) values) values)
values) values)

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 40

Example—What
Example What is an Anomaly?

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 41

NetFlow—nfdump
NetFlow nfdump and nfsen

Source: http://nfsen.sourceforge.net, ev. http://software.uninett.no/stager/

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 42
Arbor Peakflow SP — Application Distribution

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 43

Example—Arbor
Example Arbor Peakflow SP DoS Module

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 44

 BGP Next Hop TOS Aggregation
Typical Example
AS1 AS2 AS3 AS4 AS5

C PE PE C
u u
s MPLS Core s
t PE or PE t
o IP Core
C with
ith BGP R
Routes
t O Only
l o
m m
e PE e
r PE r
s s
PoP PoP

Server Farm 1 Server Farm 2

Internal Traffic: “PoP to PoP”
External Traffic Matrix PoP to BGP AS
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 45
Dropping a DDoS
attack:

BGP Blackholing

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 46

 Customer is DOSed
Before

Peer A
IXP-W
IXP W
A Peer B
IXP-E

Upstream A D
Upstream
B
A C

Upstream
Upstream
B
E B

Target

NOC
G
F POP Target is taken
out
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 47
 Customer is DOSed
Before – Co
Co-Lateral
Lateral Damage

Peer A
IXP-W
IXP W
A Peer B
IXP-E

Upstream A D
Upstream
B
A C

Upstream
Upstream
B
E B

Target
Customers

NOC
G
F POP Attack causes
Co-Lateral
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal
Damage 48
 Customer is DOSed
After – Packet Drops Pushed to the Edge

Peer A
IXP-W
IXP W
A Peer B
IXP-E

Upstream A D
Upstream
B
A C

Upstream
Upstream
B
E B

iBGP
Target
Advertises
List of Black
Holed
Prefixes
NOC
G
F POP

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 49

 BGP Blackholing: Reacting to an Attack
BGP Sent – 171.68.1.0/24 Next-Hop = 192.0.2.1

Static Route in Edge Router – 192.0.2.1 = Null0

171.68.1.0/24 = 192.0.2.1 = Null0

Next hop of 171.68.1.0/24 is now equal

to Null0

ƒ Remote Triggered Black Hole filtering is the foundation for a whole series
of techniques to traceback and react to DDOS attacks on an ISP’s network.
ƒ Easy preparation, does not effect ISP operations or performance.
ƒ It does adds the option to an ISP’s security toolkit.
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 50
 BGP Blackholing: IOS configuration

• place a host-route
host route to Null on every BGP router
ip route 192.0.2.1 255.255.255.255 Null0

• prepare a injection into BGP with the blackhole next-hop

router bgp 10
redistribute static route-map set-blackhole

route-map set-blackhole permit 10

match tag 666
set ip
i next-hop
h 192 0 2 1
192.0.2.1
set community 10:666 no-export
set local-preference 50

• simply filter it out everywhere by one command:

BH(config)# ip route 1.2.2.2 255.255.255.255 Null0 tag 666
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 51
 BGP Blackholing: Filtering on source IP
address

• loose uRPF (unicast reverse path forwarding)

ip route 192.0.2.2 255.255.255.255 Null0
int PoS 1/0/0
ip verify unicast source reachable-via any

!!! packet with source IP prefix pointing to Null0 will be dropped !!!

• prepare a injection into BGP with the blackhole next

next-hop
hop
route-map set-blackhole permit 20
match tag 667
set ip next-hop
next hop 192.0.2.2
set community 10:667 no-export
set local-preference 50

• simply filter it out everywhere by one command:

BH(config)# ip route 1.2.2.3 255.255.255.255 Null0 tag 667
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 52
 BGP Triggered Rate Limiting
QPPB (QoS Policy Propagation via BGP)
router bgp 10
table-map DOS-Activate
neighbor 200
200.200.14.4
200 14 4 remote-as
remote as 10
neighbor 200.200.14.4 update-source Loopback 0
neighbor 200.200.14.4 send-community
!
ip bgp-community new-format
!
ip community-list 1 permit 10:666
!
route-map DOS-Activate permit 10
match community 1
set ip qos
qos-group
group 66
!
route-map DOS-Activate permit 20
!
interface PoS 0/0/0
b
bgp-policy
li source i
ip-qos-map
rate-limit input qos-group 66 256000 8000 8000
conform-action transmit
exceed-action drop
ƒ QPPB marking is done before rate-limit or policing
ƒ hardware support in Cisco 10000, 12000, CRS-1
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 53
Dark IP space:
p

Sinkholes

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 54

 Default Route & the Internet
BHole(config-router)# default-information originate always

ƒ Advertising Default from the Sink

Hole will pull down all sort of junk Router
traffic. Advertises
Default
Customer Traffic when circuits flap.
Network Scans
Failed Attacks
Code Red/NIMDA Sink Hole
Network
Backscatter
ƒ Can place tracking tools (Netflow
cache) and IDS in the Sink Hole Customers
172.168.20.0/24 – target’s network
network to monitor the noise.
172.168.20.1 is attacked
ƒ BCP: Default should be always a
blackhole (Null0 or Static ARP) !!

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 55

 Target Routers are Expendable
# ip route 0.0.0.0 0.0.0.0 192.0.2.253
To ISP Backbone # arp 192.0.2.253 0007.ecbd.e000 arpa

Sink Hole Gateway Target Router

To ISP
Backbone

Sniffers and
Analyzers
To ISP Backbone

ƒ Sink Hole Gateway Generates the more specific iBGP

Announcement.
ƒ Pull the DOS/DDOS attack to the sink hole and forwards the
attack to the target router.
ƒ St
Static
ti ARP to
t the
th target
t t router
t keeps
k the
th Sink
Si k Hole
H l Operational
O ti l–
Target Router can crash from the attack and the static ARP will
keep the gateway forwarding traffic to the ethernet switch.
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 56
What to Monitor in a Sinkhole?
ƒ Scans on dark IP (allocated and announced but
unassigned address space)
Who is scoping out the network—pre-attack planning, worms…
ƒ Scans on bogons (unallocated)
Worms, infected machines, and Bot creation
ƒ Backscatter from spoofed attacks
Who is getting attacked
don’t use “no ip icmp unreachables”
use “ip icmp rate-limit unreachables”

ƒ Backscatter from garbage traffic (RFC-1918 leaks)

Which customers have mis-configuration
mis configuration or “leaking” networks

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 57

Summary &
Resources

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 58

Summary
ƒ Transit vs. Peering
g
ƒ The importance of IXP

ƒ Anatomy of the ISP Edge

ƒ Cisco peering platforms and features

ƒ The
Th importance
i t off Netflow
N tfl
ƒ Basic ISP cecurity techniques

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 59

 Cisco Networkers
25-28. januar 2010.
Barselona
R i t jt se
Registrujte

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 60

© 2006 Cisco Systems, Inc. All rights reserved. Cisco Internal 61