16 February 2013 - DRAFT

National Grid Company S.A.

Dammam Smart Grid Trial Project
Dammam Smart Grid Trial Project – 1 Year Plan
Wireless Network Platform to Facilitate Smart Grid Application Testing

Wireless Access to Existing Tier 1 Fiber/SDH Infrastructure

Fixed and Mobile Services, Wired and Wireless IED Interfaces

Power Distribution/Field Area Network Infrastructure (Tier 2)

Wireless Broadband Mesh (ABB Tropos Product Portfolio)
Connect HV, MV, and LV Substations
System Deployed Since 2008 across Abu Dhabi Emirate for ADWEA

Implementation in Close Proximity to SEC Dammam HQ

ABB Has Received Type Approval from CITC
Six (6) months, with ability to extend another six months for the Dammam trial
Smart Grid Applications - Start Testing Goal 16 March

Eventual Transition to Production Use & Regional Expansion

© ABB Group
16 February | Slide 2
2013
ABB Tropos Solution – Virtual Networks for Multiple Applications
Broadband throughput enables smarter grid applications

Distribution Automation
& Control

Renewables
Automated Integration
Metering

Field Data
One Network Demand
Applications
Many Applications Response

Power
Outage Quality &
Management Planning

PHEV Integration
© ABB Group
16 February | Slide 3
2013
ABB Tropos Broadband Wireless Mesh Solution
Infrastructure
Integrated with ABB FOX Optical Product Portfolio
Proven, Cost Effective, Open Standards
Hardware, Software, Management, Analysis Tools
Ruggedized, Weatherized (70o C)
Endpoint Comprehensive Network Management (FCAPS/ISO)
Secure (Military & Critical Infrastructure Compliance)
Embedded
Modules High Availability for Mission Critical Applications

Vehicles
One Network for all Smart Grid Applications
High Throughput (multi-megabits per second)
Indoor Low Latency (1 millisecond per hop)
Scalable Capacity for Future Applications
Management
Simple to Install, Operate, Maintain, Expand
DMS (including Volt/VAR, Voltage Optimization, Outage
Management, Distributed Generation, Demand
Response, etc.), AMI, SCADA, Security, Access Control,
Video Surveillance, Voice, Mobile Workforce, Leased
Directional Line Replacement, etc.
© ABB Group
16 February | Slide 4
Microwave
2013
Smart Grid Network Architecture Reference Model
Remote Monitoring
and Technical Support
DG DVO DMS OMS Other
VPN
SA Firewall Internet
DR DNS

AMI
Data Center(s) AAA Network Services
Network DHCP and Security
Tropos
Management Control IDS

Scalable Wired, Wireless, Fixed, and Mobile Services

Tier 1 Wired
(Core) Tier 1: NG-SA Fiber Infrastructure (Core)/MPLS

Tier 2
(Field Area
Tier 2: Wireless Broadband Access Infrastructure Wired
ABB Tropos Wireless Routers & AMI Communication Modules
& AMI)

Tier 3 Tier 3: Home Area Network Wired

(Home) In-Home Displays, Appliances, Smart Home, Active Home

16
© ABB
February
Group | Slide 5
2013
Example Smart Grid Communication Network
Multiple tiers aggregate large numbers of connections
Many Millions 100s to 1,000s
Millions

Utility Core
Systems

Fiber

Corp Office
Smart Meters

MV & LV
Substations Transmission
Power Distribution System & RMUs
System Power Plant

Home Area Network Field Area & AMI Network Core Utility Network
Tier 3 Tier 2 Tier 1
(ABB
© ABB Group
16 February
CEWE)
| Slide 6
(ABB Tropos Wireless) (ABB FOX Optical)
2013
 Connect to Mesh Router Gateways, Nodes,
Mobile Nodes, Endpoints, AMI Comm Modules

Tropos Mesh Terminology using wired or wireless Ethernet.

Tier 1: National Grid SA Fiber Infrastructure (Core)

Tier 2: ABB Tropos

Broadband Wireless
Access Network
Tropos Mesh Router Gateways,
Nodes, Endpoints, Smart Meter
Comm Modules
Gateway
Gateway Gateway

Node
Mesh

Node
Link

Node
Cluster

Node
Node
AMI Comm Node
Node Module

Alternate
Path
Endpoint AMI Comm
Endpoint Module AMI Comm
Module
Tier 1 & 2 Network Topology
Seamless Extension of Existing Fiber Network
Gateway Router Optical Optical Optical
Gateway
System System System
Routers
Node Router at HV/MV
Substations
Endpoint Router Optical Optical
System System
Endpoint IED Endpoint IED
Connected
to Infrastructure
Optical
Node Router
Optical System
System
Gateways at
all Tier 1
Locations

Primary Optical
Optical
System
System
Data Center Node
Routers
at MV/LV Endpoint Router
Substations Connects via
Wireless to
Optical Mesh Optical
System Infrastructure System
Endpoint Router
Serial or
Ethernet
Endpoint
Optical IED Optical Secondary
Everything System System
Centrally Data Center
Managed Using
Tropos Control Optical Optical Optical
& AFS Manager System System System

• Tier 1 Fiber Rings

• SDH/MPLS Optical System
© ABB Group
16 February | Slide 8
2013
 Embedded
Trial Network Design (Twenty Sites Planned) Router for
RMU Cabinets
Seamless Extension of Existing Optical Network AMI Comm
Module for
Smart Meters

Fiber Fiber
Fiber HV S/S SEC-EOA HQ HV S/S Fiber
HV S/S HV S/S

Tier 1 Optical Network Infrastructure

HV S/S HV S/S

MV S/S Fiber Fiber MV S/S

HV S/S Fiber Fiber
MV S/S HV S/S
MV S/S MV S/S

MV S/S MV S/S

LV S/S LV S/S LV S/S LV S/S LV S/S LV S/S LV S/S LV S/S LV S/S

RMU RMU RMU RMU RMU RMU RMU RMU RMU RMU

Tier 2 Wireless Broadband Mesh Infrastructure

RMU RMU RMU RMU RMU RMU RMU RMU RMU RMU

RMU RMU RMU RMU RMU

© ABB Group
16 February | Slide 9
2013
Installation Configurations
HV Substation
Tier 1 and Tier 2
MV Substation
Tier 1 and Tier 2
LV Substation
Tier 2
RMU Cabinet
Tier 2, with and without adjacent Node Router
Street Light/other intermediate Tier 2 node location
Traffic Signal Pole (municipality)?
Commercial & Industrial Meter
Residential Meter
HV Substation: Tier 1 (Ibn Khaldoun, Al Khobar)
Redundant Gateway Routers
used (best practice) at Tier 1 Locations

Tropos
Infrastructure Three (3) inch
Gateway Steel Pipe
Tropos Router(s)
Infrastructure
Node Router

Flexible
Substation or other Conduit

Facility with Tier 1 (Core) Junction Box with

Point-of-Presence (PoP)

AC Power
Lightening Isolator

Ethernet (CAT5)

MPLS Router (e.g. Juniper)

Demarcation
Tier 1 (Core) & PDH/SDH Service
SDH Optical System E1, nxE1, E3, or
Tier 2 (FAN)
[802.1q VLAN Trunk Ports]
Site A STM1

to Site B Tier 1 Network Traffic Tier 1 Network Traffic to Site C

Fiber (NGSA SDH MPLS Service) Fiber (NGSA SDH MPLS Service)
HV Substation: Tier 2
Dammam, Dammam PP, Rakha, Thuqba

Tropos
Infrastructure
Node
Tropos Router
Infrastructure
Node Router

HV Substation

Ethernet Switch

Site A
 Tier 2 Site: Adjacent RMU

Tropos
Node
Router

To Adjacent
RMU / Node Router,
Endpoint Router or
AMI Comm Module

To Adjacent
Shielded RMU / Node Router,
Ethernet RMU Endpoint Router or
Cable Supplies AMI Comm Module
In Pole DC Power
RMU (PoE)
(Cabinet, Pad Mounted) To Node

Up to 100 meters

Shielded Ethernet Cable in existing Conduit

 Tier 2 Site: RMU with Endpoint Router

Tropos
Node
Router

To Adjacent
RMU / Node Router,
Endpoint Router or
AMI Comm Module

To Adjacent
RMU / Node Router,
Endpoint Router or
AMI Comm Module

AC Power or
Hardened
AC to DC
Tropos Power Supply
1420 Edge Router In Base of Pole
Embedded in RMU

RMU Existing Conduit to Near By AC Power Source

(Cabinet, Pad Mounted) (Utilize Utility GIS Data for Pole Selection)
 Proposed Site Selection &
Estimated Mesh Paths

Group 1: IBN Khaldoun (73203130)

1. GW01 IBN Khaldoun 230 KV
2. GW02 IBN Khaldoun 230 KV
3. ND01 Adamah 69 KV
4. ND02 Al-Zuhur 69 KV
5. ND03 Borshid 69 KV
6. ND04 Dammam 230 KV
7. ND05 Dammam P/P 230 KV BSP
8. ND06 Khudria 69 KV
9. ND07 Nabia 69 KV
10.ND08 Shauba 69 KV

Group 2: Khobar (63203030, 14101000)

1. GW03 Khobar 230 KV BSP
2. GW04 Khobar 230 KV BSP
3. ND09 Al-Hada 69 KV
4. ND10 Al-Mutlaq 69 KV
5. ND11 Bayonia S/S-2 69 KV
6. ND12 Khobar Industrial 69 KV
7. ND13 Meridian 69 KV
8. ND14 Rakha 230 KV BSP
9. ND15 Talimi 69 KV
10.ND16 Thuqba 230 KV BSP
11.BR01 Bayonia S/S-1 69 KV
Next Steps
Prototype Installation (Al Mutlaq 69 kV Substation)
National Grid Inspection of Prototype Installation
Equipment/Customs
Installation Materials Acquisition
Installation
Optimization
Commissioning (Ready for Application Testing)
Smart Grid Application Testing
Project Team
National Grid
Company SA

ABB ABB/Smart Grid NGSA &

Smart Grid Communications Application Providers Distribution
Network Platform (HW & SW Application Solutions) Guidance/Support
Automated Metering Infrastructure (AMI)
Smart Meters
Meter Data Management System
Billing and Customer Data
SCADA
Distribution Management (DMS/OMS)
VOLT-VAR
DVO/CVR)
Security Monitoring
Video Surveillance
Access Control
Voice over IP (VoIP)
Mobile Workforce (Work Order Dispatch, etc.)
Leased Line Replacement (Cost Savings)
Others as Required
PWRP: Self-organizing
The routers automatically discover one another, intelligently
choosing optimal paths back to the wired connection.

MESH MESH
NODE NODE

MESH
NODE MESH
NODE
MESH
NODE

MESH
NODE MESH
NODE
MESH
NODE

Wired backhaul
Optimal routing paths
Backup links

© ABB Group
16 February | Slide 18
2013
PWRP: Optimal path selection
Optimal path is determined by end-to-end quality, not hops.

Wired backhaul
Optimal routing paths
Backup links

© ABB Group
16 February | Slide 19
2013
PWRP: Auto-discovery
Newly added routers participate in auto-discovery,

MESH MESH
MESH NODE NODE
NODE
MESH
NODE

MESH
NODE
MESH
MESH
NODE MESH NODE
NODE
MESH
NODE

MESH
NODE MESH
NODE
MESH
NODE

MESH MESH
NODE NODE
Wired backhaul
Optimal routing paths
Backup links

© ABB Group
16 February | Slide 20
2013
PWRP: Optimal path recalculation
Newly added routers participate in auto-discovery,
recalculating the optimal paths to the wired connection.

MESH MESH
MESH NODE NODE
NODE
MESH
NODE

MESH
NODE
MESH
MESH
NODE MESH NODE
NODE
MESH
NODE

MESH
NODE MESH
NODE
MESH
NODE

MESH MESH
NODE NODE
Wired backhaul
Optimal routing paths
Backup links

© ABB Group
16 February | Slide 21
2013
PWRP: Adding backhaul adds capacity
Capacity can be dynamically upgraded by
adding backhaul connections as needed.

MESH MESH
MESH NODE NODE
NODE
MESH
NODE

MESH
NODE
MESH
MESH
NODE MESH NODE
NODE
MESH
NODE

MESH
NODE MESH
NODE
MESH
NODE

MESH MESH
NODE NODE
Wired backhaul
Optimal routing paths
Backup links

© ABB Group
16 February | Slide 22
2013
PWRP: Automatic clustering
Network automatically re-clusters to take advantage of additional backhaul.

MESH MESH
MESH NODE NODE
NODE
MESH
NODE

MESH
NODE
MESH
MESH
NODE MESH NODE
NODE
MESH
NODE

MESH
NODE MESH
NODE
MESH
NODE

MESH MESH
NODE NODE
Wired backhaul
Optimal routing paths
Backup links

© ABB Group
16 February | Slide 23
2013
Importance of Wi-Fi airtime
Airtime is fundamental, must be monitored and controlled
54 Mbps transmission
54 Mbps &
1 Mbps transmission 1Mbps Clients
Tropos time
802.11g Radio

Wi-Fi is a shared medium; only one device can transmit at a time

Wi-Fi devices connect to network at various data rates
Low rate connections require more airtime than high rate
connections
Low rate users penalize high rate users
Low quality user links consume an inordinate amount of airtime as it
takes longer to send a bit of data at a lower rate
Traditional traffic rate-limits either at the node or on the back-end
don’t help
This phenomenon results in
Network congestion
Unfair distribution of bandwidth between users (i.e. bandwidth hogging)
© ABB Group
16 February | Slide 24
2013
Airtime Congestion Control (ACC)
Extends capacity and provides determinism

• Airtime utilization is continuously monitored

• When congestion is detected Airtime Congestion Control
Neighboring routers are alerted Extends Wi-Fi Network Capacity
Client traffic pre-emptively adjusted
Airtime allocations are applied equally to all

Network Capacity
clients in the affected area Ideal
Airtime allocation enables usable network capacity Airtime Congestion
of Tropos networks to surpass traditional Wi-Fi Control
network capacity
Provides more consistent network access for
users and fair sharing of resources
How ACC is unique Wi-Fi

Monitors airtime, not merely traffic levels

Clients given airtime, not throughput limits light
Network Load
congested

Client limits activated only during high

congestion

© ABB Group
16 February | Slide 25
2013
PowerCurve Dynamic Power Control
Lower power and higher data rates for short, unobstructed links enable concurrent
transmission, increasing overall capacity

MESH MESH
NODE NODE MESH
NODE

MESH
NODE
MESH
NODE

MESH
NODE MESH
NODE

MESH
NODE

MESH MESH
NODE NODE
Wired backhaul

Optimal routing paths

Alternate back-up routes
© ABB Group
16 February | Slide 26
2013
PowerCurve Dynamic Power Control
Higher power and lower data rates for long or obstructed links maximize
transmission success, increasing reliability

MESH MESH
NODE NODE
MESH
NODE

MESH
NODE
MESH
NODE

MESH
NODE MESH
NODE

MESH
NODE

MESH MESH
NODE NODE
Wired backhaul

Optimal routing paths

Alternate back-up routes
© ABB Group
16 February | Slide 27
2013
PowerCurve and PWRP work together
If a different path will support higher overall throughput because of higher data
rates and greater spatial reuse, PWRP will select another path to maximize end-
to-end throughput and capacity.

MESH
NODE
MESH MESH
NODE NODE
MESH
NODE

MESH
NODE
MESH MESH
NODE NODE

MESH
NODE MESH
NODE

MESH
NODE

MESH MESH
NODE NODE
Wired backhaul

Optimal routing paths

Alternate back-up routes
© ABB Group
16 February | Slide 28
2013
PWRP: Self-healing
If a backhaul link fails,

MESH MESH
MESH NODE NODE
NODE
MESH
NODE

MESH
NODE
MESH
MESH
NODE MESH NODE
NODE
MESH
NODE

MESH
NODE MESH
NODE
MESH
NODE

MESH MESH
NODE NODE
Wired backhaul
Optimal routing paths
Backup links

© ABB Group
16 February | Slide 29
2013
PWRP: Reconfigure around backhaul failure

If a backhaul link fails, the network automatically re-optimizes in real-time.

MESH MESH
MESH NODE NODE
NODE
MESH
NODE

MESH
NODE
MESH
MESH
NODE MESH NODE
NODE
MESH
NODE

MESH
NODE MESH
NODE
MESH
NODE

MESH MESH
NODE NODE
Wired backhaul
Optimal routing paths
Backup links

© ABB Group
16 February | Slide 30
2013
PWRP: Self-healing
Similarly, if interference causes a path to fail,

MESH MESH
MESH NODE NODE
NODE
MESH
NODE

MESH
NODE
MESH
MESH
NODE MESH NODE
NODE
MESH
NODE

MESH
NODE MESH
NODE
MESH
NODE

MESH MESH
NODE NODE
Wired backhaul
Optimal routing paths
Backup links

© ABB Group
16 February | Slide 31
2013
PWRP: Reconfigure around interference
Similarly, if interference causes a path to fail, the network re-configures to route
around the obstruction.

MESH MESH
MESH NODE NODE
NODE
MESH
NODE

MESH
NODE
MESH
MESH
NODE MESH NODE
NODE
MESH
NODE

MESH
NODE MESH
NODE
MESH
NODE

MESH MESH
NODE NODE
Wired backhaul
Optimal routing paths
Backup links

© ABB Group
16 February | Slide 32
2013
PWRP: Optimizes across path, channels and bands
Tropos mesh software leverages multiple paths, channels and bands to
create robust, high performance networks
Transmit power and connection speed
are dynamically optimized to maximize
Mesh links dynamically choose the
throughput and minimize latency
frequency band and the channel
within the band on a per hop basis
MESH MESH
MESH NODE NODE
NODE
MESH
NODE

MESH
NODE
MESH
MESH
NODE MESH NODE
NODE
MESH
NODE

MESH
NODE MESH
NODE
MESH
NODE

MESH MESH
NODE NODE
Wired backhaul

2.4 GHz Link Optimal routing paths

5.X GHz Link Backup links

© ABB Group
16 February | Slide 33
2013
Adaptive Noise Immunity (ANI)

Chip-level packet detection parameters are adjusted

in real time to
Minimize false detection events
Maximize sensitivity
Tropos radios have been tested to perform in hostile
interference environments where other devices stall,
reset, or lose sensitivity

© ABB Group
16 February | Slide 34
2013
Functional requirements for field network security
Multi-layer enterprise network security model meets these needs

Network access control (RADIUS or LDAP)

Network resource and remote endpoint protection
User and device identification and authentication
Secure end-to-end data transmission
Traffic segregation & prioritization of applications
Secure network management
Audit and accountability (logging server)
Availability and performance

© ABB Group
16 February | Slide 35
2013
Extending the enterprise model to the edge

• End-to-end secure IPsec VPN tunnels RADIUS IDS/IPS NMS

• 802.1x for authentication and access control

Primary Data Center
• Traffic segmentation using VLANs
• Firewalls for endpoint security

RADIUS IDS/IPS NMS

802.1x
Backup Data Center

802.1x
DA VLAN (tunnel 1)
802.1x
Backup DA VLAN (tunnel 2)
Recloser AMI VLAN (tunnel 3)
Backup AMI VLAN (tunnel 4)

AMI Collector

© ABB Group
16 February | Slide 36
2013
Two Groups (4 Clusters because of redundant
Gateways)
Transmit power and connection speed Mesh links dynamically choose the
are dynamically optimized to maximize frequency band and the channel
throughput and minimize latency within the band on a per hop basis

Wired backhaul (Tier 1)

Optimal routing paths
Backup links
© ABB Group
16 February | Slide 37
2013
Security Compliant for Utilities & Government
Critical Infrastructure Protection Requirements

Application HTTPS, SNMPv3, XML/SSL

Transport SSL/TLS
NERC CIP

FIPS

Network IPSec, Firewall, IP ACLs

802.1x access control, 802.11i authentication,

Link AES encryption, MAC ACLs and whitelists/
blacklists, DoS detection and mitigation

Hardened outdoor enclosure, tamper-detection, encrypted

Physical file system, hardware authentication, protection of critical
security parameters

© ABB Group
16 February | Slide 38
2013
Tropos 7320 mesh router

Dual-radio (2.4 GHz & 5 GHz)

Two wired Ethernet connections
30 W PoE sourcing capability, 12/24/48 VDC
Optional internal battery backup
Flexible input power options
AC powered version: 100 – 480 VAC
DC powered version: 12 – 60 VAC
Modular antennas
Supports all Tropos Mesh OS software
features
available in other Tropos routers

Next generation wireless

Supports 802.11a/b/g/n clients
Maximal Ratio Combining (MRC) receiver
for 2.4 GHz radio
Enhances rate-at-range
Increases mesh capacity

IEEE 1613 for substation installations

© ABB Group
16 February | Slide 39
2013
Tropos 63x0 mesh router

Single (2.4 GHz) and dual-radio (2.4

GHz and 5 GHz) versions
Two wired Ethernet connections (backhaul
and wired device support)
Powered by PoE
Supports all Tropos Mesh OS software
features available in other Tropos routers
Next generation wireless
Compatibility with 802.11b/g/a/n clients
Maximal Ratio Combining (MRC) receiver for
2.4 GHz radio
Enhances rate-at-range
Increases mesh capacity
Small, lightweight form factor
Reduces weight and wind loading concerns
on mounting assets
Skyline gray radome

© ABB Group
16 February | Slide 40
2013
Tropos 1410 automation bridge & router
Hardened cost-effective end point

Wireless mesh router and bridge

Configurable via software load
Available in ruggedized, weatherized
enclosures or as an embeddable module
Class 1, Div 2 certified (C1D2)
802.11b/g/n with 2x2 MIMO
Integrated firewall and IPsec VPN
Wired Ethernet or serial connection
to field automation devices
Supports automation protocols including
DNP3 and IEC 61850 as well as terminal
server capability
Facilitates integration of field
automation devices
10x the bandwidth at half the cost per
connected endpoint of competitive
offerings
© ABB Group
16 February | Slide 41
2013
Tropos 4310 mobile mesh router

Meshing capability optimized for mobility

Extends the mesh infrastructure
Fast roaming (100ms handoff)
Trunk or dash-mounted with external antennas
Optional GPS
Each vehicle is a mobile, high-powered hot zone
Connect Wi-Fi devices in or around the
vehicle
Connect in-vehicle Ethernet devices
Fully manageable by Tropos Control
Bulk upgrade and provisioning
Performance and fault monitoring
Location tracking on Google Maps or with
AVL server
Extends IPSec tunneling security to the vehicle
(mobile VPN)

© ABB Group
16 February | Slide 42
2013
Tropos Smart Grid Network Solution
FCAPS Management Platform
ISO telecommunications network management model and
framework (http://en.wikipedia.org/wiki/FCAPS)
Fault management
Configuration management
Accounting management
Performance management
Security management
Essential for large scale systems

Simple to design, deploy, operate, and maintain

Tropos Control
Carrier-class network management

Management designed for wireless

networks
Scalable thousands of devices
Performance management to avoid
outages
Fast identification and correction of
network faults
Identify and isolate client device issues
Asset management

© ABB Group
16 February | Slide 44
2013