LLD Template 7April05
Short Description
LLD...
Description
Cisco Systems Advanced Services Low Level Design Template Version 0.1
Corporate Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387) Fax: 408 526-4100
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS. THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY. The following information is for FCC compliance of Class A devices: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to part 15 of the FCC rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio-frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications. Operation of this equipment in a residential area is likely to cause harmful interference, in which case users will be required to correct the interference at their own expense. The following information is for FCC compliance of Class B devices: The equipment described in this manual generates and may radiate radio-frequency energy. If it is not installed in accordance with Cisco’s installation instructions, it may cause interference with radio and television reception. This equipment has been tested and found to comply with the limits for a Class B digital device in accordance with the specifications in part 15 of the FCC rules. These specifications are designed to provide reasonable protection against such interference in a residential installation. However, there is no guarantee that interference will not occur in a particular installation. You can determine whether your equipment is causing interference by turning it off. If the interference stops, it was probably caused by the Cisco equipment or one of its peripheral devices. If the equipment causes interference to radio or television reception, try to correct the interference by using one or more of the following measures: Turn the television or radio antenna until the interference stops. Move the equipment to one side or the other of the television or radio. Move the equipment farther away from the television or radio. Plug the equipment into an outlet that is on a different circuit from the television or radio. (That is, make certain the equipment and the television or radio are on circuits controlled by different circuit breakers or fuses.) Modifications to this product not authorized by Cisco Systems, Inc. could void the FCC approval and negate your authority to operate the product. The following third-party software may be included with your product and will be subject to the software license agreement: CiscoWorks software and documentation are based in part on HP OpenView under license from the Hewlett-Packard Company. HP OpenView is a trademark of the Hewlett-Packard Company. Copyright 1992, 1993 Hewlett-Packard Company. The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright 1981, Regents of the University of California. Network Time Protocol (NTP). Copyright 1992, David L. Mills. The University of Delaware makes no representations about the suitability of this software for any purpose. Point-to-Point Protocol. Copyright 1989, Carnegie-Mellon University. All rights reserved. The name of the University may not be used to endorse or promote products derived from this software without specific prior written permission. The Cisco implementation of TN3270 is an adaptation of the TN3270, curses, and termcap programs developed by the University of California, Berkeley (UCB) as part of the UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright 1981-1988, Regents of the University of California. Cisco incorporates Fastmac and TrueView software and the RingRunner chip in some Token Ring products. Fastmac software is licensed to Cisco by Madge Networks Limited, and the RingRunner chip is licensed to Cisco by Madge NV. Fastmac, RingRunner, and TrueView are trademarks and in some jurisdictions registered trademarks of Madge Networks Limited. Copyright 1995, Madge Networks Limited. All rights reserved. Xremote is a trademark of Network Computing Devices, Inc. Copyright 1989, Network Computing Devices, Inc., Mountain View, California. NCD makes no representations about the suitability of this software for any purpose. The X Window System is a trademark of the X Consortium, Cambridge, Massachusetts. All rights reserved. NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PRACTICAL PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE. IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. AccessPath, AtmDirector, Browse with Me, CCDE, CCIP, CCSI, CD-PAC, CiscoLink, the Cisco NetWorks logo, the Cisco Powered Network logo, Cisco Systems Networking Academy, Fast Step, Follow Me Browsing, FormShare, FrameShare, GigaStack, IGX, Internet Quotient, IP/VC, iQ Breakthrough, iQ Expertise, iQ FastTrack, the iQ logo, iQ Net Readiness Scorecard, MGX, the Networkers logo, Packet, RateMUX, ScriptBuilder, ScriptShare, SlideCast, SMARTnet, TransPath, Unity, Voice LAN, Wavelength Router, and WebViewer are trademarks of Cisco Systems, Inc.; Changing the Way We Work, Live, Play, and Learn, Discover All That’s Possible, and Empowering the Internet Generation, are service marks of Cisco Systems, Inc.; and Aironet, ASIST, BPX, Catalyst, CCDA, CCDP, CCIE, CCNA, CCNP, Cisco, the Cisco Certified Internetwork Expert Logo, Cisco IOS, the Cisco IOS logo, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Enterprise/Solver, EtherChannel, EtherSwitch, FastHub, FastSwitch, IOS, IP/TV, LightStream, MICA, Network Registrar, PIX, Post-Routing, Pre-Routing, Registrar, StrataView Plus, Stratm, SwitchProbe, TeleRouter, and VCO are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the U.S. and certain other countries. All other trademarks mentioned in this document or Web site are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0105R) INTELLECTUAL PROPERTY RIGHTS: THIS DOCUMENT CONTAINS VALUABLE TRADE SECRETS AND CONFIDENTIAL INFORMATION OF CISCO SYSTEMS, INC. AND IT’S SUPPLIERS, AND SHALL NOT BE DISCLOSED TO ANY PERSON, ORGANIZATION, OR ENTITY UNLESS SUCH DISCLOSURE IS SUBJECT TO THE PROVISIONS OF A WRITTEN NON-DISCLOSURE AND PROPRIETARY RIGHTS AGREEMENT OR INTELLECTUAL PROPERTY LICENSE AGREEMENT APPROVED BY CISCO SYSTEMS, INC. THE DISTRIBUTION OF THIS DOCUMENT DOES NOT GRANT ANY LICENSE IN OR RIGHTS, IN WHOLE OR IN PART, TO THE CONTENT, THE PRODUCT(S), TECHNOLOGY OF INTELLECTUAL PROPERTY DESCRIBED HEREIN. Low Level Design Template Copyright 2001-2, Cisco Systems, Inc. All rights reserved. COMMERCIAL IN CONFIDENCE.
2
Contents Contents........................................................................................................................................ 3 Tables............................................................................................................................................ 8 Figures......................................................................................................................................... 10 Document Control...................................................................................................................... 13 History.................................................................................................................................... 13 Review.................................................................................................................................... 14 Design Acceptance............................................................................................................... 15 About This Design Document................................................................................................... 16 Document Purpose................................................................................................................ 16 Scope...................................................................................................................................... 16 Document Usage Guidelines................................................................................................ 16 Assumptions and Caveats.................................................................................................... 17 Related Documents............................................................................................................... 17 Network Overview...................................................................................................................... 18 Network Topology................................................................................................................. 18 WAN Overview..................................................................................................................................18 Network Infrastructure.......................................................................................................... 18 Core.....................................................................................................................................................18 Edge ...................................................................................................................................................18 Access.................................................................................................................................................18 Traffic Flow and Characteristic............................................................................................ 19 Existing Services and SLAs................................................................................................. 19 Proposed Network Architecture................................................................................................ 20 Design Considerations......................................................................................................... 20 MPLS Network Architecture..............................................................................................................20 Quality-of-Service...............................................................................................................................21 MPLS/VPN Services..........................................................................................................................21
Contents
Detailed Naming and Addressing Specifications...............................................................22 BGP AS Number.................................................................................................................................22 IP Addressing......................................................................................................................................22 MPLS/VPN Attributes........................................................................................................................23 ..................................................................................................................................................... 25 Deployment Guidelines.............................................................................................................. 25 Physical Network Design...................................................................................................... 25 Layer-2 Transport Media....................................................................................................................25 Central Office Bratislava [BA]...........................................................................................................26 Central Office Banska Bystrica [BB].................................................................................................38 Central Office Kosice [KE]................................................................................................................42 Regional PoPs.....................................................................................................................................46 Hardware/Software Release Table......................................................................................................50 Logical Network Design............................................................................................................. 52 IGP Routing – (OSPF or ISIS)............................................................................................... 52 The Role of OSPF in MPLS network..................................................................52 OSPF Areas.........................................................................................................................................53 OSPF Authentication..........................................................................................................................54 Loopback Addresses...........................................................................................................................54 OSPF Area Summarization.................................................................................................................54 OSPF Costs.........................................................................................................................................54 Designated and Backup Designated Routers......................................................................................55 Default Routes....................................................................................................................................55 OSPF Convergence.............................................................................................................................55 OSPF DNS Lookup.............................................................................................................................57 OSPF Configuration Template...........................................................................................................57 OSPF Deployment Recommendations Summary for the network.............57 Backbone Routing and Label Distribution Protocols.........................................................58 Cisco Express Forwarding (CEF) Switching......................................................................................58 Label Distribution Protocol (LDP).....................................................................................................58 Network Services........................................................................................................................ 62 MPLS/VPN Services.............................................................................................................. 62 MPLS-VPN.........................................................................................................................................62 MP-iBGP4 (Multi-protocol iBGP) Implementation...........................................................................63 Creating VRF Definitions...................................................................................................... 70 VRF Name..........................................................................................................................................70 Route-Distinguisher............................................................................................................................70 VPN Route Target Communities........................................................................................................72 4
Contents
VPN Topologies..................................................................................................................... 73 Full Mesh............................................................................................................................................73 Hub and Spoke....................................................................................................................................73 Exranets...............................................................................................................................................74 Customers with Unique Addresses...............................................................................................74 Customers with Overlapping Addresses.......................................................................................74 Extranet NAT at a Common Service Point...................................................................................74 Extranet NAT at Customer Edge..................................................................................................74 Controlling route exports in extranets...........................................................................................74 ..................................................................................................................................................... 76 PE-CE Routing Implementation................................................................................................. 76 Connectivity via Static Routing............................................................................................ 76 RIPv2 configuration (PE to CE)............................................................................................ 77 PE Configuration................................................................................................................................77 CE Configuration................................................................................................................................78 eBGP configuration (PE to CE)............................................................................................ 78 Configuration at the PE.......................................................................................................................78 Controlling number of VRF routes.....................................................................................................82 ..................................................................................................................................................... 84 Additional MPLS VPN Services................................................................................................. 84 Internet Access for MPLS/VPN customers..........................................................................84 Separate CEs for Internet Access and VPN Access............................................................................84 Low-cost Internet Access (1CE + one/two access links)....................................................................85 Shared vrf-aware services.................................................................................................... 87 Network Address Translation for MPLS/VPN customers..................................................................87 Connecting Downstream ISPs to PE routers......................................................................................88 Remote Access (ASWAN/Security, Dial, DSL, Cable)........................................................89 Wireless.................................................................................................................................. 89 VOIP........................................................................................................................................ 89 Inter-AS/CsC.......................................................................................................................... 89 ..................................................................................................................................................... 90 Traffic Engineering and Fast Reroute Technology Overview.................................................90 Traffic Engineering Basics................................................................................................... 90 Traffic Trunk Attributes......................................................................................................... 92 Bandwidth...........................................................................................................................................92 Path Selection Policy..........................................................................................................................92 Resource Class Affinity......................................................................................................................92 Adaptability.........................................................................................................................................92 5
Contents
Resilience............................................................................................................................................92 Priority................................................................................................................................................93 Resource Attributes.............................................................................................................. 93 Available Bandwidth..........................................................................................................................93 Resource Class....................................................................................................................................93 Path Selection........................................................................................................................ 93 Path Setup ............................................................................................................................. 94 Link Protection (FRR) Basics.............................................................................................. 95 Increased Reliability for IP Services...................................................................................................97 High Scalability Solution....................................................................................................................97 TE/TE-FRR Design ..................................................................................................................... 98 Deciding on the tunnel topology and tunnel types............................................................98 How to Route Traffic Into TE Tunnels..................................................................................98 Policy Based Routing.........................................................................................................................98 Static Routing Into Tunnels................................................................................................................98 Auto-Route.........................................................................................................................................99 Forwarding Adjacency.....................................................................................................................101 Using Directed LDP Sessions............................................................................................ 102 Number of Protected Prefixes............................................................................................ 103 “3” Implementation Of TE-FRR............................................................................................... 105 “3” Network Architecture.................................................................................................... 105 Introduction.......................................................................................................................................105 TE-FRR Design................................................................................................................................106 Primary Tunnels..........................................................................................................................107 Backup Tunnels...........................................................................................................................107 Sample configurations......................................................................................................................109 Generic Global Commands.........................................................................................................109 Birmingham P Router..................................................................................................................109 Quality of Service................................................................................................................ 112 Introduction.......................................................................................................................................112 Differentiated Services Model – Introduction..................................................................................113 DiffServ Aware TE...........................................................................................................................124 ST QoS design – An Overview.........................................................................................................124 CE-to-PE QoS mechanisms (applied on the CE) – PPP or HDLC...................................................126 CE-to-PE QoS mechanisms (applied on the PE) – PPP or HDLC...................................................144 PE-to-P QoS mechanisms (applied on the PE).................................................................................146 PE-P, P-P and P-PE QoS mechanisms (applied on the P)................................................................148 PE to CE QoS mechanisms (applied on the PE)...............................................................................154 QoS mechanisms on ATM PVCs (applied on the CE and PE).........................................................155 6
Contents
..........................................................................................................................................................157 ................................................................................................................................................... 157 High Availability........................................................................................................................ 158 ................................................................................................................................................... 159 Security .................................................................................................................................... 159 Password Management.....................................................................................................................159 Console Ports....................................................................................................................................160 Controlling TTY’s............................................................................................................................160 Controlling VTYs and Ensuring VTY Availability..........................................................................160 Logging.............................................................................................................................................161 Anti-spoofing....................................................................................................................................162 Controlling Directed Broadcasts.......................................................................................................164 IP Source Routing.............................................................................................................................164 ICMP Redirects.................................................................................................................................165 CDP...................................................................................................................................................165 NTP...................................................................................................................................................165 ................................................................................................................................................... 167 Network Management............................................................................................................... 167 ................................................................................................................................................... 168 Appendix I................................................................................................................................. 168 Appendix II................................................................................................................................ 169
7
Tables Table 1 Revision History............................................................................................................ 13 Table 2 Revision Review............................................................................................................ 14 Table 3 PE-P Connectivity in CO BA......................................................................................... 32 Table 4 Software Release Table................................................................................................ 50 Table 5 Proposed OSPF Metrics.............................................................................................. 54 Table 6 OSPF Timer Default Values......................................................................................... 56 Table 7 RT/RD Allocation.......................................................................................................... 73 Table 8 BGP Timer Definitions................................................................................................. 82 Table 9
Tunnel Provisioning.............................................................................................. 108
Table 10 Class-Selector PHBs................................................................................................. 116 Table 11 Serialisation delay [ms] as function of link speed and packet size......................119 Table 12 Recommended fragment size...................................................................................121 Table 13 The components of the end-to-end delay model....................................................122 Table 14 CoS Mechanisms Overview......................................................................................125 Table 15 NB and EB settings................................................................................................... 132 Table 16 WRED Settings for Business Class.........................................................................139 Table 17 WRED Settings for Streaming Class.......................................................................139 Table 18 WRED Settings for Standard Class.........................................................................140 Table 19 WRED - exponential weighting constant.................................................................142 Table 20 MDRR weights........................................................................................................... 150 Table 21 WRED Setings for Business Class (ENG-2 GSR) ..................................................153 Table 22 WRED Setings for Streaming Class (ENG-2 GSR)..................................................153
Tables
Table 23 WRED Setings for Standard Class (ENG-2 GSR)....................................................153 Table 24 ATM Overhead........................................................................................................... 156 Table 25 LLQ bandwidths and ATM........................................................................................ 156
9
Figures Figure 1 network – WAN topology..........................................................18 Figure 2 Architecture of CO BA................................................................................................. 27 Figure 3 HW configuration of ba2-igw-2...................................................................................28 Figure 4 HW configuration of ba1-igw-1...................................................................................29 Figure 5 HW configuration of ba-six-1......................................................................................29 Figure 6 HW configuration of ba1-p-1 and ba2-p-2..................................................................31 Figure 7 HW Configuration of ba2-pe-4....................................................................................33 Figure 8 HW Configuration of ba1-pe-5....................................................................................33 Figure 9 HW Configuration of ba1-pe-6 and ba1-pe-7.............................................................35 Figure 10 Architecture of CO BB............................................................................................... 38 Figure 11 HW configuration of bb1-p-1.....................................................................................39 Figure 12 HW configuration of bb2-p-2.....................................................................................39 Figure 13 HW configuration of bb1-pe-1...................................................................................40 Figure 14 HW Configuration of bb2-pe-2..................................................................................40 Figure 15 HW Configuration of bb1-cat-1 and bb1-cat-2.........................................................41 Figure 16 Architecture of CO KE............................................................................................... 42 Figure 17 HW configuration of ke1-p-1.....................................................................................43 Figure 18 HW configuration of ke2-p-2.....................................................................................43 Figure 19 HW Configuration of ke1-pe-1.................................................................................44 Figure 20 HW Configuration of ke2-pe-2..................................................................................44 Figure 21 HW Configuration of ke1-cat-1 and ke1-cat-2..........................................................45 Figure 22 New Architecture of Regional PoPs (10k based)....................................................47
Figures
Figure 23 New Architecture of regional PoPs (7206VXR based)............................................47 Figure 24 HW Configuration of 10008 in Regional PoPs (ZA, NI, TT, PO)..............................48 Figure 25 HW Configuration of 10005 in Regional PoP Trencin (TN).....................................48 Figure 26 HW Configuration of 7206VXRs in Regional PoP..................................................48 Figure 27 OSPF enabled links................................................................................................... 53 Figure 28 Layer 2 Frame with 2 MPLS Labels.........................................................................59 Figure 29 MP-BGP VPN Route Distribution example...............................................................63 Figure 30 VPN route distribution using partitioned RRs.........................................................64 Figure 31 Route Reflector Redundancy in the Networks.......................65 Figure 32 Redundant Route Reflectors with same cluster-id................................................67 Figure 33 Unique RD per each VPN.......................................................................................... 71 Figure 34 Unique RD per site for each VPN.............................................................................72 Figure 35 PE-CE eBGP with unique AS....................................................................................79 Figure 36 PE-CE eBGP with single network wide AS..............................................................80 Figure 37 Internet Access from a VPN using separate CEs....................................................84 Figure 38 Internet Access from a VPN – Single CE (two links in CEred, single link on CEblue)........................................................................................................................................ 86 Figure 39 NAT in CE router........................................................................................................ 88 Figure 40 - Traffic Engineering Mechanisms...........................................................................91 Figure 41 - Traffic Engineering Path Setup..............................................................................94 Figure 42 - TE FRR Example...................................................................................................... 96 Figure 43 - Topology Without Tunnels.....................................................................................99 Figure 44 - R1 Routing Table – No MPLS TE..........................................................................100 Figure 45 – Topology With TE Tunnels..................................................................................100 Figure 46 - R1 Routing Table With Autoroute Announce......................................................100 Figure 47 - Forwarding Adjacency Topology.........................................................................101 Figure 48 - "3" Core Network Architecture.............................................................................106 Figure 49 - Illustration of Primary and Backup TE Tunnels..................................................107 Figure 50 Various interpretations of the TOS field................................................................114 Figure 51 DSCP Interpretation................................................................................................ 117
11
Figures
Figure 52 Adaptive jitter buffer................................................................................................ 120 Figure 53 - Call admission control.......................................................................................... 120 Figure 54 LFI to reduce frame delay and jitter.......................................................................121 Figure 55 Overview of end-to-end delay segments...............................................................123 Figure 56 DSCP to EXP mapping............................................................................................ 124 Figure 57 DSCP / MPLS Headers............................................................................................. 124 Figure 58 QoS mechanisms overview....................................................................................126 Figure 59 In/Out-contract Marking and Policing (example for Business class) .................130 Figure 60 CAR based In/Out-contract Marking and Policing ...............................................131 Figure 61 WRED Algorithm...................................................................................................... 136
12
Document Control Authors: Change Authority: Reference Number: EDCS-xxxx
History Table 1 Version No.
Revision History Issue Date
Status
Reason for Change
Document Control
Review Table 2
Revision Review
Reviewer’s Details
Version No.
Date
Change Forecast: Medium This document will be kept under revision control.
14
Document Control
Design Acceptance The signatories below confirm that the design meets the requirements specified. The design is subject to change during or following staging.
CISCO SYSTEMS
Slovak Telecom
By:__________________________________
By:_____________________________________
Name:
Name:
Title:
Title:
Date:________________________________
Date:___________________________________
15
About This Design Document Document Purpose The purpose of this document is to outline the Cisco Systems recommended Low Level Design (LLD) for . It details the physical and logical requirements and how we will accomplish these requirements. The document is split into following main sections, •
Current Architecture and Network Design
•
Planned Services
•
Proposed Design and Architecture
•
OSS
Note: The above sections may change depending on the customer’s needs The document provides sufficient detail to derive the device configurations that will be documented in the Network Implementation Plan. Some parameters may be determined during network deployment.
Scope Please refer to Statement of Work documents for exact definition of project deliverables.
Document Usage Guidelines The document should be used as a guideline for deriving the necessary information to ultimately create the configurations that allow the network to provide the necessary services. The more theoretical sections should be used in conjunction with the practical sections in order to allow the deployment engineer to understand the service requirements behind the configurations. This will also allow the deployment engineer to take certain decisions when deploying and configuring the network. As long as the Low Level Design document is in a draft format, it is susceptible to modifications and additions initiated by Cisco Systems or by the customer. After acceptance of the LLD by the customer, the LLD document is still a living document that will be updated by experiences gained throughout the deployment and testing phases.
About This Design Document
Assumptions and Caveats •
Based on the input from CRDR ,HLD,SOW and Site Survey write down the necessary assumptions and caveats
•
It is assumed the reader is familiar with the service requirements. Furthermore it is also assumed the reader is familiar with Cisco IOS and has a basic understanding of the network and technologies that will be used to fulfil the customer requirements.
Related Documents Write down the links to CRD, CRDR, SOW, HLD and Site Survey
17
Network Overview Describe what kind of customer and their core business. Also at a high level explain their current architecture with more details in the following section. This information can be collected from CRD and HLD
Network Topology WAN Overview The following figure is provided for illustrative purposes and depicts a high-level view of network. Picture is simplified for easier understanding of WAN network topology. Figure 1
network – WAN topology
Network Infrastructure Core If possible provide the details of current core network. The platforms used, kind of links, what routing protocol etc.
Edge The following types of devices are installed in network as Provider Edge (PE) routers:
Access Customer Edge (CE) routers are classical IPv4 routers and will interconnect customer sites with PE routers via leased lines (as described in Error: Reference source not found chapter below). CE routers usually reside in customer premises. CE router can be managed by or by the customer.
Traffic Flow and Characteristic In this section explainand characterize the custoemnr traffic. Explain the load that major pops(or even links) are carrying. What additional traffic is expected with the new deployment.
Existing Services and SLAs In this section explain the current services that the customer is offering and the SLAs ,if any , associated with these services
19
Proposed Network Architecture In this chapter we highlight the architecture that is being proposed for the customer based on the requirements listed in the CRD. Following chapter would go into the details of how this architecture would be deployed.
Design Considerations This chapter summarizes the design objectives that have been followed throughout the LLD, and the design rules we have taken to meet these objectives.. Following are the list of these objectives as dictated by the customer
MPLS Network Architecture Following are just some of the examples. Customize this section based on your customer requirements •
Fast convergence Fast convergence and network stability are two orthogonal components in any network design. Accurately measuring and interpreting the convergence time in complex topologies is somewhat like rocket science as many factors are involved. Improving the overall convergency by tuning the relevant parameters is a complex task that requires in depth analysis of all side effects (e.g. CPU utilization). We recommend to tune the convergence of routing protocols in a separate project. The scope of this project should be exclusively the optimization of convergency in ST MPLS network, by introducing new features (e.g. Traffic Engineering Fast Reroute) and tuning of routing protocol timers (see OSPF Convergence chapter)
•
Network Stability and Scalability Any routing protocol would scale well, if the routing information is stable. Stability of backbone IGP was one of the main concerns in the former ST network. For this reason the following changes have been made during previous project phases : o
Offload of any customer routes from backbone OSPF into BGP.
o Subnet aggregation of unstable leased-line connections and redistribution as static route into BGP. o Subnet aggregation the of dial-up customers with fixed addresses on VPDN tunnel concentrator, and redistribution as static routes into BGP •
Network resilience ST MPLS network has been designed for high availability. Physical and logical design ensures that primary and backup path exists between any two core routers. Core routers are equipped with primary
and backup route processors. Resilient connections between regional PoPs and Core locations will be rolled out in project phase 8A. •
Network security Cisco has implemented best-practices security mechanisms on ST routers to protect the network. Customer security and managed firewall service was not in the scope of any Cisco project.
•
Simplicity ST MPLS network design is clean and simple to understand. Any feature or design element that would increase network complexity - but have a limited overall benefit - has been avoided. ST has decided to clean-up the existing IP addressing scheme and migrate from multi-area OSPF into single-area design.
•
MPLS LDP has been chosen for label distribution in ST MPLS network. LDP is enabled on all core links (P-P, P-PE, P-RR, P-iGW).
Quality-of-Service •
Traffic prioritisation The following Classes of Service are implemented in the network: Voice, Streaming, Business, Standard. Each class has different QoS attributes and guaranteed (configured) bandwidth that cannot be utilised by any other class during congestion periods. Backbone links must be provisioned with sufficient capacity for each of the classes!
•
Flexibility Modular QoS CLI allows to map traffic flows of customers in one of the Classes of Service. Such classification and marking is extremely flexible (different customers can map different applications in any of the classes), but requires the understanding of traffic profiles (e.g. SMTP or any other data traffic must not be mixed with delay-sensitive VoIP packets).
•
Scaleable implementation The customer-specific QoS configuration is implemented on CE routers – QoS configuration template on PE and P devices will remain stable and the same for all ST customers. VPNSC shall be used for accurate provisioning of QoS parameters on access (PE-CE) connections.
MPLS/VPN Services
1
•
Flexible and scalable managed IP VPN service Achieved through MPLS technology, properly applied MPLS/VPN functionality and VPNSC management system1.
•
Service resilience Resilient MPLS backbone, redundant route reflectors and the possibility of fully resilient connectivity scenarios on access-layer (2CE-2PE) in all PoPs, are necessary building blocks for high availability MPLS/VPN service.
•
End-to-end Quality of Service Achieved through the use of various Diffserv mechanisms: classification, marking, policing, queuing and dropping. QoS is implemented on access layer and in the backbone.
VPNSC and NMS design is covered in separate LLD document. 21
•
Internet Access for MPLS/VPN customers Internet access from the MPLS/VPN is provided for customers with such requirement. For security reasons we only recommend to implement the Internet connection through a dedicated CE router and dedicated access-layer circuit (see chapter for detailed description)
•
Security Assuming that MPLS core is secure, the MPLS/VPN solution offers same level of security as the traditional layer-2 VPN networks.
Detailed Naming and Addressing Specifications BGP AS Number AS number for MPLS network and inter-domain routing is
Routers Naming convention for backbone routers has been defined by as part of current deployment. Explain in detail the nomenclature used by the customer or will be used by the customer
Customer Edge (CE) Routers Naming convention for CE routers shall in addition to backbone naming scheme incorporate the customer name or customer ID.
DNS domain name The domain name for networking equipment shall be
IP Addressing Explain in detail the IP addressing scheme in customer’s network
Loopbacks The following address block is used for numbering of Loopback02 interfaces in MPLS network: X.X.X.X. Give any additional details as it relates to loopback addresses 2
Loopback interfaces are used primarily for OSPF stability, creation of iBGP and LDP neighborships. 22
Backbone Links Backbone connections are numbered with IP addresses from the following subnets: X.X.X.X Explain in detail how these subnets would be allocated to different links
Existing Routers In this section explain any renumbering of ip addresses that might be needed for any existing infrastructure
MPLS/VPN Access Connections has allocated the address pool x.x.x.x/y for numbering of MPLS/VPN access connections.
Internet Access Connections has allocated the following address pool for numbering of access connections for Internet customers: x.x.x.x/y
IP-SLA Depending on how IP-SLA is deployed , explain addressing scheme that would be required
MPLS/VPN Attributes VPN naming and addressing is described by four attributes. These are the VRF name, RD, RT and SOO. Detailed naming of MPLS/VPN attributes are given in the NMS document (see Error: Reference source not found)
VRF Name The VRF name is locally significant on the PE router. Service providers that provision the MPLS/VPNs manually (via CLI on router console) are advised to use a VRF name that is recognisable across the whole network in order to aid troubleshooting. The name should identify the predominant function of the CEs
23
connected to the VRF, this can be a word to identify the customer or, in the case of a VRF that is shared by multiple customers, the type of service offered by the attached CEs. Explain how VRF name would be configured. Also explain if any provisioning tool like ISC is being used
RD Explain how RD is being assigned. Also explain if any provisioning tool like ISC is being used
RT RTs define the VPNs, the value is therefore significant across the whole network. A different RT is required for each VPN, hub-and-spoke VPNs can be considered as 2 uni-directional VPNs and therefore need 2 RTs if bi-directional routing is required. Explain how RT is being assigned. Also explain if any provisioning tool like ISC is being used
SOO SOO is required in order to provide loop avoidance on multi-homed sites. The same SOO attribute should be added to every routing update originating on the multi-homed site. Routes received by a VRF carrying a SOO also exported by the VRF should be filtered out so that they are not imported. Only multi-homed sites require the SOO attribute so SOO values will only be allocated as required. Explain how SOO is being assigned. Also explain if any provisioning tool like ISC is being used
BGP AS Numbers When peering with a customer CE router, the customer can use his registered ASN if he possesses one. If not, private ASNs (64512 to 65535) can be used on the CEs. The same ASN can be used for all the sites of a VPN to conserve the number of ASNs; this is recommended and also allows for VPNs that have more than 1024 sites. The “as-override” is used on the PEs in order to reuse the same ASN for all the sites.
24
Deployment Guidelines In this chapter we’ll exaplain you how to deploy the architecture proposed in the previous chapter
Physical Network Design The content is this section is only for reference purpose. You need to add details specific to your own customers. This is confidential information In no way should this information be shared with non-cisco folks
Layer-2 Transport Media The following summarizes the physical layer transport in ST MPLS network. •
Inter-site connectivity o POS STM-16 over DWDM is used to interconnect: Core COs BA, BB and KE 7304 SIX router and P router in BA. o
•
POS STM-1 over SDH interconnects regional PoPs (PEs) with core COs (Ps)
Intra-site connectivity o Back-to-back POS STM-16 is used for connectivity between P and collocated IGW routers in BA PoP o Back-to-back GE connections are deployed for the following device pairs: 10008 (PE) -12410 (P) ; Bratislava, Banska Bystrica, Kosice 7600 (PE) -12410 (P) ; Bratislava ERX (DSL) -12410 (P) ; Bratislava 10008 (PE) -12012 (P) ; Banska Bystrica o Back-to-back POS STM-1 is used for the following links: 12406 (iGW) -12008 (iGW) ; Bratislava (2 x STM-1) (PE) (P) ; Any other collocated PEs in central offices 7204VXR (RR) -12xxx (P) ; BA & BB o Switched FE connections are used to connect existing ST IP (CE, NAS) routers that are cascaded behind new PEs. Layer 2 switches (Cisco 4503/3550-24) are used for port aggregation. 25
Central Office Bratislava [BA] The chapters below depict the architecture of the three central offices and a typical implementation of a regional PoP in ST MPLS network. Central Office in Bratislava is a major hub in ST network, because of the largest concentration of customer base in that area. For this reason, ST decided to build the BA CO resiliently and with powerful routers. The devices in BA CO can be logically grouped into four layers: Peering, Core, Aggregation and Access. Two Firesections The Bratislava CO will be divided into two physically separated firesections (1 and 2). Main components of peering, core and aggregation layer will be deployed in different locations to achieve network resilience in case of a major disaster. Customers can be dualhomed to PE routers in both firesection in order to provide them a maximum of redundancy. The architecture of BA CO is depicted on the following figure.
26
Figure 2
Architecture of CO BA
Peering Layer Internet Gateway Routers (ba1-igw-1, ba2-igw-2) Two routers (Cisco 12406 and Cisco 12008) will be installed in BA CO for IP connectivity between ST MPLS network and: •
Downstream ISPs (eg. local ASP - Application Service Provider) that pay for transit service to ST – these are in fact customers of ST.
•
Upstream ISPs (eg. Deutsche Telekom, UTA) that provide global Internet reachability for customers of ST.
Each iGW will have a POS STM-16 back-to-back connection with a different P router. Interconnections with ISPs can be either POS STM-1 or E3 leased lines. Both iGWs are equipped with powerful route processors (primary and redundant) that can handle large number of BGP sessions, and will have installed sufficient amount of memory to carry one or more copies of full Internet routing table. Back-to-back links between iGW routers
27
Both iGWs will inject a BGP default route towards PE routers. A PE router will select the default route based on IGP distance to originating iGW, and eventually send all Internet traffic to the closest iGW. However, this iGW may not be the best exit point for a given Internet destination, so the packets would have to be re-routed to the neighbouring iGW to be delivered to the upstream ISP. For this reason, two POS STM-1 back-to-back links are installed between iGWs. No other traffic (eg. packets between two ST PoPs) are passing these two links. An alternative solution would be to download full Internet routing table to any PE router, which can in turn deliver the Internet traffic to the right iGW. This would result in more optimal traffic flows across ST core, and enable “distributed” peering system, with possibility of connecting ISP circuits in any PoP. Assuming that BGP dampening is enabled on border routers, and number of routes that can be accepted from any ISP is limited3, the major drawback is memory requirements (min. 256 MB) on all PE routers due to large number of routes in the global Internet routing table. Figure 3
HW configuration of ba2-igw-2
0
1 x POS STM-16
1
8 x POS STM-1
2
6 x E3
3 4
4 x POS STM-1 GRP Redundant
5
GRP
P router
Upstream/Downstream ISP
CSC
SFC
CSC
SFC
Alarm
Alarm Power
12406
SFC Power
3
It is a good practice to define the maximum number of prefixes that can be accepted from any eBGP peer. This is for example to prevent the situation where a peering partner at SIX advertises the full Internet routes to ST. 28
Figure 4
HW configuration of ba1-igw-1 P router
5
6
7
12008
4 x POS STM-1
4
Power
-
3
4 x POS STM-1
-
2
-
1 x POS STM-16
1
CSC 1
GRP Redundant
0
CSC 0
GRP
Power
Upstream/Downstream ISP
SIX Internet Gateway Router (ba-six-1) One 7304 router (ba-six-1) is collocated at SIX premises, for mutual and free-of-charge exchange of customer traffic between peering partners at Slovak Exchange Point. ba-six-1 router is attached to the SIX switch with a GE interface, and interconnected with ST core router ba2-p-2 via a POS STM-16 connection. Figure 5
HW configuration of ba-six-1
4
-
5
1 x POS STM-16
2
-
3
-
1
-
0
GE0 GE1
NSE100
P router
7304
Peering partners @ SIX
Resiliency in Peering Layer It is recommended and a good decision to terminate at least one upstream ISP connection on each iGW. This will protect from failures on a single peering circuit, and/or major failure of a single peering router
29
(either ST’s or the one of upstream ISP). Having two redundant Internet connections on separate routers will also permit software and hardware upgrades on iGWs without long downtimes.4 The two iGWs distribute BGP routes (default route and full Internet table if required) to other BGP neighbors in ST network via two redundant route reflectors. The Internet connectivity scheme with physically separated IGW routers protects against the failure or major disaster in one of the Bratislava firesections. Internet connectivity will remain through the backup upstream ISP in the other firesection. There’s currently a single router installed at SIX premises. If this router or a link between ba2-p-1 and SIX router fails, the direct connectivity with SIX participants will be lost. Nevertheless, this does not represent a single point of failure, because the peering partners’ networks can be during failure reached 5 across upstream ISPs.
Core Layer MPLS P Routers (ba1-p-1, ba2-p-2) ST has selected two Cisco 12410 routers for MPLS P devices in Bratislava CO. These P routers do not perform any aggregation layer services (eg. termination of customer links, or peering circuits, BGP routing, etc.) P routers are also not routing the IP packets across ST core; the only6 task of P routers is to label switch the MPLS frames through high speed links, respecting the queuing and dropping attributes which are encoded in the EXP bits of MPLS labels. Each P router links the following devices:
4
•
Remote P router - GSR. POS STM-16 links interconnect the P routers into a high-speed backbone. Underlying technology is DWDM.
•
Remote PE. Regional PEs in remote PoPs interconnect with P routers via POS STM-1 links.
•
Collocated PEs are attached with P routers either via a back-to-back POS interfaces, or using a GE technology in a back-to-back mode.
A short downtime will occur because of eBGP convergence throughout the Internet.
5
Most likely this would introduce higher RTT and jitter, and increased load on generally very expensive transit connections with upstream ISPs. 6
This is not entirely true because the locally sourced packets will be subject to label imposition (eg. OSPF updates) 30
HW configuration of ba1-p-1 and ba2-p-27
Figure 6
P, IGW and SIX router RR, MPE, Regional PEs, 7513 PE router Collocated PEs
GRP Redundant
GRP
8
9
Power
12410
Alrm 1
4 x POS STM-16 7
Alrm 0
4 x GE 6
SFC 4
5
SFC 3
4 x GE 4
SFC - 2
8 x POS STM-1 3
SFC 1
8 x POS STM-1 2
SFC 0
4 x POS STM-16 1
CSC 1
0 CSC 0
-
P router
Power
Resiliency in Core Layer Dual triangle topology of ST core network is resistant to failures on a single P-P link. When a P-P link fails, or when a port on line card fails, the transit traffic between the two P routers can be rerouted across longer path around the core ring. This should not be very frequent event, also because of layer-2 resiliency built in DWDM ring. Nevertheless, a failure on P-P link has to be considered, because it would affect the cumulative backbone throughput and quality of MPLS/VPN services. The ST core network is also protected against node failures in Bratislava, Banska Bystrica and Kosice. If one node fails, the traffic is rerouted across the second triangle. The higher RTT and jitter because of increased number of hops is not questionable, because there’re currently only four P routers in the core ring, and they’re linked with high-speed 2.5 Gbps connections. However, if the core links are well utilised (eg. 70-80%) during normal network operation (although that is not expected in ST MPLS network very soon), the failure on one P-P link might cause temporary congestions and packet loss on the alternate path. This can be dangerous if for example the SLAs of Business data class guarantee certain max. % of packet loss at any time. In this case the workaround
7
New four-port STM-16 LC will be installed in ba1-p-2 during Phase 8A Project. Three-port GE LCs will be replaced by new four-port Eng3 LCs. This will allow proper QoS implementation on P-PE connection (on GSR side). 31
solution may be to overprovision the class bandwidth for Business data class, on account of under provisioned class bandwidth in Standard class.
Aggregation Layer 10008 MPLS PE - Concentration of Leased Line Customers (ba1-pe-5, ba2-pe-4) Two ESR 10008 routers are installed in Bratislava CO for termination of leased-line customer connections (MPLS/VPN and Internet customers) and linking the non-upgraded regional PoPs of ST with MPLS core. For this, the 10008s are equipped with 24 E1 ports and 6 POS STM-1 ports. The 10008 PEs are connected to P routers with two GE uplinks in a back-to-back mode, as shown on the following table. LX long haul GBICs connected via single mode dark fiber (10/9 micron) are used for longer distances (< 6,2miles/10 km) between firesections. Table 3
PE-P Connectivity in CO BA
PE Device
PE Name
Primary P
Backup P
ESR 10008
ba1-pe-5
ba1-p-1
ba2-p-2
ESR 10008
ba2-pe-4
ba2-p-2
ba1-p-1
7609
ba1-pe-6
ba1-p-1
ba2-p-2
7609
ba1-pe-7
ba1-p-1
ba2-p-2
32
HW Configuration of ba2-pe-48
Figure 7
Customers + Regional PoPs (CEs) Backup P
-
1 x GE -
24 x E1 4
8 x E3
24 x E1 3
PRE B
6 x POS STM-1 2
PRE A
6 x POS STM-1 1
5
6
7
1 x GE
Redundant PEM
10008
Primary PEM
Primary P + Backup P
8
HW Configuration of ba1-pe-59
Figure 8
Customers + Regional PoPs (CEs) Backup P
-
1 x GE
1 x GE
24 x E1 4
8 x E3
24 x E1 3
PRE B
6 x POS STM-1 2
PRE A
6 x POS STM-1 1
5
6
7
8
Redundant PEM
10008
Primary PEM
Primary P
8
New 10008 chassis with 6xPOS STM-1LC, 24xE1LC and two half-slot GE LCs will be installed in BA firesection 2. One 6xPOS STM-1 LC will be taken from current ba-pe-4 router. One 8xE3 LC will be taken from ke1-pe-1 router. 9
One 8xE3 LC will be taken from bb1-pe-1 router. 33
7609 MPLS PE - Collocation of Server Farms (ba1-pe-6, ba1-pe-7) Two 7609 routers in Bratislava CO provide Internet connectivity for collocated server farms of ST customers. Moreover ST IP (CE, NAS) routers is directly connected via FE. Currently each 7609 is equipped with the following modules: •
one 48 port 10/100 catalyst module o
with daughter Distributed Forwarding Card - DFC
•
one OSM 4-GE module
•
two supervisor SUP2 modules (primary and redundant) each with
•
o
PFC2 and MFC2
o
2 x GE ports on MFC2 - these are active on redundant SUP2 too.
one Switch Fabric Module – SFM2
Each 7609 has a primary and backup GE uplink towards the P routers in BA PoP.
34
HW Configuration of ba1-pe-6 and ba1-pe-710
Figure 9
NAS (5850) Backup/Primary P CE/NAS
PS
7609
3
2 x GE
4
2
1
(PFC2 + MSFC2)
OSM 4 x GE
5
SUP2
SFM2 (Switch Fabric Module)
6
2 x GE 16 x GE
7
SUP2 Redun. (PFC2 + MSFC2)
-
8
48 x 10/100 RJ-21
9
(DFC)
PIX Firewalls (BA Datacenter) + NAS (5850)
PS Redundant
* Linecards only in slots 3, 4, 6-9
10008 / 7206VXR MPLS PE - Regional (remote) PoPs The following regional PoPs are attached to Bratislava core PoP: Trnava, Nitra, Trencin, Senica, Topolcany, Trencin, Levice, Nove Zamky and Dunajska Streda. Depending on the PoP location, there will be either one 10008 and one 7206VXR PE router installed, or two 7206VXR PE routers. Each regional PoP will be dualhomed to both P routers in Bratislava via POS STM-1 links. Please find more details of a typical regional PoP architecture in chapter Regional PoPs
7206VXR - MP-BGP Route Reflectors Two MP-BGP RRs are hosted in Bratislava CO. One 7204VXR IPv4 RR (ba1-irr-1) is connected with the collocated ba1-p-1 router with a single POS STM-1 interface. The other VPNv4 RR (ba2-vrr-2) is connected with the collocated ba2-p-2 router. There’s no need for redundant connection with a second P router, because the resiliency is achieved on BGP layer (two RRs).
10
New 16 port GE module will be installed in ba1-pe-6 and ba1-pe-7 durin Project Phase 8A 35
The 7204VXR RR will not forward any customer traffic.
75xx - BA-Border1, BA-Border2 The former BA-Border1 router in Bratislava PoP has been been upgraded to support MPLS/VPN technology. It is now used as PE device in new ST MPLS network (ba1-pe-2). The former BA-Border2 router has not sufficient hardware capabilities to support MPLS/VPN services. It will be used as CE router instead. As per MPLS/VPN terminology, any remote PoP connected to ba1-pe-2, represents a MPLS/VPN CE router in the MPLS network.
Resiliency in Aggregation Layer Since the core layer in centeral offices consists of two P routers, the aggregation layer routers will be resiliently connected with the rest of the ST network. When the port on P or PE router fails, or if the whole P router goes down, the traffic would be rerouted via backup PE-P uplink as per OSPF calculation. If for any reason it is not possible to interconnect a PE router with both P devices, service resiliency can still be achieved on access layer: with two CE routers interconnecting with two PE routers via two separate connections.
Access Layer Customer Edge – CE Routers There are two type of CE routers connected to ST PE routers: customer managed and ST managed CE devices. Cisco recommends to limit the number of various platforms and interfaces, in order to minimize operational and management costs. For example: •
Low-end CE router: 800, 1700 and 2600 series
•
Mid-range CE router: 3600 series
•
High-capacity CE router: 7200 (or more powerful device when required)
Customer managed CE router can be any device, which supports the leased line technology and protocols offered by ST.
Existing ST IP Access Connections In theory, ST could re-home all customer leased lines from CE routers to MPLS PEs, but this task is already a huge project, requires lots of coordination with customers. A more elegant approach has been chosen by ST. The former distribution layer was cascaded behind the MPLS PEs, in order to leave hundreds of customers’ leased lines intact. In MPLS/VPN terminology, the old distribution layer routers have become CE devices; that’s why ST has renamed them tinto ‘CE’ routers.
36
The following routers in Bratislava PoP are connected with a single FE back-to-back connection to the 7609 PEs: ba2-ce-1, ba2-ce-2, ba2-ce-3, ba2-nas-1, ba2-nas-2. New 5850 access servers will rolled out in a separate project. Two logical separated access servers (ba2nas-3 and ba2-nas-4) on the same physical device will be installed in Bratislava firesection 2. They will be dualhomed to the 7609 PEs via two GE uplinks. Alternative to migration of existing leased-line customers ST could progressively turn the old IP routers into MPLS PEs and attach them directly to P routers. For this, the old 7206 and 3640 routers shall be SW upgraded with a more recent (and stable) IOS release, which offers required MPLS/QoS functionality on given hardware. Assuming that CPU utilisation on existing CE routers is not problematic, this might be a more appropriate alternative than migration of hundreds of leased lines to the new platforms.
Resiliency in Access Layer Having two PE routers in each regional PoP, and two resilient connections between regional and core PoP provides the possibility to dualhome access devices to both PE routers. Redundant L2 access switches in core PoPs Banska Bystrica and Kosice (4503) and regional PoPs (3550-24) are used to aggregate FE connections from access devices.
37
Central Office Banska Bystrica [BB] New 12410 P and 10008 PE router will be installed in a separate location (firesection) in Banska Bystrica. The currently installed 10005 router will be moved to regional PoP Trencin, while the existing 10008 router remains in the current location. New 4503 aggregation switches will be deployed in one firesection to connect existing ST IP devices and new customers via FE. ST IP devices will be dualhomed to switches via back-to-back FE links, in cases where additional LAN interfaces are available (e.g. NAS). The architecture of CO BB is depicted on the following figure. Figure 10
Architecture of CO BB
38
HW configuration of bb1-p-111
Figure 11
P router Regional PEs
Alrm
8
GRP Redundant
7
9
12
6
11
5
-
4
4 x GE
3
10
-
8 x POS STM-1
2
1 x POS STM-16
-
1
-
1 x POS STM-16
0
8 x POS STM-1
GRP
1 x POS STM-16
BB Aggregation
CSC 0 CSC 1 SFC 0 SFC 1 SFC 2
12012 [BB]
Power A
Power B
HW configuration of bb2-p-212
Figure 12
P routers Regional PEs, 7507 PE router
PRP Redundant
PRP
8
9
SFC 0
CSC 1
Power
12410
Alrm 1
7
Alrm 0
6
SFC 4
5
SFC 3
4 SFC - 2
-
4 x GE
3
2
SFC 1
1
16 x POS STM-1
0 CSC 0
4 x POS STM-16
BB Aggregation
Power
11
New 1 port STM-16 and 4 port GE LC will be installed in bb1-p-1 during Phase 8A Project. 1xGE LC will be removed
12
New installed during Phase 8A Project 39
Figure 13
HW configuration of bb1-pe-113 Customers + Regional PoPs (CEs) ST IP devices (CAR, Dist) via 4503 Backup P
1 x GE
7
8
1 x GE
1 x GE
1 x GE 8 x E3 4
-
24 x E1 3
PRE B
5
PRE A
6 x POS STM-1 1
Redundant PEM
10008
Primary PEM
Primary P
Figure 14
5
6
HW Configuration of bb2-pe-214 Customers + Regional PoPs (CEs) Primary P + Backup P
6
1 x GE 1
2
-
1 x GE -
5
1 x GE
24 x E1
4
PRE B
8 x E3
3
PRE A
24 x E1
1 x GE 6 x POS STM-1
Redundant PEM
10008
Primary PEM
ST IP devices (CAR, Dist) via 4503
7
8
13
Two new half-slot and one single GE LC will be installed in bb1-pe-1 during Phase 8A Project. The second single slot GE LC will be moved from existing 10005 to bb1-pe-1. One 8x E3 LC moved to ba1-pe-5 router. 14
New 10008 will replace existing 10005 (moved to Trencin) during Phase 8A Project 40
Figure 15
HW Configuration of bb1-cat-1 and bb1-cat-215
4503 1
SUP IV 2 x GE
2
6 x GE
3
48 x FE
15
GE Uplinks to PEs and other Cats ST IP devices (CAR, Dist)
New installed during Phase 8A Project 41
Central Office Kosice [KE] New 12410 P and 10008 PE router will be installed in a separate location (firesection) in Kosice. ST IP devices will be migrated to new installed 4003 aggregation switch in current location. Another switch will be installed in the new location (firesection 2). Both switches will be dualhomed to 10008 PE routers via GE uplinks. The architecture of CO KE is depicted on the following figure: Figure 16
Architecture of CO KE
42
HW configuration of ke1-p-116
Figure 17
P router Regional PEs
-
Alrm 12
9
GRP Redundant
8
11
7
10
6
-
5
-
4
1 x POS STM-16
3
4 x GE
2
8 x POS STM-1
1
-
1 x POS STM-16
0
4 x POS STM-1
GRP
1 x POS STM-16
KE Aggregation
CSC 0 CSC 1 SFC 0 SFC 1 SFC 2
12012 [ke_p_1]
Power A
Power B
HW configuration of ke2-p-217
Figure 18
P routers Regional PEs, 7507 PE router
PRP Redundant
PRP
8
9
SFC 0
CSC 1
Power
12410
Alrm 1
7
Alrm 0
6
SFC 4
5
SFC 3
4 x GE 4
SFC - 2
3
2
SFC 1
-
1
16 x POS STM-1
-
0 CSC 0
4 x POS STM-16
KE Aggregation
Power
16
New 1 port STM-16 and 4 port GE LC will be installed in ke1-p-1 during Phase 8A Project
17
New installed during Phase 8A Project 43
Figure 19
HW Configuration of ke1-pe-118 Customers + Regional PoPs (CEs) Primary P + Backup P
6
1 x GE 1
Figure 20
2
-
1 x GE -
5
1 x GE
-
4
PRE B
8 x E3
3
PRE A
24 x E1
1 x GE 6 x POS STM-1
Redundant PEM
10008
Primary PEM
ST IP devices (CAR, Dist) via 3550
7
8
HW Configuration of ke2-pe-219 Customers + Regional PoPs (CEs) Primary P + Backup P
6
1 x GE 1
2
-
1 x GE -
5
1 x GE
24 x E1
4
PRE B
8 x E3
3
PRE A
24 x E1
1 x GE 6 x POS STM-1
Redundant PEM
10008
Primary PEM
ST IP devices (CAR, Dist) via 4503
7
8
18
Four new half-slot GE LCs will be installed in ke1-pe-1 during Phase 8A Project. One 8x E3 LC moved to ba2-pe-4.
19
New installed during Phase 8A Project 44
Figure 21
HW Configuration of ke1-cat-1 and ke1-cat-220
4503
20
1
SUP IV 2 x GE
2
6 x GE
3
48 x FE
GE Uplinks to PEs and other Cats ST IP devices (CAR, Dist)
New installed during Phase 8A Project 45
Regional PoPs During transition phase of ST IP network to MPLS/VPN technology, the old 7206 (CAR/POP), 3640 (CAR) and AS5300 (NAS) routers have been cascaded behind 7206VXRs as CE devices. Each 7206VXR router have been attached with a single POS STM-1 uplink to the closest P router. In the scope of phase-2 of MPLS/VPN project (deployment of 17 new PoPs) ST decided to improve the design of remote PoPs. The design change introduces a new Catalyst 3550 that will aggregate local LANattached devices. This will allow direct attachment of old STIP routers to the new PE router. Connection between a Catalyst and PE router will be configured with encapsulation dot1Q and several logical sub-interfaces on PE router. Each sub-interface can be assigned to a different VRF, or to the global routing table, which makes deployment of co-located CEs or customers’ web servers straightforward. Number of VLANs in the Catalyst switch will correspond to number of VRFs implemented on the PE router (+1 for global RT). Part of Project Phase 8A is to achieve higher service availability in the regional PoPs by deploying second PE routers. Two groups of Regional PoP will be deployed: •
A new C10k PE router will be implemented next to the current C7206VXR in 5 PoPs: (Zilina, Nitra, Trnava, Presov, Trencin)
•
A new 7206VXR will be implemented next to the current C7206VXR in 17 PoPs: (Senica, Topolcany, Dunjaska Streda, Nove Zamky, Prividza, Martin, Provazska Bystrica, Liptovsky Mikulas, Levice,Spisska Nova Ves, Bardejov, Poprad, Humenne, Michalovce, Roznava,Lucenec, Zvolen)
The following drawing shows the new design for the remote PoPs where a new 10k PE router will be deployed; Nitra PoP has been taken as a typical example. Note: The new regional PoP design is described in Appendix II
46
Figure 22
New Architecture of Regional PoPs (10k based)
The following drawing shows the new design for the remote PoPs where a new 7206VXR PE router will be deployed; Senica PoP has been taken as a typical example. Figure 23
New Architecture of regional PoPs (7206VXR based)
47
Figure 24 HW Configuration of 10008 in Regional PoPs (ZA, NI, TT, PO) 21 Primary P + Backup P Customers + Regional PoPs (CEs)
5
6
-
1 x GE -
8 x E3 4
24 x E1
24 x E1 3
PRE B
2
PRE A
6 x POS STM-1 1
1 x GE
Redundant PEM
10008
Primary PEM
ST IP devices (CAR, Dist) via 3550
7
8
Figure 25
HW Configuration of 10005 in Regional PoP Trencin (TN) 22
Figure 26
HW Configuration of 7206VXRs in Regional PoP
New5installed during Phase 8A Project
22
Customer CEs Two3newPA-8T-X21 half-slot GE LCsPA-MC-8E1/120 will be installed4in tn-pe-2 (moved from BB PoP) during Phase 8A Project
7206VXR
21
1 0
PA-2E3
6
PA-POS-OC3SMI 2
Uplink to P router
C7200-I/O-2FE/E ST IP devices (CAR, Dist) via 3550
48
49
Hardware/Software Release Table The following table summarizes the IOS releases for different platforms in ST MPLS network. Table 4
Software Release Table
Device
Role in ST MPLS network
IOS release
12xxx-GRP
existing P, iGW
12xxx-PRP
new P
1000x
PE
12.0(25)S1
c10k-p10-mz.12.0-25.S1
7609
PE
7304
SIX
7204VXR
RR
12.0(25)S2
c7200-p-mz.120-25.S2
7206VXR
PE
12.1(11b)E12
c7200-p-mz.12.1-11b.E12
2620XM
SAA
12.2(8)T5
c2600-is-mz.122-8.T5
4503
Aggregation Switch
12.1(19)EW1
cat4000-i9s-mz.121-19.EW1
3550-24
Aggregation Switch
12.1(9)EA1c
cc3550-i9q3l2-mz.121-9.EA1c
12.0(25)S2
Image Name gsr-p-mz.120-25.S2 c12kprp-p-mz,120-25.S2
New Engine 3-based 4 port GigE linecards (4GE-SFP-LC, aka “Tetra”) will be implemented in 12012 and 12410 P routers in central offices. An IOS upgrade to 12.0(25)S is required to support this new hardware. 12.0(25)S is the first release supporting this new linecard. Apart from the new hardware, which is the main factor for the IOS upgrade, some of the IOS releases in ST's network are vulnerable to Denial of Service (DoS) attacks. Refer to the following URL for more information : http://www.cisco.com/warp/public/707/cisco-sa-20030717-blocked.shtml The core release 12.0(25)S2 is the third maintenance release of 12.0(25)S and should be deployed after as much as possible testing (during staging phase) on all 12xxx core routers and 7204VXR routers (acting as BGP RRs). Due to IOS bug CSCec57264, the 12.0(25)S2 release is not avilable for 1000x routers (already running 12.0(25)S1). 1000x routers should be upgraded to a later maintenance release of 12.0(26)S. The new edge release 12.1(11b)E12 includes several bug fixes compared to the currently used 12.1(11b)E4b. It should be deployed on 7206VXR PE routers in order to improve stability and to fix vulnerability. Once satisfied, the next step should be to deploy the code on one or two devices in a redundant and noncritical area of the network for one week pilot phase. During this period, ST should monitor the status of these devices to insure successful deployment The code should then be rolled out in a controlled and logical fashion. From a strategic standpoint, Cisco recommends to use as few different IOS releases as possible. Using just one IOS release across all platforms is sometimes not achievable, due to the differences in HW architecture and feature requirements. Nevertheless, deploying 12.0(25)S2 on two platforms will be a first step towards the limitation of different IOS releases in ST’s network. In the medium term (3 to 6 months from now), Cisco recommends to do a new IOS evaluation. The aim of this evaluation should be the definition of a long-term IOS strategy for ST’s network. The IOS strategy 50
highly depends on the network evolution concerning new features and hardware deployments. Therefore it should be covered by ongoing Cisco Network Optimization Support (NOS).
51
Logical Network Design IGP Routing – (OSPF or ISIS) OSPF is a link state routing protocol. It is called as such because it sends link states advertisement (LSA) to all the routers within the same hierarchical area. All the OSPF routing information is passed within these LSAs. After the routers receive that information they run the SPF algorithm to calculate the shortest path to each destination. When an SPF router powers up all the routing protocol data structures are initialised and then the process waits for the interfaces to be functional. Once the interfaces are functioning the devices use the OSPF hello protocol to establish neighbourships. Once the hello exchange has finished and the neighbourship is established, hello packets are used as keepalives to identify which devices are active. When the link state databases of two neighbours are synchronised, they are said to be adjacent. Distribution of routing information is only performed between adjacent routers. Each router sends periodically LSAs and also when a router's state changes. The OSPF database contents are compared with the received LSAs to identify possible topology changes. The following is the generic OSPF router configuration. ! router ospf 1 log-adjacency-changes passive-interface Loopback0 network x.x.x.x y.y.y.y area 0 !
The Role of OSPF in MPLS network The MPLS network requires an underlying Interior Gateway Protocol (IGP) to be enabled for the following reasons: •
BGP next-hop reachability.
•
Fast convergence after failure of backbone node or core link.
•
Shortest path routing across backbone.
is already running OSPF in its current network. Therefore is very familiar with OSPF operation, and has gained lots of experiences in OSPF troubleshooting. has therefore requested to preserve the OSPF as IGP in current MPLS network. The choice of OSPF is a very good one as it is standardised, scales well and converges quickly. OSPF is responsible for interior routing only ! It is not used to carry any customer addresses or linksNetwork addresses of the following links are carried in the OSPF LSAs: backbone P-P links
•
distribution layer PE-P links
•
loopback0 interfaces
•
RR-P links
•
CE, NAS connections*
NAS 5800
POP 7200
Loop
P GSR
Loop
Regional NAS 5300
PE 7609
RR 7200VXR
Loop Regional PE 7206VXR
Loop
Loop
Server farms
Existing LL customers
Loop
Leasd line customers Dialup users
Loop
OSPF Area 0
Loop
P GSR
Loop
PE 10008
LL customers
Loop CAR 3640
Dialup users
OSPF enabled links
Existing LL customers
Figure 27
•
Loop
iGW GSR
Peering circuits
In addition to backbone links, the subnets allocated to NMS VLANs and VPNSC LAN are redistributed in the backbone OSPF as connected/static routes. This is required to establish connectivity between NOC sites and P routers, which do not run BGP.
OSPF Areas Single area or Multiarea OSPF would be implemented in the network. This decision is based on: •
Give reasons here. Also discuss in this sections how you are going to number OSPF areas
•
Discuss in detail the max number of routers that would there in an area. Talk about the scale numbers of ABRs, ASBRs
53
OSPF Authentication It is possible to authenticate the OSPF packets such that routers can participate in routing domains based on predefined passwords. By default, a router uses a Null authentication, which means that routing exchanges over a network are not authenticated. Two other authentication methods exist: Simple password authentication and Message Digest authentication (md5). Authentication does not need to be set, but we strongly recommended for security purposes. And we are recommending MD5 as the authentication method since it is provided higher security than plain text authentication method. Message Digest Authentication is a cryptographic authentication. A key (password) and key-id are configured on each router. The router uses an algorithm based on the OSPF packet, the key, and the key-id to generate a “message digest” that gets appended to the packet. Unlike the simple authentication, the key is not exchanged over the wire. A non-decreasing sequence number is also included in each OSPF packet to protect against replay attacks. This method also allows for uninterrupted transitions between keys. This is helpful for administrators who wish to change the OSPF password without disrupting communication. If an interface is configured with a new key, the router will send multiple copies of the same packet, each authenticated by different keys. The router will stop sending duplicate packets once it detects that all of its neighbors have adopted the new key. Following are the commands used for message digest authentication: interface ip ospf message-digest-key keyid md5 Router ospf 19 area authentication message-digest
Loopback Addresses Each of the OSPF speakers has a Loopback address configured. These are used to force stability of the routers OSPF ID. These loopback addresses are in OSPF passive mode to optimise the routing process.
OSPF Area Summarization Discuss the details of summarization plans if any
OSPF Costs •
Table 5
Discuss here the costs of ospf for different links. Expalin any considerations kept in mind when deciding ospf costs. Following table can be used to define the costs
Proposed OSPF Metrics
Bandwidth [Mbps]
CE-PE PE-PE (none MPLS)
P-P iGW1-iGW2
PE-P primary RR-P, iGWx-P
PE-P backup RR-P iGWx-P PE-PE (MPLS)
E1
2
31300
-
-
-
E3
34
31200
-
-
-
FE
100
31100
-
5700
10700 54
STM-1
155
31000
600
5600
10600
STM-4
622
30900
500
5500
10500
GE
1000
30800
400
5400
10400
STM-16
2500
-
300
5300
10300
STM-64
10 [Gbps]
-
200
5200
10200
STM-256
40 [Gbps]
-
100
5100
10100
Designated and Backup Designated Routers On a multi-access media such as Ethernet it is a good idea to force the designated router and backup designated router to be routers that have more memory and greater processing power than the other routers in the area. Under the default election scheme, each router has the default priority of 1, therefore the router with the highest router id (i.e. Loopback IP address) becomes the designated router for the segment. It is not mandatory to enforce the DR selection on multi-access media segments with just two OSPF speakers (e.g. GigE PE-P uplinks). Therefore these kind of Fast/Gigabit Ethernet interfaces in network are defined as OSPF point-to-point links to prevent election of DR/BDR routers.
interface G 5/0 ip ospf network point-to-point
Default Routes If there are any default routes then explain how and where are
they being injected
OSPF Convergence Resiliency and redundancy to circuit failure is provided by the convergence capabilities of OSPF at layer 3. There are two components to OSPF routing convergence: detection of topology changes and recalculation of routes. Detection of topology changes is supported in two ways by OSPF. The first, and quickest, is a failure or change of status on the physical interface, such as Loss of Carrier. The second is a timeout of the OSPF hello timer. An OSPF neighbor is deemed to have failed if the time to wait for a hello packet exceeds the dead timer, which defaults to four times the value of the hello timer. On a Serial, Fast Ethernet or Gigabit Ethernet interface, the default hello timer is set to 10 seconds; therefore the dead timer is 40 seconds Recalculation of routes is done by each router after a failure has been detected. A link-state advertisement (LSA) is sent to all routers in the OSPF area to signal a change in topology. This causes all routers to recalculate all of their routes using the Djikstra (SPF) algorithm. This is a CPU intensive task, and a large network, with unreliable links, could cause a CPU overload. 55
When link goes down and if layer2 is not able to detect the failure, convergence in the core can be improved by decreasing the value of the hello timer. The timer should not be set too low as this may cause phantom failures, hence unnecessary topology recalculations. Remember that these timers are used to detect failures that are not at the physical level. For example, carrier still exists but there is some sort of failure in the intermediate network. Once a topology change has been detected, LSA is generated and flooded to rest of the devices in the network. Recalculation of the routes will not occur until the spf timer has expired. The default value of this timer is 5 seconds. An spf hold time is also used to delay consecutive SPF calculations (give the router some breathing space). The default for this value is 10 seconds. As a result, the min time for the routes to converge in case of failure is always going to be more than 5 secs unless the SPF timers are tuned using OSPF throttle timers. As a result, it is now possible to schedule spf run right after flooding the LSA information but this can potentially cause the instabilities in the network e.g. even a flash congestion in the network for a very short duration could trigger declare the link down and trigger the SPF run. These timers will be left alone in the initial implementation especially because in the next phase of this project, MPLS Traffic Engineering with Fast-ReRoute (FRR) capability will be deployed. Once FRR is implemented, tweaking OSPF timers become less of a concern. A keepalive timer is also associated with the interface that will detect failure at a level lower that OSPF. The default for this timer is 10 seconds; again this will be left as default initially. In the initial deployment of the Core network, all timers will be left at their default values as shown below. These could be slowly lowered and behavior of the network monitored if faster convergence is required. If the timers are not default then explain why they are being changed and also the values used and configurations Discuss in detail the scaling issues. For example what are the max number of prefixes that can converge in a given time. Any other related work that may have been documented in DIG from SPSE or from inhouse testing of the architecture
Table 6
OSPF Timer Default Values
Timer
Default Value
ip ospf dead-interval
4 x hello interval (40 sec)
ip ospf hello-interval
10 sec
ip ospf retransmit-interval
5 sec
ip ospf transmit-delay
1 sec
timers throttle spf
5000 msec 10000msec 10000msec
56
OSPF DNS Lookup A global configuration command “ip ospf name-lookup” would cause the router to look up the Domain Name System (DNS) names for all OSPF show commands, making it easier to identify devices. vallentin#sh ip ospf nei Neighbor ID nice lisjak
Pri 1 1
State FULL/DR FULL/ -
Dead Time 00:00:37 00:00:30
Address 10.0.4.6 10.0.4.2
Interface Ethernet0/0 Serial5/0
However, this may result in slow response of various show commands, because of slow response times on DNS queries. Operations team or Cisco’s deployment team should enable OSPF namelookup on a few routers initially, and observe the responsiveness of DNS system.
OSPF Configuration Template The following generic OSPF configuration could be used on every router in the network – just the router-id and hostname needs to be changed. hostname ! interface Loopback0 ip address no ip directed-broadcast ! interface pos0/0 description ip address ip ospf network point-to-point ip ospf message-digest-key 1 md5 ! interface GigabitEthernet1/0 description ip address ip ospf network point-to-point ip ospf message-digest-key 1 md5 router ospf 100 router-id log-adjacency-changes passive-interface Loopback0 auto-cost reference-bandwidth 10000 area 0 authentication message-digest network statements for loopback and core link addresses
OSPF Deployment Recommendations Summary for the network List down any additional recommendations for ospf and try to summarize the main points
57
Backbone Routing and Label Distribution Protocols The three protocols are required in the core to provide a functional MPLS network include OSPF, LDP, and MP-BGP. OSPF provides IP Connectivity amongst the various end points and has already been discussed in the previous section. LDP is needed to distribute the necessary label information required to establish the label switched paths between the PE routers. MPLS in general, depends on IGP or in this case OSPF along with Cisco Express Forwarding to create the necessary forwarding table. CEF, LDP, and MP-BGP will be explored in greater detail in this section. Lastly, MP-BGP is needed to exchange the VPN routing information between VPN customer sites. On the PE routers, VPN Customer routes are kept in separate routing tables, known as VPN Routing and Forwarding tables (VRFs). Routes in the global routing table are not reachable by routes in VRFs or vice versa.
Cisco Express Forwarding (CEF) Switching Cisco Express Forwarding (CEF) is advanced Layer 3 IP switching technology. CEF optimises network performance and scalability for networks with large and dynamic traffic patterns by essentially distilling the routing information into a forwarding database known as the FIB, Forwarding Information Database. Cisco Express Forwarding or CEF switching is a pre-requisite for MPLS to function properly. Therefore CEF must be configured on all the PE and P devices in the networks. By default CEF and/or Distributed CEF is enabled on GSR and ESR platforms. It is possible that due to memory or some other software related issues, CEF may get disabled. In that scenario, cef can manually be enabled by entering the following command. ip cef distributed * * show ip cef and show cef line can be used to verify if CEF is operational.
Mention any scaling limitations w.r.t CEF or dCEF
Label Distribution Protocol (LDP) LDP is responsible for distributing the labels for IP destination prefix in the MPLS network. Labels are assigned to every IGP learnt prefix that is in the global routing table. This global routing table is created and maintained by the IGP, which, in network, is selected to be OSPF. Essentially all IP destination prefixes will be either a loopback or circuit interface address. No customer IP addresses will be maintained in the global routing table. The core P routers only have an understanding of labels that are associated with IP destinations in the OSPF internal routing table. They have no knowledge of labels related to routes in customer VPNs as these are created and distributed by the MP-BGP process between PEs only. Therefore, in the P network, a labelled IP packet is switched to the next-hop based only on the outer label (the one allocated by LDP from the global routing table) until it finally reaches its destination. In an MPLS-VPN network, this final destination will always be the egress- PE that originated the VPN route.
58
Interface MTU size As mentioned earlier, two levels of labels are needed to deliver MPLS-VPN services. The first level label is distributed by the LDP protocol, whilst the second level label is created by MP-BGP for VPN distribution as discussed in the next section. When these two labels are placed into the frame they increase the frame size by 8 bytes (4 bytes per label). This can be problematic particularly on Ethernet interfaces which have a default Maximum Transmission Unit of 1500 bytes; bigger frame sized packets will be dropped if packets arrive with no fragment bit set. Note that with Ethernet encapsulation without dot1q encap, the actual layer 2 frame size is 1518 while with dot1q encap, the actual layer 2 frame size is 1522. With two label impostion, the actual layer 2 frame size becomes 1526 (or 1530 with dot1q encap) as shown in the Figure 28. Figure 28 Layer 2 Frame with 2 MPLS Labels
However, it is possible to increase the MPLS mtu on an interface to accommodate the switching of packets bigger than 1500 size. The default MTU on Serial and POS interfaces is 4470bytes so frame increase of 8 bytes is not a big concern on these interfaces. The following command can be used on gigabit Ethernet interfaces in the network. mpls mtu 1516
This will allow an MPLS frame with upto 4 labels (16 bytes) over the link. If any Ethernet switches are added into the core carrying MPLS frames they must also have their MTU increased.
4 labels have been allowed to cater for future services on the network such as traffic engineering & FRR etc. In general, each additional service may require an increase in the label stack from 2 to something greater.
Not e
LDP Configuration and Design Recommendations In this section list all the design and recommendations and configurations related to customer;s networks. Following are just example of them and may or may not apply to your customer Mention any scaling limitations w.r.t LDP. Give details on any convergence results that you got from SPSE DIG or inhouse testing
•
To activating Label Switching on a router mpls ip command must be issued on each interface that connects P and PEs routers together. This should not be enabled on PECE connections.
•
By default Cisco router will enable TDP (Tag Distribution Protocol) when mpls ip is enabled on an interface. It is recommended to enable LDP globally by entering mpls label protocol LDP command globally using Cisco IOS CLI 59
•
For proper operation of MPLS, LDP chooses an ip address as a router-id. It is important to note that the ip address chosen as router-id is routable, otherwise LDP will not be able to form the neighbor relationship with the adjacent nodes. On Cisco router, LDP router ID is, by default, determined as follows: o
The IP addresses of all operational interfaces are examined.
o
If these IP addresses include loopback interface addresses, the largest such loopback address is selected as the LDP router ID.
o
Otherwise, the largest IP address pertaining to an operational interface is selected as the LDP router ID.
•
However, the normal (default) method for determining the LDP router ID may result in a router ID that is not usable in certain situations. For example, an IP address selected as the LDP router ID might not be advertisable by the routing protocol to a neighboring router. Therefore, for network, it is recommended to manually set the router-id by entering the mpls ldp router-id command. The specified interface must be operational for its IP address to be used as the LDP router ID. In addition, force keyword should be entered to make sure the router-id takes effect immediately upon entering this command. However, care should be taken using this command since all the existing LDP sessions will be torn down if the router-id of the existing sessions is different from the newly selected ID.
•
It is recommended to enable logging of LDP neighbor state change using mpls ldp logging neighbor-changes.
•
As with OSPF, MD5 based authentication could be enabled on each link where LDP will be used to prevent any DoS attacks, and to help with configuration errors.
Not e
The IOS CLI accepts “tag” and “mpls” command interchangeably for most cases. For example, “show tag tdp neighbor” and “show mpls ldp neighbor” produce identical output. In some cases, there is only an “mpls” command such as “mpls label protocol ldp”. It is recommended to use newer “mpls” form of the commands in the network.
Not e
The “mpls ip” command merely enables LDP on an interface, and it does not control whether a packet is switched using the MPLS Label at all – this is determined by the Ethertype on an incoming frame & via various forwarding tables.
Following example shows the sample configuration for enabling MPLS in the network. Below is the sample configs . Please use your cusomter specific configs hostname mpls label protocol ldp ! mpls ldp logging neighbor-changes mpls ldp router-id Loopback0 force ! interface Loopback0 ip address no ip directed-broadcast ! ! inteface pos 0/0
60
description ip address ip ospf message-digest-key 1 md5 mpls ip mpls ldp neighbor password ! inteface gig 2/0 description ip address ip ospf message-digest-key 1 md5 mpls ip mpls ldp neighbor password mpls mtu 1516
61
Network Services MPLS/VPN Services This section describes how the VPN services are offered by using the MPLS-VPN concept.
MPLS-VPN In MPLS VPN terminology the term PE (Provider Edge) refers to the provider edge router, where the CE (Customer Edge) connects to and the VPN are created. Each VPN is associated with one or more VPN routing / forwarding instances (VRFs). A VRF consists of an IP routing table, a derived Cisco Express Forwarding (CEF) table, a set of interfaces that use the forwarding table, and a set of rules and routing protocol parameters that control the information that is included into the routing table. A one-to-one relationship does not necessarily exist between customer sites and VPNs. A given site can be a member of multiple VPNs. A customer site's VRF contains all the routes available to the site from the VPNs of which it is a member. Packet forwarding information is stored in the IP routing table and the CEF table for each VRF. A separate set of routing and CEF tables is maintained for each VRF. These tables prevent information from being forwarded outside a VPN, and also prevent packets that are outside a VPN from being forwarded to a router within the VPN. All MPLS VPN configurations are done at the PE router. The rest of the network merely switches labels and is not aware of the VPN structure or logical separation of customers. The core network is referred to as the P network in an MPLS VPN. In order to enable MPLS VPN there are several implementation steps: •
MP-iBGP Implementation
•
VPN Routing & Forwarding Table Definitions
•
PE to CE Routing Definition
The following sections discuss each of these areas in more detail and provide recommendations and design guidelines and configuration examples for network.
MP-iBGP4 (Multi-protocol iBGP) Implementation BGP is one of the vital components in enabling MPLS VPN Service. It is used to propagate VPN routing information. Various BGP attributes and extensions are used to distribute the VPN routes.
Distribution of VPN Routing Information A service provider edge (PE) router can learn an IP prefix from a customer edge (CE) router via either static or dynamic routing protocols. In the most basic configuration, static routes can be configured on both the CE and PE router configuration. Alternatively, dynamic routing protocols, including BGP, RIP, EIGRP or OSPF can be used to share IP prefix information between provider and customer networks. The IP prefix is a member of the IPv4 address family. After it learns the IP prefix, the PE converts it into a VPN-IPv4 prefix by combining it with an 8-byte route distinguisher (RD). The generated prefix is a member of the VPN-IPv4 address family. It serves to uniquely identify the customer address, even if the customer site is using globally non-unique (unregistered private) IP addresses. The route distinguisher used to generate the VPN-IPv4 prefix is specified by a configuration command associated with the VRF on the PE router. BGP distributes reachability information for VPN-IPv4 prefixes for each VPN. Since these are not IPv4 addresses, BGP provides Multi-Protocol extensions (see RFC 2283, Multiprotocol Extensions for BGP-4) which defines support for address families other than IPv4 and allows the distribution of these VPN-IPv4 routes. It propagates VPNv4 reachability information, among the PE (or RR) routers only. The reachability information for a given VPN is propagated only to other members of that VPN. The BGP multi-protocol extensions identify the valid recipients for VPN routing information. All the members of the VPN learn routes from other members enabling them to communicate with each other. The entire operation of distributing the VPN routes is illustrated in the Figure 29 MP-BGP VPN Route Distribution example. Figure 29 MP-BGP VPN Route Distribution example
63
BGP communication takes place at two levels: within IP domains, known as autonomous systems (interior BGP or IBGP) and between autonomous systems (external BGP or EBGP). PEPE or PE-RR (route reflector) sessions are IBGP sessions, and PE-CE sessions are EBGP sessions. In addition, a PE router binds a label to each customer prefix learned from a CE router and includes the label in the network reachability information for the prefix that it advertises to other PE routers. When a PE router forwards a packet received from a CE router across the provider network it labels the packet with the label learned from the destination PE router. When the destination PE router receives the labelled packet, it does a MPLS lookup for the corresponding vrf and it pops the label and uses it to direct the packet to the correct CE router
Use of VPNv4 Route Reflectors In this section you should also mention the number of recommended IBGP and MPiBGP sessions that an RR can handle. Also scale number for number of routes. The scale numbers should be in conjunction with scale numbers provided for IGP/LDP The ability for route reflectors to adequately cater for all PE’s in the network is a function of the number of VPNV4 routes the RR has to hold, the number of PE peerings and the frequency of churn (VPNv4 routes being advertised and withdrawn). When RRs are used to peer the PEs in a MPLS/BGP network, the RRs will hold all the VPNV4 routes advertised by all the PE’s. In other words every route belonging to each customer network must be held in the RR for distribution to all other PEs or RRs. Scalability problems could arise if the number of VPN routes were very large as RR’s could potentially exhaust memory resources. Figure 30 VPN route distribution using partitioned RRs shows one of the possible solution to address this problem and make MPLS VPN deployment scalable is to use route reflectors, but partition them in such a way that each partition would carry routes for a subset of the VPN’s provided by the network. Thus, no single Route Reflector is required to maintain all routes for all the VPNs. The figure below is for a particular customer. In your LLD you should use naming convention used by your customer Figure 30 VPN route distribution using partitioned RRs
The mechanism for partitioning RR’s is via the route-target using a BGP command called bgp rrgroup. With this command, each RR will only hold routes that match the specified route-targets. If RR’s are to be partitioned several design issues must be considered in the network; Location of Route Reflectors – ideally it would be ideal to deploy reflectors in various physical locations so that a single failure would not impact operations. 64
Partitioning of Route Reflectors – How do you decide on which route reflectors carry which partitions (route targets)? Ultimately in a very large network, there will be many route reflectors each carrying a subset of the VPN partitions as shown below. Below is the sample configs . Please use your cusomter specific configs
Route Reflector Configuration ----------------------------ip extcommunity-list 1 permit rt 23756:1001 router bgp 23756 address family vpnv4 bgp rr-group 1 neighbor activate neighbor route-reflector-client neighbor send-community extended PE Router Configuration ----------------------ip vrf custA rd 23756:100 route-target both 23756:100 route-target export 23756:1001
Route Reflector redundancy – There would need to be at least two route reflectors holding the same information, in the event there is a failure of one, the other can still provide VPN route information as shown in Figure 31 Route Reflector Redundancy in the Networks. The figure below is for a particular customer. In your LLD you should use naming convention used by your customer
Figure 31 Route Reflector Redundancy in the Networks
65
There are a total of RR in the network. Each RR is a . We recommend deploying RR partitioned into two groups (This may change with some customers) with two RR in each group in the network. Each group of RR can be assigned to serve few regions or partitioned can be made based on the route-targets that each RR will serve in the network. This way each group of RRs will serve only a certain number of VPN customers and carries only a subset of routes instead of carrying the routes for all the customers. The PE routers could then connect to the two RRs in the corresponding group for the VPN information they require which would cut down the overhead of each RR holding all routes distributing all VPN routes to all peers. Doing this would provide with a scalable solution as the network grows. Alternately, a full mesh can be created between route-reflector if partition is not desired at this time. In addition, it is recommended to configure both route-reflectors within each group with different cluster ids which otherwise may create issues if the IBGP sessions between PE and RR fail. The figure below is for a particular customer. In your LLD you should use naming convention used by your customer
66
Figure 32 Redundant Route Reflectors with same cluster-id.
The paragraph below is for a particular customer. In your LLD you should use naming convention used by your customer In the above example, if iBGP session between JRC edge router and RR2 and KMR edge router and RR1 fails, the VPNv4 routes received by RR1 will be forwarded to RR2 but updates will be rejected due to same cluster ID. It is very unlikely that such a double failure will occur in the network but as a best practice, it is commended to place both RR in different clusters. By default, RR Cluster ID is chosen as the BGP Router-ID. However, it is advisable to set Cluster-ID manually on RR using the cluster-id configuration command. Below is the sample configs . Please use your cusomter specific configs On Route-Reflectors router bgp 23756 bgp cluster-id
Autonomous System Number An autonomous system (AS) number is required for MP-BGP peerings. By convention this value is also used in Route Distinguishers to create IP-VPNv4 addresses and route-targets, although it is not necessary they be the same as the AS number. will be using
MP-iBGP Authentication Cisco implementation of BGP allows for MD5 authentication between BGP peers. This authentication provides some protection against accidental or malicious BGP peering in the network. It is possible to configure a unique password for every peer. However this may be administratively difficult to manage, particularly for eBGP links. Hence, a single password for all internal peering only is recommended. neighbor password
67
Use of BGP Peer-groups BGP peer-groups provide a way to group individual BGP peers with common policies to enable efficient update calculation and simplifies configuration. This method of grouping neighbors together for BGP update message generation reduces the amount of system processing resources needed to process the routing table. This method, however, has the following limitations: •
All neighbors that shared the same peer-group configuration also had to share the same outbound routing policies.
•
All neighbors had to belong to the same peer-group and address-family. Neighbors configured in different peer-groups cannot belong to different addressfamilies.
These limitations existed to balance optimal update generation and replication against peer-group configuration. These limitations also caused the network operator to configure smaller peer-groups, which reduced the efficiency of update message generation. The introduction of the BGP Dynamic Update Peer-Groups feature separates BGP update generation from peer-group configuration. The BGP Dynamic Update Peer-Groups feature introduces an algorithm that dynamically calculates BGP update-group membership based on outbound routing policies. This feature does not require any configuration by the network operator. Optimal BGP update message generation occurs automatically and independently. BGP neighbor configuration is no longer restricted by outbound routing policies, and update-groups can belong to different address families. As dynamic peer-groups take care of the update generation, simplification of the configuration can be achieved using either standard peer-group configuration or peer-templates. We therefore recommend the dynamic peergroups (for update generation efficiency) and standard peer-group configuration for the network for the MPLS VPN deployment. You need to make sure you clearly articulate what is bein recommended for this specific customer
Use of Path MTU discovery Every TCP session has a limit in terms of how much data it can transport in a single packet. This limit is defined as the Maximum Segment Size (MSS) and is 536 bytes by default on the PE-routers. This means TCP will take all of the data in a transmit queue and break it up into 536 byte chunks before passing packets down to the IP layer. Using a MSS of 536 bytes ensures that the packet will not be fragmented before it gets to its destination because most links have a MTU of at least 1500 bytes. The problem is that using such a small MSS value creates a large amount of TCP/IP overhead, especially when TCP has a lot of data to transport like it does with BGP in the MPLS VPN environment. The solution is to dynamically determine how large the MSS value can be without creating packets that will need to be fragmented. This is accomplished by enabling "ip tcp path-mtu-discovery" (a.k.a. PMTU). PMTU allows TCP to determine the smallest MTU size among all links between the ends of a TCP session. TCP will then use this MTU value minus room for the IP and TCP headers, as the MSS for the session. If a TCP session only traverses Ethernet segments then the MSS will be 1460 bytes. If it only traverses POS segments then the MSS will be 4430 bytes. The increase in MSS from 536 to 1460 or 4430 bytes reduces TCP/IP overhead, which helps BGP converge faster.
Increasing Interface Input queue and SPD thresholds Large numbers of interface input queue drops are a very common problem for PE routers with large number of BGP peers. When MP-BGP is advertising thousands of VPN routes to many peers TCP will transmit thousands of packets in a short period of time. Other PE routers will receive these packets and send TCP acknowledgements to the advertising PE router. If the acks come in at a receiving rate faster than the route-processor can handle, it will cause the packets to be 68
buffered in the interface input queue. This queue by default is only 75 packets deep. In addition, there is a SPD (Selective Packet Discard) algorithm, which plays an important role in selectively removing the non-routing packets from processor's input queue in the case of congestion so that only routing packets can be processed. However, SPD queue is by default is only 100 deep. Therefore, TCP acks can potentially fill the 175 spots of input buffering leading to large number of dropped packets. Increasing the interface input queue depth (hold-queue in) will help reduce the number of dropped TCP acknowledgements which reduces the amount of work BGP has to do to converge. Similarly the SPD parameters can be tuned using the following commands Configure router for more SPD headroom and SPD extended headroom ip ip ip ip ip
spd spd spd spd spd
mode aggressive headroom extended-headroom queue min-threshold queue max-threshold
For a large MPLS VPN network, it is recommended to extend both the input queue and spd thresholds. As a intitial value, input hold queue can be increased to 1500-2000. However, the optimum values can be obtained by constantly monitoring and tuning the input queue drops until drops have stopped. . To check if there is any drop following command can be used: Router#show int | include input queue
MP-BGP Configuration Once neighbors are listed under the BGP process, Cisco IOS by default considers the neighbors to exchang ipv4 NLRIs. However, for MPLS VPN, VPNv4 prefixes need to be exchanged and for that neighbors explicitly need to be activated under address-family. However, it is recommended to disable the default BGP behavior using no bgp default ipv4-unicast. The following shows a generic MP-BGP configuration that could be applied to each PE router in the network. hostname ! interface Loopback0 ip address no ip directed-broadcast ! router bgp 23756 no synchronization no bgp default ipv4-unicast no auto-summary bgp router-id bgp log-neighbor-changes ! xxxPE peer group (xxx – Site / Region name) neighbor peer-group neighbor remote-as 23756 neighbor update-source Loopback0 neighbor password ! xxxPE peer groups definition (complete with PE’s loopback0 address) neighbor peer group neighbor peer group ! - - - List all PEs in the cluster / region “peer-group xxxPE” - - - no auto-summary address-family vpnv4 neighbor xxxPE activate neighbor xxxPE send-community extended exit address-family
69
Creating VRF Definitions In this section give scaling numbers for number of vrfs that can be configured on a given PE The VRF is the basis for providing VPN services in MPLS. There will be a VRF table defined on every interface that connects to one of the edge services in the network. Each VRF definition includes the following components; •
A case sensitive name which identifies the VRF
•
A Route Distinguisher to uniquely identify IPv4 routes coming in on this customer interface
•
Import and export route targets defining export and import attributes for routes
•
Optional route-maps to further define the granularity of route export/imports
VRF Name The VRF name is simply a unique name used to identify the VRF and the routes it contains. It is suggested the name be short and lower case if possible for operational ease. In addition, a description can be added to the vrf using the description command in the vrf sub-mode. ip vrf vpna description customerA vpn
Route-Distinguisher Each VRF must have a unique RD (Route Distinguisher) which will have the following format: : where, is the 16-bit autonomous system number allocated to the network. This will be set to the value and will also be used for all MP-BGP configurations A unique 32 bit number within the AS that is allocated by In the situation where multiple sites of the same customer are connecting to the same PE router, each interface will use the same VRF definition, as they would normally be part of the same routing policy. This would give the customer sites peer access to each other via the PE.
Route-distinguisher Allocation schemes and Recommendations There are three different approaches to allocate route-distinguishers for a given VPN in the MPLS VPN network. Approach #1 - Unique RD for each VPN A unique RD value can be assigned for each VPN. For example: If there are three sites belonging to customer A connected to three different PEs, a same Rd value e.g. :100 can be assigned at each location as shown in Figure 33. Though this looks like a simple and straight forward approach, unfortunately, this option prevents from offering load sharing to the VPN client in the presence of route-reflectors which is the case in the 70
network. If load sharing is not a requirement, then this scheme may be useful (as it reduces the memory requirements at the PE routers). The figure below is for a particular customer. In your LLD you should use naming convention used by your customer
Figure 33 Unique RD per each VPN
Approach#2 - Unique RD per PE for each VPN An alternative to the first approach is to assign a unique RD per PE for each VPN. In other words, for a given VPN, a unique RD value will be assigned on each PE. This is illustrated in theFigure 34. Note that, with this approach, routes received from multiple interfaces belonging to the same VPN on a particular PE will share the same RD value. However, each PE will assign a unique RD. The main advantage of this approach is that it allows iBGP load balancing. However, the drawback of this scheme is that extra memory is required to hold the additional paths at the PE-routers. This is the recommended scheme in the case where Route Reflectors are deployed. The figure below is for a particular customer. In your LLD you should use naming convention used by your customer
71
Figure 34 Unique RD per site for each VPN
Approach# 3 - Unique RD per PE per interface for each VPN Approaches 1 and 2 could be used in the simple or overlapping VPN requiring any-to-any connectivity. However, implementing topologies such as hub and spoke etc. is not easy using approach 1 or 2. For Central or hub and spoke topologies, a PE may have more than one interface belonging to the same VPN but the connectivity requirement on one interface is different from the other interfaces. Approach 3 offers a solution to this problem by assigning a unique RD for each VRF per interface. The main advantage of this approach is to uniquely identify the site that has originated a route and enables the implementation of complex topologies. However, this capability comes at a relatively higher cost in terms of memory consumption and the number of VRFs to be configured. Because of these issues, this method is not recommended for simple VPNs. Moreover, BGP communities and Site-of-Origin (SOO) may be used to identify where a particular route originated. This scheme is only recommended for Hub & Spoke scenarios where multiple spoke sites are connected to the same PE router. For network, we recommend to use scheme
VPN Route Target Communities The distribution of VPN routing information is controlled through the use of VPN route target communities, implemented by border gateway protocol (BGP) extended communities. Distribution of VPN routing information works as follows: When a VPN route learned from a CE router is injected into BGP, a list of VPN route target extended community attributes are associated with it. Typically the list of route target community values is set from an export list of route targets associated with the VRF from which the route was learned. An import list of route target extended communities is associated with each VRF. The import list defines route target extended community attributes a route must have for the route to be imported into the VRF. For example, if the import list for a particular VRF includes route target communities A, B, and C, then any VPN route that carries any of those route target extended communities --- A, B, or C --- is imported into the VRF. The import and export values for route-targets can match the RD value for the VRF although they don’t need to be the same. The RD uniquely identifies customer IPv4 routes and the routetargets define import export policies for routes into and out of the VRF. Using the same routetarget and RD values simplifies the configuration and management of it. 72
The default route-target for VRFs will be equal to the RD presented above. Additional route-targets may be required given the route import & export policies needed. The Table below is for a particular customer. In your LLD you should use the right nomenclature
Table 7 RT/RD Allocation VRF Name
RD
Default
Import
Default
RT
RT
custA
23756:100
23756:100
23756:100
cust2
23756:101
23756:101
23756:101
Cust3l
23756:102
23756:102
23756:102
Export
2-15
VPN Topologies Full Mesh An Intranet VPN is the simplest way of deploying a VPN using MPLS. It essentially consists of all sites of the same customer to directly peering with each other. From the customer's perspective, all of its sites appear one hop away from each other. In reality a customer's IP packet may transit more than one core node, though the customer will not see this. Each of the sites exchanges VRF routes directly with its peer. Note that only routes that originate from that VRF are exchanged. The result is that the customer's VRF table in each PE holds an identical set of routes and each customer route is reachable via the next hop PE.
Hub and Spoke One of the advantages of MPLS VPNs is the full peering that is available between customer sites. However this is not always the ideal situation for some customers who may require a hub and spoke topology where all traffic between spokes must pass through the hub. The hub will have the knowledge of every destination, whilst the spoke will send traffic to the hub site for any destination. The hub therefore is the central transit point between spoke sites. It can then control access between spoke sites. Hub and spoke topologies require a special configuration. The Hub site requires two connections (sub-interfaces) to the PE. One will be to import all spoke routes into the hub while the other will be used export hub routes back to the spokes. This simple example concentrates on using dynamic routing for the distribution of all routing information. Static routing could also be used just as effectively, by placing a default route at spoke, which would then be imported to all the spoke VRF's. Each spoke VRF would then only need a single route to get to the hub.
73
Exranets Customers with Unique Addresses The creation of an Extranet is simply a matter of importing/exporting routes between the VRF's of two or more customers. If IP address overlap between customers is not an issue, that is, the IP address space is unique between customers, then routes could be imported directly between the VPN_ VRF tables.
Customers with Overlapping Addresses If customers wishing to participate in an Extranet share the same address space, or there is the possibility at some stage at new Extranet members will cause addressing problems, then address translation to unique addresses (provided by the service provider) must occur before traffic is allowed into the Extranet.
Extranet NAT at a Common Service Point NAT can be done at a central point managed by Service Provider. Each customer will have a physically separate NAT gateway which is connected to a VRF in their respective Intranet VPN's. The VRF connected to the NAT gateway will have the route of the translated addresses from the other customer injected into it. So a route is injected into the VRF of Customer 2 Site B and a separated route is injected into the VRF of Customer 1 Site A. This way each of the customers can participate in an Intranet, via the two NAT gateways. The NAT gateways could also be firewalls, with a NAT function. Therefore additional security could be provided between the Extranet customers.
Extranet NAT at Customer Edge NAT can also be done at the customer edge. The example used here is that the CE can only connect to the PE using a single 10BaseT/FL interface. So Extranet/NAT and Intranet/non-NAT traffic must travel over the same interface to conserve hardware resources at the PE. In most situations the PE/CE connection would be over a physical interface of some sort which could support sub-interfaces (NAT and non-NAT) If the CEs were owned by the customer, then they would be responsible for creating the translations on the interfaces going to the Extranet VRF, and agreeing on the addresses to be used. Service Provider would be responsible for creating the VRF's and injecting the translated routes, if static routing was being used. A more desirable situation would be if Service Provider provided a managed router service. This would mean Service Provider would have control all the way to the CE, and could provide and end-to-end managed NAT service between the CEs. Both customers VRF tables will have the translated routes injected into them so that packets can be routed in the Extranet. A special route-map and virtual interface on each of the CE NAT routers, prevent any translation occurring for traffic destined to their own Intranets. Intranet traffic would be classified as any packet with a destination address in the customer intranet space. Since translations are being done at both sites into the Extranet, static NAT translations would be required for each host address that required Extranet communication. If dynamic translation was done, there would be no way of knowing what NAT address was allocated to each inside host.
Controlling route exports in extranets Route-maps are very useful if you want to avoid populating the Customer VRF's with unnecessary Extranet routes. This assists in conserving memory and provides a basic form of security (no route no access). Each customer VRF will have its standard route-targets to import/export routes for their 74
Intranets. These are the first two route-target commands shown in the configuration for each VRF. Next, each VRF has an export map defined. This export map will set a specific route-target value (referred to as an extended community attribute in BGP) for the Extranet route defined. By using route-targets, we can selectively import the only the routes the CE needs to participate in an Extranet. Individual host addresses could also be explicitly specified and exported using route-maps.
75
PE-CE Routing Implementation In this section you need to also highlight scale numbers for different routiing protocols between PE-CE. For example the max number of seesions for OSPF. Or max number of routes that would be accepted per session etc. If dampening is used then parameters for it. A VPN Routing and Forwarding table (VRF) is associated with each CE interface on the PE and contains the routing information associated with that site. A PE-CE routing protocol is necessary so that the PE table can be populated with the customer’s routes. The following routing protocols are available to operate between the PE and CE in a MPLS VPN environment; •
Static
•
RIPv2
•
eBGP
•
OSPF
•
EIGRP
BGP, RIP, and EIGRP protocols have been modified to understand VRF tables by the use of a feature called address families. Address families define the VRF contexts that the routing protocol will operate in. Note that the routing protocol that operates between the PE-CE is independent of any IGP that may run inside the VPN customers network. Routes learnt at the local VPN site by the customer IGP will be redistributed into the PE-CE routing protocol to populate the VRF. It is important to understand that no special MPLS configurations are needed at the Customer Edge. Only standard IOS routing commands are required. is planning to use for the PECE routing protocols
Connectivity via Static Routing It is recommended to use static routing if the customer is small or is a stub site and the IP addressing of devices is unlikely to change. The CE router would have a default route pointing in the direction of the MPLS cloud, whilst the PE would require a similar static route inserted into the appropriate VRF table associated with the customer interface. In order to tell the remote VPN sites (or PE routers) about the local VPN routes, these customer specific vrf static routes on the each PE router need to be redistributed in the customer specific BGP address-family.
76
The static route in the PE would have the following format indicating name of the VRF table and also the outgoing interface and its IP address.
Static routing configuration - PE ip route vrf CustomerA 10.0.0.0 255.0.0.0 serial1/0 [permanent] router bgp 23756 address-family ipv4 vrf customerA redistribute static
The CE route would consist of the default route pointing to the next hop ip address/interface of the PE.
Static routing configuration – CE ip route 0.0.0.0 0.0.0.0
Routing Stability With static routing, if the PE-CE link fails, the static route associated with the interface will be removed from the routing table. In the case of the PE, this will cause an MP-iBGP routing update to be forwarded to all other PE peers. To prevent such a behavior, the keyword “permanent” can be appended when configuring the static route. This will cause the static route to remain in the routing table regardless of the interface status. This obviously reduces the BGP update messages and improves VPN route convergence, however, such an improvement comes at the cost of unnecessary backbone bandwidth utilization. This is because the packet will get forwarded through the core all the way to the remote PE and only then will get dropped if the directly connected link is down.
RIPv2 configuration (PE to CE) RIPv2 could be used as a PE-CE routing protocol and is included here as an example. RIPv2 is a distance vector protocol and will periodically send the whole routing table to each neighbor to maintain synchronisation of routes. For managed CE routers, dual homing and more sophisticated routing policy eBGP should be used as the PE-CE protocol.
PE Configuration The following example shows the configuration of the PE side of the RIPv2 circuit. In the example, the CE device is connected to a PE via a serial link.
router rip version 2 ! address-family ipv4 vrf CustomerA version 2 redistribute bgp 23756 metric 5 network no auto-summary exit-address-family ! ! interface serial3/0 Description Circuit to Customer1 ip vrf forwarding CustomerA ip address a.b.c.1 router bgp 23756 address-family ipv4 vrf customerA
77
redistibute rip
…
The address-family under RIP process indicates RIPv2 that this routing instance is associated with a VRF for CustomerA. Any interface on this PE that has this VRF defined will participate in RIPv2 routing if they are part of network. The redistribute bgp command allows routes BGP has learnt from other VRF’s that have been brought into the VRF CustomerA (subject to the policies, route-targets etc…) to be redistributed to the RIPv2 routing instance for forwarding to the CE. Similarly, these local site routes need to be redistributed into MP-BGP so that these are advertised to the remote PE and ultimately to the remote VPN sites.
CE Configuration The CE uses a standard RIPv2 configuration. No special VRF configurations are necessary
RIPv2 CE Configuration router rip version 2 network redistribute interface serial0 Description Connection to PE router ip address a.b.c.2
…
eBGP configuration (PE to CE) eBGP routing would be the most appropriate protocol to use if the CE was dual homed to multiple PE’s or extensive policy routing features are needed. By using eBGP between the PE and CE routing loops can be avoided using various mechanism within BGP. No routing information is lost as BGP (either eBGP or MP-iBGP) is used along all the paths, i.e. between PE-CE and PE-PE. is planning to use EBGP to inject routes from larger customers. eBGP also has the ability to automatically prevent routing loops. The BGP configuration of the PE depends on whether the customer has many sites using eBGP as the PE-CE protocol and whether those CE sites are using unique AS numbers or are all using the same AS number.
Configuration at the PE Unique AS per customer site The example in this section shows the BGP configuration for connecting CE’s from one customer, each of which uses a unique AS. shows a number of CE networks each with a different AS number. Therefore if the network at CE A wished to talk to the network at CE B it would have to pass via the MPLS-VPN core and the AS_PATH followed would be 23756 65001. The AS number 23756 will appear in the AS_PATH as the CE packet transits the core.
78
In this scenario if one (or more) of the CE’s were dual homed, routing loops would be avoided due to the standard AS path check done on incoming routes to the CE from the PE. The figure below and the above last two paragraphs are for a particular customer. In your LLD you should use naming convention used by your customer
Figure 35 PE-CE eBGP with unique AS
The eBGP configuration for the PE-CE link is shown in the following diagram. The configs below is for a particular customer. In your LLD you should use customer specific configs
router bgp 23756 …. address-family ipv4 vrf CustomerA neighbor remote-as 65001 neighbor activate no auto-summary no synchronization exit-address-family ! …. interface serial3/0 Description Circuit to CE A ip vrf forwarding CustomerA ip
Note that the above configlet is showing only the IPV4 address family section of the BGP configuration related to the VRF. For every CE that requires a BGP peering there must be a corresponding address family with appropriate neighbor commands. The configuration at the CE is standard BGP (as if it were connecting to another CE).
Single AS for all customer sites There may be occasion where customers wish to use the same AS number at all their sites. This would be typical in an existing BGP customer network where the customer is migrating to an MPLS-VPN network and does not want to have to change their BGP configurations. 79
The figure below is for a particular customer. In your LLD you should use naming convention used by your customer
Figure 36 PE-CE eBGP with single network wide AS
As shown in the Figure 36, CE B rejects the routes coming from CE A when it sees its own AS number in the BGP AS Path. This is standard BGP loop prevention mechanism. As a result, CE B will not be able to communicate with CE A.
AS-Override To solve this problem, the PE can be instructed to override the customer’s AS number before forwarding the BGP update to the customer. This can be achieved by using BGP neighbor parameter “as-override” configuration command. This is illustrated in the following configuration example: The configs e below is for a particular customer. In your LLD you should use customerspecific configs
router bgp 23756 rddress-family ipv4 vrf customerA reighbor as-override
This configuration is needed at the PE which then replaces the customers AS number (in this case AS 65001) with the providers AS number (AS 23756) so that the receiving CE will accept the routes as it will not see its own AS in the path. With ASN override configured, the PE does the following: •
If the last ASN in the AS_PATH is equal to the neighboring one, it is replaced by the provider ASN
•
If last ASN has multiple occurrences (due to AS_PATH prepend) all the occurrences are replaced with provider-ASN value
•
After this operation, normal eBGP operation will occur and the provider AS will be added 80
to the AS_PATH
Site-of-Origin By enabling as-override feature, loop detection using the AS_PATH is disabled. This obviously will cause problems if the CE is dual-homed, as is the case for CE B in Figure 36. A BGP extended community attribute, referred to as the Siteof-Origin (SOO) addresses this issue. The SOO prevents routing loops when a site is multi-homed and the as-override feature is being also being used. This is achieved by identifying each customer site with a unique SOO. The SOO, similar to route-target is a BGP extended community and is denoted in the same format as route-target. All routes originating from a customer site are identified with a SOO by the eBGP process on ingress to the PE. If those routes for some reason end up back at the originating PE, they will not be re-advertised to the CE as the SOO will match that of the site. Note that a site may consist of many routers each containing the same routing information. If several of these routers are connected to the MPLS-VPN backbone as CE’s, they will still use the same SOO. Only when the sites are different will a different SOO be used. The configs below is for a particular customer. In your LLD you should use customer specific configs
router bgp 23756 …. address-family ipv4 vrf CustomerA neighbor remote-as 65001 neighbor activate neighbor as-override neighbor routemap setsoo in no auto-summary no synchronization exit-address-family ! …. interface serial9/0 Description Circuit to CE A ip vrf forwarding CustomerA ip route-map setsoo permit 10 set extcommunity soo 23756:1002
Above example shows the PE configuration when using as-override and SOO. The neighbor as-override command causes AS 65001 in the AS_PATH to be replaced with AS 23756. The neighbor routemap setsoo in command causes all incoming routes from the CE (CE A in this case) to have SOO 23756:1002 set in the extended community attribute.
Note
What is not obvious is that the same command neighbor routemap setsoo in also causes the PE to check routes it is distributing to the CE for the same SOO. If there is an SOO match then the routes are not re-advertised to the CE.
81
Routing Stability The eBGP route dampening feature can control flapping routes from the CE. The “maximum route limit” command described in the following section and the BGP “neighbor x.x.x.x prefix-limit” command will allow the limiting of the number of routes installed in the VRF and redistributed in MP-iBGP.
Controlling number of VRF routes It is possible that an excessive number of routes get distributed into the VRF due to some problem in the customer network. In the MPLs VPN network, multiple customers connect to the same provider edge router. Therefore it is very important to protect the resources such as memory, and CPU on the PE routers. If the PE-CE protocol is BGP, the numbers of routes received from the CE can be controlled at each site by using the maximum-prefix command as shown below.
The configs below is for a particular customer. In your LLD you should use customer specific configs
router bgp 23756 address-family ipv4 vrf customerA neighbor {maximum-prefix maximum [threshold]} [restart restart-interval] [warning-only]
Table 8 BGP Timer Definitions maximum Maximum number of prefixes allowed from the specified neighbor. The number of prefixes that can be configured is limited only by the available system resources on a router. threshold (Optional) Integer specifying at what percentage of the maximum-prefix limit the router starts to generate a warning message. The range is from 1 to 100; the default is 75. restart
(Optional) Configures the router that is running BGP to automatically reestablish a peering session that has been disabled because the maximum-prefix limit has been exceeded. The restart timer is configured with the restart-interval argument.
With the maximum prefix command, when the threshold is reached, the BGP session is terminated. Alternately, maximum-prefix command can be augmented with warning-only key word. This allows the router to generate a log message but keeps the bgp session up instead of terminating it when the threshold is reached. If warning-only key word is not configured, BGP session is torn down as a result of reaching the threshold and will stay down indefinitely. Manual intervention is required to bring the session up unless restart-interval is configured to bring the session up automatically after the restart-interval has elapsed. There is no default limit on the number of prefixes that can be configured with this command. Limitations on the number of prefixes that can be configured are determined by the amount of available system resources and are configured by the network operator. Peering sessions will be disabled (by default) when the configured maximum number of prefixes has been exceeded.
82
The BGP maximum route knob allows to control the routes if the PE-CE protocol is BGP. However, there is no such per neighbour capability available in the other dynamic protocols to control the routes received from the CE sites. However, alternately, the total number of routes in a customer VRF can be controlled by using the “maximum routes” command inside the VRF configuration as follows. The configs below is for a particular customer. In your LLD you should use customer specific configs
ip vrf CustomerA rd 23756:1000 route-target both 23756:1000 maximum routes 1000 warn-only | warn-threshold
In the above example, the number of routes allowed in the VRF is limited to 1000. The “warn” keyword does the following: warn-threshold
Rejects routes when the threshold limit is reached. The threshold limit is a percentage of the limit specified, from 1 to 100.
warn-only
Issues a SYSLOG error message when the maximum number of routes allowed for a VRF exceeds the threshold. However, additional routes are still allowed.
For network, we recommend that the both the BGP per neighbor and VRF maximum routes command should be included with every PE-CE BGP session and VRF definition.
83
Additional MPLS VPN Services Internet Access for MPLS/VPN customers There are two basic design models for combining Internet Access with MPLS / VPN services. •
Internet access is offered through global routing on the PE routers. There are 2 implementation options. o A first one is to implement packet leaking shortcut between a VRF and the global routing table. This option has a number drawbacks and must be avoided. o A second implementation option is to use separate physical or logical interfaces for VPN and for Internet access. The physical or logical interface meant for Internet access will be placed in the global routing table. Ideally, the Internet interface (also called IPv4 link) will be implemented on a separate CE router, which permits to put the FW in customer site.
•
Internet access is offered through yet another VPN. This is called the Internet VPN (and associated Internet VRF). This solution has the advantage that the provider’s backbone is isolated from the Internet, resulting in improved security. A drawback is that full Internet routing cannot be implemented because of scalability problems and is therefore not recommended solution for ST.
Separate CEs for Internet Access and VPN Access From the point of view of the VPN customer, the “separate CE” design model maps ideally on the situation where the VPN customer wants centralised and firewalled access to the Internet The customer managed firewall can provide NAT services in between the private VPN addressing and the public Internet addressing. The central customer site firewall gives the customer the ability to control security and Internet service policies. A drawback is that all the Internet traffic must flow through a central site. For example, a large bank with hundreds of branches would not want to implement Internet access directly from each of the branches, as this would imply management of strict security policies at every site The fig below is for a particular customer. In your LLD you should use customer specific figs
Figure 37
Internet Access from a VPN using separate CEs 84
(difficult and expensive). The centralised FW approach with two CE routers is more appropriate solution. Region. Site
CE1
Internet Internet
MPLS Network
PE1
PE2
PE3
CE1
CE2
FW Central Site
Default route injected into VPN Data forwarding path from regional sites to Internet VRF_RED interface (VPNv4) Global routing table interface (IPv4)
It is worth to mention that default static routes will be injected into VPN and used by regional sites, but the default route can not be used for VPN traffic on central site. On the drawing above, the CE2 will be configured with a default route pointing to PE3 via IPv4 interface. For this reason, the CE1 (and CE2) have to have all the VPN routes in the routing table. Central site shall learn the VPN routes dynamically with BGP4 or RIPv2 between CE1 and PE3. This is recommended approach as it allows greater flexibility and redundancy. For example, customer may want to implement two VPN CEs in central site to improve service availability. In case of small number of regional prefixes, or if all regional prefixes can be summarized in a single aggregate route, static route can be implemented from CE1 to PE3 for VPN traffic.
Low-cost Internet Access (1CE + one/two access links) The low-cost solution described in this chapter is not as secure as the one with two CE routers and firewall in customer site. The low-cost solution can therefore become very expensive if the security is compromised and intruder gains the access into customer’s VPN. Customers with sensitive data shall subscribe to secure Internet access from their VPN. Therefore, the single-CE design for Internet&VPN access shall not be recommended to ST customers! Two options exist to provide Internet connectivity from a single CE router: •
Single access-layer connection for Internet and VPN traffic, and packet leaking on the PE.
•
Two logical PVCs or two physical connections between the CE and PE; one for VPN traffic (VPNv4 link) and one for Internet traffic (IPv4 link).
Single link option The option with single link for VPN and Internet traffic represents serious risk for that VPN because of the “shortcut” that has to be created between the global routing table on the PE (i.e. the Internet) and the VRF. 85
No security mechanisms (e.g. packet filtering) are available on this shortcut. CE_Blue on Figure 38 below depicts this situation. Packet leaking between a VRF and the global routing table is implemented with two IOS mechanisms: •
A static route with a global next-hop can be configured in a VRF. Packets following this static route will end in the global address space at the next-hop router. Traffic originated at a customer site can thus be forwarded into the Internet.
•
Global static route can be defined pointing to a connected interface, which belongs to a VRF. This static route is further redistributed into IGP or BGP. Packets originated in the global address space will follow this route (in the global routing table) and will eventually be forwarded toward a CE router. Traffic originating in the Internet can thus be forwarded to the CE router.
Since the default route in the VPN points to the Internet, no additional default routing can be used in the customer VPN. In addition, when a customer site looses connectivity to the MPLS / VPN backbone, packets from other sites destined for the failed VPN site will be leaked to the Internet. This is another major security issue. In general, this option is also fairly complex to implement. VPNv4 and IPv4 links The two links between CE and PE can be implemented as two separate physical circuits (e.g. two E1 circuits) or as a two logical connections - for example the ATM PVCs. IPv4 link will terminate in the Global Routing table on the PE router, VPNv4 link will be assigned to the customer’s VPN. Static default route will be configured on the CE for Internet access and it will point towards PE via IPv4 link. VPN routes will be in most cases uploaded to the CE with dynamic routing protocol (eBGP, RIPv2), but can be statically configured on the CE if number of prefixes is small. The single-CE solution implemented with separated links for VPN and Internet traffic allows configuring packet filtering on IPv4 link on the CE router, but does not offer logical separation of two security zones (MPLS/VPN and Internet) with a firewall. It is mandatory to define a strict packet filtering rules in both directions: to and from the Internet. Outbound filter must for example prevent VPN packets to be leaked in the Internet (via default route) when VPNv4 connection fails. Inbound filter must clearly define the list of hosts and applications that can be reached from the global Internet. It is up to customer and service provider (ST) to define and implement desired security policy (i.e. packet filters) on a managed CE router. If the customer uses private IP addresses, NAT would have to be implemented on the IPv4 link. Please note that static “one-to-one” translation is needed only for Internet servers, whereas the clients can be dynamically translated in a pool of IP addresses in a PAT-like mode. Figure 38
Internet Access from a VPN – Single CE (two links in CEred, single link on CEblue)
In the end explain which option is being used
86
CE RED VPNv4 link
IPv4 link
vrf_red
global_rt
CE BLUE VPNv4 & IPv4 link
! ip route vrf BLUE 0.0.0.0 0.0.0.0 global !
vrf_blue
PE
MP-BGP
Shared vrf-aware services Network Address Translation for MPLS/VPN customers The following configuration template can be used on customer’s CE router in case of private IP addressing in customer site. The example below shows two types of NAT translations: •
Static on-to-one translation for servers in customer site, that must be reachable from the Internet
•
Dynamic NAT in overload mode (PAT) for PC clients.
Please note that NAT is only required on IPv4 link. The config below is for a particular customer. In your LLD you should use customer specific configs hostname CE ! interface Ethernet0 description Customer site x ip address 10.10.10.254 255.255.255.0 !--- This is the inside local IP address and it's a private IP address. ip nat inside ! interface Serial0 description CE-PE Internet link ip address 213.x.x.x 255.255.255.252 !--- This is the inside global IP address. !--- This is public IP address and it is provided by ST.
87
ip nat outside ! interface Serial1 description CE-PE VPN link ip address 213.x.x.x 255.255.255.252 !--- NAT is not performed on the VPNv4 link ! !--- This statement makes the router perform PAT to overload the Serial0 !--- IP address for all the End Stations behind the Ethernet interface !--- that are using private IP addresses defined in access list #1. ip nat inside source list 1 interface Serial0 overload ! !--- This statement performs the static address translation for the Web server. !--- With this statement, users trying to reach 171.68.1.1 port 80 (www) will be !--- automatically redirected to 10.10.10.5 port 80 (www), which in this case !--- is the Web server. ip nat inside source static tcp 10.10.10.5 80 171.68.1.1 80 ! !--- This access list defines the private network !--- that will be network address translated using PAT overload mode. access-list 1 deny host 10.10.10.5 access-list 1 permit 10.10.10.0 0.0.0.255 ! ip route 0.0.0.0 0.0.0.0 Serial0 !
The fig below is for a particular customer. In your LLD you should use customer specific figs Figure 39
NAT in CE router
VRF ip route 10.10.10.0/24 -> S1@CE1
Static NAT translation 10.10.10.5 171.68.1.1 10.10.10.5/24 VPNv4 link
S1
E0 .254
PE
IPv4 link
S0
Web serv.
CE 10.10.10.x/24
Global RT ip route 171.68.1.1/32 -> S0@CE1
Dynamic NAT in overload mode
PC
Connecting Downstream ISPs to PE routers Internet customers that require full Internet routing table (eg. a downstream ISP or multi-homed customer) to implement primary/backup or any other inter-domain routing policy will be in most occasions attached to the two iGW routers. If there’s a need to interconnect such customer in regional PoP, ST will install a PE router with sufficient memory and CPU power to hold the full Internet routes. Otherwise, the customer would have to install two eBGP sessions: one with iGW to download full Internet routes and another with the PE router to advertise its customers’ routes. This is required because next-hop-self feature is systematically applied on PE-RR and iGW-RR BGP neighborships.
88
Remote Access (ASWAN/Security, Dial, DSL, Cable) Wireless VOIP Inter-AS/CsC
89
Traffic Engineering and Fast Reroute Technology Overview Traffic Engineering Basics Traffic Engineering is a powerful MPLS-based tool, which can be used not only to reduce cost for service providers (SPs) but to generate new revenues as well. One of the key functions of Traffic Engineering is to maximize the utilization of network resources. By making the SP’s network more efficient, Traffic Engineering reduces the cost of the network. Another function of Traffic Engineering is restoration. While delivering the same level of protection as SONET APS, Traffic Engineering restoration is more flexible and less costly. With Traffic Engineering, the SP may choose to protect only the set of links that are most vital to the entire network, and only the traffic which requires low loss probability. Traffic Engineering restoration increases the reliability of the SP’s network and improves the quality of the SP’s service. Alternatively, the SP may sell Traffic Engineering restoration as a premium service. Traffic Engineering helps the SP generate new revenue because it enables the SP to offer new services. First, we introduce the concept of traffic trunks. Traffic trunks are aggregated micro-flows 23 that share a common path. In the context of this document, a "common path" does not refer to the end-to-end path of the flows, but a portion of the end-to-end path within the service provider's network. Typically, the common path originates from the ingress of the service provider's wide area network to the egress of the service provider's wide area network. For example, all traffic originating from an IP address in San Jose and destined for an address in New York City may constitute a traffic trunk, and all traffic between an address in Palo Alto and an address in Washington D.C. another. Optionally, we may require that all packets within a traffic trunk have the same class of service. For example, all ftp and telnet (priority 1) traffic between San Francisco and New York City may be considered a trunk, and all VoIP (priority 5) traffic between San Francisco and New York City another one.
23
A micro-flow refers to the packets travelling from a source to a destination using the same transport protocol and the same port number. For example, an ftp session between two IP hosts constitutes two micro-flows, one from the client to the server, and the other from the server to the client. 90
Traffic Engineering creates one or more explicit paths with bandwidth assurances for each traffic trunk. It takes into consideration the policy constraints associated with the traffic trunks, and the physical network resources, as well as the topology of the network. This way, packets are no longer routed just based on destination, but also based on resource availability, and policy. The following section describes the operation of Traffic Engineering. Figure 1 illustrates the operation of Traffic Engineering. Each step shown in the diagram is explained below.
traffic statistics
Creation or update traffic model (using off-line optimization tools)
resource attributes
trunk attributes
Input resource constraints throughout network
Input traffic model at the head-ends of traffic trunks
topology and resource information
Path selection at traffic trunk head-ends Explicit routes
Path maintenance
Path admission, reservation, and/or LSP creation for calculated paths (via extended RSVP)
Figure 40 - Traffic Engineering Mechanisms The network operator must create a traffic model. Based on statistics collected from the routers, as well as administrative policies, the network operator needs to identify the traffic trunks within the network, and decide how these traffic trunks should be routed. The operator can use an off-line tool to optimize the traffic model. This does not mean that the operator is required to use the off-line tool to determine the routes for all traffic trunks. Typically, the operator identifies a full mesh of traffic trunks but administratively routes only the "top" N traffic trunks. On-line procedures are used for the rest of the trunks, as well as to handle failure situations. Traffic trunks could also be forwarded along routes computed by conventional IGP. The router uses RSVP to set up Label Switching Paths (LSPs) and to reserve bandwidth at each hop along the LSPs. During the LSP setup process, any router within the network must perform admission control and/or preemption to ensure that resources are available to honor the reservation. After the paths are set up, the head-end routers forward the packets belonging to traffic trunks by placing them into the appropriate LSPs. The following section breaks down Traffic Engineering into components and describes each component.
91
Traffic Trunk Attributes Traffic trunk attributes allow the network operator to describe the characteristics of traffic trunks. They must be granular enough to account for the different types of packets traversing the network, and detailed enough to specify the desired behaviour in failure situations. There are six traffic trunk attributes and each is described below.
Bandwidth This attribute specifies the amount of bandwidth the traffic trunk requires.
Path Selection Policy This attribute gives the network operator the option to specify the order in which the head-end routers should select explicit paths for traffic trunks. Explicit paths may be either administratively specified or dynamically computed.
Resource Class Affinity This attribute is used to allow the network operator to apply path selection policies by administratively including or excluding network links. As will be shown later, each link on the network may be assigned a resource class as one of the resource attributes. Resource class affinity specifies whether to include or exclude links with resource classes in the path selection process. It takes the form of the tuple . The "resource class mask" attribute indicates which bits in the resource class need to be inspected, and the "resource affinity" attribute indicates whether to explicitly include or explicitly exclude the links.
Adaptability This attribute indicates whether the traffic trunk should be re-optimized. The re-optimization procedure is discussed in a later section.
Resilience This attribute specifies the desired behavior under fault conditions, i.e., the path carrying the traffic trunk no longer exists due to either network failures or preemption. Traffic Engineering's restoration operation is discussed in a later section. 92
Priority Priority is the mechanism by which the operator controls access to resources when the resources are under contention. It is a required function to place all traffic trunks. Another important application of the priority mechanism is supporting multiple classes of services. We assign two types of priorities to each traffic trunk: holding priority, and setup priority. Holding priority determines whether the traffic trunk has the right to hold a resource reservation when other traffic trunks attempt to take away its existing reservation. Setup priority determines whether the traffic trunk as the right to take over the resources already reserved by other traffic trunks.
Resource Attributes Resource attributes are used to describe the network links used for path calculations. There are three resource attributes, each of which is described below.
Available Bandwidth This attribute describes the amount of bandwidth available at each setup priority. Note that the available bandwidth for the higher setup priority is always larger than that for the lower setup priority. This attribute needs not necessarily reflect the actual available bandwidth. In some cases, the network operator may oversubscribe a link by assigning a value that is larger than the actual bandwidth, e.g., 49.5 Mbps for a DS-3 link.
Resource Class This attribute indicates the resource class of a link. Recall that the trunk attribute, resource class affinity, is used to allow the operator to administratively include or exclude links in path calculations. This capability is achieved by matching the resource class attribute of links with resource class affinity of traffic trunks. The resource class is a 32-bit value. The resource class affinity contains a 32-bit resource affinity attribute and an associated 32-bit resource class mask. .
Path Selection Path selection for a traffic trunk takes place at the head-end routers of traffic trunks. Using extended ISIS/OSPF, the edge routers have knowledge of both network topology and link resources. For each traffic trunk, the router starts from the destination of the trunk and attempts to find the shortest path toward the source (i.e., using the shortest path first (SPF) algorithm). The SPF calculation does not consider the links which are explicitly excluded by the resource class affinities of the trunk, as well as the links which have 93
insufficient bandwidth. The output of the path selection process is an explicit route consisting of a sequence of label switching routers. This path is used as the input to the path setup procedure.
Path Setup Path setup is initiated by the head-end routers. RSVP24 is the protocol which establishes the forwarding state along the path computed in the path selection process. The head-end router sends a PATH message for each traffic trunk it originates. The PATH message carries the explicit route computed for this traffic trunk. As a result the PATH message always follows this explicit route. Each intermediate router along the path performs trunk admission control after receiving the PATH message. Once the router at the end of the path receives the PATH message, it sends a RESV message in the reverse direction towards the head-end of the traffic trunk. As the RESV message flows toward the sender, each intermediate node reserves bandwidth and allocates labels for the trunk. Thus when the RESV message reaches the sender, the LSP is already established. The following diagram is an example of the path setup procedure.
R9
R8 R3 R4 R2
Pop
R5
R1
32 49 17
R6
R7 22
Setup: Path (R1->R2->R6->R7->R4->R9) Reply: Resv communicates Tags and reserves bandwidth on each link
Figure 41 - Traffic Engineering Path Setup Once you’ve decided to set up an LSP for a tunnel, you do that using RSVP with certain extensions to support this feature. In RSVP, the forward leg of the signaling message is called the path message, and the reverse leg is called the reservation message. So one of the extensions is that the path message can carry the source route in the new object. Resources are actually allocated on the reverse leg with the reservation message. In addition to bandwidth, which is an existing RSVP resource, there are extensions so that labels can be allocated and transmitted in the reverse direction on the reservation message.
24
Note that the usage of RSVP in Traffic Engineering deviates from the original design goal of RSVP. Extensions to RSVP and the justification for using RSVP are discussed in a later section. 94
In Figure 2 we’re establishing a tunnel from R1 to R9 along the path shown in the slide here. That path is included in the path message that is generated by R1, and it directs the path along the yellow arrows from the head of the tunnel to the tail. In the reverse direction, the reservation message flows back on whatever series of hops was established by the path. At each hop the tag from the hop closer to the tail is received and programmed into the MPLS forwarding table. A new tag is allocated, and that new tag or label is sent upstream towards the head until eventually we get back to the head and the head knows that to send traffic down the tunnel, it should use label 49.
One feature of interest about the resulting LSP and about the MPLS tunnels under IOS in general is that they’re unidirectional. Traffic flows from the head to the tail, but there’s no automatic reverse direction. So you couldn’t for instance run an adjacency over one of these MPLS tunnels because the traffic’s one way.
Link Protection (FRR) Basics Regular MPLS traffic engineering automatically establishes and maintains label-switched paths (LSPs) across the backbone using Resource Reservation Protocol (RSVP). The path used by a given LSP at any point in time is based upon the LSP resource requirements and available network resources such as bandwidth. Available resources are flooded via extensions to a link-state based Interior Gateway Protocol (IGP), such as IS-IS or OSPF. Paths for LSPs are calculated at the LSP headend. Under failure conditions, the headend determines a new route for the LSP. Recovery at the headend provides for the optimal use of resources. However, due to messaging delays, the headend cannot recover as fast as possible by making a repair at the point of failure. Fast Reroute provides link protection to LSPs. This enables all traffic carried by LSPs that traverse a failed link to be rerouted around the failure. The reroute decision is completely controlled locally by the router interfacing the failed link. The headend of the tunnel is also notified of the link failure through the IGP or through RSVP; the headend then attempts to establish a new LSP that bypasses the failure. Local reroute prevents any further packet loss caused by the failed link. This gives the headend of the tunnel time to re-establish the tunnel along a new, optimal route. If the headend still cannot find another path to take, it will continue using the backup tunnel.
95
Figure 42 - TE FRR Example
R8
R9
Pop 14
Swap 37->14 Push 17
R4
R2 Push 37 R5
R1 R7
R6 Swap 17->22
Label Stack:
Pop 22
R1
R2 37
R6 17 14
R7 22 14
R4 14
R9 None
The example in Figure 1 illustrates how Fast Reroute link protection is used to protect traffic carried in a TE tunnel between devices R1 and R9, as it traverses the mid-point link between devices R2 and R4. [The TE tunnel from R1 to R9 is considered to be the primary tunnel and is defined by labels 37, 14, and Pop.] To protect that R2-R4 link, you create a backup tunnel that runs from R2 to R4 by way of R6 and R7. This backup tunnel is defined by labels 17, 22, and Pop. When R2 is notified that the link between it and R4 is no longer available, it simply forwards traffic destined for R4 through the backup tunnel. That is accomplished by pushing label 17 onto packets destined to R4 after the normal swap operation (which replaces label 37 with label 14) has been performed. Pushing label 17 onto packets forwards them along the backup tunnel, thereby routing traffic around the failed link. The decision to reroute packets from the primary tunnel to the backup tunnel is made solely by R2 upon detection of link failure. The Fast Reroute feature has two noticeable benefits. •
Increased reliability and minimal traffic loss it gives to IP traffic service during link loss. 96
•
High scalability inherent in its design.
Increased Reliability for IP Services MPLS traffic engineering with Fast Reroute uses fail over times that match the capabilities of SONET link restoration. This leverages a very high degree of resiliency for IP traffic that flows over a service provider's backbone, leading to more robust IP services and higher end-customer satisfaction.
High Scalability Solution The Fast Reroute feature uses the highest degree of scalability by supporting the mapping of all primary tunnels that traverse a link onto a single backup tunnel. This capability bounds the growth of backup tunnels to the number of links in the backbone rather than the number of TE tunnels that run across the backbone.
97
TE/TE-FRR Design
Deciding on the tunnel topology and tunnel types How to Route Traffic Into TE Tunnels
Policy Based Routing You can use PBR to send traffic down a TE tunnel. However you cannot apply policy routing to an MPLSVPN interface as the Hardware and IOS software for the VRF interface is not PBR aware. This enhancement may be added in future line cards and IOS software. So for normal Ipv4 interface you just set the outgoing interface in the policy map as the tunnel interface. RtrA(config)#int s0 RtrA(config-if)#ip policy route-map set-tunnel RtrA(config)#route-map set-tunnel RtrA(config-route-map)#match ip address 101 RtrA(config-route-map)#set interface Tunnel1
Static Routing Into Tunnels You can manually send traffic down specific TE tunnels using static routes. In this case the destination interface is the tunnel interface. This is the simplest method of “steering” traffic into a tunnel and many service providers use this method in relatively simple topologies.
However this method is obviously un-scalable in larger, more complex topologies and can be prone to “routing loops” unless careful provisioning is adhered to. An example syntax is:ip route H.H.H.H 255.255.255.255 Tunnel1 (where X.X.X.X is the I.P Destination)
Auto-Route Cisco IOS MPLS Autoroute Announce installs the routes announced by the tail-end router and its downstream routers into the routing table (forwarding table) of the head-end router as directly reachable through the tunnel. The Constrained Based Routing Algorithm allows MPLS TE to establish a Label Switch Path from the head-end to the tail-end node. By default, those paths will not be announced to the IGP routing protocol. Hence, any prefixes/networks announced by the tail end router and its downstream routers would not be "visible" through those paths. For every MPLS TE tunnel configured with Autoroute Announce, the link state IGP will install the routes announced by the tail-end router and its downstream routers into the RIB. Therefore, all the traffic directed to prefixes topologically behind the tunnel head-end is pushed onto the tunnel. To have a better understanding of this feature, consider an example with and without Autoroute Announce enabled. Consider the topology of Figure 4. For the sake of simplicity, assume that Ri's loopback address is i.i.i.i.
Figure 43 - Topology Without Tunnels The corresponding routing table on Router R1 with normal IGP and no MPLS TE looks like the following.
99
Figure 44 - R1 Routing Table – No MPLS TE Considering the same topology as in Figure 4, now let us introduce two MPLS Traffic Engineering tunnels T1 and T2 respectively. Tunnel T1 will originate in R1 and its tail end is R4. Tunnel T2 will originate in R1 and its tail end is R5. MPLS TE Autoroute Announce will be enabled on the two tunnels. Similarly, R1 routing table entries are given in Figure 7.
Figure 45 – Topology With TE Tunnels
Figure 46 - R1 Routing Table With Autoroute Announce
100
The routing tables (Figure 5 and Figure 7) demonstrate that R4 and R5 are directly reachable through tunnel T1 (resp. T2) with MPLS TE Autoroute Announce. Similarly, R8 is now reachable through the tunnel T1 via R4 instead of the "physical" connection. Without Cisco MPLS TE Autoroute Announce, even though Tunnel T1 is up, route to R8 is done via the "physical" connection (as in Figure 5).
Forwarding Adjacency The MPLS TE Forwarding Adjacency feature allows a network administrator to handle a traffic engineering, label-switched path (LSP) tunnel as a link in an Interior Gateway Protocol (IGP) network based on the Shortest Path First (SPF) algorithm. A forwarding adjacency can be created between routers regardless of their location in the network. The routers can be located multiple hops from each other, as shown in Figure 8.
Figure 47 - Forwarding Adjacency Topology As a result, a TE tunnel is advertised as a link in an IGP network with the link's cost associated with it. Routers outside of the TE domain see the TE tunnel and use it to compute the shortest path for routing traffic throughout the network. Benefits •
TE Tunnel Interfaces Advertised for SPF
•
TE tunnel interfaces are advertised in the IGP network just like any other links. Routers can then use these advertisements in their IGPs to compute the SPF even if they are not the head end of any TE tunnels.
Restrictions •
Using the MPLS TE Forwarding Adjacency feature increases the size of the IGP database by advertising a TE tunnel as a link.
101
•
The MPLS TE Forwarding Adjacency feature is supported by Intermediate System-toIntermediate System (IS-IS). Open Shortest Path First (OSPF) support will be available in a future release.
•
When the MPLS TE Forwarding Adjacency feature is enabled on a TE tunnel, the link is advertised in the IGP network as a Type Length Value (TLV) 22 without any TE sub-TLV.
•
MPLS TE forwarding adjacency tunnels must be configured bidirectionally.
•
Do not use the tunnel mpls traffic-eng autoroute announce statement in your configuration when you are using forwarding adjacency.
Using Directed LDP Sessions If you are using TE in conjunction with RFC2547 L3 VPN’s then an extra configuration step may be needed on the primary tunnel interface. When the TE tunnel is terminated on the egress PE, the MPLS VPN and the TE work together without any additional configuration. When the TE tunnel is terminated on any P routers (before the PE in the core), the MPLS VPN traffic forwarding fails because packets arrive with VPN labels as the outer labels, which are not in the LFIBs of these devices. Therefore, these intermediate routers are not able to forward packets to the final destination, the VPN customer network. In such a case, LDP/TDP should be enabled on the TE tunnel to solve the problem. Below is an example of the extra configuration step required:P1#show run int tu0 interface Tunnel0 ip unnumbered Loopback0 no ip directed-broadcast ip route-cache distributed tag-switching ip *this enables tdp/ldp on the tunnel interface tunnel destination 10.5.5.5 tunnel mode mpls traffic-eng tunnel mpls traffic-eng autoroute announce tunnel mpls traffic-eng path-option 10 dynamic end !
102
Number of Protected Prefixes It is possible that if a customer has hundreds of prefixes in his FRR database that he may wish to prioritise which order prefixes get re-written. This way you can manually configure certain prefixes to be re-written on FRR switchover as a priority to ensure LSA’s are met. The MPLS TE—FRR Prefix Ordering Using an ACL feature allows you to prioritize the FRR database according to a single ACL ID. This feature was introduced in IOS 12.0(17)ST7. The ACL ID can contain many networks and hosts. A match in the ACL simply gives precedence to the prefix and places this prefix earlier in the database to provide faster switchover time in the event of a failure. Benefits •
FRR Database Sorting. This feature adds a modified software sorting function for the FRR database based on the existence of a configured ACL. As a result, matching prefixes receive higher priority during a failure and fewer packets are lost.
Restrictions •
This feature is limited to FRR functionality and the order of the failed-over routing prefixes.
•
This feature does not add, delete, or modify the routing prefixes in the FRR database; it just resorts them.
The following command output shows the FRR database before it is reordered: Router# show mpls traffic-eng fast-reroute database
Tunnel head fast reroute information: Prefix Tunnel In-label Out intf/label FRR intf/label Status 10.0.6.1/32 Tu3 12307 PO1/0:Pop tag Tu10:tag-implicit ready 10.0.7.1/32 Tu3 12306 PO1/0:12305 Tu10:tag-implicit ready 10.0.8.1/32 Tu3 12304 PO1/0:12304 Tu10:tag-implicit ready 10.0.0.36/30 Tu3 12314 PO1/0:Pop tag Tu10:tag-implicit ready 10.0.0.40/30 Tu3 12312 PO1/0:Pop tag Tu10:tag-implicit ready 10.0.0.48/30 Tu3 12316 PO1/0:Pop tag Tu10:tag-implicit ready 10.0.0.52/30 Tu3 12317 PO1/0:12307 Tu10:tag-implicit ready 10.0.0.60/30 Tu3 12315 PO1/0:Pop tag Tu10:tag-implicit ready 10.0.0.64/30 Tu3 12318 PO1/0:12308 Tu10:tag-implicit ready In the following command output, the last prefix, which is 10.0.0.64/30, is placed first in the FRR database:
103
Router# configure terminal Router(config)# access-list 1 permit 10.0.0.64 0.0.0.3
In the following command output, the ACL is applied globally: Router(config)# mpls traffic-eng fast-reroute acl 1
In the following command output, the 10.0.0.64/30 prefix has been reordered and now appears first in the FRR database: Router# show mpls traffic-eng fast-reroute database
Tunnel head fast reroute information:Acl in use 1 Prefix Tunnel In-label Out intf/label FRR intf/label Status 10.0.0.64/30 Tu3 12318 PO1/0:12308 Tu10:tag-implicit ready 10.0.6.1/32 Tu3 12307 PO1/0:Pop tag Tu10:tag-implicit ready 10.0.7.1/32 Tu3 12306 PO1/0:12305 Tu10:tag-implicit ready 10.0.8.1/32 Tu3 12304 PO1/0:12304 Tu10:tag-implicit ready 10.0.0.36/30 Tu3 12314 PO1/0:Pop tag Tu10:tag-implicit ready 10.0.0.40/30 Tu3 12312 PO1/0:Pop tag Tu10:tag-implicit ready 10.0.0.48/30 Tu3 12316 PO1/0:Pop tag Tu10:tag-implicit ready 10.0.0.52/30 Tu3 12317 PO1/0:12307 Tu10:tag-implicit ready 10.0.0.60/30 Tu3 12315 PO1/0:Pop tag Tu10:tag-implicit ready LSP midpoint frr information: LSP identifier In-label Out intf/label FRR intf/label Status
104
“3” Implementation Of TE-FRR “3” Network Architecture Introduction The core network of “3” is illustrated in Figure 9 below. It consists of 3 major POP’s deployed in major cities within the U.K.
Figure 48 - "3" Core Network Architecture The core network is built entirely out of 124XX routers with 7200’s used as Route Reflectors. The network utilises MPLS-VPN L3 RFC2547. Cisco 12416’s are used as core switching routers and interface to a Nortel Optera DWDM network for Optical Transport. OC-192 POS linecards are used to buid a 10G network infrastructure and these nodes are used as “P” devices in the context of the MPLS-VPN. Cisco 12410’s are used as edge routers (PE) and are inter-connected via OC-48 POS lincards to the P routers within the POP. VPN interface’s are present on the GigE cards within these routers. Initially Trident (3 X GigE) linecards were used and later these were swapped out for the new Tango (10 X GigE) linecards. The design uses a wide range of PE-CE connection models for various VPN’s:•
Static
•
Connected
•
OSPF
TE-FRR Design In the design it was decided to only protect the core OC-192 POS (Inter-POP) links as these had the greatest chance of failure compared to the Intra-POP links. Obviously TE-FRR provides a very cost effective mechanism of link protection compared to Sonet APS. In the design IP traffic will be protected in the core by Fast Re-Route (FRR) for link protection for sub 50ms performance. Tunnel Engineering aims to optimize network resource usage by directing traffic onto LSP tunnels established according to criteria other than lowest cost or fewest hops, which existing routing protocols use today. For example, to minimize congestion and maximize performance, an ISP might want all traffic destined for a particular network to use the path with maximum bandwidth. Fast restoration is possible within 50 milliseconds. This is because no signaling is required, the backup tunnel is already in place, and the ingress to the back-up tunnel can be co-located on the device that detects the failure. Protection and restoration span is flexible. Backup LSP tunnels can be set up to protect individual links. MPLS-TE FRR will be used to protect all the OC-192 POS links between the 3 x GSRs in the test network. In the event of a link failure, the backup FRR tunnels will provide an immediate local path around the failure until the primary tunnel has re-optimised.
106
Primary Tunnels So in the design we have a number of 1-Hop Primary tunnels going between the POP’s. This makes a total of 6 Primary tunnels in the design. The primary tunnels are dynamically routed to the TE loopback address of its neighbouring 2 POP’s. Initially auto-route was used as the mechanism for injecting traffic into the tunnels, however this was replaced with “Forwarding Adjacency” during system testing dues to un-expected traffic loss. (See Sec XXX) Its important to note that because of the use of 1-Hop tunnels that the tunnel head end is also the point of local repair (PLR) so after an FRR operation the primary tunnel will re-route across the 2-Hop link. This will happen after the fast re-write operation.
Backup Tunnels So each protected link has a 2-Hop backup tunnel provisioned as the alternate path when FRR-LP kicks in. Each backup tunnel is explicitly configured to go via the alternate POP to reach the original POP destination. Figure 10 gives an example of the tunnel provisioning. Obviously explicit backup tunnel configuration is sensible as you obviously provision the backup tunnels to cross a specific 2 hop path
Manchester –GSR2
Hemel GSR1 Birmingham – GSR3
Primary link used by primary tunnel, backed up by FRR FRR backup Tunnel via alternative STM4 interface Figure 49 - Illustration of Primary and Backup TE Tunnels 107
i.p addresses
Table 9
Source Router
Description
Tunnel Number
Explicit/ Dynamic
Final Destination
GSR1
Primary 1-2
1
Dynamic
GSR2
GSR1
Primary 1-3
2
Dynamic
GSR3
GSR1
Backup of 1-2
11
Explicit via GSR3
GSR2
GSR1
Backup of 1-3
12
Explicit via GSR2
GSR3
GSR2
Primary 2-1
1
Dynamic
GSR1
GSR2
Primary 2-3
2
Dynamic
GSR3
GSR2
Backup of 2-1
11
Explicit via GSR3
GSR1
GSR2
Backup of 2-3
12
Explicit via GSR1
GSR3
GSR3
Primary 3-1
1
Dynamic
GSR1
GSR3
Primary 3-2
2
Dynamic
GSR2
GSR3
Backup of 3-1
11
Explicit via GSR2
GSR1
GSR3
Backup of 3-2
12
Explicit via GSR1
GSR2
Tunnel Provisioning All Primary TE tunnel parameters will be as follows: ·
IP Unnumbered to Loopback 0
·
Path option - Dynamic
·
Autoroute announce
·
Priority 5 5
·
Bandwidth 1
·
Fast Re-Route enabled
All FRR backup TE tunnel parameters will be as follows: 108
·
IP Unnumbered to Loopback 0
·
Path option - Explicit path
·
Priority 0 0
·
Bandwidth 0
POS interface specifics: ·
Enable AIS alarm when interface shutdown
·
IP RSVP bandwidth to match link speed
Sample configurations
Generic Global Commands mpls traffic-eng tunnels no tag-switching ip propagate-ttl forwarded tag-switching tdp router-id Loopback0 router isis passive-interface Loopback0 mpls traffic-eng router-id Loopback0 mpls traffic-eng level-2 net 49.4401.1720.3125.0254.00 is-type level-2-only domain-password vlPhuj8p5 metric-style wide level-2 max-lsp-lifetime 65535 lsp-refresh-interval 65000 no hello padding log-adjacency-changes
Birmingham P Router interface Tunnel1001 description from bm0gsr01 tunnel1001 to hh0gsr01 tunnel1002, Primary ip unnumbered Loopback0 no ip directed-broadcast mpls label protocol tdp tag-switching ip 109
tunnel destination 172.31.252.254 tunnel mode mpls traffic-eng tunnel mpls traffic-eng autoroute announce tunnel mpls traffic-eng forwarding-adjacency tunnel mpls traffic-eng priority 5 5 tunnel mpls traffic-eng bandwidth 1 tunnel mpls traffic-eng path-option 1 dynamic tunnel mpls traffic-eng record-route tunnel mpls traffic-eng fast-reroute interface Tunnel1002 description from bmgsr01 tunnel1002 to mr0gsr01 tunnel1002, Primary ip unnumbered Loopback0 no ip directed-broadcast mpls label protocol tdp tag-switching ip tunnel destination 172.31.248.254 tunnel mode mpls traffic-eng tunnel mpls traffic-eng autoroute announce tunnel mpls traffic-eng forwarding-adjacency tunnel mpls traffic-eng priority 5 5 tunnel mpls traffic-eng bandwidth 1 tunnel mpls traffic-eng path-option 1 dynamic tunnel mpls traffic-eng record-route tunnel mpls traffic-eng fast-reroute ! interface Tunnel2001 description from bm0gsr01 tunnel2001 via mr0gsr01 to hh0gsr01 tunnel2002, Backup of pos3/0 ip unnumbered Loopback0 no ip directed-broadcast tunnel destination 172.31.252.254 tunnel mode mpls traffic-eng tunnel mpls traffic-eng priority 0 0 tunnel mpls traffic-eng path-option 1 explicit name backup-to-hh01-via-mr01 tunnel mpls traffic-eng record-route ! interface Tunnel2002 description from bm0gsr01 tunnel2002 via hh0gsr01 to mr0gsr01 tunnel2002, Backup of pos12/0 ip unnumbered Loopback0 no ip directed-broadcast tunnel destination 172.31.248.254 tunnel mode mpls traffic-eng 110
tunnel mpls traffic-eng priority 0 0 tunnel mpls traffic-eng path-option 1 explicit name backup-to-mr01-via-hh01 tunnel mpls traffic-eng record-route interface POS3/0 description from bm0gsr01 pos 3/0 to hh0gsr01 pos 12/0 STM-64 ip address 172.31.254.6 255.255.255.252 no ip directed-broadcast no ip proxy-arp ip router isis encapsulation ppp carrier-delay msec 0 mpls label protocol tdp mpls traffic-eng tunnels mpls traffic-eng backup-path Tunnel2001 tag-switching ip no peer neighbor-route crc 32 clock source internal pos ais-shut pos framing sdh pos report lrdi pos flag s1s0 2 tx-cos STM64-TX no cdp enable isis circuit-type level-2-only isis metric 100 level-2 isis password vlPhuj8p5 level-2 ip rsvp bandwidth 10000000 10000000 interface POS12/0 description from bm0gsr01 pos 12/0 to mr0gsr01 pos 12/0 STM-64 ip address 172.31.254.17 255.255.255.252 no ip directed-broadcast no ip proxy-arp ip router isis encapsulation ppp carrier-delay msec 0 mpls label protocol tdp mpls traffic-eng tunnels mpls traffic-eng backup-path Tunnel2002 tag-switching ip 111
no peer neighbor-route crc 32 clock source internal pos ais-shut pos framing sdh pos report lrdi pos flag s1s0 2 tx-cos STM64-TX no cdp enable isis circuit-type level-2-only isis metric 100 level-2 isis password vlPhuj8p5 level-2 ip rsvp bandwidth 10000000 10000000 ip explicit-path name backup-to-hh01-via-mr01 enable next-address 172.31.254.18 next-address 172.31.254.1 ! ip explicit-path name backup-to-mr01-via-hh01 enable next-address 172.31.254.5 next-address 172.31.254.2
•
The configurations are in principle identical for Hemel and Manchester apart from the I.P addresses.
Quality of Service Introduction In order to fulfil ST requirements of having four distinct classes of service, each with their specific service characteristics, QoS mechanisms are deployed on the access layer and backbone links. The following section describes the technical implementation and features that form the basis for a set of new innovative products. Scalability and stability are the main criteria for any extension of the network. It is absolutely necessary to aggregate IP streams with identical flow characteristic. The expression used for this solution is “service classes”. Dedicated handling of single streams is only meaningful in special cases when high bandwidths are involved, and there are no plans for this solution to be introduced in the first instance. The number of service classes should be strictly limited from the technical point of view. This is not a restriction to construct various commercial products on top of it. Service level agreements (SLA) form the definition interface for the service that will be delivered to the customer by ST. Parameters should describe a probability for a certain service and will be reported on a per class base.
112
For ST MPLS backbone network a robust solution that aligns to base ideas of IETF's DiffServ approach would appear to be practicable at present. With respect to the intended MPLS solution, a maximum of 8 code points per path can be supported. These are distinguished using the three experimental bits of the MPLS shim header. A large part of best effort background traffic is required to produce efficient high quality service classes because DiffServ is based on relative priorities. The strength of a large IP backbone network is to be seen in the fact that high-priority and low-priority traffic is merged on a single network platform. This results in synergy that permits optimum resource utilisation. The bundling of many different traffic streams (statistical multiplexing) smoothes individual bursts.
Differentiated Services Model – Introduction This section is intended as an introduction to the Differentiated Services (DiffServ) reference model. DiffServ is a new model by which traffic is treated by intermediate systems with relative priorities based on the type of services (ToS) or Differentiated Services Code Point (DSCP) field. Defined in RFC’s 2474 and 2475, the DiffServ standard supersedes the original specification for defining packet priority described in RFC 791. The new DiffServ standard proposes a new way of interpreting a field that has always been part of an IP packet. In the DiffServ standard, the ToS field will be renamed to Differentiated Services Code Point (DSCP) and will have new meaning. The DiffServ standard proposes to increase the number of definable priority levels by re-allocating bits of an IP packet for priority marking. As per RFC 791, the ToS field describes one entire byte (eight bits) of an IP packet. Precedence refers to the three most significant bits of the ToS field---that is, [XXX]XXXXX. There may be some confusion because the RFC 1349 defines a new 4-bit ToS XXX[XXXX]X as shown on the following picture.
113
Figure 50
Various interpretations of the TOS field
The three most significant bits of the RFC-791 ToS field - the precedence bits - define the IP packet priority or importance. XXX00000 Bits 0,1,2 = Precedence, where: 111 = Network Control = Precedence 7 110 = Internetwork Control = Precedence 6 101 = CRITIC/ECP = Precedence 5 100 = Flash Override = Precedence 4 011 = Flash = Precedence 3 010 = Immediate = Precedence 2 001 = Priority = Precedence 1 000 = Routine = Precedence 0 The four bits of the RFC-1349 TOS are used in IOS configuration and have the following semantics: 000XXXX0 Bits 3, 4, 5, 6: 1000 = Minimize delay 0100 = Maximize throughput 0010 = Maximize reliability 0001 = Minimize monetary cost 0000 = Normal service
0000000X Bit 7: Reserved for future use This one-byte ToS field has been almost completely unused since it was proposed almost 20 years ago. Only in the last few years have Cisco and other router companies begun utilising the Precedence bits for making forwarding decisions. The DiffServ standard follows a similar scheme to RFC 791, but utilises more bits for setting priority. The new standard maintains backward compatibility with RFC 791 implementations, but allows more efficient use of bits 3, 4, and 5. (Bits 6 and 7 will still be reserved for future development.) With the additional 3 bits, there are now a total of 64 classes instead of the previous 7 classes.
114
RFC 2475 defines Per Hop Behaviour (PHB) as the externally observable forwarding behaviour applied at a DiffServ-compliant node to a DiffServ Behaviour Aggregate (BA). With the ability of the system to mark packets according to DSCP setting, collections of packets with the same DSCP setting and sent in a particular direction can be grouped into a BA. Packets from multiple sources or applications can belong to the same BA. In other words, a PHB refers to the packet scheduling, queuing, policing, or shaping behaviour of a node on any given packet belonging to a BA, as configured by a service level agreement (SLA) or a policy map. The following sections describe the four available standard PHBs: •
Default PHB (as defined in RFC 2474)
•
Class-Selector PHB (as defined in RFC 2474)
•
Assured Forwarding (AFxy) PHB (as defined in RFC 2597)
•
Expedited Forwarding (EF) PHB (as defined in RFC 2598)
Default PHB The default PHB essentially specifies that a packet marked with a DSCP value of 000000 (recommended) receives the traditional best-effort service from a DS-compliant node (that is, a network node that complies with all of the core DiffServ requirements). Also, if a packet arrives at a DS-compliant node, and the DSCP value is not mapped to any other PHB, the packet will get mapped to the default PHB. For more information about default PHB, refer to RFC 2474, Definition of the Differentiated Services Field in IPv4 and IPv6 Headers.
Class-Selector PHB: To preserve backward-compatibility with any IP Precedence scheme currently in use on the network, DiffServ has defined a DSCP value in the form xxx000, where x is either 0 or 1. These DSCP values are called Class-Selector Code Points. (The DSCP value for a packet with default PHB 000000 is also called the Class-Selector Code Point.) The PHB associated with a Class-Selector Code Point is a Class-Selector PHB. These Class-Selector PHBs retain most of the forwarding behaviour as nodes that implement IP Precedence-based classification and forwarding. For example, packets with a DSCP value of 110000 (the equivalent of the IP Precedence-based value of 110) have preferential forwarding treatment (for scheduling, queuing, and so on), as compared to packets with a DSCP value of 100000 (the equivalent of the IP Precedence-based value of 100). These ClassSelector PHBs ensure that DS-compliant nodes can coexist with IP Precedence-based nodes. The DiffServ standard utilises the same precedence bits (the most significant bits: 0, 1, and 2) for priority setting, but further clarifies their functions/definitions, plus offers finer priority granularity through use of the next three bits in the ToS field. DiffServ reorganises (and renames) the precedence levels (still defined by the three most significant bits of the ToS field) into the following categories:
115
Table 10
Class-Selector PHBs
Precedence 7
Stays the same (link layer and routing protocol keep alive)
Precedence 6
Stays the same (used for IP routing protocols)
Precedence 5
Class 5
Precedence 4
Class 4
Precedence 3
Class 3
Precedence 2
Class 2
Precedence 1
Class 1
Precedence 0
Best effort
For more information about class-selector PHB, refer to RFC 2474, Definition of the Differentiated Services Field in IPv4 and IPv6 Headers.
Assured Forwarding PHB Assured Forwarding PHB is nearly equivalent to Controlled Load Service available in the integrated services model. AFxy PHB defines a method by which BAs can be given different forwarding assurances. For example, network traffic can be divided into the following classes: •
Gold: Traffic in this category is allocated 50 percent of the available bandwidth.
•
Silver: Traffic in this category is allocated 30 percent of the available bandwidth.
•
Bronze: Traffic in this category is allocated 20 percent of the available bandwidth.
Further, the AFxy PHB defines four AF classes: AF1, AF2, AF3, and AF4. Each class is assigned a specific amount of buffer space and interface bandwidth, according to the SLA with the service provider or policy map. Within each AF class, you can specify three drop precedence (dP) values: 1, 2, and 3. Assured Forwarding PHB can be expressed as shown in the following example: AFxy In this example, x represents the AF class number (1, 2, or 3) and y represents the dP value (1, 2, or 3) within the AFx class. In instances of network traffic congestion, if packets in a particular AF class (for example, AF1) need to be dropped, packets in the AF1 class will be dropped according to the following guideline: dP(AFx1)
View more...
Comments
