BGP Cheat Sheet Sections: BGP4 - An Overview The Community Attribute Brief Overview of BGP Routing Instability Route Flap Damping Internet Service Providers - The Tier Hierarchy BGP Security Threats Some Terms References
BGP4 - An Overview BGP Attributes Well-known Mandatory Attributes Next Hop AS Path Origin Well-known Discretionary Attributes Local Preference (32 bits) Atomic Aggregate Optional Transitive Attributes Aggregator Community Optional Non-transitive Attributes MED - 'Multi Exit Discriminator' or 'Metric' (32 bits)
Four Message Types 1. OPEN 2. KEEP-ALIVES Defaults: Hold Time - 180s, Keep-alive Intervals - 60s 3. UPDATES Each Update can include several prefixes, but only one only one path. 4. NOTIFICATIONS
BGP States 1. 2. 3. 4. 5. 6.
Idle Connect Active OpenSent OpenConfirm Established (BGP session)
BGP Best Path Selection 1. 'Synchronisation' or 'No Synchronisation' with IGP, i.e. should inaccessible next hops
2. 3. 4. 5. 6. 7.
8. 9. 10. 11. 12. 13.
in IGP still be considered for best path selection Cisco only: 'Cisco Weight', only relevant locally, higher values are better 'Local Preference', propagated only among IBGP peers, higher values are better (inverse value to 'pref' in an AUT-NUM object) 'Originated Locally', i.e. network or aggregate command in router configuration AS Path/Sequence Length (shorter preferred) 'Origin' code, i>e>? 'MED', advertised to EBGP peers, used only within AS, not propagated to other ASes, lower values are better (note implications of 'Always-compare-MED' and 'Deterministic' MED) EBGP is preferred over IBGP, better to choose directly connected EBGP peers than sending packet through local AS 'IGP Metric', choose closest egress point out of local AS Oldest route, sign of stability (can be disabled) Lowest Router ID, i.e. highest IP on router's interfaces (loopback preferred) Shortest cluster-ID Lowest IP used in neighbor statement
Keep in mind that synchronisation and administrative distance, preferences related to how a route is learnt by the router, determines whether the selected path is injected into the IP forwarding table. It is this table that is used by the router to determine the next hop of actual packets.
The Community Attribute Format: AS:Community (16 + 16 = 32bits) Reserved ranges: 0000:0000-0000:FFFF FFFF:0000-FFFF:FFFF In theory, communities, where the first sixteen bits correspond to private AS numbers, should also be considered reserved.
Examples of Well-known Communities No_Export (FFFF:FF01): Don't advertise to EBGP peers No_Advertise (FFFF:FF02): Don't advertise, full Stop No_Export_Subconfed (FFFF:FF03): Only advertise to peers within subconfederation Local_AS: Advertise within local AS, used with confederations Internet: Advertise to the entire Internet
Usage of Communities Route Tagging
Type of Peer: customer, peer, upstream via private peering or public IXP Geographic Location Interconnection Point Traffic Engineering No announcements to specified peers AS Prepending Setting local preference
Brief Overview of BGP Routing Instability Ideally, a stable Internet generates legitimate BGP updates when the following happens: 1. 2. 3. 4.
Policy changes Network failures Addition of new networks Maintenance
The Essential Updates Announcements: A BGP speaker has either learnt or made a new policy decision that it prefers another best path. Withdrawals: A BGP speaker has concluded that a network (prefix) is no longer reachable and withdraws it from service. Explicit withdrawal : Prefix is withdrawn from service by a withdrawal message. Implicit withdrawal : Path and/or other attributes are changed by a new announcement message. Example:
AS1 - AS2 - AS3 If we assume that AS1 becomes unreachable and then reachable again, then AS2 will see an implicit withdrawal, a new announcement without a preceding withdrawal, and AS3 will see an explicit withdrawal, a withdrawal message from AS2 including AS1's prefixes, before being notified that AS1's networks are reachable again. However, if AS2 knows another path to AS1, then AS3 will also see an implicit withdrawal. These updates can be grouped into three categories: 1. Forwarding instabilities 2. Policy changes 3. Pathological updates (redundant) Pathological updates are relatively benign, because they usually do not affect the IP forwarding table. The question is the impact of excessive updates on the network infrastructure. Updates that do repeatedly change the forwarding path due to policy changes or forwarding instabilities are more serious.
Labovitz et al have constructed a well-known taxonomy for analyzing BGP routing instabilities: WAdiff Explicit withdrawal followed by an announcement that replaces the original path. Reflects legitimate forwarding instabilities. AAdiff Implicit withdrawal, where the original AS_PATH is replaced. Reflects legitimate forwarding instabilities. WAdup Explicit withdrawal followed by an announcement that reinstates the same AS_PATH. Could either be legitimate or pathological. AAdup Two sequential announcements, implying an implicit withdrawal that replaces the original AS_PATH with the same one again. Could either be legitimate updates related to policy changes or pathological. WWdup Repeated pathological, duplicate withdrawals. NOTE: Routes are the same if they have the same AS_Path and NEXT_HOP.
Effects of BGP Routing Instability 1. Increased delays and packet loss to unstable destination networks 2. Delays in convergence, could cause routing loops etc. 3. Additional overhead on Internet infrastructure (bandwidth, router hardware, administration etc.) The causes could be: 1. Unstable physical links along the AS path 2. Failing interfaces along the AS path 3. Humans... redistributing carelessly from IGP. changing the state of interfaces. clearing BGP sessions. aggregating routes improperly.
Preventive Actions 1. 2. 3. 4. 5.
Route flap damping/dampening Techniques that allow resetting sessions without tearing them down, e.g. RFC2918 Keep some state of recently sent messages to peers Implementing Route Servers Proper aggregation
Aggregation cannot always be optimal due to multihoming practices and provider-independent address space.
Route-flap Damping There are two approaches according to the recommendations in RIPE-229: "Progressive" approach: Start with suppressing 4th flap /24 or longer prefix length: Max = Min 60 minutes /22, /23: Max 45 minutes, Min 30 minutes Others: Max 30 minutes, Min 10 minutes "Flat & Gentle" approach: Start with suppressing 4th flap Suppress route: Max 30 minutes, Min 10 minutes Always exclude Golden Networks (e.g. root name servers and G-TLD name servers) from dampening. Why does RIPE-229 recommend suppressing routes from the fourth flap? 1. IOS upgrade and reload 2. Failed IOS upgrade & reload 3. Old working IOS image again Hypothetically, dampening will not be enforced if the scenario above occurs.
Cisco's Flap Dampening Parameters (by default) Flap: 1000 per flap (penalty for withdrawal) Half-life: 15 minutes (penalty reduced to half at end of each 15 min interval) Suppress: 2000 (dampening starts when half-life reduction is above this value) Reuse: 750 (route advertised again when penalty reaches below this value) Max suppress time: 60 minutes (4x half-life)
Juniper's Flap Dampening Parameters (by default) Flap: 1000 per flap (penalty for withdrawal) Half-life: 15 minutes (penalty reduced to half at end of each 15 min interval) Suppress: 3000 (dampening starts when half-life reduction is above this value) Reuse: 750 (route advertised again when penalty reaches below this value) Max suppress time: 60 minutes (4x half-life) Naturally, suppressed routes can be cleared manually.
Internet Service Providers (ISP) - The Tier Hierarchy Tier 1 Providers Tier 1 ISPs are large and hold all the world's Internet routes, also referred to as the Default Free Zone (DFZ). Their mutual peering agreements give each other access to all Internet routes. Think of 'Global Networks' (tens), eg AT&T, C&W, MCI, Equant, Qwest, Sprint, Level 3
etc.
Tier 2 Providers Tier 2 providers purchase upstream transit to the world's Internet routes from one or more tier 1 ISPs, which in turn makes their IP networks a sub-set of those tier 1 networks. To minimize the amount of traffic that needs to tranverse the tier 1's network ($$$), the tier 2 ISPs will set up peerings with each other, either over private links or at Internet Exchange Points (IXP). Think of 'Regional Networks' (thousands) in a certain part of the world, eg national telecoms and their competitors. In theory, tier 3 providers buy transit from tier 2 ISPs and so on. However, this model is becoming increasingly vague as the Internet structure becomes more and more mesh-like. For instance, an ISP may have bought transit from both a tier 1 and tier 2 provider, while having peering agreements with a tier 2 and a tier 3. Today, the term 'tier' is primarily used to differentiate between tier 1 providers that do not need to buy transit due to peerings with other tier 1 ISPs and the rest, ie tier 2 and below.
BGP Security Threats These include Lack of Filtering Usually implemented at edge of Internet 'Smart' to filter prefixes based on route objects in IRR, eg RIPE database, or apply Reverse Path Forwarding (RPF) checks or similar mechanisms 'Blackholing' Route to nowhere Often problem in conjuction with EBGP-multihop IP Spoofing of Updates Not possible against modern IOS 1. MD5 Authentication 2. Randomized TCP sequence numbers Misconfigurations Classic Example: ASxx gets full or partial BGP feeds from two neighboring ASes: ASyy and ASzz. ASxx then redistributes the full or partial BGP table into IGP and back again into BGP. This results in ASyy and ASzz believing that all traffic to the Internet should be routed via ASxx.
Some terms: BGP Storm: Excessive updates impact the router's CPU and memory, usually at the Internet's periphery, that leads to keep-alives being dropped and BGP sessions going down. This behaviour propagates throughout the Internet causing a BGP update "storm". EBGP-multihop
BGP assumes that neighbor is NOT directly connected (TTL=255 by default, normally TTL=0). Internet Weather: End-to-End connectivity issues that are noticed by Internet end-users. Route Flaps: Withdrawal followed by an announcement. There is a distinction between "Route Oscilliations", which are periodic, and "Route Flaps", that are not. Route Server: 'Route Reflector' for EBGP peers, often used at IXPs to solve n^2-related configuration issues.
References: "CS 268: Lecture 8 (Routing Behavior in the Internet)", Ion Stoica, University of California, Berkeley, February 8, 2001 "A survey of the utilization of bgp communities", draft-quoitin-bgp-comm-survey-00.txt, B. Quoitin and O. Bonaventure, Internet draft, work in progress, February 2002. "Cisco ISP Essentials", Barry Raveendran Greene & Philip Smith, CiscoPress, 2002 "Internet Routing Instability", Craig Labovitz, G. Robert Malan & Farnam Jahanian, U niversity of Michigan, Department of Electrical Engineering and Computer Science Ripe-229: "RIPE Routing-WG Recommendations for Coordinated Route-flap Damping Parameters", Christian Panigl, Joachim Schmitz, Philip Smith & Cristina Vistoli, October 22, 2001 "Routing TCP/IP, Volume Two", Jeff Doyle & Jennifer DeHaven Carroll, CiscoPress, 2001 "The Basics of BGP Routing and its Performance in Today's Internet", Nina Taft, Sprint, Advanced Technology Labs, California, May 2001 "Route Flap Damping Exacerbates Internet Routing Convergence", Zhuoqing Morley Mao, Ramesh Govindan, George Varghese & Randy H. Katz, August 2002
Please send comments to
[email protected]
