Introduction
This sample configuration demonstrates how to run Border Gateway Protocol (BGP) across a Security Appliance (PIX/ASA) and how to achieve redundancy in a multihomed BGP and PIX environment. With a network diagram as an example, this document explains how to automatically route traffic to Internet service provider B (ISP-B) when AS 10 loses connectivity to ISP-A (or vice versa), through the use of dynamic routing protocols that run between all routers in AS 10.
Because BGP uses unicast TCP packets on port 179 to communicate with its peers, you can configure PIX1 and PIX2 to allow unicast traffic on TCP port 179. This way, BGP peering can be established between the routers that are connected through the firewall. Redundancy and the desired routing policies can be achieved through the manipulation of the BGP attributes.
Prerequisites
Requirements
Readers of this document should be familiar with Configuring BGP and Basic Firewall Configuration.
Components Used
The example scenarios in this document are based on these software versions:
-
Cisco 2600 routers with Cisco IOS® Software Release 12.2(27)
-
PIX 515 with Cisco PIX Firewall Version 6.3(3) and later
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
Related Products
This configuration can also be used with these hardware and software versions:
-
Cisco Adaptive Security Appliance (ASA) 5500 Series with 7.x version and later
Conventions
Refer to Cisco Technical Tips Conventions for more information on document conventions.
Configure
This section provides information to configure the features described in this document.
Note: To find additional information about the commands in this document, use the Command Lookup Tool ( registered customers only) .
Network Diagram
This document uses this network setup:
In this network setup, Router12 and Router22 (which belong to AS 10) are multihomed to Router14 (ISP-A) and Router24 (ISP-B) respectively, for redundancy. The internal network 192.168.10.0/24 is on the inside of the firewall. Router11 and Router21 connect to Router12 and Router22 through the firewall. PIX1 and PIX2 are not configured to perform Network Address Translation (NAT).
Scenario 1
In this scenario, Router12 in AS 10 does external BGP (eBGP) peering with Router14 (ISP-A) in AS 20. Router12 also does internal BGP (iBGP) peering with Router11 through PIX1. If eBGP learned routes from ISP-A are present, Router12 announces a default route 0.0.0.0/0 on iBGP to Router11. If the link to ISP-A fails, Router12 stops announcing the default route.
Similarly, Router22 in AS 10 does eBGP peering with Router24 (ISP-B) in AS 30 and announces a default route on iBGP to Router21 conditionally based on the presence of ISP-B routes in its routing table.
Through the use of an access list, PIX1 and PIX2 are configured to allow the BGP traffic (TCP, port 179) between iBGP peers. This is because PIX interfaces have an associated security level. By default, the inside interface (ethernet1) has a security level 100 and the outside interface (ethernet0) has a security level 0. Connections and traffic are normally permitted from higher to lower security level interfaces. To permit traffic from a lower security level interface to a higher security level interface, however, you must explicitly define an access list on the PIX. Also, you must configure a static NAT translation on PIX1 and PIX2, to allow routers on the outside to initiate a BGP session with routers on the inside of PIX.
Both Router11 and Router21 conditionally announce the default route into the Open Shortest Path First (OSPF) domain based on the iBGP-learned default route. Router11 announces the default route into the OSPF domain with a metric of 5, Router21 announces the default route with a metric of 30, and therefore the default route from Router11 is preferred. This configuration helps propagate only the default route 0.0.0.0/0 to Router11 and Router21, which conserves memory consumption on the inside routers and achieves optimum performance.
Thus, to summarize these conditions, this is the routing policy for AS 10:
-
AS 10 prefers the link from Router12 to ISP-A for all outbound traffic (from 192.168.10.0/24 to the Internet).
-
If connectivity to ISP-A fails, all traffic is routed via the link from Router22 to ISP-B.
-
All traffic that comes from the Internet to 192.168.10.0/24 uses the link from ISP-A to Router12.
-
If the link from ISP-A to Router12 fails, all inbound traffic is routed via the link from ISP-B to Router22.
Configurations
This scenario uses these configurations:
| Router11 |
|---|
hostname Router11 |
| Router12 |
|---|
hostname Router12 |
| Router14 (ISP-A) |
|---|
hostname Router14 |
| Router21 |
|---|
hostname Router21 |
| Router22 |
|---|
hostname Router22 |
| Router24 (ISP-B) |
|---|
hostname Router24 |
| PIX1 |
|---|
nameif ethernet0 outside security0 |
| PIX2 |
|---|
nameif ethernet0 outside security0 |
Verify
Use this section to confirm that your configuration works properly.
The Output Interpreter Tool ( registered customers only) (OIT) supports certain show commands. Use the OIT to view an analysis of show command output.
When both BGP sessions are up, you can expect all packets to be routed via ISP-A. Consider the BGP table on Router11. It learns a default route 0.0.0.0/0 from Router12 with the next hop 172.16.12.2.
Router11# show ip bgp
BGP table version is 14, local router ID is 192.168.10.1
Status codes: s suppressed, d damped, h history, * valid, > best, i -
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*>i0.0.0.0 172.16.12.2 100 0 i
*> 192.168.10.0 0.0.0.0 0 32768 i
The 0.0.0.0/0 default route that is learned via BGP is installed in the routing table, as shown in the output of show ip route on Router11.
Router11# show ip route
Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default, U - per-user static route, o - ODR
P - periodic downloaded static route
Gateway of last resort is 172.16.12.2 to network 0.0.0.0
C 192.168.10.0/24 is directly connected, FastEthernet0/0
172.16.0.0/24 is subnetted, 2 subnets
S 172.16.12.0 [1/0] via 172.16.11.10
C 172.16.11.0 is directly connected, FastEthernet0/1
B* 0.0.0.0/0 [105/0] via 172.16.12.2, 00:27:24
Now consider the BGP table on Router21. It also learns the default route via Router22.
Router21# show ip bgp
BGP table version is 8, local router ID is 192.168.10.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*>i0.0.0.0 172.16.22.2 100 0 i
*> 192.168.10.0 0.0.0.0 0 32768
Now see if this BGP-learned default route gets installed in the routing table of Router21.
Router21# show ip route
Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default, U - per-user static route, o - ODR
P - periodic downloaded static route
Gateway of last resort is 192.168.10.1 to network 0.0.0.0
C 192.168.10.0/24 is directly connected, FastEthernet0/0
172.16.0.0/24 is subnetted, 2 subnets
C 172.16.21.0 is directly connected, FastEthernet0/1
S 172.16.22.0 [1/0] via 172.16.21.10
O*E2 0.0.0.0/0 [110/5] via 192.168.10.1, 00:27:06, FastEthernet0/0
The default route in Router21 is learned via OSPF (note the O prefix on the 0.0.0.0/0 route). It is interesting to note that there is a default route learned via BGP from Router22, but the show ip route output shows the default route learned via OSPF.
The OSPF default route was installed in Router21 because Router21 learns the default route from two sources: Router22 via iBGP and Router11 via OSPF. The route selection process installs the route with a better administrative distance into the routing table. The administrative distance of OSPF is 110 while the administrative distance of iBGP is 200. Therefore, the OSPF-learned default route gets installed in the routing table, because 110 is less than 200. For more information on route selection, refer to Route Selection in Cisco Routers.
Troubleshoot
Use this section to troubleshoot your configuration.
Bring down the BGP session between Router12 and ISP-A.
Router22(config)# interface fas 0/0
b8-12-2621(config-if)# shut
1w0d: %LINK-5-CHANGED: Interface FastEthernet0/0,
changed state to administratively down
1w0d: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0,
changed state to down
Router11 does not have the default route learned via BGP from Router12.
Router11# show ip bgp
BGP table version is 16, local router ID is 192.168.10.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 192.168.10.0 0.0.0.0 0
Check the routing table on Router11. The default route is learned via OSPF (administrative distance of 110) with a next hop of Router21.
Router11# show ip route
!--- Output suppressed.
Gateway of last resort is 192.168.10.2 to network 0.0.0.0
C 192.168.10.0/24 is directly connected, FastEthernet0/0
172.16.0.0/24 is subnetted, 2 subnets
S 172.16.12.0 [1/0] via 172.16.11.10
C 172.16.11.0 is directly connected, FastEthernet0/1
O*E2 0.0.0.0/0 [110/30] via 192.168.10.2, 00:00:09, FastEthernet0/0
This output is expected as per the pre-defined policies. At this point, however, it is important to understand the distance bgp 20 105 200 configuration command in Router11 and how it influences the route selection on Router11.
The default values of this command are distance bgp 20 200 200, where eBGP-learned routes have an administrative distance of 20, iBGP-learned routes have an administrative distance of 200, and local BGP routes have an administrative distance of 200.
When the link between Router12 and ISP-A comes up again, Router11 learns the default route via iBGP from Router12. However, because the default administrative distance of this iBGP-learned route is 200, it will not replace the OSPF-learned route (because 110 is less than 200). This forces all of the outbound traffic to the link from Router21 to Router22 to ISP-B, even though the link from Router12 to ISP-A is up again. To solve this issue, change the administrative distance of the iBGP-learned route to a value less than the Interior Gateway Protocol (IGP) used. In this example, the IGP is OSPF, so a distance of 105 was chosen (because 105 is less than 110).
For more information on the distance bgp command, refer to BGP Commands. For more information on multihoming with BGP, refer to Load Sharing with BGP in Single and Multihomed Environments: Sample Configurations.
Scenario 2
In this scenario, Router11 is directly eBGP peering with Router 14 (ISP-A), and Router21 is directly eBGP peering with Router24 (ISP-B). Router12 and Router22 do not participate in BGP peering, but they do provide the IP connectivity to the ISPs. Because the eBGP peers are not directly connected neighbors, the neighbor ebgp-multihop command is used on the participating routers. The neighbor ebgp-multihop command enables BGP to override the default one hop eBGP limit because it changes the Time to Live (TTL) of eBGP packets from the default value of 1. In this scenario, the eBGP neighbor is 3 hops away, so neighbor ebgp-multihop 3 is configured on the participating routers so that the TTL value is changed to 3. Also, static routes are configured on the routers and PIX to ensure that Router11 can ping the Router14 (ISP-A) address 172.16.13.4 and to ensure that Router21 can ping the Router24 (ISP-B) address 172.16.23.4.
By default, PIX does not allow Internet Control Message Protocol (ICMP) packets (sent when you issue the ping command) to pass through. To allow ICMP packets, use the access-list command as shown in in the next PIX configuration. For more information on the access-list command, refer to the PIX Firewall A through B Commands.
The routing policy is the same as in Scenario 1: the link between Router12 and ISP-A is preferred over the link between Router22 and ISP-B, and when the ISP-A link goes down the ISP-B link is used for all inbound and outbound traffic.
Configurations
This scenario uses these configurations:
| Router11 |
|---|
hostname Router11 |
| Router12 |
|---|
hostname Router12 |
| Router14 (ISP-A) |
|---|
hostname Router14 |
| Router21 |
|---|
hostname Router21 |
| Router22 |
|---|
hostname Router22 |
| Router24 (ISP-B) |
|---|
hostname Router24 |
| PIX1 |
|---|
nameif ethernet0 outside security0 |
| PIX2 |
|---|
nameif ethernet0 outside security0 |
Verify
Router11# show ip bgp summary
BGP router identifier 192.168.10.1, local AS number 10
BGP table version is 13, main routing table version 13
4 network entries and 5 paths using 568 bytes of memory
7 BGP path attribute entries using 420 bytes of memory
2 BGP AS-PATH entries using 48 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP activity 43/264 prefixes, 75/70 paths, scan interval 15 secs
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
172.16.13.4 4 20 1627 1623 13 0 0 02:13:36 2
192.168.10.2 4 10 1596 1601 13 0 0 02:08:47 2
Router21# show ip bgp summary
!--- Output suppressed.
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
172.16.23.4 4 30 1610 1606 8 0 0 02:06:22 2
192.168.10.1 4 10 1603 1598 8 0 0 02:10:16 3
The BGP table on Router11 shows the default route (0.0.0.0/0) toward the next hop ISP-A 172.16.13.4.
Router11# show ip bgp
BGP table version is 13, local router ID is 192.168.10.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 0.0.0.0 172.16.13.4 200 0 20 i
*> 10.10.20.0/24 172.16.13.4 0 200 0 20 i
*>i10.10.30.0/24 192.168.10.2 0 100 0 30 i
* i192.168.10.0 192.168.10.2 0 100 0 i
*> 0.0.0.0 0 32768 i
Now check the BGP table on Router21. It has two 0.0.0.0/0 routes: one learned from ISP-B with a next hop of 172.16.23.4 on eBGP, and the other learned via iBGP with a local-preference of 200. Router21 prefers iBGP-learned routes because of the higher local-preference attribute, so it installs that route in the routing table. For more information on BGP path selection, refer to BGP Best Path Selection Algorithm.
Router21# show ip bgp
BGP table version is 8, local router ID is 192.168.10.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
* 0.0.0.0 172.16.23.4 0 30 i
*>i 192.168.10.1 200 0 20 i
*>i10.10.20.0/24 192.168.10.1 0 200 0 20 i
*> 10.10.30.0/24 172.16.23.4 0 0 30 i
*> 192.168.10.0 0.0.0.0 0 32768 i
* i 192.168.10.1 0 100 0 i
Troubleshoot
Bring down the Router11 and ISP-A BGP session.
Router11(config)# interface fas 0/1
Router11(config-if)# shut
4w2d: %LINK-5-CHANGED: Interface FastEthernet0/1,
changed state to administratively down
4w2d: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1,
changed state to down
4w2d: %BGP-5-ADJCHANGE: neighbor 172.16.13.4 Down BGP Notification sent
4w2d: %BGP-3-NOTIFICATION: sent to neighbor 172.16.13.4 4/0 (hold time expired)0 bytes
The eBGP session to ISP-A goes down when the hold-down timer (180 seconds) expires.
Router11# show ip bgp summary
!--- Output suppressed.
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
172.16.13.4 4 20 1633 1632 0 0 0 00:00:58 Active
192.168.10.2 4 10 1609 1615 21 0 0 02:18:09
With the link to ISP-A down, Router11 installs 0.0.0.0/0 with a next hop of 192.168.10.2 (Router21), which is learned via iBGP in its routing table. This pushes all outbound traffic through Router21 and then to ISP-B, as shown in this output:
Router11# show ip bgp
BGP table version is 21, local router ID is 192.168.10.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*>i0.0.0.0 192.168.10.2 100 0 30 i
*>i10.10.30.0/24 192.168.10.2 0 100 0 30 i
* i192.168.10.0 192.168.10.2 0 100 0 i
*> 0.0.0.0 0 32768 i
Router21# show ip bgp
BGP table version is 14, local router ID is 192.168.10.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 0.0.0.0 172.16.23.4 0 30 i
*> 10.10.30.0/24 172.16.23.4 0 0 30 i
*> 192.168.10.0 0.0.0.0 0 32768 i
* i 192.168.10.1 0 100 0 i
MD5 Authentication for BGP Neighbors through the PIX
PIX 6.x Configuration
Just like any other routing protocol, BGP can be configured for authentication. You can configure MD5 authentication between two BGP peers, which means that each segment sent on the TCP connection between the peers is verified. MD5 authentication must be configured with the same password on both BGP peers; otherwise, the connection between them will not be made. The configuration of MD5 authentication causes Cisco IOS software to generate and check the MD5 digest of every segment sent on the TCP connection. If authentication is invoked and a segment fails authentication, then an error message will be displayed in the console.
When you are configuring BGP peers with MD5 authentication that pass through a PIX firewall, it is important to configure the PIX between the BGP neighbors so that the sequence numbers for the TCP flows between the BGP neighbors are not random. This is because the TCP random sequence number feature on the PIX firewall is enabled by default, and it changes the TCP sequence number of the incoming packets before it forwards them.
MD5 authentication is applied on the TCP psuedo-IP header, TCP header and data (refer to RFC 2385 
). TCP uses this data—which includes the TCP sequence and ACK numbers—along with the BGP neighbor password to create a 128 bit hash number. The hash number is included in the packet in a TCP header option field. By default, the PIX offsets the sequence number by a random number, per TCP flow. On the sending BGP peer, TCP uses the original sequence number to create the 128 bit MD5 hash number and includes this hash number in the packet. When the receiving BGP peer gets the packet, TCP uses the PIX-modified sequence number to create a 128 bit MD5 hash number and compares it to the hash number that is included in the packet.
The hash number is different because the TCP sequence value was changed by the PIX, and TCP on the BGP neighbor drops the packet and logs an MD5 failed message similar to this one:
%TCP-6-BADAUTH: Invalid MD5 digest from 172.16.11.1:1778 to 172.16.12.2:179
Use the norandomseq keyword with the static (inside,outside) 172.16.11.1 172.16.11.1 netmask 255.255.255.0 norandomseq command to solve this problem and to stop the PIX from offsetting the TCP sequence number. This example illustrates the use of the norandomseq keyword:
| Router11 |
|---|
hostname Router11 |
| Router12 |
|---|
hostname Router12 |
| PIX1 |
|---|
nameif ethernet0 outside security0 |
PIX / ASA 7.x and Later
In order to pass BGP updates through ASA/PIX 7.x and later, you must allow TCP option 19 in BGP. By default, option 19 is not permitted to pass through a PIX/ASA that runs 7.0 or later.
In order to permit traffic with these TCP options, issue the tcp-map command to enter tcp-map configuration mode. Issue the tcp-options command in tcp-map configuration mode in order to clear selective-acknowledgement, window-scale, and timestamp TCP options. You can also clear or drop packets with options that are not very well defined.
The tcp-map command is used along with the Modular Policy Framework infrastructure. Define the class of traffic that uses the class-map command and customize the TCP inspection with tcp-map commands. Apply the new TCP map with the policy-map command. Activate TCP inspection with service-policy commands.
| PIX/ASA 7.x/8.x |
|---|
access-list BGP-MD5-ACL remark *** Allow BGP MD5 Authentication **** |
Note: The addresses of the devices that run BGP cannot be NATed, because the MD5 hash takes IP header as well as the TCP header, which means none of that information can be changed.
Verify
Output from the show ip bgp command indicates that the authentication is successful and that the BGP session is established on Router11.
Router11# show ip bgp
BGP table version is 14, local router ID is 192.168.10.1
Status codes: s suppressed, d damped, h history, * valid, > best, i -
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*>i0.0.0.0 172.16.12.2 100 0 i
*> 192.168.10.0 0.0.0.0 0 32768 i