Note: in my articles, any time I give a config example, parts you need to replace *should* be in red text.
Here’s the CliffsNotes version of this article for those that don’t like reading a lot and just have a dead OSPFv3 session on a Brocade to third party neighbor where you’re trying to get IPv6 routing going:
- You of course want to do OSPFv3 with some sort of security right? Brocade OSPFv3 only gives you one option, ESP with SHA1 and no (null) encryption. I’d give you the specific Cisco config you need here but it’s not that simple if you’re doing things at the area-level too (which Brocade doesn’t support), or encryption, so you’ll have to read further. The hint version is: Cisco config “ipv6 ospf encryption ipsec spi ### esp null sha1 KEY“
- If you’re doing switched links (not route-only), you’ll need to set the MTU for both ip and ipv6 at the interface or ‘ve’ level on the Brocade side to match the actual VLAN from the Cisco’s perspective; i.e. hard code even if it’s 1500. The commands are simply “ip mtu 1500” and “ipv6 mtu 1500” or whatever the correct MTU would be for you.
Now, the long version:
I had a network deployment that involved adding some new routers from Brocade to an existing Cisco-based network. The preexisting equipment was a network of Cisco ASR1000-series routers and Cisco Catalyst 4900M layer 3 switches. What I was involved in adding were Brocade MLXe routers and Brocade FastIron SX800 layer 3 switches. The network was dual stack and the devices were talking OSPF (v2) and OSPFv3.
You’d think adding a couple new routers would not be that big of a deal; you’d be half correct.
At the time of this article, the Brocade devices were running code versions V5.4.0bT163 (aka 05400c) on the MLXe’s and 7.4.00cT3e3 (aka 07400c) on the FastIron SX 800’s.
As you know, or maybe you don’t if you haven’t worked with Brocade (and Foundry) equipment in the past, their switching platforms, or at least the one’s I’ve worked with (old jetcore-based BigIron and NetIron, old jetcore FastIron chassis, FastIron SX, MLXe, ADX), do not let you set the MTU on a per-VLAN basis or even a specific MTU. You have to set it globally to ‘jumbo’ on the device and then if specific VLANs have nodes that choose to talk using a certain MTU, that MTU just has to be less than ‘jumbo’ if you’re going to expect it to handle and/or forward/route the packets. I believe, currently, that ‘jumbo’ is defined as 10240. In contrast, with Cisco you set the MTU to a specific number on a per-VLAN basis, which I like, because if I have a VLAN I explicitly do not want jumbo frames on, that gives me the control to have them dropped on just that VLAN. I like having control over each VLAN.
Here’s a run down on the config options I was dealing with on both sides:
- My configuration had a 9000-byte MTU on some VLANs so I had ‘jumbo’ in the config on the Brocade devices and the relevant MTU defined at the VLAN level on the Cisco side (4900M’s) or interface level on the ASR’s since they don’t do switching.
- The Cisco devices were doing OSPFv2 and OSPFv3 with a few unique parameters (the non-relevant pieces have been removed):
-
router ospf 65535 router-id 192.0.2.1 auto-cost reference-bandwidth 10000 area 0 authentication message-digest passive-interface default ! ipv6 router ospf 65535 router-id 192.0.2.1 auto-cost reference-bandwidth 10000 area 0 authentication ipsec spi ### sha1 KEY passive-interface default
- To elaborate on the above for those not familiar with the config:
- This network had a mix of 10gig and gig links. If you don’t explicitly set the reference bandwidth to 10000 your 10gig and gig links will have the same cost from an OSPF perspective since 10gig was way beyond what they had anticipated when the protocol was developed. If your network has 40 or 100 gig planned, you may want to set it even higher. Just be careful when integrating because you need to set the reference to be the same on all your routers in the OSPF area or you could accidentally disrupt your network when adding new equipment. For example, this particular network deployment had some routers talking LACP over two gig links and we did not, yet, want the 10gig links of these new routers to take over, so I not only had to alter the reference bandwidth on those existing routers, I had to also hard code OSPF costs onto the existing links to ensure the 10gig links did not have a lower cost when bringing them online.
- The ‘area 0 authentication message-digest’ piece of the OSPFv2 part means we are doing md5 authentication for the entire OSPF area by default. This prevents you from accidentally forgetting that on certain links, or from unauthorized routers on the other side of participating links from messing things up.
- Similar to the above, the ‘area 0 auth ipsec spi….’ stuff on the OSPFv3 section ensures any new router in that area is only communicated with if it’s talking AH with SHA1 and the matching key and SPI number.
- Both OSPFv2 and v3 are set to ‘passive-interface default’. This is a cool Cisco-specific command that I don’t find a lot of people using, for reasons unknown. What it does is it ensures the only interfaces that will be participating in your OSPF network are ones you explicitly define via “no passive-interface INT“. This makes it much easier to not accidentally have OSPF traffic going on non-transit links. In contrast, Brocade works like this by default and has no option to have links participate in OSPF automatically (which is a good thing).
-
- I of course wanted to talk the same on the Brocade devices going in so there’d be no change needed from this client’s existing config.
OSPFv2 was easy. I adjusted the costs on all the links that would be participating to ensure the new routers did not take over for the existing ones, yet. Defined my MD5 keys for the links. In this particular case it was a VLAN interface on the Cisco side talking to a ‘ve’ interface on the Brocade side. Defined the interfaces on the Cisco side as not passive.
This brought OSPF up and IPv4 routing was working fine.
Next up, OSPFv3. I ran into one weird thing after another; lots of debugging commands, watching packets, etc. I simply could not get things working. My first suspicion was it had something to do with authentication (AH), and I was partially correct; had to solve that problem later though. With this being my working suspicion, I wanted to just take all authentication and encryption off and see if the adjacency would establish. This was easier said than done since the existing routers were doing authentication at the area level. I was able to use “ipv6 ospf authentication null” at the interface level on the Cisco side, and Brocade, if no config related to authentication is present, is always none, so no special config was needed there.
Still no luck. Why wasn’t the session coming up? Debugging it, it was at least looking a bit better now. Both sides knew about each other but were getting stuck in an ExStart state. A little bit of searching ultimately lead me to a Cisco paper titled “Why are OSPF Neighbors Stuck in
Exstart/Exchange State?” In case the article goes away, I have a copy here. To quote:
Neighbors Stuck in Exstart/Exchange State – The problem occurs most frequently when attempting to run OSPF between a Cisco router and another vendor’s router. The problem occurs when the maximum transmission unit (MTU) settings for neighboring router interfaces don’t match.
Interesting. I never expected this as the issue since the OSPFv2 session came up fine without any special config, and on the Cisco side the VLAN is set to 1500. On top of that, how do I even set a matching MTU on the Brocade side since your only option is ‘jumbo’ or no jumbo?! Turns out you are able to set an MTU specific to IP and IPv6 traffic at the interface (physical or ve) level on the Brocade side. So I gave that a shot: “ipv6 mtu 1500”. No luck. I wasted a lot more time searching and debugging after that and ultimately decided to just throw “ip mtu 1500” on there too even though OSPFv2 was working fine. Voilà, OSPFv3 adjacency established!
With IPv6 routing now working, time to go back to the authentication stuff. I figured the MTU was the problem all along now so I just throw my configs back in and we’re good to go; nope, session go bye bye.
I ultimately found the problem here and part of it was me just not paying close enough attention to what I was reading, and also not realizing that Brocade’s own documentation is wrong in some places with regard to OSPFv3 and authentication/encryption. For example, in the Brocade docs for this OS release (file name FastIron_07400_ConfigGuide.pdf), it only mentions OSPFv3 authentication briefly in a table named “Area configuration of IPSec.” This table contains:
Shows new SPI (if changed), authentication algorithm (currently ESP only), encryption algorithm (currently SHA1 only), and the new key.
Why is this wrong? ESP is not an authentication or encryption algorithm, so suggesting ESP is an authentication algorithm is not correct. SHA1 is not an encryption algorithm, so same issue there. The chart just served to confuse me more, slowing down my finally realizing what was going on, which is the fact that Brocade uses packets (ESP) typically only used for IPSec with encryption, but in their case no encryption is applied. They support ‘authentication’ via SHA1, but on the Cisco side, ‘authentication’ means AH packets which Brocade doesn’t support. On top of that, things were complicated further by the fact that the OSPFv3 area authentication on the Cisco side had a default configuration that is not compatible with Brocade.
I’ll spare you the lengthy explanation of how I finally arrived at a working configuration and I’ll just sum it up for you instead with a happy table:
| Cisco Supports? | Brocade Supports? | |
| IPSec using AH and MD5 authentication | Yes | No |
| IPSec using AH and SHA1 authentication | Yes | No |
| IPSec using ESP with NULL (i.e. no) encryption and SHA1 authentication | Yes | Yes |
| IPSec using ESP with 3DES encryption and SHA1 authentication | Yes | No |
| IPSec using ESP with AES-CBC encryption and SHA1 authentication | Yes | No |
| IPSec using ESP with DES encryption and SHA1 authentication | Yes | No |
| IPSec using ESP with 3DES encryption and MD5 authentication | Yes | No |
| IPSec using ESP with AES-CBC encryption and MD5 authentication | Yes | No |
| IPSec using ESP with DES encryption and MD5 authentication | Yes | No |
Lots of options huh? Well, at least in one column lol. So, if you’re doing OSPFv3 between Brocade and Cisco and you want security, here’s what you’ll actually need on the Cisco side (if you’re doing area-level authentication or encryption):
router ospf 65535 auto-cost reference-bandwidth 10000 area 0 authentication message-digest passive-interface default no passive-interface Vlan# interface Vlan# bandwidth 10000000 ip ospf authentication message-digest ip ospf message-digest-key ## md5 KEY ip ospf 65535 area 0 ipv6 ospf authentication null ipv6 ospf 65535 area 0 ipv6 ospf encryption ipsec spi ### esp null sha1 KEY
And the Brocade side:
router ospf area 0 auto-cost reference-bandwidth 10000 ! interface ve # ip mtu 1500 ip ospf area 0 ip ospf md5-authentication key-id ## key KEY ipv6 mtu 1500 ipv6 ospf area 0 ipv6 ospf authentication ipsec spi ### esp sha1 encryptb64 KEY !