Key-based SSH authentication to IOS XR devices

Are you used to doing this in Cisco IOS?

ip ssh pubkey-chain
dev-1(conf-ssh-pubkey)#username sysadmin1
dev-1(conf-ssh-pubkey-data)#ssh-rsa AAAA....

and what you end up with is an easy and obvious local user who can authenticate over SSH using their key instead of password.  Well, IOS XR is different, and, of course, more convoluted, and, of course, has absolutely no consistency from one platform to another, one version to the next, so on and so forth.  Srsly, Cisco?  I now almost feel bad about complaining I had to chop my key up into small bits for IOS to import it, but XR makes that feel like a pleasant experience.

Anyway, so your XR user is defined, and you see it in the config:

RP/0/RP0/CPU0:rtr1#show run
username routeradmin
group root-lr
group cisco-support
secret 5 $1$asdf/

To add an ssh key for that user, you might need to use the command “crypto key import authentication rsa” followed by the user, or perhaps the location of the key file, but that depends entirely on your platform and version of XR.  I can tell you that the NCS platform wants just a file as the next argument, while other XR platforms expect user-related data before the key, so the key is actually associated with that username.  Completely inconsistent, and in the case of NCS, undesirable as now you have keys floating about which can’t be attributed to any specific user other than a tag for who installed it.  I’ll elaborate on that later.

In relation to the above, what does seem to be consistent is that XR does NOT want your key in base64 format like IOS and every other system in the universe, it wants you to import the binary key data.  To do this, you’ll need to decode the base64 portion of your public key file, e.g. the middle part of the following between the ssh-rsa tag and the comment (if there is a comment), separated by spaces: “ssh-rsa AAAAblah== key-comments”.  Assuming you already have a handy ssh key generated via ssh-keygen, then on a Mac, you’d use the base64 command with the -D (capital) option to decode the base64 to binary:

cat | cut -f 2 -d ' ' | base64 -D > mykey.bin

On Linux, same thing, but use -d instead of -D to ‘decode’.

I did the above and am sitting with my binary key data in my new mykey.bin file.  I happen to be running 6.2.25 on the NCS platform and I couldn’t get the key to import no matter what I tried.  My attempts were:

  • Copying it to the device under the disk0:/ filesystem, then ‘crypto key import authentication rsa KEY’ like the CLI guides me to, only to continuously get “Cannot execute the command : Invalid argument”.  For example, here’s me confirming my key file is there as both the current directory and the full path:
    Sat Jan 6 16:51:04.699 UTC
    Directory of /misc/scratch
    71 -rw-r--r-- 1 535 Jan 6 16:51 mykey.bin
    RP/0/RP0/CPU0:rtr1#dir mykey.bin
    Sat Jan 6 16:53:27.898 UTC
    Directory of disk0:mykey.bin
    71 -rw-r--r-- 1 535 Jan 6 16:51 mykey.bin

    So the above confirms it’s there, and even gives me the path to it as /misc/scratch even though I copied it to disk0:/, so /misc/scratch appears to be a mount for disk0.  Anyway, since dir can see it, one would logically assume the key import utility would see it in the same place?  Nope:

    RP/0/RP0/CPU0:rtr1#crypto key import authentication rsa mykey.bin
    Fri Jan 5 23:28:24.392 UTC
    Cannot execute the command : Invalid argument
  • Okay, so maybe it doesn’t like current directory for imports and needs the full path which dir already confirmed is /misc/scratch/mykey.bin?
    RP/0/RP0/CPU0:rtr1#crypto key import authentication rsa /misc/scratch/mykey.bin
    Fri Jan 5 23:30:54.153 UTC
    Cannot execute the command : Operation not permitted

    Hmm, well that’s mildly better, at least it said operation not permitted vs invalid. Not sure how to get around it not allowing me to import though lol.

  • I tried the import from within the ‘admin’ side, because I found at least one reference to someone else using XR doing it that way successfully (, but nope, not on my NCS’ XR release:
    Fri Jan 5 15:56:01.311 UTC
    routeradmin connected from using console on rtr1
    sysadmin-vm:0_RP0# crypto?
    sysadmin-vm:0_RP0# conf
    Fri Jan 5 15:56:14.409 UTC
    Entering configuration mode terminal
    sysadmin-vm:0_RP0(config)# crypto ?
    % Invalid input detected at '^' marker.
    sysadmin-vm:0_RP0(config)# exit
    Fri Jan 5 15:56:26.719 UTC
    sysadmin-vm:0_RP0# exit
    Fri Jan 5 15:56:27.838 UTC
  • Even though the command line didn’t tell me it was available, I tried variations where I specify the username the key is supposed to be associated with, since that only makes sense and seems to exist on other IOS XR platforms, i.e. “crypto key import auth username routeradmin” but any variation of that that just gives syntax error.  That apparently is the correct thing to do on 9k platform though.
  • Tried multiple off-device paths including ftp:// and tftp:// for the crypto key import location, all resulted in “Cannot execute the command : Invalid argument”
  • Happened to “show run” while in ‘admin’ mode, and noticed this:
    aaa authentication users user routeradmin
     uid 9000
     gid 100
     password $1$xxxx/
     ssh_keydir /var/confd/homes/routeradmin/.ssh
     homedir /var/confd/homes/routeradmin

    Hmm well that’s interesting, a linux style home directory and .ssh subdirectory. Maybe I can get my key in there?

    sysadmin-vm:0_RP0# dir /var
    Fri Jan 5 16:52:24.466 UTC
    total 40
    8380 lrwxrwxrwx 1 12 Dec 13 03:52 tmp -> volatile/tmp
    8375 lrwxrwxrwx 1 6 Dec 13 03:52 run -> ../run
    8376 drwxr-xr-x 5 4096 Dec 13 03:52 spool
    5951 lrwxrwxrwx 1 9 Dec 13 04:04 lock -> /run/lock
    8304 drwxr-xr-x 17 4096 Jan 5 16:50 lib
    8302 drwxr-xr-x 3 4096 Dec 13 03:52 db
    8367 drwxr-xr-x 2 4096 Dec 13 03:52 local
    17011 drwxr-xr-x 4 80 Dec 13 04:04 volatile
    16878 drwxr-xr-x 2 4096 Dec 13 04:04 empty
    16879 drwxr-xr-x 2 4096 Dec 13 04:04 dhclient
    8296 drwxr-xr-x 2 4096 Dec 13 03:52 backups
    8373 drwxr-xr-x 3 4096 Dec 13 03:52 opt
    8297 drwxr-xr-x 6 4096 Dec 13 04:04 cache
    2 drwxr-xr-x 15 4096 Jan 5 16:50 log
    2453228 kbytes total (1411980 kbytes free)

    Cute, /var/confd/ doesn’t actually exist.  Let’s create it:

    sysadmin-vm:0_RP0# mkdir /var/confd
    Fri Jan 5 16:45:49.666 UTC
    mkdir : /var/confd : Path outside filesystem disallowing access

    ROFL, how hard does it need to be to get key-based auth working?

At this point I gave up and opened a ticket.  TAC admitted the docs are wrong and/or misleading on two topics.  The first is that if importing from local device, you need the device name, a full path to the file, even if it would work for other commands, is not sufficient.  Second, the docs state that a 2048-bit key is the largest you can use, but I’ve demonstrated that is wrong because my key is 4096-bit.  I didn’t delve into why the remote copy wasn’t working since I already had my key on the device, so perhaps there’s a workaround to that as well, but tftp/sftp/ftp copying was broken for me.  Here’s what worked:

scp mykey.bin routeradmin@

RP/0/RP0/CPU0:rtr1#crypto key import authentication rsa disk0:/mykey.bin
Sat Jan 6 17:15:27.422 UTC
RP/0/RP0/CPU0:rtr1#show crypto key authentication rsa
Sat Jan 6 17:16:36.829 UTC
Key label: routeradmin
Type : RSA public key authentication
Size : 4096
Imported : 17:15:27 UTC Sat Jan 06 2018
Data :....

There are three annoying aspects of this setup, as opposed to IOS:

  1. Any given user can only have one ssh key.  If you try to install a second one for yourself, you’ll get this:
    RP/0/RP0/CPU0:rtr-1#crypto key import authentication rsa disk0:/key2.bin
    Sat Jan 6 17:19:09.598 UTC
    % You already have RSA keys defined for routeradmin
    Do you really want to replace them? [yes/no]: no
  2. With that limitation, this also means your setup staff member can’t import other users’ keys.  With such a limitation, now you’re left with the choice of adding users with known passwords so the setup person can log in as each person to import their key, then have those people change the passwords later, or, every network admin has to log on to the new device to import their own key before we lock the device down against password-based access.
  3. Who knows where the keys are actually stored.  They do not show up in the running config, nor does the /var/confd/ directory exist in admin mode, where that config seems to suggest the key should be.  This means you can’t easily audit what keys are on a device unless you have a task that logs in and runs ‘show crypto key authentication rsa’ and not only looks at the usernames but also looks at the Data field, which is more complicated than just running something like rancid against your device configs daily and looking for the IOS key data:
    ip ssh pubkey-chain
    username routeradmin
    key-hash ssh-rsa 4A3xxxx

Leave a Reply

Your email address will not be published. Required fields are marked *