Here’s my play by play for installing a Thales nShield Connect network-based HSM into a CentOS linux environment:
- Edit your ~/.bash_profile to include the following additional directory in your path and then re-exec bash: /opt/nfast/bin
- Get the Thales nShield client software from Thales support. They should end up providing you with a zip file that has an iso file inside it (?!), which you then mount to get at tar files. It will be named something like SecWorld-linux64-user-12.10.01.zip with SecWorld-linux64-user-12.10.01.iso.
- Mount the ISO locally, such as
mount -o loop /root/SecWorld-linux64-user-12.10.01.iso /mnt
or burn it to an actual CD/DVD since we already have this old school theme going.
- This next part is weird. You need to be in the root directory, aka /, because the tar files have the expectation you’ll be extracting them from that point so the files end up in the correct places. You will likely need to extract more than one tar file depending on which components you need. Consult the nShileld Connect Installation Guide for the list of the components. I’m doing straight OpenSSL CHIL interface, but also want to use the random number generator (requires PKCS#11), so I’m going to extract the following, relative to my mount point of the ISO:
- Hardware Support (mandatory even for non-PCI nShilelds): linux/libc6_11/amd64/nfast/hwsp/agg.tar
- Core Tools: linux/libc6_11/amd64/nfast/ctls/agg.tar
- Firmware Files: linux/libc6_11/amd64/nfast/nhfw/agg.tar
- CHIL interface: linux/libc6_11/amd64/nfast/hwcrhk/gnupg.tar AND linux/libc6_11/amd64/nfast/hwcrhk/user.tar
- SNMP agent: linux/libc6_11/amd64/nfast/ncsnmp/user.tar
- PKCS#11 Library: linux/libc6_11/amd64/nfast/pkcs11/user.tar
- Switch to /opt/nfast/sbin and run ./installI was expecting this to be a disaster as the experience has been a little kludgy but it ran without issue:
/opt/nfast/sbin# ./install ---- Stopping any nCipher servers ---- No nCipher init scripts installed. ---- Cleaning up any old install ---- No nCipher components requiring cleanup found. ---- Installing ---- -- Running install fragment 10nfastug Checking for user 'nfast' in group 'nfast' Creating nfast group. Creating nfast user. useradd: warning: the home directory already exists. Not copying any file from skel directory into it. Checking user 'nfast' is in correct group 'nfast' users created correctly -- Running install fragment 15makefiles Setting up directories. Making default config file. Making default cardlist file -- Running install fragment 45drivers Unloading old nCipher PCI nfp driver. Checking for PCI nfp hardware. Warning: No suitable pre-built PCI driver available. No nCipher PCI nfp devices found. Installing startup scripts for 'drivers'. Not linking in init scripts or loading drivers. -- Running install fragment 50hardserver Making privconn setuid and root. Configuring hardserver privileges. cap_net_bind_service capability set on hardserver file. ls: cannot access /dev/nfastpci*: No such file or directory Installing startup scripts for 'hardserver'. Linking in init scripts Warning: Installed, but no directly attached hardware was found. If you have an nCipher PCI card , re-run 'install' script with hardware attached, or with '-d' option, or consult nCipher support. Starting nCipher 'hardserver' server process. waiting for nCipher server to become operational ... nCipher server now running -- Running install fragment 60cmdadp -- Running install fragment 70edgecfg -- Running install fragment 90ncsnmpd Checking for user 'ncsnmpd' in group 'ncsnmpd' Creating ncsnmpd group. Creating ncsnmpd user. useradd: warning: the home directory already exists. Not copying any file from skel directory into it. Checking user 'ncsnmpd' is in correct group 'ncsnmpd' users created correctly Installing startup scripts for 'ncsnmpd'. Creating wait script for 'ncsnmpd' Linking in init scripts Starting nCipher 'ncsnmpd' server process. 'ncsnmpd' server now running ---- Installation complete ----
- You should be able to run the ‘enquiry’ command now, which will be in your new path:
# enquiry Server: enquiry reply flags none enquiry reply level Six serial number mode operational version 3.21.3 speed index 0 rec. queue 2..50 level one flags none version string 3.21.3cam7, checked in XXXX Wed Jul 16 08:38:45 2008 level two flags none max. write size 8192 level three flags none level four flags ServerHasPollCmds ServerHasLongJobs ServerHasCreateClient module type code 0 product name nFast server device name EnquirySix version 4 impath kx groups feature ctrl flags none features enabled none version serial 0 remote server port 9004
- The Thales nShield Connect needs a Remote File System (RFS) server where the nShield will store and access critical files. This is typically recommended to be a client that is also participating in transaction processing, but can also be a standalone system not running the full nShield client. I personally recommend a third option. Have an additional system that is configured identically to one of your transactional servers, with full nShield client and RFS, but is not sent any transactions. The reason for this is that if a transactional client is also the RFS for your nShield Connects, and even if you have redundant nShield Connects, you will still take a service affecting outage when upgrading firmware. If you use a third system that is running the full client install, you can upgrade your standby nShield Connect first, push config changes to your active clients, they begin using it, then upgrade what is now the standby nShield Connect, before switching back. You can’t do this if your RFS is also processing transactions, nor can you do this is you have a system being solely the RFS without the normal full client install.The system we just installed the full client software on is set to be an RFS as-is.
- In lieu of the above, your system is now listening for connections on all interfaces to port 9004. I recommend adjusting the relevant firewall settings on the server to limit that to just the IP addresses of nShield Connects.
- You should be able to hook a USB keyboard to your nShield Connect now, unless you’re a sadist and want to enter all the settings using four menu buttons. You’ll want to enter the relevant IPv4/IPv6 addresses, gateway, subnet mask.
- From the client side, and you ONLY do this if your network is secure between the client and the nShield Connect, run the ‘anonkneti’ utility against the IP of your nShield Connect; for example:
anonkneti 192.0.2.1
It will spit back something that looks like this (don’t worry, this isn’t from a real nShield):
A285-4F5A-7500 2418ec85c86027eb2d5959fef35edc5e1b3b698f
which is your unit’s electronic serial number followed by a hash specific to your unit.
- Now you’ll use the rfs-setup utility to initialize the RFS on this specific client and permit your nShield to connect to it:
# rfs-setup --force 192.0.2.1 A285-4F5A-7500 2418ec85c86027eb2d5959fef35edc5e1b3b698f Adding read-only remote_file_system entries Ensuring the directory /opt/nfast/kmdata/local exists Adding new writable remote_file_system entries Ensuring the directory /opt/nfast/kmdata/hsm-A285-4F5A-7500 exists Ensuring the directory /opt/nfast/femcerts exists Ensuring the directory /opt/nfast/kmdata/hsm-A285-4F5A-7500/features exists Ensuring the directory /opt/nfast/kmdata/hsm-A285-4F5A-7500/config exists Ensuring the directory /opt/nfast/log/hsm-A285-4F5A-7500 exists Saving the new config file and configuring the hardserver Done
- Enroll this client to the HSM (with unprivileged command access):
# nethsmenroll 192.0.2.1 A285-4F5A-7500 2418ec85c86027eb2d5959fef35edc5e1b3b698f OK configuring hardserver's nethsm imports
If you need privileged access, add a –privileged flag before the IP. You must also set the privileged option for this client from the front panel.
- Now, just for the sake of mentioning it, you need to be very very sure you always have a backup copy of the /opt/nfast/kmdata/ directory structure from this particular client since it is the RSM and contains critical data specific to the security world you’ll be creating; i.e. your encryption data. If your HSM dies, or you add more, you need your administrator card set (smart cards) and the kmdata directory, to add the same security world to another device.
- From the front panel of your Connect, set it to auto update config and give this system’s address. This will let you make some types of changes to the HSM config remotely. Otherwise, for added security, don’t do that, and you have to initiate a pull from the front panel.
- Enroll this client to the Connect via front panel. If you use nToken for auth, you can enable that as well, and confirm your nToken credentials spit out by
ntokenenroll -H
on the client side. If not, choose no, and add.
- The ‘enqiry’ command should spit out details of your Connect now:
# enquiry -m 1 Module #1: enquiry reply flags UnprivOnly enquiry reply level Six serial number A285-4F5A-7500 mode operational version 2.55.1 speed index 4512 rec. queue 19..152 level one flags Hardware HasTokens version string 2.55.1cam7 built on Jan 01 2015 10:10:10, 3.22.2cam3 checked in XXXX Mon Jun 16 11:19:23 2008 level two flags none max. write size 8192 level three flags KeyStorage level four flags OrderlyClearUnit HasRTC HasNVRAM HasNSOPermsCmd ServerHasPollCmds FastPollSlotList HasSEE HasKLF HasShareACL HasFeatureEnable HasFileOp HasPCIPush HasKernelInterface HasLongJobs ServerHasLongJobs AESModuleKeys NTokenCmds JobFragmentation LongJobsPreferred Type2Smartcard ServerHasCreateClient HasInitialiseUnitEx module type code 7 product name device name Rt2 EnquirySix version 6 impath kx groups DHPrime1024 DHPrime3072 feature ctrl flags LongTerm features enabled StandardKM version serial 26 connection status OK connection info esn = ; addr = INET/192.0.2.1/9004; ku hash = XXXX, mech = Any; time-limit = 24h; data-limit = 8MB image version 12.14.9cam6 max exported modules 3 rec. LongJobs queue 18 SEE machine type PowerPCSXF supported KML types DSAp1024s160 DSAp3072s256 using impath kx grp DHPrime3072 hardware status unsupported firmware
- Notice the “unsupported firmware” in the above? Your nShield probably came with older firmware than the client software. No problem; you extracted what you need earlier, so find it using your RFS server’s IP (not the nShield):
nethsmadmin --list-images 192.0.2.2 Initiating RFS nethsm image check on 192.0.2.2 Checking the nethsm-firmware directory on the RFS. nethsm-firmware/12.23.1cam3/nCx3N.nff nethsm-firmware/12.23.1cam3-fips/nCx3N.nff Images were successfully found on the RFS (192.0.2.2).
- Now, if you enabled remote upgrade via the front panel of your nShield Connect, and are coming from a privileged client, upgrade it:
nethsmadmin -i nethsm-firmware/12.23.1cam3/nCx3N.nff
Otherwise, upgrade it from the front panel; it will be able to see the images on the RFS. A word of warning; it takes a LONG time to upgrade. You’ll be staring at a particular downloading file screen on the HSM for what feels like forever; don’t get antsy and pull the plug. Give it a solid 30 minutes. Additionally, after upgrade, the front panel interface will be VERY slow, and if you’re pressing, thinking it didn’t register, and pressing some more, you could easily perform an unintended operation, so be very very slow and patient with the front interface after upgrade. I suspect it’s doing something internally, because even after rebooting it like the user manual says, it’s still slow at the 60 minute post-upgrade mark. I’ll update later if it speeds back up.
- Time to create a security world. Put the HSM into pre-init mode via front panel, or privileged client:
nopclearfail -I -m
- Once it boots into that mode, enquiry will show ‘pre-initialization’ status. You can create the world now:
new-world -i -c DLf3072s256mRijndael --acs-quorum=1/1
This example creates a simple world using a cipher suite compliant with SP800-131 and having, and requiring, just one smart card in the admin key set. The utility will do some init stuff:
Create Security World: Module 2: 0 cards of 1 written Module 2 slot 0: empty
and you won’t realize it, but it wants you to input the card in the front of the unit now. Do that, then enter a pass phrase. In my ssh session, since they must use some kind of shell script loop for input, it was flashing like crazy and was very difficult to see what it wanted me to do. It was then showing “(Wait…)” for a few minutes after typing the pass phrase. I ended up giving up, reboot the client computer, tried again, everything went smoothly:
# new-world -i -c DLf3072s256mRijndael --acs-quorum=1/1 Create Security World: Module 1: 0 cards of 1 written Module 1 slot 0: empty Module 1 slot 0: unknown card Module 1 slot 0:- passphrase specified - overwriting card Card writing complete. security world generated on module #0
- If you add a second HSM and need to copy your security world to it:
new-world -l -m 2
where -m 2 specifies the 2nd module from the enquiry command.
Connecting additional HSM clients to both the HSM(s) and RFS server:
- Perform nearly the same nfast software setup on the new client with the exception of the SNMP and firmware file extraction since you’d only be using those on your RFS server where you’d be performing monitoring and upgrade operations against the HSM(s). Run the ./install after extraction. Run ‘enquiry’ after install to ensure this local client is good to go.
- Time to connect the client to the HSMs. Add this client’s IP to the HSM via front panel, or, beyond the scope of this article, you could copy the file /opt/nfast/kmdata/hsm-ESN/config/config on your RFS server to ‘config.new’ in the same directory, add the new clients to the hs_clients section, then use the cfg-pushnethsm command to push the updated config file to your HSMs. Just be careful to not screw it up because pushing a bad config to the HSM could leave your RFS without access to it and requiring a trip to the data center to re-authorize the RFS server as a client. If the push is successful, the HSM will also immediately write back the original config file. You should compare them with diff to ensure it happened successfully. The command would look something like
cfg-pushnethsm --address=192.0.2.1 -f -n /opt/nfast/kmdata/hsm-ESN/config/config.new
- With the client’s IP authorized on the HSM, run this from the new client: nethsmenroll 192.0.2.1 `anonkneti 192.0.2.1`
This will enroll the client to the HSM located at IP 192.0.2.1, using the serial number generated by the command “anonkneti 192.0.2.1”. Only do this if the client and HSM talk across a known-secure network. You can enter the serial info manually otherwise. - On your RFS server, authorize the client IP’s to connect to it without write authorization:
rfs-setup –gang-client –write-noauth 192.0.2.10where 192.0.2.10 is the IP of the new RFS client #1. - On the new RFS client #1, give it unauthenticated access to the RFS via:
rfs-sync –setup –no-authenticate 192.0.2.2
where the 192.0.2.2 is the IP of the RFS server. - If everything was done correctly, you should now be able to run this on the new client: rfs-sync -U
and it will output the fact that it’s pulling down the config data for the HSM’s, the security world and any previously generated hardware keys located on the RFS server. - Your client is now connected to the HSM’s and ready to perform operations involving HSM keys.
Good comprehensive guide. Thanks.
My RFS setup always fails… have you run across this issue. everything else works.. Enquiry cmd shows module operational and ok. However, whenever, i try to list the local client ip as the rfs on the HSM.. it spits out error failed to connect to RFS… any suggestion would be great
Hi Michael, any chance your RFS is RHEL/CentOS 7? That would have firewalld enabled by default and would deny the connection even if the software were listening. That’s the only thing I can think of off-hand; rfs-setup has always worked for me.
yes… the operating systems firewall was the culprit. Thanks again Mom :)
Thanks for the article. how do we uninstall SNMP agent that installed on Ubuntu Linux?