Using SSL Wildcard certificates from a pfx file on FME Server (Tomcat)

UPDATE:

I just installed FME Server 2018 today, and the procedure is still more or less the same. To specifically choose pkcs12 as your keystone format use:

keytool -importkeystore -srckeystore c:\temp\my_keystore.pfx -srcstoretype pkcs12 -destkeystore c:\temp\tomcat.keystore -deststoretype pkcs12


-----


I recently installed an instance of FME Server (version 2015.1.3.1) on a Windows 2012R2 server.

The default installation of FME Server uses http so I decided to install the Wildcard SSL certificate we use in my organization to improve security some. Most of our servers are IIS so I only had a .pfx file accessible.

In Safes documentation library I found this description on how to configure FME Server for https:

http://docs.safe.com/fme/2016.0/html/FME_Server_Documentation/Content/AdminGuide/configuring_for_https.htm

This describes how to use a self signed certificate or regular CA-authorized certificate from a CSR. None of which applied to my exact need. So instead I created new JKS keystore by importing the .pfx-keychain with "keytool.exe" (the documentation says you need JDK, but I used the JRE binaries that comes with the FME Server installation).

C:\apps\FMEServer\Utilities\jre\bin>keytool -importkeystore -srckeystore c:\temp\my_keystore.pfx -srcstoretype pkcs12

Enter destination keystore password: 
Re-enter new password:
Enter source keystore password:
Entry for alias le-1234abcd-abcd-4444-a395-1234567890ab successfully imported.
Import command completed:  1 entries successfully imported, 0 entries failed or
cancelled

Next I went through Safe's documentation and altered these files accordingly:

<tomcatdir>/conf/server.xml
<tomcatdir>/conf/web.xml
<fmeserverdir>/server/fmeWebSocketConfig.txt
<fmeserverdir>/server/config/subscribers/websocket.properties
<fmeserverdir>/server/config/publishers/websocket.properties
<programdata>\Safe Software\FME Server\localization\publishers\websocket\publisherProperties.xml
<programdata>\Safe Software\FME Server\localization\subscribers\websocket\subscriberProperties.xml

Then I restarted the "FME Server Application Service" but there was no response from https. I checked <tomcatdir>/logs/catalina.<date>.log and found the following useful entries:

mar 19, 2016 6:25:44 AM org.apache.coyote.AbstractProtocol init
SEVERE: Failed to initialize end point associated with ProtocolHandler ["http-bio-443"]

java.io.IOException: Cannot recover key

mar 19, 2016 6:25:44 AM org.apache.catalina.core.StandardService initInternal
SEVERE: Failed to initialize connector [Connector[HTTP/1.1-443]]
org.apache.catalina.LifecycleException: Failed to initialize component [Connector[HTTP/1.1-443]]

I checked around and it turns out that Tomcat was not able to access my newly imported key because I had a different password set for my JKS keystore than what the imported keystore had. I mistakenly assumed that by importing the key from my existing .pfx-keystore it would inherit the password from my new local JKS keystore. Apparently not so. Anyway - by deleting my c:\users\<user>\.keystore file, running the keytool-importkeystore command and making sure both the destination keystore password AND the source keystore password matched I was able to get it running.

PS: After installing the certificate I had to run the post-configuration scripts again manually. It turned out that I wasn't able to edit service properties upon publishing workspaces to FME server:
http://docs.safe.com/fme/html/FME_Server_Documentation/Default.htm#AdminGuide/Post_Config_Steps.htm (you will have to go and manually set services back to https again after doing this though).

Thanks to the guys at Safe for their help.