Java and SSL Certificates

Overview

I have a Linux box at home, running Apache 2.2, that I use to archive pictures. I use an application called Gallery as a front end to organize and view the photos. I'm using a Java application called Gallery Remote to upload pictures to the server. I've also added SSL encryption so that the username and password, to access the site, are not sent in the clear.

Problem and Solution

The problem was that Gallery Remote wasn't able to connect to the server. It seemed to be having problems with the SSL certificate I had on the web server. I was using a self-signed SSL certificate, so that was definitely possible. I checked out the SSL certificate and found that it was expired. I regenerated a new certificate using the instructions on the Apache Website. The relevant text is shown below.

How do I create a self-signed SSL Certificate for testing purposes?

1. Make sure OpenSSL is installed and in your PATH.


2. Run the following command, to create server.key and
   server.crt files:
   $ openssl req -new -x509 -nodes -out server.crt
-keyout server.key
   These can be used as follows in your httpd.conf
   file:
   

                SSLCertificateFile    /path/to/this/server.crt
                SSLCertificateKeyFile /path/to/this/server.key
    

   
   


3. It is important that you are aware that this
   server.key does not have any passphrase.
   To add a passphrase to the key, you should run the following
   command, and enter & verify the passphrase as requested.
   

   $ openssl rsa -des3 -in server.key -out
server.key.new
$ mv server.key.new server.key

   
   Please backup the server.key file, and the passphrase
   you entered, in a secure location.
   

After restarting the webserver, I was still having problems with Gallery Remote. I then found out that Java has it's own repository of trusted SSL certificates. My SSL certificate was a self-signed certificate, so it definitely wasn't in the default SSL Certificate trust list. One method of adding the certificate is by going through the Java control panel. Another method is to add it through the command line. This was described on the Gallery Remote FAQ page. The relevant text is shown below.

Using HTTPS

You can use https:// URLs with Gallery Remote to connect to secured web sites. This functionality is only available on Java 1.4 and later. If the site you are attempting to connect to uses a server certificate that is not certified by a trusted certificate authority, Gallery Remote will be unable to connect. If this happens, you will need to add the site's certificate to the Java registry of trusted certificates:

For Windows:

1. Go to the site with Internet Explorer
   
2. Go to menu File>Properties
   
3. In the Properties window, click Certificates
   
4. On the Details tab, click Copy to File...
   
5. In the wizard, select DER-encoded X.509 certificate and save it to a
   file
   
6. Open a console window (cmd.exe)
   
7. Type the following command-line:
   

     keytool -import -trustcacerts -file path_to_cer_file 
-keystore %JAVA_HOME%/jre/lib/security/cacerts -alias arbitrary_name

You'll be prompted for the store password, which by default is

changeit

I used that and it worked. It's interesting to note that Java uses it's own keystore and that there's a default password used if using the command line.

Allowing CSA Management Center to access WSUS server

The pre-configured CSA 6.0 policy for the CSA Management Center does not allow for connectivity to the WSUS server. Rule 269 blocks the access as shown below



You'll notice that the screenshot is from the events shown on the agent GUI on the management center. This is because rule 269 does not log by default. Because of this, the denied packets do not show up in the management center event logs. In order to view the logs on the management center, you would need to do one of two things:

Explicitly turn on logging for rule 269


Enable log overrides for a particular group


Once the denied rule shows up in the management center event logs, the denied events can be viewed on the management center. This helps with the troubleshooting process.

The problem is that rule 269 blocks all network traffic not explicitly allowed by another rule. Since rule 269 applies to the "CSA MC Network Security Module", it only affects the management center. This is why WSUS updates work fine with the pre-configured server and desktop rules. In those policies, there is no rule explicitly blocking network traffic. The default action is to allow traffic, so the WSUS update traffic is allowed for desktops and servers.

There are a number of ways to fix the problem for the management center. The easiest method is to use the Wizard in the event log entry for rule 269. The Wizard provides a method of easily creating an exception rule for the specific traffic that was blocked.

The first step is to locate the rule 269 event log entry and click on the Wizard link. This is shown in the red oval in the diagram below


The next step is to click on the "Allow Operation" radio button, provide a justification and click "Finish". This is shown below.


After "Finish" is clicked, the necessary variables and rule are created. The next step is to generate the policy to install the rules. The diagram below shows the variables and rules that will be generated.


After the rule generation, there should now be an exception rule that allows access to the WSUS server to get Microsoft updates. This is shown below.


A closer inspection of the rule shows that it is a granular rule only allowing executable "svchost.exe -k netsvc" to talk to the WSUS server, as a client, on port 80/tcp. This is shown below.


To verify that the rule is really working, you can temporarily turn on logging for the exception rule. This is shown below.


A reboot of the management center should kick off the WSUS update check again. Once this is completed, something similar to the following should be in the event log


After verifying the success of the exception rule, make sure to turn off logging on the exception rule and any other logging that was turned on for troubleshooting purposes above.

Cisco CSA 6.0 Upgrade Note

When upgrading to CSA 6.0, most of the effort is concentrated on upgrading the management center. Cisco provides fairly well documented instructions in their installation guide. The part they don't talk about enough is the CSA Agent upgrade to 6.0.

One problem I ran into has to do with upgrading clients running Windows XP SP3. According to the CSA 5.2 release notes, CSA 5.2 only supports Windows XP SP 0, 1, or 2. Of course, you can surmise that they just forgot to update their release notes with XP SP3 support, since the documentation is dated 4/2/07 and Windows XP SP3 came out on 5/6/08. Unfortunately, that is not true. This can be seen when viewing the "Host Identification" information under "Systems > Hosts > [hostname]". I've shown an example below. The "unsupported" information is shown in red.



Despite this screen, the CSA 5.2 Agent works fine after the upgrade to XP SP3. The big problem comes when the Management Center is upgraded to version 6.0 and you try to do the scheduled software update to upgrade all the agents to 6.0. This does not work. The normal process is to

1. Access "Systems > Software Updates > Scheduled Software Updates"
2. Create a new Item that schedules the update for a particular group
3. The agents check in with the Management Center, download the update, and install

The problem is that the agents never download the update. The "System > Hosts > [hostname]" page always shows that the software version as "Agent is running the latest software" instead of "Update Available". Both of these screenshots are shown below.





The only workaround is to create a new CSA 6.0 agent kit and push the new agent kit to all the users via your normal application installation mechanism (Altiris, SMS,...)

Cisco NAC Manager HA log files

The NAC Manager documentation provides a number of logs that can be viewed to troubleshoot various issues. The only problem is that when a problem occurs it would be really nice to have a reference showing what good log output looks like. That's what I'd like to share here. Hopefully this will help someone troubleshooting an issue.

The log files below show the log files on a standby NAM when it becomes active
/perfigo/control/tomcat/logs/localhost_log..txt
2008-11-19 14:22:58 StandardHost[localhost]: Removing web application at context path /admin
2008-11-19 14:22:58 StandardHost[localhost]: Removing web application at context path
2008-11-19 14:23:03 WebappLoader[/admin]: Deploying class repositories to work directory /perfigo/control/tomcat/work/Standalone/localhost/admin
2008-11-19 14:23:03 WebappLoader[/admin]: Deploy JAR /WEB-INF/lib/jsf_hack_tld.jar to /perfigo/control/tomcat/webapps/admin/WEB-INF/lib/jsf_hack_tld.jar
2008-11-19 14:23:04 ContextConfig[/admin]: Configured an authenticator for method NONE
2008-11-19 14:23:04 PersistentManager[/admin]: Seeding random number generator class java.security.SecureRandom
2008-11-19 14:23:04 PersistentManager[/admin]: Seeding of random number generator has been completed
2008-11-19 14:23:04 PersistentManager[/admin]: No Store configured, persistence disabled
2008-11-19 14:23:22 StandardWrapper[/admin:default]: Loading container servlet default
2008-11-19 14:23:22 StandardWrapper[/admin:invoker]: Loading container servlet invoker
2008-11-19 14:23:22 HostConfig[localhost]: Deploying web application directory ROOT
2008-11-19 14:23:22 StandardHost[localhost]: Installing web application at context path from URL file:/perfigo/control/tomcat/normal-webapps/ROOT
2008-11-19 14:23:22 WebappLoader[]: Deploying class repositories to work directory /perfigo/control/tomcat/work/Standalone/localhost/_
2008-11-19 14:23:22 StandardManager[]: Seeding random number generator class java.security.SecureRandom
2008-11-19 14:23:22 StandardManager[]: Seeding of random number generator has been completed
2008-11-19 14:23:23 StandardWrapper[:default]: Loading container servlet default
2008-11-19 14:23:23 StandardWrapper[:invoker]: Loading container servlet invoker
2008-11-19 14:23:23 HostConfig[localhost]: Deploying web application directory upload
2008-11-19 14:23:23 StandardHost[localhost]: Installing web application at context path /upload from URL file:/perfigo/control/tomcat/normal-webapps/upload
2008-11-19 14:23:23 WebappLoader[/upload]: Deploying class repositories to work directory /perfigo/control/tomcat/work/Standalone/localhost/upload
2008-11-19 14:23:23 StandardManager[/upload]: Seeding random number generator class java.security.SecureRandom
2008-11-19 14:23:23 StandardManager[/upload]: Seeding of random number generator has been completed
2008-11-19 14:23:23 StandardWrapper[/upload:default]: Loading container servlet default
2008-11-19 14:23:23 StandardWrapper[/upload:invoker]: Loading container servlet invoker
2008-11-19 14:23:23 HostConfig[localhost]: Deploying web application directory wlan
2008-11-19 14:23:23 StandardHost[localhost]: Installing web application at context path /wlan from URL file:/perfigo/control/tomcat/normal-webapps/wlan
2008-11-19 14:23:23 WebappLoader[/wlan]: Deploying class repositories to work directory /perfigo/control/tomcat/work/Standalone/localhost/wlan
2008-11-19 14:23:23 ContextConfig[/wlan]: Configured an authenticator for method NONE
2008-11-19 14:23:23 StandardManager[/wlan]: Seeding random number generator class java.security.SecureRandom
2008-11-19 14:23:23 StandardManager[/wlan]: Seeding of random number generator has been completed
2008-11-19 14:23:23 StandardWrapper[/wlan:default]: Loading container servlet default
2008-11-19 14:23:23 StandardWrapper[/wlan:invoker]: Loading container servlet invoker
2008-11-19 14:23:23 HostConfig[localhost]: Deploying web application directory packages
2008-11-19 14:23:23 StandardHost[localhost]: Installing web application at context path /packages from URL file:/perfigo/control/tomcat/normal-webapps/packages
2008-11-19 14:23:23 WebappLoader[/packages]: Deploying class repositories to work directory /perfigo/control/tomcat/work/Standalone/localhost/packages
2008-11-19 14:23:23 StandardManager[/packages]: Seeding random number generator class java.security.SecureRandom
2008-11-19 14:23:23 StandardManager[/packages]: Seeding of random number generator has been completed
2008-11-19 14:23:24 StandardWrapper[/packages:default]: Loading container servlet default
2008-11-19 14:23:24 StandardWrapper[/packages:invoker]: Loading container servlet invoker
2008-11-19 14:23:24 HostConfig[localhost]: Deploying web application directory download
2008-11-19 14:23:24 StandardHost[localhost]: Installing web application at context path /download from URL file:/perfigo/control/tomcat/normal-webapps/download
2008-11-19 14:23:24 WebappLoader[/download]: Deploying class repositories to work directory /perfigo/control/tomcat/work/Standalone/localhost/download
2008-11-19 14:23:24 StandardManager[/download]: Seeding random number generator class java.security.SecureRandom
2008-11-19 14:23:24 StandardManager[/download]: Seeding of random number generator has been completed
2008-11-19 14:23:24 StandardWrapper[/download:default]: Loading container servlet default
2008-11-19 14:23:24 StandardWrapper[/download:invoker]: Loading container servlet invoker

/var/log/ha-log
heartbeat: 2008/11/19_14:22:58 info: Received shutdown notice from 'camanager1'.
heartbeat: 2008/11/19_14:22:58 info: Resources being acquired from camanager1.
heartbeat: 2008/11/19_14:22:58 info: acquire all HA resources (standby).
heartbeat: 2008/11/19_14:22:58 info: No local resources [/usr/lib64/heartbeat/ResourceManager listkeys camanager2] to acquire.
heartbeat: 2008/11/19_14:22:58 info: Acquiring resource group: camanager1 x.x.x.x controlsmart
heartbeat: 2008/11/19_14:22:58 info: Running /etc/ha.d/resource.d/IPaddr x.x.x.x start
heartbeat: 2008/11/19_14:22:58 info: /sbin/ifconfig eth0:0 x.x.x.x netmask 255.255.255.0 broadcast 172.31.31.255
heartbeat: 2008/11/19_14:22:58 info: Sending Gratuitous Arp for x.x.x.x on eth0:0 [eth0]
heartbeat: 2008/11/19_14:22:58 /usr/lib64/heartbeat/send_arp -i 1010 -r 5 -p /var/lib/heartbeat/rsctmp/send_arp/send_arp-x.x.x.x eth0 x.x.x.x auto x.x.x.x ffffffffffff
heartbeat: 2008/11/19_14:22:58 info: Running /perfigo/control/bin/controlsmart start
heartbeat: 2008/11/19_14:23:02 info: all HA resource acquisition completed (standby).
heartbeat: 2008/11/19_14:23:02 info: Standby resource acquisition done [all].
heartbeat: 2008/11/19_14:23:02 info: Running /etc/ha.d/rc.d/status status
heartbeat: 2008/11/19_14:23:04 info: Taking over resource group x.x.x.x
heartbeat: 2008/11/19_14:23:04 info: Acquiring resource group: camanager1 x.x.x.x controlsmart
heartbeat: 2008/11/19_14:23:04 info: Running /perfigo/control/bin/controlsmart start
heartbeat: 2008/11/19_14:23:04 info: /usr/lib64/heartbeat/mach_down: nice_failback: foreign resources acquired
heartbeat: 2008/11/19_14:23:04 info: mach_down takeover complete.
heartbeat: 2008/11/19_14:23:04 info: mach_down takeover complete for node camanager1.
heartbeat: 2008/11/19_14:23:14 WARN: node camanager1: is dead
heartbeat: 2008/11/19_14:23:14 info: Dead node camanager1 gave up resources.
heartbeat: 2008/11/19_14:23:14 info: Link camanager1:eth1 dead.

These logs show the story when a NAM starts as the active NAM and then "service perfigo stop" is entered to turn off the NAC service
/perfigo/control/tomcat/logs/localhost_log..txt
2008-11-19 14:22:55 StandardHost[localhost]: Removing web application at context path /admin
2008-11-19 14:22:55 StandardHost[localhost]: Removing web application at context path /upload
2008-11-19 14:22:55 StandardHost[localhost]: Removing web application at context path /download
2008-11-19 14:22:55 StandardHost[localhost]: Removing web application at context path /packages
2008-11-19 14:22:55 StandardHost[localhost]: Removing web application at context path /wlan
2008-11-19 14:22:55 StandardHost[localhost]: Removing web application at context path

/var/log/ha-debug
heartbeat: 2008/11/19_14:22:54 info: Heartbeat shutdown in progress. (4516)
heartbeat: 2008/11/19_14:22:54 info: Giving up all HA resources.
heartbeat: 2008/11/19_14:22:54 info: Releasing resource group: camanager1 x.x.x.x controlsmart
heartbeat: 2008/11/19_14:22:54 info: Running /perfigo/control/bin/controlsmart stop
heartbeat: 2008/11/19_14:22:58 info: Running /etc/ha.d/resource.d/IPaddr x.x.x.x stop
heartbeat: 2008/11/19_14:22:58 info: /sbin/route -n del -host x.x.x.x
heartbeat: 2008/11/19_14:22:58 info: /sbin/ifconfig eth0:0 down
heartbeat: 2008/11/19_14:22:58 info: IP Address x.x.x.x released
heartbeat: 2008/11/19_14:22:58 info: All HA resources relinquished.
heartbeat: 2008/11/19_14:22:59 info: killing HBREAD process 4521 with signal 15
heartbeat: 2008/11/19_14:22:59 info: killing HBFIFO process 4519 with signal 15
heartbeat: 2008/11/19_14:22:59 info: killing HBWRITE process 4520 with signal 15
heartbeat: 2008/11/19_14:22:59 info: Core process 4519 exited. 3 remaining
heartbeat: 2008/11/19_14:22:59 info: Core process 4520 exited. 2 remaining
heartbeat: 2008/11/19_14:22:59 info: Core process 4521 exited. 1 remaining
heartbeat: 2008/11/19_14:22:59 info: Heartbeat shutdown complete.

This show the status after "service perfigo start" is entered with another NAM active
[root@camanager1 logs]# service perfigo start
Starting High-Availability services:
[ OK ]
Please wait while bringing up service IP.
Heartbeat service is running.
Service IP is up on the peer node.
Stopping postgresql service: [ OK ]
Starting postgresql service: [ OK ]
DROP DATABASE
CREATE DATABASE
DROP DATABASE
CREATE DATABASE
Database synced
[root@camanager1 logs]#

/perfigo/control/tomcat/logs/localhost_log..txt
2008-11-19 14:30:14 WebappLoader[/admin]: Deploying class repositories to work directory /perfigo/control/tomcat/work/Standalone/localhost/admin
2008-11-19 14:30:14 WebappLoader[/admin]: Deploy JAR /WEB-INF/lib/jsf_hack_tld.jar to /perfigo/control/tomcat/webapps/admin/WEB-INF/lib/jsf_hack_tld.jar
2008-11-19 14:30:14 ContextConfig[/admin]: Configured an authenticator for method NONE
2008-11-19 14:30:14 PersistentManager[/admin]: Seeding random number generator class java.security.SecureRandom
2008-11-19 14:30:14 PersistentManager[/admin]: Seeding of random number generator has been completed
2008-11-19 14:30:14 PersistentManager[/admin]: No Store configured, persistence disabled
2008-11-19 14:30:15 StandardWrapper[/admin:default]: Loading container servlet default
2008-11-19 14:30:15 StandardWrapper[/admin:invoker]: Loading container servlet invoker
2008-11-19 14:30:15 HostConfig[localhost]: Deploying web application directory ROOT
2008-11-19 14:30:15 StandardHost[localhost]: Installing web application at context path from URL file:/perfigo/control/tomcat/admin-webapps/ROOT
2008-11-19 14:30:15 WebappLoader[]: Deploying class repositories to work directory /perfigo/control/tomcat/work/Standalone/localhost/_
2008-11-19 14:30:15 StandardManager[]: Seeding random number generator class java.security.SecureRandom
2008-11-19 14:30:15 StandardManager[]: Seeding of random number generator has been completed
2008-11-19 14:30:16 StandardWrapper[:default]: Loading container servlet default
2008-11-19 14:30:16 StandardWrapper[:invoker]: Loading container servlet invoker

/var/log/ha-log
heartbeat: 2008/11/19_14:27:25 info: **************************
heartbeat: 2008/11/19_14:27:25 info: Configuration validated. Starting heartbeat 1.2.5
heartbeat: 2008/11/19_14:27:25 info: heartbeat: version 1.2.5
heartbeat: 2008/11/19_14:27:26 info: Heartbeat generation: 44
heartbeat: 2008/11/19_14:27:26 info: ucast: write socket priority set to IPTOS_LOWDELAY on eth1
heartbeat: 2008/11/19_14:27:26 info: ucast: trying to bind: eth1

heartbeat: 2008/11/19_14:27:26 info: ucast: bound send socket to device: eth1
heartbeat: 2008/11/19_14:27:26 info: ucast: try binding receive socket to device: eth1
heartbeat: 2008/11/19_14:27:26 info: ucast: could bind receive socket to device: eth1:fe00a8c0.
heartbeat: 2008/11/19_14:27:26 info: ucast: started on port 694 interface eth1 to 192.168.0.253
heartbeat: 2008/11/19_14:27:26 notice: Using watchdog device: /dev/watchdog
heartbeat: 2008/11/19_14:27:26 info: pid 19899 locked in memory.
heartbeat: 2008/11/19_14:27:26 info: Local status now set to: 'up'
heartbeat: 2008/11/19_14:27:27 info: pid 19902 locked in memory.
heartbeat: 2008/11/19_14:27:27 info: pid 19903 locked in memory.
heartbeat: 2008/11/19_14:27:27 info: pid 19904 locked in memory.
heartbeat: 2008/11/19_14:27:27 info: Link camanager2:eth1 up.
heartbeat: 2008/11/19_14:27:27 info: Status update for node camanager2: status active
heartbeat: 2008/11/19_14:27:27 info: Local status now set to: 'active'
heartbeat: 2008/11/19_14:27:27 info: remote resource transition completed.
heartbeat: 2008/11/19_14:27:27 info: remote resource transition completed.
heartbeat: 2008/11/19_14:27:27 info: Local Resource acquisition completed. (none)
heartbeat: 2008/11/19_14:27:27 info: Initial resource acquisition complete (T_RESOURCES(them))
heartbeat: 2008/11/19_14:27:27 info: Running /etc/ha.d/rc.d/status status

Essential Cisco NAC deployment tools

When deploying NAC there are a number of very useful tools that can help with implementing and troubleshooting. Here's a list of tools I've found useful

• Wireshark - This is an open source network protocol analyzer that allows you to see exactly what traffic is going across the wire. There's a "Follow TCP Stream" feature that allows you to see the entire stream of traffic for a session. One place this can be used is when looking for certificate CRL information being sent from a client to a CA. You'll be able to see the exact URL that is being used in an easy to read manner.


• LDAP Browser - This tool allows you to browse the LDAP tree to help determine what entries you should match on.


• Kerbtray - This is a one tool in a set of Microsoft resource kit tools that is meant for Windows 2003, but also works for Windows XP. This tool provides information about Kerberos authentication. This is invaluable for troubleshooting AD SSO issues.


• Camstudio - This is an open source video creation tool that you can use to create short video tutorials showing how NAC works. It can create an AVI or Flash file of your screen while you're demonstrating different NAC features. This can be a great tool for providing a visual representation of the NAC login process during end user training


• Irfanview - This is a great tool for editing screenshots