Monday, October 21, 2013

Out-Of-The-Box JMX over Firewall in Hotspot 7

Finally! A minor feature with big impact made it into the Hotspot 7 - probably as a result of the JRockit and Hotspot convergence efforts (as JRockit has had this feature for ages). The absence of this feature has been a big pain in the past, at least for people like me that are a bit obsessed with runtime monitoring, and effectively prevented one from using JMX in production-like environments.

The problem that all hotspot JVM versions until later Java7 releases had, was that it was not possible to define the RMI port used in the stubs returned by the JMX registry. It was possible (and still is) to define the JMX registry port via command-line arguments (-Dcom.sun.management.jmxremote.port). This allows you to connect to the JMX registry, lookup the actual RMI stub and download it (done by the JVM for you). The stub unfortunately was referring to a remote object running on a second port (aka ephemeral JMX port) which was not configurable and, to make things worse, was randomly picked upon JVM startup. This made it next to impossible (unless you open your firewall completely for inbound TCP connections to ports > 1024) to actually use JMX over a firewall. This (minor) problem forced you to either write your own JMX socket factory which needs to be initialized when the JVM process was started (e.g. using weblogic startup classes, special tomcat connectors, etc.) or forget about JMX in production.

In recent versions of the JVM (at least starting from 7u25) it is also possible to configure the second (ephemeral) port. You can (and probably would want to) set it to the same port as the registry thus requiring a firewall rule for one single TCP port. The new JVM argument is -Dcom.sun.management.jmxremote.rmi.port. To enable anonymous and unencrypted JMX connections over a firewall you need the following startup arguments (note how both ports are set and also set to the same value):

-Dcom.sun.management.jmxremote.port=2811 -Dcom.sun.management.jmxremote.rmi.port=2811 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false
and an inbound firewall rule for TCP port 2811.

Have fun!

PS. In case you are also interested in open-source and very flexible production monitoring of your JVMs you can have a look at graphite and metrics-sampler.

1 comment:

  1. Thanks for this useful info. If we have 2 different instances of java process on the same host, and we want to share the rmi port between them, is there a way to do that using default jmx agent ? I tried giving different values of jmxremote.port and same value for jmxremote.rmi.port but 2nd jvm didn't start up. Can you please explain what's going on inside ?

    ReplyDelete