October 2, 2018
About 4 minutes
Java JAR Modules Jigsaw

Loading JARs in Java 9+

Dynamically adding JAR files to the class path in a forward compatible way

A lot of Java-based applications ran into trouble with Java 9 because they assume that the system class loader is a URLClassLoader and they use this to dynamically load code (such as plug-ins). This article offers a simple solution to this problem. It requires minimal changes to existing code and has essentially the safe effect under the hood, so the odds of it introducing side effects are minimal.

What’s the problem?

A lot of people have wanted to be able to append new JARs to the class path dynamically as an application is running. Many of them also independently discovered that the system class loader, as a subclass of URLClassLoader, offers a tantalizing addURL method hidden just behind a protected attribute—an attribute that is easy enough to ignore with the reflection API (exception handling omitted):

public static void addToClassPath(File jarFile) throws IOException {
  addUrlMethod.invoke(addUrlThis, jarFile.toURI().toURL());
}

private static ClassLoader addUrlThis;
private static Method addUrlMethod;

static {
    addUrlThis = ClassLoader.getSystemClassLoader();
    addUrlMethod = URLClassLoader.class.getDeclaredMethod("addURL", URL.class);
    addUrlMethod.setAccessible(true);
}

This worked fine, because the system class loader has been a URLClassLoader since time immemorial. And although this was never documented as a fact, it was a reasonable assumption since that’s the class loader that knows how to deal with JAR files. But then came Project Jigsaw, the module system introduced with Java 9. With it came a whole new hierarchy of class loaders that broke any app that assumed a URLClassLoader system class loader. If your app was one of them, this article describes the easiest and most compatible way to fix the problem.

What are the options?

Assuming you can’t rewrite your code to not add JARs dynamically, you are left with a few options:

1. Stick with Java 8 indefinitely. This is a serious option: there are multiple vendors offering free long-term support builds of OpenJDK 8 and most desktop Java apps these days bundle their own JRE.
2. Write your own class loader. There are a few approaches here, such as setting the system class loader with a -Djava.system.class.loader=MyClassLoader option, or installing one at startup before loading your main class. You can also write a separate class loader just for loading things dynamically. This can actually be an improvement because you may gain the ability to GC a plug-in once it is no longer needed, or replace/upgrade plug-ins at runtime. Depending on the assumptions of the existing code, this can be a smooth transition or it can introduce any number of errors.
3. Switch to something like OSGi for loading components at run time. This is a great option when starting a new project, but if you have already been using addURL it can be difficult to switch. (Again, it depends on the assumptions of your existing code.)
4. Use an agent to extend the system class path in an officially supported manner. Yeah, that’s right. There has been an official way to do this since Java 6, buried under the obscure java.lang.instrument package. The best part is that this method is practically identical to what you were doing before and generally requires little change to your existing code base.

Agents in Java

The java.lang.instrument package, introduced in Java 5, is meant to let you attach instrumentation tools to running programs. Agents typically do things like modify a class’s byte code to inject code at each call site to record how long the method call took for profiling. On the surface it has nothing to do with what we want to accomplish. But, Java 6 added a method appendToSystemClassLoaderSearch that does exactly what the protected addURL method was doing, but in a public way.

Creating an agent

To create an agent, you define a class to act as its entry point. This is similar to the main method in an application, but it uses the name premain and has a different signature:

public static void premain(String agentArgs, Instrumentation instrumentation)

You won’t be passing any arguments to the agent, so the first parameter is irrelevant. You will need to store a reference to the Instrumentation instance, however, since it provides the appendToSystemClassLoaderSearch method.

Agents load and run before your main application, so you need to tell the JRE to start them:

1. The JAR that contains the agent code has to have an entry in its MANIFEST.MF file specifying the agent class: Premain-Class: fully.qualified.AgentClass.
2. You have to add a VM option -javaagent:path/to/agent.jar to the command line used to start the JRE. (It can be the same as your main app JAR.)

The Java runtime sees the -javaagent argument, looks at the manifest in the JAR file that it specifies, finds the Premain-Class entry, loads the class named there, and invokes its premain method. Then it proceeds to load your application normally and invoke its main method.

Using the agent to load JARs

Once we have the VM happily loading our agent, the reflection-based code above just needs to be replaced with a call to the agent’s Instrumentation instance:

public static void addToClassPath(File jarFile) throws IOException {
  inst.appendToSystemClassLoaderSearch(new JarFile(jarFile));
}

public static void agentmain(String agentArgs, Instrumentation instrumentation) {
  inst = instrumentation;
}

private static Instrumentation inst;

And that’s that! Just replace calls to the reflection-based loader with calls to the agent-based loader and your app should load JARs under Java 9+ as happily as it does under previous versions.

Is it future-proof?

Reasonably. Although Oracle has shown less concern with maintaining compatibility than Sun Microsystems, there is little reason to remove support for either JAR files or agents in the near future and any change would likely come with plenty of notice.

Source code on GitHub

I’ve put a small (4 kiB) agent library on GitHub that makes implementing the solution in this article a snap. It also falls back to using reflection if the -javaagent VM argument is left off, so if you have not yet started transitioning to Java 9+ you can drop it in, update your code to call it instead of your current reflection-based code, and then make the switch to using the agent at your leisure.