Download to read offline
![Its a very common programming technique to create any number of
URLClassLoader instances repeatedly in order to load the new
implementation of the classes/resources from the same location with the help
of new class loader instances.
In fact, some well-known tools like plexus-compiler jar makes heavy usage of
IsolatedClassLoader to implement the above concept.
Lets focus on the actual problem.
We have a large maven project which has 200 jars in maven repo.
Any code change in source file fires – MavenBuilder …..
- which invokes JavacCompiler (plexus-compiler jar)
- in turn creates IsolatedClassLoader cl = new IsolatedClassLoader()
[extends URLClassloader] ..
- adds the list of 200 jars in urlClassPath - cl.addURL(URL)
- loads javac.Main along with all jars in classpath and reflectively fires
compilation.
So for N save-operations, N instances of IsolatedClassloaders are
created.
In principle, once the application clears all references to a loader object, the
garbage collector and finalization mechanisms will eventually ensure that all
resources (such as the JarFile objects) are released and closed.
But in reality, the application goes OutOfMemory !
We used optimal concurrent gc strategy - Xgcpolicy:optavgpause.
But still the app running out of memory very quickly.
Then Heap dump analysis showed there are some 100 instances (Uuh !) of
isolated class loaders each of which is holding onto unclaimed
ZipFileIndexEntries(.. guess what .. indexEntry for all the jars loaded on each
instance of class laoder..)
Well ! looks like.. since a new URL Class Loader is created before closing the
resources in previous loader, GC is confused and does not reclaim the
previous class loader ! This causes problems for applications which need to
be able to GC in a predictable and timely fashion. It is a particular problem on
Windows, because open files cannot be deleted or replaced.
So are we missing something trivial here ? Is plexus-jar doing something
wrong ?
A BIG Emphatic YES !
We can use sun.misc.ClassLoaderUtil to release an url class loader and file
resources held therein.](https://image.slidesharecdn.com/perilsofurlclassloader-13004343802486-phpapp01/85/Perils-Of-Url-Class-Loader-1-320.jpg)
![Its a very common programming technique to create any number of
URLClassLoader instances repeatedly in order to load the new
implementation of the classes/resources from the same location with the help
of new class loader instances.
In fact, some well-known tools like plexus-compiler jar makes heavy usage of
IsolatedClassLoader to implement the above concept.
Lets focus on the actual problem.
We have a large maven project which has 200 jars in maven repo.
Any code change in source file fires – MavenBuilder …..
- which invokes JavacCompiler (plexus-compiler jar)
- in turn creates IsolatedClassLoader cl = new IsolatedClassLoader()
[extends URLClassloader] ..
- adds the list of 200 jars in urlClassPath - cl.addURL(URL)
- loads javac.Main along with all jars in classpath and reflectively fires
compilation.
So for N save-operations, N instances of IsolatedClassloaders are
created.
In principle, once the application clears all references to a loader object, the
garbage collector and finalization mechanisms will eventually ensure that all
resources (such as the JarFile objects) are released and closed.
But in reality, the application goes OutOfMemory !
We used optimal concurrent gc strategy - Xgcpolicy:optavgpause.
But still the app running out of memory very quickly.
Then Heap dump analysis showed there are some 100 instances (Uuh !) of
isolated class loaders each of which is holding onto unclaimed
ZipFileIndexEntries(.. guess what .. indexEntry for all the jars loaded on each
instance of class laoder..)
Well ! looks like.. since a new URL Class Loader is created before closing the
resources in previous loader, GC is confused and does not reclaim the
previous class loader ! This causes problems for applications which need to
be able to GC in a predictable and timely fashion. It is a particular problem on
Windows, because open files cannot be deleted or replaced.
So are we missing something trivial here ? Is plexus-jar doing something
wrong ?
A BIG Emphatic YES !
We can use sun.misc.ClassLoaderUtil to release an url class loader and file
resources held therein.](https://image.slidesharecdn.com/perilsofurlclassloader-13004343802486-phpapp01/75/Perils-Of-Url-Class-Loader-1-2048.jpg)
Uploaded byKaniska Mandal
Perils Of Url Class Loader
The document discusses the problem of repeatedly creating new URLClassLoader instances without closing previous instances. This causes OutOfMemory errors as the classloaders accumulate and prevent garbage collection from freeing up resources like JarFile objects. The solution is to explicitly call loader.close() or sun.misc.ClassLoaderUtil.releaseLoader(loader) on each classloader instance after use to allow the resources to be released and prevent memory leaks.
Perils Of Url Class Loader
- 1. Its a verycommon programming technique to create any number of URLClassLoader instances repeatedly in order to load the new implementation of the classes/resources from the same location with the help of new class loader instances. In fact, some well-known tools like plexus-compiler jar makes heavy usage of IsolatedClassLoader to implement the above concept. Lets focus on the actual problem. We have a large maven project which has 200 jars in maven repo. Any code change in source file fires – MavenBuilder ….. - which invokes JavacCompiler (plexus-compiler jar) - in turn creates IsolatedClassLoader cl = new IsolatedClassLoader() [extends URLClassloader] .. - adds the list of 200 jars in urlClassPath - cl.addURL(URL) - loads javac.Main along with all jars in classpath and reflectively fires compilation. So for N save-operations, N instances of IsolatedClassloaders are created. In principle, once the application clears all references to a loader object, the garbage collector and finalization mechanisms will eventually ensure that all resources (such as the JarFile objects) are released and closed. But in reality, the application goes OutOfMemory ! We used optimal concurrent gc strategy - Xgcpolicy:optavgpause. But still the app running out of memory very quickly. Then Heap dump analysis showed there are some 100 instances (Uuh !) of isolated class loaders each of which is holding onto unclaimed ZipFileIndexEntries(.. guess what .. indexEntry for all the jars loaded on each instance of class laoder..) Well ! looks like.. since a new URL Class Loader is created before closing the resources in previous loader, GC is confused and does not reclaim the previous class loader ! This causes problems for applications which need to be able to GC in a predictable and timely fashion. It is a particular problem on Windows, because open files cannot be deleted or replaced. So are we missing something trivial here ? Is plexus-jar doing something wrong ? A BIG Emphatic YES ! We can use sun.misc.ClassLoaderUtil to release an url class loader and file resources held therein.
- 2. Lets now fixthe issue. org.codehaus.plexus.compiler.javac.JavacCompiler#compileInProcess(..) { IsolatedClassLoader loader= new IsolatedClassLoader(); loader.addURL(jarListoURI().toURL()) c = loader.loadClass( "com.sun.tools.javac.Main" ); ok = (Integer) compile.invoke(args) //Now that you are done with compilation, get rid of loader .. *** loader.close () Or sun.misc.ClassLoaderUtil.releaseLoader(loader) ////// -> allows graceful release of loader and all resources *** Thankfully JDK 7 URLClassloader has implemented close() method to give the caller a chance to invalidate the loader, so that no new classes can be loaded from it. It also closes any JAR files that were opened by the loader. This allows the application to properly delete or replace these files and, gracefully create new loaders using new implementations. The bottom line is wherever we create seemingly harmless URlCLassloader in a repeated manner (say to generate / compile code) holding onto a good many jars in url classpath; we should always close the loader instance.