My JVM ran out of memory!  What objects are consuming all of the memory?

By 01

Thursday, October 16, 2008

Introduction

This article isn't a comprehensive JVM Java heap sizing and garbage collection tuning document; its goal is simply to outline options for identifying which type of Object(s) are consuming the majority of memory in a running JVM--troubleshooting memory leaks in the Java heap.  As usual, I focus on two JVMs: Sun & IBM.  There are a couple of features and tools each vendor provides for identifying which types of objects are using the most memory.  So, this article will help answer the question, what is using all the memory in my JVM? 

You can answer this question with a profiling tool such as Borland OptimizeIt.  However, these cost money.  And, you generally can't use profilers in a production environment with any amount of load.  Another approach would be to use something like the Leak Hunter tool in Wily Introscope--this, too, has a significant cost associated with it.  I'm interested in general approaches that can be used anytime under just about any circumstances.

It is important to remember that sometimes, there is no memory leak to be found, even if you have received an OutOfMemoryException error.  Sometimes, the Java heap simply needs to be increased.  Guidelines for memory sizing and garbage collection tuning will be presented another day.

Sun

In a Sun Java 1.4.2 JVM & later, you can use the -XX:+PrintClassHistogram argument to produce a class histogram whenever a thread dump is generated.  More information about this parameter can be found here.  An example of this option looks like:

num   #instances    #bytes  class name
--------------------------------------
  1:      4267      414048  <no name>
  2:      4267      241344  <methodKlass>
  3:      6244      224496  <symbolKlass>
  4:       296      157480  <constantPoolKlass>
  5:       296      112792  <instanceKlassKlass>
  6:       278      110912  <constantPoolCacheKlass>
  7:       365       76872  [B
  8:       576       63232  [C
  9:       620       39376  [I
 10:       366       35136  java.lang.Class
 11:       467       27448  [[I
 12:       431       26360  [S
 13:       564       13536  java.lang.String
 14:        41       12464  <objArrayKlassKlass>
 15:       211        8272  [Ljava.lang.Object;
 16:         8        2432  <typeArrayKlassKlass>
 17:        57        1368  java.util.Hashtable$Entry
 18:        10        1200  <klassKlass>
 19:        15        1200  [Ljava.util.HashMap$Entry;

This was printed at the bottom of a thread dump generated by a Sun Java 1.5.0_16 JVM--running the DeadLock program from the last article.  The object type consuming the most memory in this container is only using ~400kb of memory.  The top six object type categories using the most memory are only consuming about 12MB of Java heap.  This JVM has a relatively small memory footprint (for a Java application).  The same example implemented in C code using a couple of semaphores to produce a similar effect would use a couple 100kb of memory.

The class histogram that is generated here orders collections of object types by total bytes consumed by all instances of that type.  It reports a ranking, number of instances, number of bytes, and a class name or other identifier.  A "class name" like "[B" represents a Byte array.  A quick reference to these primitive identifiers is given here.  It can take several minutes on large JVMs to produce this information; this may not be something you want to do in a production environment.  However, if you are in dire straights and need solve the problem, this might be a good way to go about doing it.  But, you've got to have the JVM parameter there ahead of time.  Also, this parameter isn't compatible with all garbage collectors (and garbage collector options, including logging); I generally try to use the default garbage collectors when I use this parameter.  This may change in future versions of the Sun JVM; try it with your current garbage collector settings and see what happens.

If you are using Java 1.4.2 (a recent patch) or later, you can also use a tool called jmap that ships with the Sun JDK to generate a class histogram that essentially looks the same as what I have listed in the last example.  This solution is actually better than the last because it doesn't require having a Java argument present to work.  It still suffers from the same two shortcomings of the first solution: it pauses the JVM for an extended period of time and it isn't compatible with all garbage collector settings.

IBM

For an IBM JVM, I use the IBM Heap Analyzer tool to identify what objects are consuming the most memory.  Before one can use the Heap Analyzer tool, you have to generate a Heap Dump.  In a 1.4.2 or 1.5.0 IBM JVM, you  can cause the JVM to generate a heap dump with a "kill -SIGQUIT pid" by setting the following environment variables:

IBM_HEAPDUMPIBM_HEAP_DUMP=true
IBM_HEAPDUMPDIR=path_to_heap_dump
IBM_HEAP_DUMP=true

Note, these are environment variables, NOT java system properties.  You should also be able to produce a Heap Dump with the "-Xdump:heap,opt=PHD" option being added to the JVM arguments.

Now, you should be able to generate a Heap Dump by sending QUIT signal to the java process.  Once you have this, you can load the .phd file into the IBM Heap Analyzer tool.

Launch the Heap Analyzer tool; give the Java heap plenty of memory.  Choose File->Open to load a .phd file into the tool.  Once loaded, choose View->Tree View.  This will open a new sub window in the work space.  Enlarge the Tree View window.  Right-click on any tree node.  Choose "Compile Leak Suspects".  This could take a while.  Once completed, click "Leak Suspects".  The drop down list will present you with a list of objects that are using a large percentage of memory.  If there is a single or a couple of objects that are using a lot of memory, this will go quickly.  Another common pattern is for their to be many instances of the same type of object; this will also be fairly obvious.  If there is no rhyme nor reason to it, you may simply be looking at increasing the heap space.

For more information about the IBM Heap Analyzer, watch the webcast.

Conclusion

I've presented the most flexible methodologies for identifying objects or groups of objects that are consuming memory in this article.  Each JVM vendor has its own tools and features for troubleshooting memory leaks in a JVM.  It's important to remember that memory leaks in Java are not exactly the same as memory leaks in unmanaged languages such as C & C++.  Sometimes, the answer to your memory problems isn't that there is a particular object or type of object that is consuming memory, it is simply that your Java heap needs to be larger.  Only careful analysis can determine your situation.

References

[1] http://blogs.sun.com/watt/resource/jvm-options-list.html
[2] http://www-01.ibm.com/support/docview.wss?uid=swg27006624
[3] http://www.alphaworks.ibm.com/tech/heapanalyzer

 

©2008 www.thinkmiddleware.com

All copyrights & trademarks belong to their respective owners.

The comments and opinions herein are that of the author.

Please direct all comments to 01.

While the information presented on this web site is believed to be correct, the author is not responsible for any damage, loss of data, or other issues that may arise from using the information posted here.

Made with CityDesk
Last Modified: Sunday, 09-Nov-2008 10:48:26 MST