As you know, Jelastic PaaS has automatic vertical scaling, which allows you to pay less -- to pay for your actual resource consumption, not instance size. True automatic vertical scaling is our one of key differentiators. It provides your application exactly the amount of RAM and CPU it needs, and reclaims those resources when your application no longer needs them. Other PaaS platforms do not provide true resource scaling. They do not provide the ability to scale instances vertically, which means you pay for the limits of your instances. The common name for this old tactic is overselling. The business model of the entire hosting industry and old generation PaaS solutions is based on overselling. So, the importance of fair payment for actual resource consumption is pretty obvious.
Honestly, it was not easy to create true vertical scaling, because there are several technical restrictions. One of them is related to JVM restrictions. In this article I would like to share some information which can be useful to understand these restrictions of JVM. I hope this will help more of the drivers of the java community to adapt JVM for PaaS.
In the beginning, when JVM was designed, nobody knew about the cloud or virtualization, and moreover, nobody was thinking about density in PaaS. Today virtualization has changed the game of hosting industry and the revolution is not finished yet. Today, we can use resources more efficiently and with better elasticity. Jelastic is the only PaaS which offers true automatic vertical scaling for Java and PHP applications. However, I see good movements of the key java drivers into this sphere. I talked to Mikael Vidstedt – one of the main JVM architects at Oracle, and he agreed that JVM was not designed for PaaS at all and Oracle is going to improve this. Plus guys from IBM are working hard on it as well. Some related notes to dynamic behavior of JVM can be found in IBM JavaOne Keynote 2012 Highlights.
One of the most important points of vertical scaling is understanding how JVM allocates more RAM when it’s needed and how allocated RAM is returned to the OS when it is not needed anymore. The algorithm, which provides allocation of RAM, works fine, but compactication (returning RAM to OS) does not work well today. It works if you know the details, but there are a lot of space for improvements. So, JVM must be improved significantly in this part.
When we were initially developing Jelastic, one of the most important points was understanding how vertical scaling depends on the JVM Garbage Collector (GC). We tested many different combinations and found critically important relations and restrictions which prevent applications from scaling down vertically. So the issue of garbage collections was pretty high on the list of things that needed to work well, all the time.
How Java Garbage Collection Works
Just in case you aren’t totally sure how the garbage collection works in Java, have no idea what it is, or just need a quick refresher, here is an overview of how it works. For a more in-depth view on this, check out Java Enterprise Performance Book or Javin Paul’s blog, Javarevisted.
In Java, dynamic allocation of objects is achieved using the new operator. An object, once created, uses some memory and the memory remains allocated until there are no references for the use of the object. When there are no references for an object, it is assumed to be no longer needed and the memory occupied by the object can be reclaimed. There is no explicit need to destroy an object as java handles the de-allocation automatically. Garbage Collection is the technique that accomplishes this. Programs that do not de-allocate memory can eventually crash when there is no memory left in the system to allocate. These programs are said to have "memory leaks." In Java, Garbage collection happens automatically during the lifetime of a java program, eliminating the need to de-allocate memory and avoiding memory leaks.
What kind of Garbage Collector is used with Jelastic?
As we were going about trying to decide which garbage collector to use with Jelastic by default, we had to narrow down the field. We found that the biggest issue would be one of the features within Jelastic that we are most proud of, vertical scaling. When we were deciding how to configure garbage collection in Java, this feature presented a problem.
In order to help us decide which kind of GC to use, we created an application that controlled resource usage. Turns out that the JVM doesn’t actually deal very well with vertical scaling. If an application starts with a small amount of RAM usage, then adds more, the JVM doesn’t do a good job of returning that RAM to the OS when it is not needed anymore. We found out this was initially done on purpose: the guys that designed the JVM used this approach to speed up the process of memory allocation by having it already queued. This process didn’t work with our platform if we are going to have vertical scaling, and we intended it to.
So, we started testing out different garbage collectors. We tested the Serial Garbage Collector, the Parallel Garbage Collector, the Concurrent Mark Sweep Garbage Collector and the G1 Garbage Collector. Some statistical information, which was collected when we started to work on Jelastic, is included below.
Serial Garbage Collector (-XX:+UseSerialGC)
First, we tried out the Serial Garbage Collector. It uses a single thread to perform all garbage collection work. It is a stop-the-world collector and has very specific limits.
Below, you can see simple tests in Java 6 and Java 7.
Test for Serial Garbage Collector
public class Memoryleak {
public static void main(String[] args) {
System.out.println("START....");
while (true) {
System.out.println("next loop...");
try {
int count = 1000 * 1024;
byte [] array = new byte[1024 * count];
Thread.sleep(5000);
array = null;
System.gc();
System.gc();
Thread.sleep(5000);
}
catch (InterruptedException ex) {
}
}
}
}
We ran the JVM with these parameters:
-XX:+UseSerialGC -Xmx1024m -Xmn64m -Xms128m -Xminf0.1 -Xmaxf0.3where
- -XX:+UseSerialGC — use Serial Garbage Collector (this parameter will be changed in the next tests);
- -Xmx1024m - max RAM usage - 1024 MB;
- -Xmn64m — the size of the heap for the young generation - 64MB;
- -Xms128m – initial java heap size - 128 MB;
- -Xminf0.1 – this parameter controls minimum free space in the heap and instructs the JVM to expand the heap, if after performing garbage collection it does not have at least 10% of free space;
- -Xmaxf0.3 – this parameter controls how the heap is expanded and instructs the JVM to compact the heap if the amount of free space exceeds 30%.
The defaults for -Xminf and Xmaxf are 0.3 and 0.6, respectively, so the JVM tries to maintain a heap that is between 30 and 60 percent free at all times. But we set these parameters to the more aggressive limits - it enlarges the amplitude of the vertical scaling
As you can see in the chart below heap memory is dynamically allocated and released.
The following chart shows the total memory consumption by OS.
As you can see, vertical scaling works fine in this case. Unfortunately, the Serial Garbage Collector is meant to be used by small applications, GC runs on a single thread.
Pros and Cons for Serial Garbage Collector
Pros:
- It shows good results in scaling
- It can do memory defragmentation and returns the unused resources back to the OS
- Great for applications with small data sets
Cons:
- Big pauses when it works with big data sets
- Big applications are a no-go
Parallel Garbage Collector (-XX:+UseParallelGC)
The Parallel Garbage Collector performs minor garbage collections in parallel, which can significantly reduce garbage collection overhead. It is useful for applications with medium to large-sized data sets that are run on a multiprocessor or multithreaded hardware.We repeated our test from before so we could compare the results.
Test for Parallel Garbage Collector
Single process:
Below is the total memory consumption by the OS:
The Parallel Garbage Collector has many advantages over the Serial Garbage Collector. It can work with multithreaded applications and multiprocessor machines. It also works quite well with large data sets. But, as you can see in the above charts, it doesn't do well in returning resources to the OS. So, Parallel GC was not suitable for vertical scaling.
Pros and Cons for Parallel Garbage Collector
Pros:
- Works well with large data sets and applications
- It works great with multithreaded applications and multiprocessor machines
Cons:
- Doesn't do a good job of returning resources to the OS
- Works fine as long as you don't need vertical scaling
Concurrent Mark Sweep Garbage Collector
(-XX:+UseConcMarkSweepGC)
The Concurrent Mark Sweep Garbage Collector performs most of its work concurrently to keep garbage collection pauses short. It is designed for applications with medium to large-sized data sets for which response time is more important than overall throughput.
We repeated the same tests as before and got the following results.
Test for Concurrent Mark Sweep Garbage Collector
Single process:
Total memory consumption by the OS:
While the Concurrent Mark Sweep Garbage Collector has it's advantages with certain kinds of applications, we ran into basically the same issues as we did with the Parallel Garbage Collector - it is not suitable for vertical scaling.
Pros and Cons for Concurrent Mark Sweep Garbage Collector
Pros:
- Works well with large data sets and applications
- Work well with multithreaded applications and multiprocessor machines
- Response time
Cons:
- Doesn't do a good job of returning resources to the OS
- Throughput is a lower priority
- Vertical scaling doesn't work well with it
G1 Garbage Collector (-XX:+UseG1GC)
The G1 ("Garbage First") Garbage Collector was first introduced with Java 7 and then in subsequent Java 6 updates. It splits the heap up into fixed-sized regions and tracks the live data in those regions. When garbage collection is necessary, it collects from the regions with less live data first. It has all the advantages of the Parallel GC and Mark Sweep GC and meets all our requirements.
But when we did our tests we discovered that, in Java 6, after a long period of work there was a constant and stable memory leak.
Test for G1 Garbage Collector (Java 6)
Single process:
Total memory consumption by the OS:
When we found this issue, we reached out to the guys at Oracle. We were the first to find and report the problem with the memory leak in Java 6 with respect to the G1 GC. Based on our input, they fixed this issue "in-flight" within the JVM 7. So, below, you can see our tests after the fix that we asked them to do.
Test for G1 Garbage Collector (Java 7)
As you can see, the fix that the Oracles guys did really improved G1 Garbage Collector. We are very grateful to them for their help is getting this sorted out.
We are hoping that soon, Java 6 will also have its memory leak issue fixed as well. If you are currently using the JVM 6, you can test it out and contact Oracle support like we did to help speed up the process.
Pros and Cons for G1 Garbage Collector
Cons:
- Still have an issue with Java 6 (hopefully resolved soon)
Pros:
- Works well with both large and small data sets and applications
- Work well with multithreaded applications and multiprocessor machines
- Returns resources to the OS well and in a timely manner
- Good response time
- Good throughput
- Works great for vertical scaling
Conclusion
As you can understand, the ability to only pay for the actual resources used is very important for every customer and company. They should not be oversold and overpay. However, there are still several blockers which prevent faster development in this direction and must be fixed. Jelastic is the only platform pioneering true automatic vertical scaling. We have already changed the game and we will do our best to continue to revolutionize the hosting industry.
The video below explains how automatic vertical scaling works in Jelastic:
* For some reasons the official bug according to our report about G1 accumulative memory leak was removed from Oracle bug database. More details about this bug can be found here.
