Reposting my posts I have in the Paper & Spigot community here!
Many months ago, I did an extremely extensive study into Garbage Collection and Minecraft, and got a really deep understanding on Minecrafts allocation behavior, general object life expectancy and collection rates. I studied my server for about an entire month.
I tweaked various settings, studying the GC log files, studying the various flags documentation, and other performance tuning case studies, and then even better: using the VisualVM plugin VisualGC, that lets you visually see how each region of the garbage collector fills up and collects.
This insight was amazing and let me truly understand how the GC was behaving, and learn how to overall improve the garbage collector efficiency.
These flags are based on my study, and have strong reasoning behind them to actually have an intended behavior change of the Garbage Collector and Performance Settings.
Many servers in the Paper & Spigot community have started using these and reported improvements (And those who don't notice, probally wasn't doing too bad to begin with)
The goal of these flags is not to reduce overall CPU usage, as G1 does use more CPU than CMS, but this CPU usage is parallel, so not 'directly' main thread. The goal here is to provide CONSISTENT, low pauses, and avoid Old Gen collections as much as possible. Improperly tuned GC that ends up collecting Old Gen could see many 1 second long GC collections, which your players will notice!
Also NOTE: Memory usage will be a saw blade / zig zag! G1 does not constantly collect. It only collects when one region fills up, so this is normal:
(Graph provided by Zabbix Server Monitoring Software with JMX agent integration)
During low activity, it will appear as memory is higher, as it simply takes longer between every GC pause - THIS IS A VERY GOOD THING!
Going 3 minutes between GC pauses means your CPU spent 0 time on GC for 3 minutes.
Here are the flags!
These flags help keep your server running CONSISTENT without any large spikes. CPU may be slightly higher, but your server will be overall more reliable and stable TPS.
If you are running with 10GB or less memory for MC, you should not adjust these parameters. (I use 10GB myself - pauses are 100ms or less)
If you for sure need more than 10GB (Hopefully you are 150+ player server, but maybe less with Forge Mods) use these changes:
You may need to tune these to your needs for larger memory servers (these reduce memory in new gen and give more back to old gen)
Explanation of flags:
-Xms matching -Xmx - Why: You should never run your server with the case that -Xmx can run the system completely out of memory. Your server should always be expected to use the entire -Xmx! You should then ensure the OS has extra memory on top of that Xmx for non MC/OS level things. Therefore, you should never run MC with -Xmx settings you can't support if java uses it all.
Now, that means if -Xms is lower than -Xmx - YOU HAVE UNUSED MEMORY! Unused memory is wasted memory. G1 (and probably even CMS to a certain threshhold, but I'm only stating what I'm sure about) operates better with the more memory its given. G1 adaptively chooses how much memory to give to each region to optimize pause time. If you have more memory than it needs to reach an optimal pause time, G1 will simply push that extra into the old generation and it will not hurt you (This may not be the case for CMS, but is the case for G1)
The fundamental idea of improving GC behavior is to ensure short lived objects die young and never get promoted. With the more memory G1 has, the better assurance you will get that objects are not getting prematurely promoted to the old generation.
G1 Operates differently than previous collectors and is able to handle larger heaps more efficiently. If it does not need the memory given to it, it will not use it. The entire engine operates differently and does not suffer from too large of heaps.
UnlockExperimentalVMOptions - needed for some of the others specified
TargetSurvivorRatio: I'm sure your all use to seeing this one suggested. Good news! It's actually a good flag to use This setting controls how much of the Survivor space is ABLE to be used before promotion. If survivor gets too full, stuff starts promoting to Old Gen. The reason behind this is to be able to handle memory allocation spikes.
However, MC allocation rate for most part is pretty steady (steadily high.....), and when its steady its safe to raise this value to avoid premature promotions.
G1NewSize Percent: These are the important ones. In CMS and other Generations, tweaking the New Generation results in FIXED SIZE New Gen and usually is done through explicit size setting with -Xmn.
With G1, things are better! You now can specify percentages of an overall desired range for the new generation.
With these settings, we tell G1 to not use its default 5% for new gen, and instead give it 50% at least!
Minecraft has an extremely high memory allocation rate, ranging to at least 800 Megabytes a second on a 30 player server! And this is mostly short lived objects (BlockPosition)
now, this means MC REALLY needs more focus on New Generation to be able to even support this allocation rate. If your new gen is too small, you will be running new gen collections 1-2+ times per second!!!
Then combine the fact that objects will now promote faster, resulting in your Old Gen growing faster.... This is bad and needs to be avoided.
Given more NewGen, we are able to slow down the intervals of Young Gen collections, resulting in more time for short lived objects to die young and overall more efficient GC behavior.
if you run with larger heaps (15GB+), you may want to lower the minimum to say 30%, but don't go lower than 30%. This will let G1 have more power in its own assumptions.
InitiatingHeapOccupancyPercent/G1MixedGCLiveThresholdPercent: Controls when to include Mixed GC's in the Young GC collection, keeping OldGen tidy without doing a normal OldGen GC collection.
On larger heaps(10GB+), you can raise InitiatingHeap to around 20 to reduce CPU usage, but I wouldn't go higher than that. And you also need to REDUCE the Maximum New Percentage to around 60. If you use 80% NewGen Max, you must keep this at 10 then. It doesn't hurt to leave it at 10, but "efficiency wise" you can improve it to 20 if you reduce your new gen size, but if you start seeing Old Gen GC, lower it back.
AlwaysPreTouch: AlwaysPreTouch gets the memory setup and reserved at process start ensuring it is contiguous, improving the efficiency of it more.
Also for Large Pages - It's even more important to use -Xms = -Xmx! That memory CAN NOT be used by the OS anyways, so let something use it!
Additionally use these flags (Metaspace is Java8 Only, don't use it for Java7):