Upstream/Paper
1
0
Fork 0
mirror of https://github.com/PaperMC/Paper.git synced 2024-11-27 04:55:47 +01:00
Code Issues Projects Releases Wiki Activity
5b352d9b33
Paper/Spigot-Server-Patches/0484-Optimize-NibbleArray-to-use-pooled-buffers.patch

376 lines
17 KiB
Diff
Raw Normal View History

Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Aikar <aikar@aikar.co>
Date: Wed, 6 May 2020 23:30:30 -0400
Subject: [PATCH] Optimize NibbleArray to use pooled buffers
Massively reduces memory allocation of 2048 byte buffers by using
an object pool for these.
Uses lots of advanced new capabilities of the Paper codebase :)
diff --git a/src/main/java/net/minecraft/server/ChunkRegionLoader.java b/src/main/java/net/minecraft/server/ChunkRegionLoader.java
1.16.2 Release (#4123) PaperMC believes that 1.16.2 is now ready for general release as we fixed the main issue plagueing the 1.16.x release, the MapLike data conversion issues. Until now, it was not safe for a server to convert a world to 1.16.2 without data conversion issues around villages and potentially other things. If you did, those MapLike errors meant something went wrong. This is now resolved. Big thanks to all those that helped, notably @BillyGalbreath and @Proximyst who did large parts of the update process with me. Please as always, backup your worlds and test before updating to 1.16.2! If you update to 1.16.2, there is no going back to an older build than this. --------------------------------- Co-authored-by: William Blake Galbreath <Blake.Galbreath@GMail.com> Co-authored-by: Mariell Hoversholm <proximyst@proximyst.com> Co-authored-by: krolik-exe <69214078+krolik-exe@users.noreply.github.com> Co-authored-by: BillyGalbreath <BillyGalbreath@users.noreply.github.com> Co-authored-by: stonar96 <minecraft.stonar96@gmail.com> Co-authored-by: Shane Freeder <theboyetronic@gmail.com> Co-authored-by: Jason <jasonpenilla2@me.com> Co-authored-by: kashike <kashike@vq.lc> Co-authored-by: Aurora <21148213+aurorasmiles@users.noreply.github.com> Co-authored-by: KennyTV <kennytv@t-online.de> Co-authored-by: commandblockguy <commandblockguy1@gmail.com> Co-authored-by: DigitalRegent <misterwener@gmail.com> Co-authored-by: ishland <ishlandmc@yeah.net>
2020-08-25 04:22:08 +02:00
index 937bd0daf84f6b5f382dea73e20f2fd6ad903000..3d06c87b64c209526bee2ea11b58d30fe9b1d062 100644
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
--- a/src/main/java/net/minecraft/server/ChunkRegionLoader.java
+++ b/src/main/java/net/minecraft/server/ChunkRegionLoader.java
Current non compilable status of all patches - THIS IS NOT READY THERE IS STILL NO ETA. GOBLINS WILL EAT YOU.
2020-06-26 03:58:00 +02:00
@@ -386,11 +386,11 @@ public class ChunkRegionLoader {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
}
if (nibblearray != null && !nibblearray.c()) {
- nbttagcompound2.setByteArray("BlockLight", nibblearray.asBytes());
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
+ nbttagcompound2.setByteArray("BlockLight", nibblearray.asBytesPoolSafe().clone()); // Paper
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
}
if (nibblearray1 != null && !nibblearray1.c()) {
- nbttagcompound2.setByteArray("SkyLight", nibblearray1.asBytes());
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
+ nbttagcompound2.setByteArray("SkyLight", nibblearray1.asBytesPoolSafe().clone()); // Paper
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
}
nbttaglist.add(nbttagcompound2);
diff --git a/src/main/java/net/minecraft/server/LightEngineStorage.java b/src/main/java/net/minecraft/server/LightEngineStorage.java
Current non compilable status of all patches - THIS IS NOT READY THERE IS STILL NO ETA. GOBLINS WILL EAT YOU.
2020-06-26 03:58:00 +02:00
index 6c7c4e75670a7e08ba10c0231a2510bf985dab6b..b8b06c790adfa0246b1a6fb5eab1f63bf5ef8b0b 100644
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
--- a/src/main/java/net/minecraft/server/LightEngineStorage.java
+++ b/src/main/java/net/minecraft/server/LightEngineStorage.java
Current non compilable status of all patches - THIS IS NOT READY THERE IS STILL NO ETA. GOBLINS WILL EAT YOU.
2020-06-26 03:58:00 +02:00
@@ -149,7 +149,7 @@ public abstract class LightEngineStorage<M extends LightEngineStorageArray<M>> e
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
protected NibbleArray j(long i) {
NibbleArray nibblearray = (NibbleArray) this.i.get(i);
- return nibblearray != null ? nibblearray : new NibbleArray();
+ return nibblearray != null ? nibblearray : new NibbleArray().markPoolSafe(); // Paper
}
protected void a(LightEngineLayer<?, ?> lightenginelayer, long i) {
Current non compilable status of all patches - THIS IS NOT READY THERE IS STILL NO ETA. GOBLINS WILL EAT YOU.
2020-06-26 03:58:00 +02:00
@@ -331,12 +331,12 @@ public abstract class LightEngineStorage<M extends LightEngineStorageArray<M>> e
Optimize Light Engine Massive update to light to improve performance and chunk loading/generation. 1) Massive bit packing/unpacking optimizations and inlining. A lot of performance has to do with constant packing and unpacking of bits. We now inline a most bit operations, and re-use base x/y/z bits in many places. This helps with cpu level processing to just do all the math at once instead of having to jump in and out of function calls. This much logic also is likely over the JVM Inline limit for JIT too. 2) Applied a few of JellySquid's Phosphor mod optimizations such as - ensuring we don't notify neighbor chunks when neighbor chunk doesn't need to be notified - reduce hasLight checks in initializing light, and prob some more, they are tagged JellySquid where phosphor influence was used. 3) Optimize hot path accesses to getting updating chunk to have less branching 4) Optimize getBlock accesses to have less branching, and less unpacking 5) Have a separate urgent bucket for chunk light tasks. These tasks will always cut in line over non blocking light tasks. 6) Retain chunk priority while light tasks are enqueued. So if a task comes in at high priority but the queue is full of tasks already at a lower priority, before the task was simply added to the end. Now it can cut in line to the front. this applies for both urgent and non urgent tasks. 7) Buffer non urgent tasks even if queueUpdate is called multiple times to improve efficiency. 8) Fix NPE risk that crashes server in getting nibble data Fixes #3489 Fixes #3363
2020-06-05 07:25:11 +02:00
Current non compilable status of all patches - THIS IS NOT READY THERE IS STILL NO ETA. GOBLINS WILL EAT YOU.
2020-06-26 03:58:00 +02:00
protected void a(long i, @Nullable NibbleArray nibblearray, boolean flag) {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
if (nibblearray != null) {
Optimize Light Engine Massive update to light to improve performance and chunk loading/generation. 1) Massive bit packing/unpacking optimizations and inlining. A lot of performance has to do with constant packing and unpacking of bits. We now inline a most bit operations, and re-use base x/y/z bits in many places. This helps with cpu level processing to just do all the math at once instead of having to jump in and out of function calls. This much logic also is likely over the JVM Inline limit for JIT too. 2) Applied a few of JellySquid's Phosphor mod optimizations such as - ensuring we don't notify neighbor chunks when neighbor chunk doesn't need to be notified - reduce hasLight checks in initializing light, and prob some more, they are tagged JellySquid where phosphor influence was used. 3) Optimize hot path accesses to getting updating chunk to have less branching 4) Optimize getBlock accesses to have less branching, and less unpacking 5) Have a separate urgent bucket for chunk light tasks. These tasks will always cut in line over non blocking light tasks. 6) Retain chunk priority while light tasks are enqueued. So if a task comes in at high priority but the queue is full of tasks already at a lower priority, before the task was simply added to the end. Now it can cut in line to the front. this applies for both urgent and non urgent tasks. 7) Buffer non urgent tasks even if queueUpdate is called multiple times to improve efficiency. 8) Fix NPE risk that crashes server in getting nibble data Fixes #3489 Fixes #3363
2020-06-05 07:25:11 +02:00
- this.i.put(i, nibblearray);
+ NibbleArray remove = this.i.put(i, nibblearray); if (remove != null && remove.cleaner != null) remove.cleaner.run(); // Paper - clean up when removed
Current non compilable status of all patches - THIS IS NOT READY THERE IS STILL NO ETA. GOBLINS WILL EAT YOU.
2020-06-26 03:58:00 +02:00
if (!flag) {
this.n.add(i);
}
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
} else {
- this.i.remove(i);
+ NibbleArray remove = this.i.remove(i); if (remove != null && remove.cleaner != null) remove.cleaner.run(); // Paper - clean up when removed
}
}
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
diff --git a/src/main/java/net/minecraft/server/LightEngineStorageArray.java b/src/main/java/net/minecraft/server/LightEngineStorageArray.java
Improve Chunk Prioritization and Internal Scheduler In previous MC versions, we had a rather simple internal scheduler for delayed tasks that would just keep pushing task back until desired tick was reached. The method it called to schedule the task changed behavior in 1.14, and now this scheduler is not working nowhere near what it was supposed to be doing. This was causing long delayed task to eat up CPU (In Oversleep for example) Rewrite this to just use the CraftScheduler for scheduling delayed tasks. Once this was fixed, it became quite clear the code that delayed ticket additions for chunks based on distance was clearly not right, as it was tested on the previous broken logic. So the ticket delay process has been vastly revamped to be even smarter. Chunks behind the player can load slower than the chunks in front of the player. We also can delay ticket adding until one of its neighbors has loaded, as this lets us get a smoother spiral out for the chunks (minus frustum intent). Additionally on frustum previous commit inadvertently broke frustum trying to fix an issue when the real fix lied elsewhere, so restore chunk priority so it works again.
2020-06-09 09:17:25 +02:00
index 53199595da71a25710bd1ff8ee2868ee63edc0e1..37c44a89f28c44915fcae5a7e2c4797b1c123723 100644
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
--- a/src/main/java/net/minecraft/server/LightEngineStorageArray.java
+++ b/src/main/java/net/minecraft/server/LightEngineStorageArray.java
Drop remove leaked chunk patch - causing many issues I'm hoping the other fix in 324 for the level map getting corrupted fixes the real issue and this isn't needed anymore, but i suspect it is will wait until more study can be done though. Fixes #3469
2020-05-29 09:30:58 +02:00
@@ -33,7 +33,9 @@ public abstract class LightEngineStorageArray<M extends LightEngineStorageArray<
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
public void a(long i) {
if (this.isVisible) { throw new IllegalStateException("writing to visible data"); } // Paper - avoid copying light data
- this.data.queueUpdate(i, ((NibbleArray) this.data.getUpdating(i)).b()); // Paper - avoid copying light data
Fix concurrency issue in light engine (Vanilla bug) Mojang implemented a cache like chunks have, but this cache is accessed by multiple threads and is totally not safe. So just remove it Fixes #3466 Also missed a pooled nibble release, so slid that in there too.
2020-05-28 22:59:38 +02:00
+ NibbleArray updating = this.data.getUpdating(i); // Paper - pool nibbles
+ this.data.queueUpdate(i, new NibbleArray().markPoolSafe(updating.getCloneIfSet())); // Paper - avoid copying light data - pool safe clone
Improve Chunk Prioritization and Internal Scheduler In previous MC versions, we had a rather simple internal scheduler for delayed tasks that would just keep pushing task back until desired tick was reached. The method it called to schedule the task changed behavior in 1.14, and now this scheduler is not working nowhere near what it was supposed to be doing. This was causing long delayed task to eat up CPU (In Oversleep for example) Rewrite this to just use the CraftScheduler for scheduling delayed tasks. Once this was fixed, it became quite clear the code that delayed ticket additions for chunks based on distance was clearly not right, as it was tested on the previous broken logic. So the ticket delay process has been vastly revamped to be even smarter. Chunks behind the player can load slower than the chunks in front of the player. We also can delay ticket adding until one of its neighbors has loaded, as this lets us get a smoother spiral out for the chunks (minus frustum intent). Additionally on frustum previous commit inadvertently broke frustum trying to fix an issue when the real fix lied elsewhere, so restore chunk priority so it works again.
2020-06-09 09:17:25 +02:00
+ if (updating.cleaner != null) MCUtil.scheduleTask(2, updating.cleaner, "Light Engine Release"); // Paper - delay clean incase anything holding ref was still using it
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
this.c();
}
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
diff --git a/src/main/java/net/minecraft/server/LightEngineStorageSky.java b/src/main/java/net/minecraft/server/LightEngineStorageSky.java
Updated Upstream (Bukkit/CraftBukkit) (#3980) Upstream has released updates that appears to apply and compile correctly. This update has not been tested by PaperMC and as with ANY update, please do your own testing Bukkit Changes: 09f10fd9 SPIGOT-5950: Add PrepareSmithingEvent event CraftBukkit Changes: 7c03d257 SPIGOT-6011: End Gateways do not work on Non-Main End Worlds d492e363 SPIGOT-6015: Small Armor Stand doesn't drop items 5db13eea SPIGOT-5950: Add PrepareSmithingEvent event
2020-07-23 01:35:44 +02:00
index 7cec18fcfc311d20beca244c0affe5d6c1849e46..a35e7b392c74fadf2760d1fc2021e98d33858cb5 100644
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
--- a/src/main/java/net/minecraft/server/LightEngineStorageSky.java
+++ b/src/main/java/net/minecraft/server/LightEngineStorageSky.java
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
@@ -166,9 +166,9 @@ public class LightEngineStorageSky extends LightEngineStorage<LightEngineStorage
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
j = SectionPosition.a(j, EnumDirection.UP);
}
- return new NibbleArray((new NibbleArrayFlat(nibblearray1, 0)).asBytes());
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
+ return new NibbleArray().markPoolSafe(new NibbleArrayFlat(nibblearray1, 0).asBytes()); // Paper - mark pool use as safe (no auto cleaner)
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
} else {
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
- return new NibbleArray();
+ return new NibbleArray().markPoolSafe(); // Paper - mark pool use as safe (no auto cleaner)
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
}
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
}
}
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
@@ -197,7 +197,7 @@ public class LightEngineStorageSky extends LightEngineStorage<LightEngineStorage
((LightEngineStorageSky.a) this.f).a(i);
}
- Arrays.fill(this.a(i, true).asBytes(), (byte) -1);
+ Arrays.fill(this.a(i, true).asBytesPoolSafe(), (byte) -1); // Paper
k = SectionPosition.c(SectionPosition.b(i));
l = SectionPosition.c(SectionPosition.c(i));
int i1 = SectionPosition.c(SectionPosition.d(i));
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
diff --git a/src/main/java/net/minecraft/server/NibbleArray.java b/src/main/java/net/minecraft/server/NibbleArray.java
1.16.2 Release (#4123) PaperMC believes that 1.16.2 is now ready for general release as we fixed the main issue plagueing the 1.16.x release, the MapLike data conversion issues. Until now, it was not safe for a server to convert a world to 1.16.2 without data conversion issues around villages and potentially other things. If you did, those MapLike errors meant something went wrong. This is now resolved. Big thanks to all those that helped, notably @BillyGalbreath and @Proximyst who did large parts of the update process with me. Please as always, backup your worlds and test before updating to 1.16.2! If you update to 1.16.2, there is no going back to an older build than this. --------------------------------- Co-authored-by: William Blake Galbreath <Blake.Galbreath@GMail.com> Co-authored-by: Mariell Hoversholm <proximyst@proximyst.com> Co-authored-by: krolik-exe <69214078+krolik-exe@users.noreply.github.com> Co-authored-by: BillyGalbreath <BillyGalbreath@users.noreply.github.com> Co-authored-by: stonar96 <minecraft.stonar96@gmail.com> Co-authored-by: Shane Freeder <theboyetronic@gmail.com> Co-authored-by: Jason <jasonpenilla2@me.com> Co-authored-by: kashike <kashike@vq.lc> Co-authored-by: Aurora <21148213+aurorasmiles@users.noreply.github.com> Co-authored-by: KennyTV <kennytv@t-online.de> Co-authored-by: commandblockguy <commandblockguy1@gmail.com> Co-authored-by: DigitalRegent <misterwener@gmail.com> Co-authored-by: ishland <ishlandmc@yeah.net>
2020-08-25 04:22:08 +02:00
index 6d608dfa52fffd79878f3bdebcda9fab176e413e..ff00830b95a17f66d0c913087492dbf4b066df8a 100644
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
--- a/src/main/java/net/minecraft/server/NibbleArray.java
+++ b/src/main/java/net/minecraft/server/NibbleArray.java
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
@@ -1,16 +1,75 @@
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
package net.minecraft.server;
+import com.destroystokyo.paper.util.pooled.PooledObjects; // Paper
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
+
+import javax.annotation.Nonnull;
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
import javax.annotation.Nullable;
public class NibbleArray {
- @Nullable
- protected byte[] a;
+ // Paper start
+ public static byte[] EMPTY_NIBBLE = new byte[2048];
+ private static final int nibbleBucketSizeMultiplier = Integer.getInteger("Paper.nibbleBucketSize", 3072);
Allow server startup for those poor people running <1G Xmx Clamp logic for poolsize so we dont pass zero
2020-05-17 06:06:19 +02:00
+ private static final int maxPoolSize = Integer.getInteger("Paper.maxNibblePoolSize", (int) Math.min(6, Math.max(1, Runtime.getRuntime().maxMemory() / 1024 / 1024 / 1024)) * (nibbleBucketSizeMultiplier * 8));
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
+ public static final PooledObjects<byte[]> BYTE_2048 = new PooledObjects<>(() -> new byte[2048], maxPoolSize);
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
+ public static void releaseBytes(byte[] bytes) {
+ if (bytes != null && bytes != EMPTY_NIBBLE && bytes.length == 2048) {
+ System.arraycopy(EMPTY_NIBBLE, 0, bytes, 0, 2048);
+ BYTE_2048.release(bytes);
+ }
+ }
+
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
+ public NibbleArray markPoolSafe(byte[] bytes) {
+ if (bytes != EMPTY_NIBBLE) this.a = bytes;
+ return markPoolSafe();
+ }
+ public NibbleArray markPoolSafe() {
+ poolSafe = true;
+ return this;
+ }
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
+ public byte[] getIfSet() {
+ return this.a != null ? this.a : EMPTY_NIBBLE;
+ }
+ public byte[] getCloneIfSet() {
+ if (a == null) {
+ return EMPTY_NIBBLE;
+ }
+ byte[] ret = BYTE_2048.acquire();
+ System.arraycopy(getIfSet(), 0, ret, 0, 2048);
+ return ret;
+ }
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
+
+ public NibbleArray cloneAndSet(byte[] bytes) {
+ if (bytes != null && bytes != EMPTY_NIBBLE) {
+ this.a = BYTE_2048.acquire();
+ System.arraycopy(bytes, 0, this.a, 0, 2048);
+ }
+ return this;
+ }
+ boolean poolSafe = false;
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
+ public java.lang.Runnable cleaner;
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
+ private void registerCleaner() {
+ if (!poolSafe) {
+ cleaner = MCUtil.registerCleaner(this, this.a, NibbleArray::releaseBytes);
+ } else {
+ cleaner = MCUtil.once(() -> NibbleArray.releaseBytes(this.a));
+ }
+ }
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
+ // Paper end
+ @Nullable protected byte[] a;
+
public NibbleArray() {}
public NibbleArray(byte[] abyte) {
+ // Paper start
+ this(abyte, false);
+ }
+ public NibbleArray(byte[] abyte, boolean isSafe) {
this.a = abyte;
+ if (!isSafe) this.a = getCloneIfSet(); // Paper - clone for safety
+ registerCleaner();
+ // Paper end
if (abyte.length != 2048) {
1.16.2 Release (#4123) PaperMC believes that 1.16.2 is now ready for general release as we fixed the main issue plagueing the 1.16.x release, the MapLike data conversion issues. Until now, it was not safe for a server to convert a world to 1.16.2 without data conversion issues around villages and potentially other things. If you did, those MapLike errors meant something went wrong. This is now resolved. Big thanks to all those that helped, notably @BillyGalbreath and @Proximyst who did large parts of the update process with me. Please as always, backup your worlds and test before updating to 1.16.2! If you update to 1.16.2, there is no going back to an older build than this. --------------------------------- Co-authored-by: William Blake Galbreath <Blake.Galbreath@GMail.com> Co-authored-by: Mariell Hoversholm <proximyst@proximyst.com> Co-authored-by: krolik-exe <69214078+krolik-exe@users.noreply.github.com> Co-authored-by: BillyGalbreath <BillyGalbreath@users.noreply.github.com> Co-authored-by: stonar96 <minecraft.stonar96@gmail.com> Co-authored-by: Shane Freeder <theboyetronic@gmail.com> Co-authored-by: Jason <jasonpenilla2@me.com> Co-authored-by: kashike <kashike@vq.lc> Co-authored-by: Aurora <21148213+aurorasmiles@users.noreply.github.com> Co-authored-by: KennyTV <kennytv@t-online.de> Co-authored-by: commandblockguy <commandblockguy1@gmail.com> Co-authored-by: DigitalRegent <misterwener@gmail.com> Co-authored-by: ishland <ishlandmc@yeah.net>
2020-08-25 04:22:08 +02:00
throw (IllegalArgumentException) SystemUtils.c((Throwable) (new IllegalArgumentException("ChunkNibbleArrays should be 2048 bytes not: " + abyte.length)));
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
}
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
@@ -44,7 +103,8 @@ public class NibbleArray {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
public void a(int i, int j) { // PAIL: private -> public
if (this.a == null) {
- this.a = new byte[2048];
+ this.a = BYTE_2048.acquire(); // Paper
+ registerCleaner();// Paper
}
int k = this.d(i);
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
@@ -66,14 +126,36 @@ public class NibbleArray {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
public byte[] asBytes() {
if (this.a == null) {
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
this.a = new byte[2048];
+ } else { // Paper start
+ // Accessor may need this object past garbage collection so need to clone it and return pooled value
+ // If we know its safe for pre GC access, use asBytesPoolSafe(). If you just need read, use getIfSet()
+ Runnable cleaner = this.cleaner;
+ if (cleaner != null) {
+ this.a = this.a.clone();
+ cleaner.run(); // release the previously pooled value
+ this.cleaner = null;
+ }
+ }
+ // Paper end
+
+ return this.a;
+ }
+
+ @Nonnull
+ public byte[] asBytesPoolSafe() {
+ if (this.a == null) {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
+ this.a = BYTE_2048.acquire(); // Paper
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
+ registerCleaner(); // Paper
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
}
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
+ //noinspection ConstantConditions
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
return this.a;
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
}
+ // Paper end
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
public NibbleArray copy() { return this.b(); } // Paper - OBFHELPER
public NibbleArray b() {
- return this.a == null ? new NibbleArray() : new NibbleArray((byte[]) this.a.clone());
+ return this.a == null ? new NibbleArray() : new NibbleArray(this.a); // Paper - clone in ctor
}
public String toString() {
diff --git a/src/main/java/net/minecraft/server/NibbleArrayFlat.java b/src/main/java/net/minecraft/server/NibbleArrayFlat.java
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
index 67c960292db9d99ac85b5d0dda50ae48ef942c1b..5e3efa1fa6c089df35971ce5c83da384f7dbd402 100644
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
--- a/src/main/java/net/minecraft/server/NibbleArrayFlat.java
+++ b/src/main/java/net/minecraft/server/NibbleArrayFlat.java
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
@@ -8,7 +8,7 @@ public class NibbleArrayFlat extends NibbleArray {
public NibbleArrayFlat(NibbleArray nibblearray, int i) {
super(128);
- System.arraycopy(nibblearray.asBytes(), i * 128, this.a, 0, 128);
+ System.arraycopy(nibblearray.getIfSet(), i * 128, this.a, 0, 128); // Paper
}
@Override
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
@@ -18,7 +18,7 @@ public class NibbleArrayFlat extends NibbleArray {
@Override
public byte[] asBytes() {
- byte[] abyte = new byte[2048];
+ byte[] abyte = BYTE_2048.acquire(); // Paper
for (int i = 0; i < 16; ++i) {
System.arraycopy(this.a, 0, abyte, i * 128, 128);
diff --git a/src/main/java/net/minecraft/server/PacketPlayOutLightUpdate.java b/src/main/java/net/minecraft/server/PacketPlayOutLightUpdate.java
Latest progress - restored patches, some issues resolved
2020-06-26 08:29:44 +02:00
index 6b70df646c6a690ab9437ead96c5ff097e4e12d2..a22f0cccecc85b4e4fe4603bcfa213f15c23db69 100644
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
--- a/src/main/java/net/minecraft/server/PacketPlayOutLightUpdate.java
+++ b/src/main/java/net/minecraft/server/PacketPlayOutLightUpdate.java
Latest progress - restored patches, some issues resolved
2020-06-26 08:29:44 +02:00
@@ -1,6 +1,8 @@
package net.minecraft.server;
import com.google.common.collect.Lists;
+import io.netty.channel.ChannelFuture; // Paper
+
import java.io.IOException;
import java.util.Iterator;
import java.util.List;
@@ -17,14 +19,43 @@ public class PacketPlayOutLightUpdate implements Packet<PacketListenerPlayOut> {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
private List<byte[]> h;
Current non compilable status of all patches - THIS IS NOT READY THERE IS STILL NO ETA. GOBLINS WILL EAT YOU.
2020-06-26 03:58:00 +02:00
private boolean i;
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
+ // Paper start
+ java.lang.Runnable cleaner1;
+ java.lang.Runnable cleaner2;
+ java.util.concurrent.atomic.AtomicInteger remainingSends = new java.util.concurrent.atomic.AtomicInteger(0);
+
+ @Override
+ public void onPacketDispatch(EntityPlayer player) {
+ remainingSends.incrementAndGet();
+ }
+
+ @Override
Latest progress - restored patches, some issues resolved
2020-06-26 08:29:44 +02:00
+ public void onPacketDispatchFinish(EntityPlayer player, ChannelFuture future) {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
+ if (remainingSends.decrementAndGet() <= 0) {
+ // incase of any race conditions, schedule this delayed
Optimize Light Engine Massive update to light to improve performance and chunk loading/generation. 1) Massive bit packing/unpacking optimizations and inlining. A lot of performance has to do with constant packing and unpacking of bits. We now inline a most bit operations, and re-use base x/y/z bits in many places. This helps with cpu level processing to just do all the math at once instead of having to jump in and out of function calls. This much logic also is likely over the JVM Inline limit for JIT too. 2) Applied a few of JellySquid's Phosphor mod optimizations such as - ensuring we don't notify neighbor chunks when neighbor chunk doesn't need to be notified - reduce hasLight checks in initializing light, and prob some more, they are tagged JellySquid where phosphor influence was used. 3) Optimize hot path accesses to getting updating chunk to have less branching 4) Optimize getBlock accesses to have less branching, and less unpacking 5) Have a separate urgent bucket for chunk light tasks. These tasks will always cut in line over non blocking light tasks. 6) Retain chunk priority while light tasks are enqueued. So if a task comes in at high priority but the queue is full of tasks already at a lower priority, before the task was simply added to the end. Now it can cut in line to the front. this applies for both urgent and non urgent tasks. 7) Buffer non urgent tasks even if queueUpdate is called multiple times to improve efficiency. 8) Fix NPE risk that crashes server in getting nibble data Fixes #3489 Fixes #3363
2020-06-05 07:25:11 +02:00
+ MCUtil.scheduleTask(5, () -> {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
+ if (remainingSends.get() == 0) {
+ cleaner1.run();
+ cleaner2.run();
+ }
Improve Chunk Prioritization and Internal Scheduler In previous MC versions, we had a rather simple internal scheduler for delayed tasks that would just keep pushing task back until desired tick was reached. The method it called to schedule the task changed behavior in 1.14, and now this scheduler is not working nowhere near what it was supposed to be doing. This was causing long delayed task to eat up CPU (In Oversleep for example) Rewrite this to just use the CraftScheduler for scheduling delayed tasks. Once this was fixed, it became quite clear the code that delayed ticket additions for chunks based on distance was clearly not right, as it was tested on the previous broken logic. So the ticket delay process has been vastly revamped to be even smarter. Chunks behind the player can load slower than the chunks in front of the player. We also can delay ticket adding until one of its neighbors has loaded, as this lets us get a smoother spiral out for the chunks (minus frustum intent). Additionally on frustum previous commit inadvertently broke frustum trying to fix an issue when the real fix lied elsewhere, so restore chunk priority so it works again.
2020-06-09 09:17:25 +02:00
+ }, "Light Packet Release");
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
+ }
+ }
+
+ @Override
+ public boolean hasFinishListener() {
+ return true;
+ }
+
+ // Paper end
public PacketPlayOutLightUpdate() {}
Current non compilable status of all patches - THIS IS NOT READY THERE IS STILL NO ETA. GOBLINS WILL EAT YOU.
2020-06-26 03:58:00 +02:00
public PacketPlayOutLightUpdate(ChunkCoordIntPair chunkcoordintpair, LightEngine lightengine, boolean flag) {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
this.a = chunkcoordintpair.x;
this.b = chunkcoordintpair.z;
Current non compilable status of all patches - THIS IS NOT READY THERE IS STILL NO ETA. GOBLINS WILL EAT YOU.
2020-06-26 03:58:00 +02:00
this.i = flag;
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
- this.g = Lists.newArrayList();
- this.h = Lists.newArrayList();
+ this.g = Lists.newArrayList();cleaner1 = MCUtil.registerListCleaner(this, this.g, NibbleArray::releaseBytes); // Paper
+ this.h = Lists.newArrayList();cleaner2 = MCUtil.registerListCleaner(this, this.h, NibbleArray::releaseBytes); // Paper
for (int i = 0; i < 18; ++i) {
NibbleArray nibblearray = lightengine.a(EnumSkyBlock.SKY).a(SectionPosition.a(chunkcoordintpair, -1 + i));
Latest progress - restored patches, some issues resolved
2020-06-26 08:29:44 +02:00
@@ -35,7 +66,7 @@ public class PacketPlayOutLightUpdate implements Packet<PacketListenerPlayOut> {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
this.e |= 1 << i;
} else {
this.c |= 1 << i;
- this.g.add(nibblearray.asBytes().clone());
+ this.g.add(nibblearray.getCloneIfSet()); // Paper
}
}
Latest progress - restored patches, some issues resolved
2020-06-26 08:29:44 +02:00
@@ -44,7 +75,7 @@ public class PacketPlayOutLightUpdate implements Packet<PacketListenerPlayOut> {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
this.f |= 1 << i;
} else {
this.d |= 1 << i;
- this.h.add(nibblearray1.asBytes().clone());
+ this.h.add(nibblearray1.getCloneIfSet()); // Paper
}
}
}
Latest progress - restored patches, some issues resolved
2020-06-26 08:29:44 +02:00
@@ -57,8 +88,8 @@ public class PacketPlayOutLightUpdate implements Packet<PacketListenerPlayOut> {
Current non compilable status of all patches - THIS IS NOT READY THERE IS STILL NO ETA. GOBLINS WILL EAT YOU.
2020-06-26 03:58:00 +02:00
this.i = flag;
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
this.c = i;
this.d = j;
- this.g = Lists.newArrayList();
- this.h = Lists.newArrayList();
+ this.g = Lists.newArrayList();cleaner1 = MCUtil.registerListCleaner(this, this.g, NibbleArray::releaseBytes); // Paper
+ this.h = Lists.newArrayList();cleaner2 = MCUtil.registerListCleaner(this, this.h, NibbleArray::releaseBytes); // Paper
for (int k = 0; k < 18; ++k) {
NibbleArray nibblearray;
Latest progress - restored patches, some issues resolved
2020-06-26 08:29:44 +02:00
@@ -66,7 +97,7 @@ public class PacketPlayOutLightUpdate implements Packet<PacketListenerPlayOut> {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
if ((this.c & 1 << k) != 0) {
nibblearray = lightengine.a(EnumSkyBlock.SKY).a(SectionPosition.a(chunkcoordintpair, -1 + k));
if (nibblearray != null && !nibblearray.c()) {
- this.g.add(nibblearray.asBytes().clone());
+ this.g.add(nibblearray.getCloneIfSet()); // Paper
} else {
this.c &= ~(1 << k);
if (nibblearray != null) {
Latest progress - restored patches, some issues resolved
2020-06-26 08:29:44 +02:00
@@ -78,7 +109,7 @@ public class PacketPlayOutLightUpdate implements Packet<PacketListenerPlayOut> {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
if ((this.d & 1 << k) != 0) {
nibblearray = lightengine.a(EnumSkyBlock.BLOCK).a(SectionPosition.a(chunkcoordintpair, -1 + k));
if (nibblearray != null && !nibblearray.c()) {
- this.h.add(nibblearray.asBytes().clone());
+ this.h.add(nibblearray.getCloneIfSet()); // Paper
} else {
this.d &= ~(1 << k);
if (nibblearray != null) {
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
diff --git a/src/main/java/org/bukkit/craftbukkit/CraftChunk.java b/src/main/java/org/bukkit/craftbukkit/CraftChunk.java
1.16.2 Release (#4123) PaperMC believes that 1.16.2 is now ready for general release as we fixed the main issue plagueing the 1.16.x release, the MapLike data conversion issues. Until now, it was not safe for a server to convert a world to 1.16.2 without data conversion issues around villages and potentially other things. If you did, those MapLike errors meant something went wrong. This is now resolved. Big thanks to all those that helped, notably @BillyGalbreath and @Proximyst who did large parts of the update process with me. Please as always, backup your worlds and test before updating to 1.16.2! If you update to 1.16.2, there is no going back to an older build than this. --------------------------------- Co-authored-by: William Blake Galbreath <Blake.Galbreath@GMail.com> Co-authored-by: Mariell Hoversholm <proximyst@proximyst.com> Co-authored-by: krolik-exe <69214078+krolik-exe@users.noreply.github.com> Co-authored-by: BillyGalbreath <BillyGalbreath@users.noreply.github.com> Co-authored-by: stonar96 <minecraft.stonar96@gmail.com> Co-authored-by: Shane Freeder <theboyetronic@gmail.com> Co-authored-by: Jason <jasonpenilla2@me.com> Co-authored-by: kashike <kashike@vq.lc> Co-authored-by: Aurora <21148213+aurorasmiles@users.noreply.github.com> Co-authored-by: KennyTV <kennytv@t-online.de> Co-authored-by: commandblockguy <commandblockguy1@gmail.com> Co-authored-by: DigitalRegent <misterwener@gmail.com> Co-authored-by: ishland <ishlandmc@yeah.net>
2020-08-25 04:22:08 +02:00
index 2a08a3170fcef666cb3096511d8a592e575c9734..d2c45157a27c24651c72fd8ab9a0950dec097b84 100644
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
--- a/src/main/java/org/bukkit/craftbukkit/CraftChunk.java
+++ b/src/main/java/org/bukkit/craftbukkit/CraftChunk.java
1.16.2 Release (#4123) PaperMC believes that 1.16.2 is now ready for general release as we fixed the main issue plagueing the 1.16.x release, the MapLike data conversion issues. Until now, it was not safe for a server to convert a world to 1.16.2 without data conversion issues around villages and potentially other things. If you did, those MapLike errors meant something went wrong. This is now resolved. Big thanks to all those that helped, notably @BillyGalbreath and @Proximyst who did large parts of the update process with me. Please as always, backup your worlds and test before updating to 1.16.2! If you update to 1.16.2, there is no going back to an older build than this. --------------------------------- Co-authored-by: William Blake Galbreath <Blake.Galbreath@GMail.com> Co-authored-by: Mariell Hoversholm <proximyst@proximyst.com> Co-authored-by: krolik-exe <69214078+krolik-exe@users.noreply.github.com> Co-authored-by: BillyGalbreath <BillyGalbreath@users.noreply.github.com> Co-authored-by: stonar96 <minecraft.stonar96@gmail.com> Co-authored-by: Shane Freeder <theboyetronic@gmail.com> Co-authored-by: Jason <jasonpenilla2@me.com> Co-authored-by: kashike <kashike@vq.lc> Co-authored-by: Aurora <21148213+aurorasmiles@users.noreply.github.com> Co-authored-by: KennyTV <kennytv@t-online.de> Co-authored-by: commandblockguy <commandblockguy1@gmail.com> Co-authored-by: DigitalRegent <misterwener@gmail.com> Co-authored-by: ishland <ishlandmc@yeah.net>
2020-08-25 04:22:08 +02:00
@@ -274,14 +274,14 @@ public class CraftChunk implements Chunk {
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
sectionSkyLights[i] = emptyLight;
} else {
sectionSkyLights[i] = new byte[2048];
- System.arraycopy(skyLightArray.asBytes(), 0, sectionSkyLights[i], 0, 2048);
+ System.arraycopy(skyLightArray.getIfSet(), 0, sectionSkyLights[i], 0, 2048); // Paper
}
NibbleArray emitLightArray = lightengine.a(EnumSkyBlock.BLOCK).a(SectionPosition.a(x, i, z));
if (emitLightArray == null) {
sectionEmitLights[i] = emptyLight;
} else {
sectionEmitLights[i] = new byte[2048];
- System.arraycopy(emitLightArray.asBytes(), 0, sectionEmitLights[i], 0, 2048);
+ System.arraycopy(emitLightArray.getIfSet(), 0, sectionEmitLights[i], 0, 2048); // Paper
}
}
}
Reference in New Issue Copy Permalink