Paper/patches/server/0260-Asynchronous-chunk-IO-and-loading.patch

3725 lines
169 KiB
Diff
Raw Normal View History

2021-06-11 14:02:28 +02:00
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Spottedleaf <Spottedleaf@users.noreply.github.com>
Date: Sat, 13 Jul 2019 09:23:10 -0700
Subject: [PATCH] Asynchronous chunk IO and loading
This patch re-adds a file IO thread as well as shoving de-serializing
chunk NBT data onto worker threads. This patch also will shove
chunk data serialization onto the same worker threads when the chunk
is unloaded - this cannot be done for regular saves since that's unsafe.
The file IO Thread
Unlike 1.13 and below, the file IO thread is prioritized - IO tasks can
be reoredered, however they are "stuck" to a world & coordinate.
Scheduling IO tasks works as follows, given a world & coordinate - location:
The IO thread has been designed to ensure that reads and writes appear to
occur synchronously for a given location, however the implementation also
has the unfortunate side-effect of making every write appear as if
they occur without failure.
The IO thread has also been designed to accomodate Mojang's decision to
store chunk data and POI data separately. It can independently schedule
tasks for each.
However threads can wait for writes to complete and check if:
- The write was overwriten by another scheduler
- The write failed (however it does not indicate whether it was overwritten by another scheduler)
Scheduling reads:
- If a write task is in progress, the task is not scheduled and returns the in-progress write data
This means that readers cannot modify the NBTTagCompound returned and must clone if it they wish to write
- If a write task is not in progress but a read task is in progress, then the read task is simply chained
This means that again, readers cannot modify the NBTTagCompound returned
Scheduling writes:
- If a read task is in progress, ignore the read task and schedule the write
We cannot complete the read task since we assume it wants old data - not current
- If a write task is pending, overwrite the write data
The file IO thread does correctly handle cases where the data is overwritten when it
is writing data (before completing a task it will check if the data was overwritten and
will retry).
When the file IO thread executes a task for a location, the it will
execute the read task first (if it exists), then it will execute the
write task. This ensures that, even when scheduling at different
priorities, that reads/writes for a location act synchronously.
The downside of the file IO thread is that write failure can only be
indicated to the scheduling thread if:
- No other thread decides to schedule another write for the location
concurrently
- The scheduling thread blocks on the write to complete (however the
current implementation can be modified to indicate success
asynchronously)
The file io thread can be modified easily to provide indications
of write failure and write overwriting if needed.
The upside of the file IO thread is that if a write failures, then
chunk data is not lost until server restart. This leaves more room
for spurious failure.
Finally, the io thread will indicate to the console when reads
or writes fail - with relevant detail.
Asynchronous chunk data serialization for unloading chunks
When chunks unload they make a call to PlayerChunkMap#saveChunk(IChunkAccess).
Even if I make the IO asynchronous for this call, the data serialization
still hits pretty hard. And given that now the chunk system will
aggressively unload chunks more often (queued immediately at
ticket level 45 or higher), unloads occur more often, and
combined with our changes to the unload queue to make it
significantly more aggresive - chunk unloads can hit pretty hard.
Especially players running around with elytras and fireworks.
For serializing chunk data off main, there are some tasks which cannot be
done asynchronously. Lighting data must be saved beforehand as well as
potentially some tick lists. These are completed before scheduling the
asynchronous save.
However serializing chunk data off of the main thread is still risky.
Even though this patch schedules the save to occur after ALL references
of the chunk are removed from the world, plugins can still technically
access entities inside the chunks. For this, if the serialization task
fails for any reason, it will be re-scheduled to be serialized on the
main thread - with the hopes that the reason it failed was due to a plugin
and not an error with the save code itself. Like vanilla code - if the
serialization fails, the chunk data is lost.
Asynchronous chunk io/loading
Mojang's current implementation for loading chunk data off disk is
to return a CompletableFuture that will be completed by scheduling a
task to be executed on the world's chunk queue (which is only drained
on the main thread). This task will read the IO off disk and it will
apply data conversions & deserialization synchronously. Obviously
all 3 of these operations are expensive however all can be completed
asynchronously instead.
The solution this patch uses is as follows:
0. If an asynchronous chunk save is in progress (see above), wait
for that task to complete. It will use the serialized NBTTagCompound
created by the task. If the task fails to complete, then we would continue
with step 1. If it does not, we skip step 1. (Note: We actually load
POI data no matter what in this case).
1. Schedule an IO task to read chunk & poi data off disk.
2. The IO task will schedule a chunk load task.
3. The chunk load task executes on the async chunk loader threads
and will apply datafixers & de-serialize the chunk into a ProtoChunk
or ProtoChunkExtension.
4. The in progress chunk is then passed on to the world's chunk queue
to complete the ComletableFuture and execute any of the synchronous
tasks required to be executed by the chunk load task (i.e lighting
and some poi tasks).
diff --git a/src/main/java/co/aikar/timings/WorldTimingsHandler.java b/src/main/java/co/aikar/timings/WorldTimingsHandler.java
2021-06-13 04:16:10 +02:00
index 0fda52841b5e1643efeda92106124998abc4e0aa..fe79c0add4f7cb18d487c5bb9415c40c5b551ea2 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/co/aikar/timings/WorldTimingsHandler.java
+++ b/src/main/java/co/aikar/timings/WorldTimingsHandler.java
2021-06-13 04:16:10 +02:00
@@ -58,6 +58,16 @@ public class WorldTimingsHandler {
2021-06-11 14:02:28 +02:00
public final Timing miscMobSpawning;
+ public final Timing poiUnload;
+ public final Timing chunkUnload;
+ public final Timing poiSaveDataSerialization;
+ public final Timing chunkSave;
+ public final Timing chunkSaveDataSerialization;
+ public final Timing chunkSaveIOWait;
+ public final Timing chunkUnloadPrepareSave;
+ public final Timing chunkUnloadPOISerialization;
+ public final Timing chunkUnloadDataSave;
+
public WorldTimingsHandler(Level server) {
String name = ((PrimaryLevelData) server.getLevelData()).getLevelName() + " - ";
2021-06-13 04:16:10 +02:00
@@ -111,6 +121,16 @@ public class WorldTimingsHandler {
2021-06-11 14:02:28 +02:00
miscMobSpawning = Timings.ofSafe(name + "Mob spawning - Misc");
+
+ poiUnload = Timings.ofSafe(name + "Chunk unload - POI");
+ chunkUnload = Timings.ofSafe(name + "Chunk unload - Chunk");
+ poiSaveDataSerialization = Timings.ofSafe(name + "Chunk save - POI Data serialization");
+ chunkSave = Timings.ofSafe(name + "Chunk save - Chunk");
+ chunkSaveDataSerialization = Timings.ofSafe(name + "Chunk save - Chunk Data serialization");
+ chunkSaveIOWait = Timings.ofSafe(name + "Chunk save - Chunk IO Wait");
+ chunkUnloadPrepareSave = Timings.ofSafe(name + "Chunk unload - Async Save Prepare");
+ chunkUnloadPOISerialization = Timings.ofSafe(name + "Chunk unload - POI Data Serialization");
+ chunkUnloadDataSave = Timings.ofSafe(name + "Chunk unload - Data Serialization");
}
public static Timing getTickList(ServerLevel worldserver, String timingsType) {
diff --git a/src/main/java/com/destroystokyo/paper/PaperConfig.java b/src/main/java/com/destroystokyo/paper/PaperConfig.java
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
index dd81701230133442186524e9cd65d70232b824f3..8a1f581bd7ef8a20329813dedb5bb952e32d41ae 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/com/destroystokyo/paper/PaperConfig.java
+++ b/src/main/java/com/destroystokyo/paper/PaperConfig.java
@@ -1,5 +1,6 @@
package com.destroystokyo.paper;
+import com.destroystokyo.paper.io.chunk.ChunkTaskManager;
import com.google.common.base.Strings;
import com.google.common.base.Throwables;
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -337,4 +338,58 @@ public class PaperConfig {
2021-06-13 04:16:10 +02:00
}
tabSpamLimit = getInt("settings.spam-limiter.tab-spam-limit", tabSpamLimit);
2021-06-11 14:02:28 +02:00
}
+
+ public static boolean asyncChunks = false;
+ private static void asyncChunks() {
+ ConfigurationSection section;
+ if (version < 15) {
+ section = config.createSection("settings.async-chunks");
+ section.set("threads", -1);
+ } else {
+ section = config.getConfigurationSection("settings.async-chunks");
+ if (section == null) {
+ section = config.createSection("settings.async-chunks");
+ }
+ }
+ // Clean up old configs
+ if (section.contains("load-threads")) {
+ if (!section.contains("threads")) {
+ section.set("threads", section.get("load-threads"));
+ }
+ section.set("load-threads", null);
+ }
+ section.set("generation", null);
+ section.set("enabled", null);
+ section.set("thread-per-world-generation", null);
+
+ int threads = getInt("settings.async-chunks.threads", -1);
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
+ int cpus = Runtime.getRuntime().availableProcessors() / 2;
2021-06-11 14:02:28 +02:00
+ if (threads <= 0) {
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
+ if (cpus <= 4) {
+ threads = cpus <= 2 ? 1 : 2;
+ } else {
+ threads = (int) Math.min(Integer.getInteger("paper.maxChunkThreads", 4), cpus / 2);
+ }
2021-06-11 14:02:28 +02:00
+ }
+ if (cpus == 1 && !Boolean.getBoolean("Paper.allowAsyncChunksSingleCore")) {
+ asyncChunks = false;
+ } else {
+ asyncChunks = true;
+ }
+
+ // Let Shared Host set some limits
+ String sharedHostThreads = System.getenv("PAPER_ASYNC_CHUNKS_SHARED_HOST_THREADS");
+ if (sharedHostThreads != null) {
+ try {
+ threads = Math.max(1, Math.min(threads, Integer.parseInt(sharedHostThreads)));
+ } catch (NumberFormatException ignored) {}
+ }
+
+ if (!asyncChunks) {
+ log("Async Chunks: Disabled - Chunks will be managed synchronously, and will cause tremendous lag.");
+ } else {
+ ChunkTaskManager.initGlobalLoadThreads(threads);
+ log("Async Chunks: Enabled - Chunks will be loaded much faster, without lag.");
+ }
+ }
}
diff --git a/src/main/java/com/destroystokyo/paper/io/IOUtil.java b/src/main/java/com/destroystokyo/paper/io/IOUtil.java
new file mode 100644
index 0000000000000000000000000000000000000000..5af0ac3d9e87c06053e65433060f15779c156c2a
--- /dev/null
+++ b/src/main/java/com/destroystokyo/paper/io/IOUtil.java
@@ -0,0 +1,62 @@
+package com.destroystokyo.paper.io;
+
+import org.bukkit.Bukkit;
+
+public final class IOUtil {
+
+ /* Copied from concrete or concurrentutil */
+
+ public static long getCoordinateKey(final int x, final int z) {
+ return ((long)z << 32) | (x & 0xFFFFFFFFL);
+ }
+
+ public static int getCoordinateX(final long key) {
+ return (int)key;
+ }
+
+ public static int getCoordinateZ(final long key) {
+ return (int)(key >>> 32);
+ }
+
+ public static int getRegionCoordinate(final int chunkCoordinate) {
+ return chunkCoordinate >> 5;
+ }
+
+ public static int getChunkInRegion(final int chunkCoordinate) {
+ return chunkCoordinate & 31;
+ }
+
+ public static String genericToString(final Object object) {
+ return object == null ? "null" : object.getClass().getName() + ":" + object.toString();
+ }
+
+ public static <T> T notNull(final T obj) {
+ if (obj == null) {
+ throw new NullPointerException();
+ }
+ return obj;
+ }
+
+ public static <T> T notNull(final T obj, final String msgIfNull) {
+ if (obj == null) {
+ throw new NullPointerException(msgIfNull);
+ }
+ return obj;
+ }
+
+ public static void arrayBounds(final int off, final int len, final int arrayLength, final String msgPrefix) {
+ if (off < 0 || len < 0 || (arrayLength - off) < len) {
+ throw new ArrayIndexOutOfBoundsException(msgPrefix + ": off: " + off + ", len: " + len + ", array length: " + arrayLength);
+ }
+ }
+
+ public static int getPriorityForCurrentThread() {
+ return Bukkit.isPrimaryThread() ? PrioritizedTaskQueue.HIGHEST_PRIORITY : PrioritizedTaskQueue.NORMAL_PRIORITY;
+ }
+
+ @SuppressWarnings("unchecked")
+ public static <T extends Throwable> void rethrow(final Throwable throwable) throws T {
+ throw (T)throwable;
+ }
+
+}
diff --git a/src/main/java/com/destroystokyo/paper/io/PaperFileIOThread.java b/src/main/java/com/destroystokyo/paper/io/PaperFileIOThread.java
new file mode 100644
index 0000000000000000000000000000000000000000..a630a84b60b4517e3bc330d4983b914bd064efa4
--- /dev/null
+++ b/src/main/java/com/destroystokyo/paper/io/PaperFileIOThread.java
@@ -0,0 +1,606 @@
+package com.destroystokyo.paper.io;
+
+import net.minecraft.nbt.CompoundTag;
+import net.minecraft.server.MinecraftServer;
+import net.minecraft.server.level.ServerLevel;
+import net.minecraft.world.level.ChunkPos;
+import net.minecraft.world.level.chunk.storage.RegionFile;
+import org.apache.logging.log4j.Logger;
+
+import java.io.IOException;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.function.Consumer;
+import java.util.function.Function;
+
+/**
+ * Prioritized singleton thread responsible for all chunk IO that occurs in a minecraft server.
+ *
+ * <p>
+ * Singleton access: {@link Holder#INSTANCE}
+ * </p>
+ *
+ * <p>
+ * All functions provided are MT-Safe, however certain ordering constraints are (but not enforced):
+ * <li>
+ * Chunk saves may not occur for unloaded chunks.
+ * </li>
+ * <li>
+ * Tasks must be scheduled on the main thread.
+ * </li>
+ * </p>
+ *
+ * @see Holder#INSTANCE
+ * @see #scheduleSave(ServerLevel, int, int, CompoundTag, CompoundTag, int)
+ * @see #loadChunkDataAsync(ServerLevel, int, int, int, Consumer, boolean, boolean, boolean)
+ */
+public final class PaperFileIOThread extends QueueExecutorThread {
+
+ public static final Logger LOGGER = MinecraftServer.LOGGER;
+ public static final CompoundTag FAILURE_VALUE = new CompoundTag();
+
+ public static final class Holder {
+
+ public static final PaperFileIOThread INSTANCE = new PaperFileIOThread();
+
+ static {
+ INSTANCE.start();
+ }
+ }
+
+ private final AtomicLong writeCounter = new AtomicLong();
+
+ private PaperFileIOThread() {
+ super(new PrioritizedTaskQueue<>(), (int)(1.0e6)); // 1.0ms spinwait time
+ this.setName("Paper RegionFile IO Thread");
+ this.setPriority(Thread.NORM_PRIORITY - 1); // we keep priority close to normal because threads can wait on us
+ this.setUncaughtExceptionHandler((final Thread unused, final Throwable thr) -> {
+ LOGGER.fatal("Uncaught exception thrown from IO thread, report this!", thr);
+ });
+ }
+
+ /* run() is implemented by superclass */
+
+ /*
+ *
+ * IO thread will perform reads before writes
+ *
+ * How reads/writes are scheduled:
+ *
+ * If read in progress while scheduling write, ignore read and schedule write
+ * If read in progress while scheduling read (no write in progress), chain the read task
+ *
+ *
+ * If write in progress while scheduling read, use the pending write data and ret immediately
+ * If write in progress while scheduling write (ignore read in progress), overwrite the write in progress data
+ *
+ * This allows the reads and writes to act as if they occur synchronously to the thread scheduling them, however
+ * it fails to properly propagate write failures. When writes fail the data is kept so future reads will actually
+ * read the failed write data. This should hopefully act as a way to prevent data loss for spurious fails for writing data.
+ *
+ */
+
+ /**
+ * Attempts to bump the priority of all IO tasks for the given chunk coordinates. This has no effect if no tasks are queued.
+ * @param world Chunk's world
+ * @param chunkX Chunk's x coordinate
+ * @param chunkZ Chunk's z coordinate
+ * @param priority Priority level to try to bump to
+ */
+ public void bumpPriority(final ServerLevel world, final int chunkX, final int chunkZ, final int priority) {
+ if (!PrioritizedTaskQueue.validPriority(priority)) {
+ throw new IllegalArgumentException("Invalid priority: " + priority);
+ }
+
+ final Long key = Long.valueOf(IOUtil.getCoordinateKey(chunkX, chunkZ));
+
+ final ChunkDataTask poiTask = world.poiDataController.tasks.get(key);
+ final ChunkDataTask chunkTask = world.chunkDataController.tasks.get(key);
+
+ if (poiTask != null) {
+ poiTask.raisePriority(priority);
+ }
+ if (chunkTask != null) {
+ chunkTask.raisePriority(priority);
+ }
+ }
+
+ public CompoundTag getPendingWrite(final ServerLevel world, final int chunkX, final int chunkZ, final boolean poiData) {
+ final ChunkDataController taskController = poiData ? world.poiDataController : world.chunkDataController;
+
+ final ChunkDataTask dataTask = taskController.tasks.get(Long.valueOf(IOUtil.getCoordinateKey(chunkX, chunkZ)));
+
+ if (dataTask == null) {
+ return null;
+ }
+
+ final ChunkDataController.InProgressWrite write = dataTask.inProgressWrite;
+
+ if (write == null) {
+ return null;
+ }
+
+ return write.data;
+ }
+
+ /**
+ * Sets the priority of all IO tasks for the given chunk coordinates. This has no effect if no tasks are queued.
+ * @param world Chunk's world
+ * @param chunkX Chunk's x coordinate
+ * @param chunkZ Chunk's z coordinate
+ * @param priority Priority level to set to
+ */
+ public void setPriority(final ServerLevel world, final int chunkX, final int chunkZ, final int priority) {
+ if (!PrioritizedTaskQueue.validPriority(priority)) {
+ throw new IllegalArgumentException("Invalid priority: " + priority);
+ }
+
+ final Long key = Long.valueOf(IOUtil.getCoordinateKey(chunkX, chunkZ));
+
+ final ChunkDataTask poiTask = world.poiDataController.tasks.get(key);
+ final ChunkDataTask chunkTask = world.chunkDataController.tasks.get(key);
+
+ if (poiTask != null) {
+ poiTask.updatePriority(priority);
+ }
+ if (chunkTask != null) {
+ chunkTask.updatePriority(priority);
+ }
+ }
+
+ /**
+ * Schedules the chunk data to be written asynchronously.
+ * <p>
+ * Impl notes:
+ * </p>
+ * <li>
+ * This function presumes a chunk load for the coordinates is not called during this function (anytime after is OK). This means
+ * saves must be scheduled before a chunk is unloaded.
+ * </li>
+ * <li>
+ * Writes may be called concurrently, although only the "later" write will go through.
+ * </li>
+ * @param world Chunk's world
+ * @param chunkX Chunk's x coordinate
+ * @param chunkZ Chunk's z coordinate
+ * @param poiData Chunk point of interest data. If {@code null}, then no poi data is saved.
+ * @param chunkData Chunk data. If {@code null}, then no chunk data is saved.
+ * @param priority Priority level for this task. See {@link PrioritizedTaskQueue}
+ * @throws IllegalArgumentException If both {@code poiData} and {@code chunkData} are {@code null}.
+ * @throws IllegalStateException If the file io thread has shutdown.
+ */
+ public void scheduleSave(final ServerLevel world, final int chunkX, final int chunkZ,
+ final CompoundTag poiData, final CompoundTag chunkData,
+ final int priority) throws IllegalArgumentException {
+ if (!PrioritizedTaskQueue.validPriority(priority)) {
+ throw new IllegalArgumentException("Invalid priority: " + priority);
+ }
+
+ final long writeCounter = this.writeCounter.getAndIncrement();
+
+ if (poiData != null) {
+ this.scheduleWrite(world.poiDataController, world, chunkX, chunkZ, poiData, priority, writeCounter);
+ }
+ if (chunkData != null) {
+ this.scheduleWrite(world.chunkDataController, world, chunkX, chunkZ, chunkData, priority, writeCounter);
+ }
+ }
+
+ private void scheduleWrite(final ChunkDataController dataController, final ServerLevel world,
+ final int chunkX, final int chunkZ, final CompoundTag data, final int priority, final long writeCounter) {
+ dataController.tasks.compute(Long.valueOf(IOUtil.getCoordinateKey(chunkX, chunkZ)), (final Long keyInMap, final ChunkDataTask taskRunning) -> {
+ if (taskRunning == null) {
+ // no task is scheduled
+
+ // create task
+ final ChunkDataTask newTask = new ChunkDataTask(priority, world, chunkX, chunkZ, dataController);
+ newTask.inProgressWrite = new ChunkDataController.InProgressWrite();
+ newTask.inProgressWrite.writeCounter = writeCounter;
+ newTask.inProgressWrite.data = data;
+
+ PaperFileIOThread.this.queueTask(newTask); // schedule
+ return newTask;
+ }
+
+ taskRunning.raisePriority(priority);
+
+ if (taskRunning.inProgressWrite == null) {
+ taskRunning.inProgressWrite = new ChunkDataController.InProgressWrite();
+ }
+
+ boolean reschedule = taskRunning.inProgressWrite.writeCounter == -1L;
+
+ // synchronize for readers
+ //noinspection SynchronizationOnLocalVariableOrMethodParameter
+ synchronized (taskRunning) {
+ taskRunning.inProgressWrite.data = data;
+ taskRunning.inProgressWrite.writeCounter = writeCounter;
+ }
+
+ if (reschedule) {
+ // We need to reschedule this task since the previous one is not currently scheduled since it failed
+ taskRunning.reschedule(priority);
+ }
+
+ return taskRunning;
+ });
+ }
+
+ /**
+ * Same as {@link #loadChunkDataAsync(ServerLevel, int, int, int, Consumer, boolean, boolean, boolean)}, except this function returns
+ * a {@link CompletableFuture} which is potentially completed <b>ASYNCHRONOUSLY ON THE FILE IO THREAD</b> when the load task
+ * has completed.
+ * <p>
+ * Note that if the chunk fails to load the returned future is completed with {@code null}.
+ * </p>
+ */
+ public CompletableFuture<ChunkData> loadChunkDataAsyncFuture(final ServerLevel world, final int chunkX, final int chunkZ,
+ final int priority, final boolean readPoiData, final boolean readChunkData,
+ final boolean intendingToBlock) {
+ final CompletableFuture<ChunkData> future = new CompletableFuture<>();
+ this.loadChunkDataAsync(world, chunkX, chunkZ, priority, future::complete, readPoiData, readChunkData, intendingToBlock);
+ return future;
+ }
+
+ /**
+ * Schedules a load to be executed asynchronously.
+ * <p>
+ * Impl notes:
+ * </p>
+ * <li>
+ * If a chunk fails to load, the {@code onComplete} parameter is completed with {@code null}.
+ * </li>
+ * <li>
+ * It is possible for the {@code onComplete} parameter to be given {@link ChunkData} containing data
+ * this call did not request.
+ * </li>
+ * <li>
+ * The {@code onComplete} parameter may be completed during the execution of this function synchronously or it may
+ * be completed asynchronously on this file io thread. Interacting with the file IO thread in the completion of
+ * data is undefined behaviour, and can cause deadlock.
+ * </li>
+ * @param world Chunk's world
+ * @param chunkX Chunk's x coordinate
+ * @param chunkZ Chunk's z coordinate
+ * @param priority Priority level for this task. See {@link PrioritizedTaskQueue}
+ * @param onComplete Consumer to execute once this task has completed
+ * @param readPoiData Whether to read point of interest data. If {@code false}, the {@code NBTTagCompound} will be {@code null}.
+ * @param readChunkData Whether to read chunk data. If {@code false}, the {@code NBTTagCompound} will be {@code null}.
+ * @return The {@link PrioritizedTaskQueue.PrioritizedTask} associated with this task. Note that this task does not support
+ * cancellation.
+ */
+ public void loadChunkDataAsync(final ServerLevel world, final int chunkX, final int chunkZ,
+ final int priority, final Consumer<ChunkData> onComplete,
+ final boolean readPoiData, final boolean readChunkData,
+ final boolean intendingToBlock) {
+ if (!PrioritizedTaskQueue.validPriority(priority)) {
+ throw new IllegalArgumentException("Invalid priority: " + priority);
+ }
+
+ if (!(readPoiData | readChunkData)) {
+ throw new IllegalArgumentException("Must read chunk data or poi data");
+ }
+
+ final ChunkData complete = new ChunkData();
+ final boolean[] requireCompletion = new boolean[] { readPoiData, readChunkData };
+
+ if (readPoiData) {
+ this.scheduleRead(world.poiDataController, world, chunkX, chunkZ, (final CompoundTag poiData) -> {
+ complete.poiData = poiData;
+
+ final boolean finished;
+
+ // avoid a race condition where the file io thread completes and we complete synchronously
+ // Note: Synchronization can be elided if both of the accesses are volatile
+ synchronized (requireCompletion) {
+ requireCompletion[0] = false; // 0 -> poi data
+ finished = !requireCompletion[1]; // 1 -> chunk data
+ }
+
+ if (finished) {
+ onComplete.accept(complete);
+ }
+ }, priority, intendingToBlock);
+ }
+
+ if (readChunkData) {
+ this.scheduleRead(world.chunkDataController, world, chunkX, chunkZ, (final CompoundTag chunkData) -> {
+ complete.chunkData = chunkData;
+
+ final boolean finished;
+
+ // avoid a race condition where the file io thread completes and we complete synchronously
+ // Note: Synchronization can be elided if both of the accesses are volatile
+ synchronized (requireCompletion) {
+ requireCompletion[1] = false; // 1 -> chunk data
+ finished = !requireCompletion[0]; // 0 -> poi data
+ }
+
+ if (finished) {
+ onComplete.accept(complete);
+ }
+ }, priority, intendingToBlock);
+ }
+
+ }
+
+ // Note: the onComplete may be called asynchronously or synchronously here.
+ private void scheduleRead(final ChunkDataController dataController, final ServerLevel world,
+ final int chunkX, final int chunkZ, final Consumer<CompoundTag> onComplete, final int priority,
+ final boolean intendingToBlock) {
+
+ Function<RegionFile, Boolean> tryLoadFunction = (final RegionFile file) -> {
+ if (file == null) {
+ return Boolean.TRUE;
+ }
+ return Boolean.valueOf(file.hasChunk(new ChunkPos(chunkX, chunkZ)));
+ };
+
+ dataController.tasks.compute(Long.valueOf(IOUtil.getCoordinateKey(chunkX, chunkZ)), (final Long keyInMap, final ChunkDataTask running) -> {
+ if (running == null) {
+ // not scheduled
+
+ final Boolean shouldSchedule = intendingToBlock ? dataController.computeForRegionFile(chunkX, chunkZ, tryLoadFunction) :
+ dataController.computeForRegionFileIfLoaded(chunkX, chunkZ, tryLoadFunction);
+
+ if (shouldSchedule == Boolean.FALSE) {
+ // not on disk
+ onComplete.accept(null);
+ return null;
+ }
+
+ // set up task
+ final ChunkDataTask newTask = new ChunkDataTask(priority, world, chunkX, chunkZ, dataController);
+ newTask.inProgressRead = new ChunkDataController.InProgressRead();
+ newTask.inProgressRead.readFuture.thenAccept(onComplete);
+
+ PaperFileIOThread.this.queueTask(newTask); // schedule task
+ return newTask;
+ }
+
+ running.raisePriority(priority);
+
+ if (running.inProgressWrite == null) {
+ // chain to the read future
+ running.inProgressRead.readFuture.thenAccept(onComplete);
+ return running;
+ }
+
+ // at this stage we have to use the in progress write's data to avoid an order issue
+ // we don't synchronize since all writes to data occur in the compute() call
+ onComplete.accept(running.inProgressWrite.data);
+ return running;
+ });
+ }
+
+ /**
+ * Same as {@link #loadChunkDataAsync(ServerLevel, int, int, int, Consumer, boolean, boolean, boolean)}, except this function returns
+ * the {@link ChunkData} associated with the specified chunk when the task is complete.
+ * @return The chunk data, or {@code null} if the chunk failed to load.
+ */
+ public ChunkData loadChunkData(final ServerLevel world, final int chunkX, final int chunkZ, final int priority,
+ final boolean readPoiData, final boolean readChunkData) {
+ return this.loadChunkDataAsyncFuture(world, chunkX, chunkZ, priority, readPoiData, readChunkData, true).join();
+ }
+
+ /**
+ * Schedules the given task at the specified priority to be executed on the IO thread.
+ * <p>
+ * Internal api. Do not use.
+ * </p>
+ */
+ public void runTask(final int priority, final Runnable runnable) {
+ this.queueTask(new GeneralTask(priority, runnable));
+ }
+
+ static final class GeneralTask extends PrioritizedTaskQueue.PrioritizedTask implements Runnable {
+
+ private final Runnable run;
+
+ public GeneralTask(final int priority, final Runnable run) {
+ super(priority);
+ this.run = IOUtil.notNull(run, "Task may not be null");
+ }
+
+ @Override
+ public void run() {
+ try {
+ this.run.run();
+ } catch (final Throwable throwable) {
+ if (throwable instanceof ThreadDeath) {
+ throw (ThreadDeath)throwable;
+ }
+ LOGGER.fatal("Failed to execute general task on IO thread " + IOUtil.genericToString(this.run), throwable);
+ }
+ }
+ }
+
+ public static final class ChunkData {
+
+ public CompoundTag poiData;
+ public CompoundTag chunkData;
+
+ public ChunkData() {}
+
+ public ChunkData(final CompoundTag poiData, final CompoundTag chunkData) {
+ this.poiData = poiData;
+ this.chunkData = chunkData;
+ }
+ }
+
+ public static abstract class ChunkDataController {
+
+ // ConcurrentHashMap synchronizes per chain, so reduce the chance of task's hashes colliding.
+ public final ConcurrentHashMap<Long, ChunkDataTask> tasks = new ConcurrentHashMap<>(64, 0.5f);
+
+ public abstract void writeData(final int x, final int z, final CompoundTag compound) throws IOException;
+ public abstract CompoundTag readData(final int x, final int z) throws IOException;
+
+ public abstract <T> T computeForRegionFile(final int chunkX, final int chunkZ, final Function<RegionFile, T> function);
+ public abstract <T> T computeForRegionFileIfLoaded(final int chunkX, final int chunkZ, final Function<RegionFile, T> function);
+
+ public static final class InProgressWrite {
+ public long writeCounter;
+ public CompoundTag data;
+ }
+
+ public static final class InProgressRead {
+ public final CompletableFuture<CompoundTag> readFuture = new CompletableFuture<>();
+ }
+ }
+
+ public static final class ChunkDataTask extends PrioritizedTaskQueue.PrioritizedTask implements Runnable {
+
+ public ChunkDataController.InProgressWrite inProgressWrite;
+ public ChunkDataController.InProgressRead inProgressRead;
+
+ private final ServerLevel world;
+ private final int x;
+ private final int z;
+ private final ChunkDataController taskController;
+
+ public ChunkDataTask(final int priority, final ServerLevel world, final int x, final int z, final ChunkDataController taskController) {
+ super(priority);
+ this.world = world;
+ this.x = x;
+ this.z = z;
+ this.taskController = taskController;
+ }
+
+ @Override
+ public String toString() {
+ return "Task for world: '" + this.world.getWorld().getName() + "' at " + this.x + "," + this.z +
+ " poi: " + (this.taskController == this.world.poiDataController) + ", hash: " + this.hashCode();
+ }
+
+ /*
+ *
+ * IO thread will perform reads before writes
+ *
+ * How reads/writes are scheduled:
+ *
+ * If read in progress while scheduling write, ignore read and schedule write
+ * If read in progress while scheduling read (no write in progress), chain the read task
+ *
+ *
+ * If write in progress while scheduling read, use the pending write data and ret immediately
+ * If write in progress while scheduling write (ignore read in progress), overwrite the write in progress data
+ *
+ * This allows the reads and writes to act as if they occur synchronously to the thread scheduling them, however
+ * it fails to properly propagate write failures
+ *
+ */
+
+ void reschedule(final int priority) {
+ // priority is checked before this stage // TODO what
+ this.queue.lazySet(null);
+ this.priority.lazySet(priority);
+ PaperFileIOThread.Holder.INSTANCE.queueTask(this);
+ }
+
+ @Override
+ public void run() {
+ ChunkDataController.InProgressRead read = this.inProgressRead;
+ if (read != null) {
+ CompoundTag compound = PaperFileIOThread.FAILURE_VALUE;
+ try {
+ compound = this.taskController.readData(this.x, this.z);
+ } catch (final Throwable thr) {
+ if (thr instanceof ThreadDeath) {
+ throw (ThreadDeath)thr;
+ }
+ LOGGER.fatal("Failed to read chunk data for task: " + this.toString(), thr);
+ // fall through to complete with null data
+ }
+ read.readFuture.complete(compound);
+ }
+
+ final Long chunkKey = Long.valueOf(IOUtil.getCoordinateKey(this.x, this.z));
+
+ ChunkDataController.InProgressWrite write = this.inProgressWrite;
+
+ if (write == null) {
+ // IntelliJ warns this is invalid, however it does not consider that writes to the task map & the inProgress field can occur concurrently.
+ ChunkDataTask inMap = this.taskController.tasks.compute(chunkKey, (final Long keyInMap, final ChunkDataTask valueInMap) -> {
+ if (valueInMap == null) {
+ throw new IllegalStateException("Write completed concurrently, expected this task: " + ChunkDataTask.this.toString() + ", report this!");
+ }
+ if (valueInMap != ChunkDataTask.this) {
+ throw new IllegalStateException("Chunk task mismatch, expected this task: " + ChunkDataTask.this.toString() + ", got: " + valueInMap.toString() + ", report this!");
+ }
+ return valueInMap.inProgressWrite == null ? null : valueInMap;
+ });
+
+ if (inMap == null) {
+ return; // set the task value to null, indicating we're done
+ }
+
+ // not null, which means there was a concurrent write
+ write = this.inProgressWrite;
+ }
+
+ // check if another process is writing
+ /*try { TODO: Can we restore this?
+ ((WorldServer)this.world).checkSession();
+ } catch (final Exception ex) {
+ LOGGER.fatal("Couldn't save chunk; already in use by another instance of Minecraft?", ex);
+ // we don't need to set the write counter to -1 as we know at this stage there's no point in re-scheduling
+ // writes since they'll fail anyways.
+ return;
+ }
+*/
+ for (;;) {
+ final long writeCounter;
+ final CompoundTag data;
+
+ //noinspection SynchronizationOnLocalVariableOrMethodParameter
+ synchronized (write) {
+ writeCounter = write.writeCounter;
+ data = write.data;
+ }
+
+ boolean failedWrite = false;
+
+ try {
+ this.taskController.writeData(this.x, this.z, data);
+ } catch (final Throwable thr) {
+ if (thr instanceof ThreadDeath) {
+ throw (ThreadDeath)thr;
+ }
+ LOGGER.fatal("Failed to write chunk data for task: " + this.toString(), thr);
+ failedWrite = true;
+ }
+
+ boolean finalFailWrite = failedWrite;
+
+ ChunkDataTask inMap = this.taskController.tasks.compute(chunkKey, (final Long keyInMap, final ChunkDataTask valueInMap) -> {
+ if (valueInMap == null) {
+ throw new IllegalStateException("Write completed concurrently, expected this task: " + ChunkDataTask.this.toString() + ", report this!");
+ }
+ if (valueInMap != ChunkDataTask.this) {
+ throw new IllegalStateException("Chunk task mismatch, expected this task: " + ChunkDataTask.this.toString() + ", got: " + valueInMap.toString() + ", report this!");
+ }
+ if (valueInMap.inProgressWrite.writeCounter == writeCounter) {
+ if (finalFailWrite) {
+ valueInMap.inProgressWrite.writeCounter = -1L;
+ }
+
+ return null;
+ }
+ return valueInMap;
+ // Hack end
+ });
+
+ if (inMap == null) {
+ // write counter matched, so we wrote the most up-to-date pending data, we're done here
+ // or we failed to write and successfully set the write counter to -1
+ return; // we're done here
+ }
+
+ // fetch & write new data
+ continue;
+ }
+ }
+ }
+}
diff --git a/src/main/java/com/destroystokyo/paper/io/PrioritizedTaskQueue.java b/src/main/java/com/destroystokyo/paper/io/PrioritizedTaskQueue.java
new file mode 100644
index 0000000000000000000000000000000000000000..97f2e433c483f1ebd7500ae142269e144ef5fda4
--- /dev/null
+++ b/src/main/java/com/destroystokyo/paper/io/PrioritizedTaskQueue.java
@@ -0,0 +1,277 @@
+package com.destroystokyo.paper.io;
+
+import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+
+public class PrioritizedTaskQueue<T extends PrioritizedTaskQueue.PrioritizedTask> {
+
+ // lower numbers are a higher priority (except < 0)
+ // higher priorities are always executed before lower priorities
+
+ /**
+ * Priority value indicating the task has completed or is being completed.
+ */
+ public static final int COMPLETING_PRIORITY = -1;
+
+ /**
+ * Highest priority, should only be used for main thread tasks or tasks that are blocking the main thread.
+ */
+ public static final int HIGHEST_PRIORITY = 0;
+
+ /**
+ * Should be only used in an IO task so that chunk loads do not wait on other IO tasks.
+ * This only exists because IO tasks are scheduled before chunk load tasks to decrease IO waiting times.
+ */
+ public static final int HIGHER_PRIORITY = 1;
+
+ /**
+ * Should be used for scheduling chunk loads/generation that would increase response times to users.
+ */
+ public static final int HIGH_PRIORITY = 2;
+
+ /**
+ * Default priority.
+ */
+ public static final int NORMAL_PRIORITY = 3;
+
+ /**
+ * Use for tasks not at all critical and can potentially be delayed.
+ */
+ public static final int LOW_PRIORITY = 4;
+
+ /**
+ * Use for tasks that should "eventually" execute.
+ */
+ public static final int LOWEST_PRIORITY = 5;
+
+ private static final int TOTAL_PRIORITIES = 6;
+
+ final ConcurrentLinkedQueue<T>[] queues = (ConcurrentLinkedQueue<T>[])new ConcurrentLinkedQueue[TOTAL_PRIORITIES];
+
+ private final AtomicBoolean shutdown = new AtomicBoolean();
+
+ {
+ for (int i = 0; i < TOTAL_PRIORITIES; ++i) {
+ this.queues[i] = new ConcurrentLinkedQueue<>();
+ }
+ }
+
+ /**
+ * Returns whether the specified priority is valid
+ */
+ public static boolean validPriority(final int priority) {
+ return priority >= 0 && priority < TOTAL_PRIORITIES;
+ }
+
+ /**
+ * Queues a task.
+ * @throws IllegalStateException If the task has already been queued. Use {@link PrioritizedTask#raisePriority(int)} to
+ * raise a task's priority.
+ * This can also be thrown if the queue has shutdown.
+ */
+ public void add(final T task) throws IllegalStateException {
+ int priority = task.getPriority();
+ if (priority != COMPLETING_PRIORITY) {
+ task.setQueue(this);
+ this.queues[priority].add(task);
+ }
+ if (this.shutdown.get()) {
+ // note: we're not actually sure at this point if our task will go through
+ throw new IllegalStateException("Queue has shutdown, refusing to execute task " + IOUtil.genericToString(task));
+ }
+ }
+
+ /**
+ * Polls the highest priority task currently available. {@code null} if none.
+ */
+ public T poll() {
+ T task;
+ for (int i = 0; i < TOTAL_PRIORITIES; ++i) {
+ final ConcurrentLinkedQueue<T> queue = this.queues[i];
+
+ while ((task = queue.poll()) != null) {
+ final int prevPriority = task.tryComplete(i);
+ if (prevPriority != COMPLETING_PRIORITY && prevPriority <= i) {
+ // if the prev priority was greater-than or equal to our current priority
+ return task;
+ }
+ }
+ }
+
+ return null;
+ }
+
+ /**
+ * Returns whether this queue may have tasks queued.
+ * <p>
+ * This operation is not atomic, but is MT-Safe.
+ * </p>
+ * @return {@code true} if tasks may be queued, {@code false} otherwise
+ */
+ public boolean hasTasks() {
+ for (int i = 0; i < TOTAL_PRIORITIES; ++i) {
+ final ConcurrentLinkedQueue<T> queue = this.queues[i];
+
+ if (queue.peek() != null) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ /**
+ * Prevent further additions to this queue. Attempts to add after this call has completed (potentially during) will
+ * result in {@link IllegalStateException} being thrown.
+ * <p>
+ * This operation is atomic with respect to other shutdown calls
+ * </p>
+ * <p>
+ * After this call has completed, regardless of return value, this queue will be shutdown.
+ * </p>
+ * @return {@code true} if the queue was shutdown, {@code false} if it has shut down already
+ */
+ public boolean shutdown() {
+ return this.shutdown.getAndSet(false);
+ }
+
+ public abstract static class PrioritizedTask {
+
+ protected final AtomicReference<PrioritizedTaskQueue> queue = new AtomicReference<>();
+
+ protected final AtomicInteger priority;
+
+ protected PrioritizedTask() {
+ this(PrioritizedTaskQueue.NORMAL_PRIORITY);
+ }
+
+ protected PrioritizedTask(final int priority) {
+ if (!PrioritizedTaskQueue.validPriority(priority)) {
+ throw new IllegalArgumentException("Invalid priority " + priority);
+ }
+ this.priority = new AtomicInteger(priority);
+ }
+
+ /**
+ * Returns the current priority. Note that {@link PrioritizedTaskQueue#COMPLETING_PRIORITY} will be returned
+ * if this task is completing or has completed.
+ */
+ public final int getPriority() {
+ return this.priority.get();
+ }
+
+ /**
+ * Returns whether this task is scheduled to execute, or has been already executed.
+ */
+ public boolean isScheduled() {
+ return this.queue.get() != null;
+ }
+
+ final int tryComplete(final int minPriority) {
+ for (int curr = this.getPriorityVolatile();;) {
+ if (curr == COMPLETING_PRIORITY) {
+ return COMPLETING_PRIORITY;
+ }
+ if (curr > minPriority) {
+ // curr is lower priority
+ return curr;
+ }
+
+ if (curr == (curr = this.compareAndExchangePriorityVolatile(curr, COMPLETING_PRIORITY))) {
+ return curr;
+ }
+ continue;
+ }
+ }
+
+ /**
+ * Forces this task to be completed.
+ * @return {@code true} if the task was cancelled, {@code false} if the task has already completed or is being completed.
+ */
+ public boolean cancel() {
+ return this.exchangePriorityVolatile(PrioritizedTaskQueue.COMPLETING_PRIORITY) != PrioritizedTaskQueue.COMPLETING_PRIORITY;
+ }
+
+ /**
+ * Attempts to raise the priority to the priority level specified.
+ * @param priority Priority specified
+ * @return {@code true} if successful, {@code false} otherwise.
+ */
+ public boolean raisePriority(final int priority) {
+ if (!PrioritizedTaskQueue.validPriority(priority)) {
+ throw new IllegalArgumentException("Invalid priority");
+ }
+
+ for (int curr = this.getPriorityVolatile();;) {
+ if (curr == COMPLETING_PRIORITY) {
+ return false;
+ }
+ if (priority >= curr) {
+ return true;
+ }
+
+ if (curr == (curr = this.compareAndExchangePriorityVolatile(curr, priority))) {
+ PrioritizedTaskQueue queue = this.queue.get();
+ if (queue != null) {
+ //noinspection unchecked
+ queue.queues[priority].add(this); // silently fail on shutdown
+ }
+ return true;
+ }
+ continue;
+ }
+ }
+
+ /**
+ * Attempts to set this task's priority level to the level specified.
+ * @param priority Specified priority level.
+ * @return {@code true} if successful, {@code false} if this task is completing or has completed.
+ */
+ public boolean updatePriority(final int priority) {
+ if (!PrioritizedTaskQueue.validPriority(priority)) {
+ throw new IllegalArgumentException("Invalid priority");
+ }
+
+ for (int curr = this.getPriorityVolatile();;) {
+ if (curr == COMPLETING_PRIORITY) {
+ return false;
+ }
+ if (curr == priority) {
+ return true;
+ }
+
+ if (curr == (curr = this.compareAndExchangePriorityVolatile(curr, priority))) {
+ PrioritizedTaskQueue queue = this.queue.get();
+ if (queue != null) {
+ //noinspection unchecked
+ queue.queues[priority].add(this); // silently fail on shutdown
+ }
+ return true;
+ }
+ continue;
+ }
+ }
+
+ void setQueue(final PrioritizedTaskQueue queue) {
+ this.queue.set(queue);
+ }
+
+ /* priority */
+
+ protected final int getPriorityVolatile() {
+ return this.priority.get();
+ }
+
+ protected final int compareAndExchangePriorityVolatile(final int expect, final int update) {
+ if (this.priority.compareAndSet(expect, update)) {
+ return expect;
+ }
+ return this.priority.get();
+ }
+
+ protected final int exchangePriorityVolatile(final int value) {
+ return this.priority.getAndSet(value);
+ }
+ }
+}
diff --git a/src/main/java/com/destroystokyo/paper/io/QueueExecutorThread.java b/src/main/java/com/destroystokyo/paper/io/QueueExecutorThread.java
new file mode 100644
index 0000000000000000000000000000000000000000..ee906b594b306906c170180a29a8b61997d05168
--- /dev/null
+++ b/src/main/java/com/destroystokyo/paper/io/QueueExecutorThread.java
@@ -0,0 +1,241 @@
+package com.destroystokyo.paper.io;
+
+import net.minecraft.server.MinecraftServer;
+import org.apache.logging.log4j.Logger;
+
+import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.locks.LockSupport;
+
+public class QueueExecutorThread<T extends PrioritizedTaskQueue.PrioritizedTask & Runnable> extends Thread {
+
+ private static final Logger LOGGER = MinecraftServer.LOGGER;
+
+ protected final PrioritizedTaskQueue<T> queue;
+ protected final long spinWaitTime;
+
+ protected volatile boolean closed;
+
+ protected final AtomicBoolean parked = new AtomicBoolean();
+
+ protected volatile ConcurrentLinkedQueue<Thread> flushQueue = new ConcurrentLinkedQueue<>();
+ protected volatile long flushCycles;
+
+ public QueueExecutorThread(final PrioritizedTaskQueue<T> queue) {
+ this(queue, (int)(1.e6)); // 1.0ms
+ }
+
+ public QueueExecutorThread(final PrioritizedTaskQueue<T> queue, final long spinWaitTime) { // in ms
+ this.queue = queue;
+ this.spinWaitTime = spinWaitTime;
+ }
+
+ @Override
+ public void run() {
+ final long spinWaitTime = this.spinWaitTime;
+ main_loop:
+ for (;;) {
+ this.pollTasks(true);
+
+ // spinwait
+
+ final long start = System.nanoTime();
+
+ for (;;) {
+ // If we are interrpted for any reason, park() will always return immediately. Clear so that we don't needlessly use cpu in such an event.
+ Thread.interrupted();
+ LockSupport.parkNanos("Spinwaiting on tasks", 1000L); // 1us
+
+ if (this.pollTasks(true)) {
+ // restart loop, found tasks
+ continue main_loop;
+ }
+
+ if (this.handleClose()) {
+ return; // we're done
+ }
+
+ if ((System.nanoTime() - start) >= spinWaitTime) {
+ break;
+ }
+ }
+
+ if (this.handleClose()) {
+ return;
+ }
+
+ this.parked.set(true);
+
+ // We need to parse here to avoid a race condition where a thread queues a task before we set parked to true
+ // (i.e it will not notify us)
+ if (this.pollTasks(true)) {
+ this.parked.set(false);
+ continue;
+ }
+
+ if (this.handleClose()) {
+ return;
+ }
+
+ // we don't need to check parked before sleeping, but we do need to check parked in a do-while loop
+ // LockSupport.park() can fail for any reason
+ do {
+ Thread.interrupted();
+ LockSupport.park("Waiting on tasks");
+ } while (this.parked.get());
+ }
+ }
+
+ protected boolean handleClose() {
+ if (this.closed) {
+ this.pollTasks(true); // this ensures we've emptied the queue
+ this.handleFlushThreads(true);
+ return true;
+ }
+ return false;
+ }
+
+ protected boolean pollTasks(boolean flushTasks) {
+ Runnable task;
+ boolean ret = false;
+
+ while ((task = this.queue.poll()) != null) {
+ ret = true;
+ try {
+ task.run();
+ } catch (final Throwable throwable) {
+ if (throwable instanceof ThreadDeath) {
+ throw (ThreadDeath)throwable;
+ }
+ LOGGER.fatal("Exception thrown from prioritized runnable task in thread '" + this.getName() + "': " + IOUtil.genericToString(task), throwable);
+ }
+ }
+
+ if (flushTasks) {
+ this.handleFlushThreads(false);
+ }
+
+ return ret;
+ }
+
+ protected void handleFlushThreads(final boolean shutdown) {
+ Thread parking;
+ ConcurrentLinkedQueue<Thread> flushQueue = this.flushQueue;
+ do {
+ ++flushCycles; // may be plain read opaque write
+ while ((parking = flushQueue.poll()) != null) {
+ LockSupport.unpark(parking);
+ }
+ } while (this.pollTasks(false));
+
+ if (shutdown) {
+ this.flushQueue = null;
+
+ // defend against a race condition where a flush thread double-checks right before we set to null
+ while ((parking = flushQueue.poll()) != null) {
+ LockSupport.unpark(parking);
+ }
+ }
+ }
+
+ /**
+ * Notify's this thread that a task has been added to its queue
+ * @return {@code true} if this thread was waiting for tasks, {@code false} if it is executing tasks
+ */
+ public boolean notifyTasks() {
+ if (this.parked.get() && this.parked.getAndSet(false)) {
+ LockSupport.unpark(this);
+ return true;
+ }
+ return false;
+ }
+
+ protected void queueTask(final T task) {
+ this.queue.add(task);
+ this.notifyTasks();
+ }
+
+ /**
+ * Waits until this thread's queue is empty.
+ *
+ * @throws IllegalStateException If the current thread is {@code this} thread.
+ */
+ public void flush() {
+ final Thread currentThread = Thread.currentThread();
+
+ if (currentThread == this) {
+ // avoid deadlock
+ throw new IllegalStateException("Cannot flush the queue executor thread while on the queue executor thread");
+ }
+
+ // order is important
+
+ int successes = 0;
+ long lastCycle = -1L;
+
+ do {
+ final ConcurrentLinkedQueue<Thread> flushQueue = this.flushQueue;
+ if (flushQueue == null) {
+ return;
+ }
+
+ flushQueue.add(currentThread);
+
+ // double check flush queue
+ if (this.flushQueue == null) {
+ return;
+ }
+
+ final long currentCycle = this.flushCycles; // may be opaque read
+
+ if (currentCycle == lastCycle) {
+ Thread.yield();
+ continue;
+ }
+
+ // force response
+ this.parked.set(false);
+ LockSupport.unpark(this);
+
+ LockSupport.park("flushing queue executor thread");
+
+ // returns whether there are tasks queued, does not return whether there are tasks executing
+ // this is why we cycle twice twice through flush (we know a pollTask call is made after a flush cycle)
+ // we really only need to guarantee that the tasks this thread has queued has gone through, and can leave
+ // tasks queued concurrently that are unsychronized with this thread as undefined behavior
+ if (this.queue.hasTasks()) {
+ successes = 0;
+ } else {
+ ++successes;
+ }
+
+ } while (successes != 2);
+
+ }
+
+ /**
+ * Closes this queue executor's queue and optionally waits for it to empty.
+ * <p>
+ * If wait is {@code true}, then the queue will be empty by the time this call completes.
+ * </p>
+ * <p>
+ * This function is MT-Safe.
+ * </p>
+ * @param wait If this call is to wait until the queue is empty
+ * @param killQueue Whether to shutdown this thread's queue
+ * @return whether this thread shut down the queue
+ */
+ public boolean close(final boolean wait, final boolean killQueue) {
+ boolean ret = !killQueue ? false : this.queue.shutdown();
+ this.closed = true;
+
+ // force thread to respond to the shutdown
+ this.parked.set(false);
+ LockSupport.unpark(this);
+
+ if (wait) {
+ this.flush();
+ }
+ return ret;
+ }
+}
diff --git a/src/main/java/com/destroystokyo/paper/io/chunk/ChunkLoadTask.java b/src/main/java/com/destroystokyo/paper/io/chunk/ChunkLoadTask.java
new file mode 100644
2021-06-17 19:11:00 +02:00
index 0000000000000000000000000000000000000000..0eaa4f3cd476c64feab358879eee3d67f46092f4
2021-06-11 14:02:28 +02:00
--- /dev/null
+++ b/src/main/java/com/destroystokyo/paper/io/chunk/ChunkLoadTask.java
2021-06-13 04:16:10 +02:00
@@ -0,0 +1,138 @@
2021-06-11 14:02:28 +02:00
+package com.destroystokyo.paper.io.chunk;
+
+import co.aikar.timings.Timing;
+import com.destroystokyo.paper.io.PaperFileIOThread;
+import com.destroystokyo.paper.io.IOUtil;
+import java.util.ArrayDeque;
+import java.util.function.Consumer;
+import net.minecraft.server.level.ChunkMap;
+import net.minecraft.server.level.ServerLevel;
+import net.minecraft.world.level.ChunkPos;
+import net.minecraft.world.level.chunk.storage.ChunkSerializer;
+
+public final class ChunkLoadTask extends ChunkTask {
+
+ public boolean cancelled;
+
+ Consumer<ChunkSerializer.InProgressChunkHolder> onComplete;
+ public PaperFileIOThread.ChunkData chunkData;
+
+ private boolean hasCompleted;
+
+ public ChunkLoadTask(final ServerLevel world, final int chunkX, final int chunkZ, final int priority,
+ final ChunkTaskManager taskManager,
+ final Consumer<ChunkSerializer.InProgressChunkHolder> onComplete) {
+ super(world, chunkX, chunkZ, priority, taskManager);
+ this.onComplete = onComplete;
+ }
+
+ private static final ArrayDeque<Runnable> EMPTY_QUEUE = new ArrayDeque<>();
+
+ private static ChunkSerializer.InProgressChunkHolder createEmptyHolder() {
+ return new ChunkSerializer.InProgressChunkHolder(null, EMPTY_QUEUE);
+ }
+
+ @Override
+ public void run() {
+ try {
+ this.executeTask();
+ } catch (final Throwable ex) {
+ PaperFileIOThread.LOGGER.error("Failed to execute chunk load task: " + this.toString(), ex);
+ if (!this.hasCompleted) {
+ this.complete(ChunkLoadTask.createEmptyHolder());
+ }
+ }
+ }
+
+ private boolean checkCancelled() {
+ if (this.cancelled) {
+ // IntelliJ does not understand writes may occur to cancelled concurrently.
+ return this.taskManager.chunkLoadTasks.compute(Long.valueOf(IOUtil.getCoordinateKey(this.chunkX, this.chunkZ)), (final Long keyInMap, final ChunkLoadTask valueInMap) -> {
+ if (valueInMap != ChunkLoadTask.this) {
+ throw new IllegalStateException("Expected this task to be scheduled, but another was! Other: " + valueInMap + ", current: " + ChunkLoadTask.this);
+ }
+
+ if (valueInMap.cancelled) {
+ return null;
+ }
+ return valueInMap;
+ }) == null;
+ }
+ return false;
+ }
+
+ public void executeTask() {
+ if (this.checkCancelled()) {
+ return;
+ }
+
+ // either executed synchronously or asynchronously
+ final PaperFileIOThread.ChunkData chunkData = this.chunkData;
+
+ if (chunkData.poiData == PaperFileIOThread.FAILURE_VALUE || chunkData.chunkData == PaperFileIOThread.FAILURE_VALUE) {
+ PaperFileIOThread.LOGGER.error("Could not load chunk for task: " + this.toString() + ", file IO thread has dumped the relevant exception above");
+ this.complete(ChunkLoadTask.createEmptyHolder());
+ return;
+ }
+
+ if (chunkData.chunkData == null) {
+ // not on disk
+ this.complete(ChunkLoadTask.createEmptyHolder());
+ return;
+ }
+
+ final ChunkPos chunkPos = new ChunkPos(this.chunkX, this.chunkZ);
+
+ final ChunkMap chunkManager = this.world.getChunkSource().chunkMap;
+
+ try (Timing ignored = this.world.timings.chunkLoadLevelTimer.startTimingIfSync()) {
+ final ChunkSerializer.InProgressChunkHolder chunkHolder;
+
+ // apply fixes
+
+ try {
+ chunkData.chunkData = chunkManager.getChunkData(this.world.getTypeKey(),
2021-06-17 19:11:00 +02:00
+ chunkManager.overworldDataStorage, chunkData.chunkData, chunkPos, this.world); // clone data for safety, file IO thread does not clone
2021-06-11 14:02:28 +02:00
+ } catch (final Throwable ex) {
+ PaperFileIOThread.LOGGER.error("Could not apply datafixers for chunk task: " + this.toString(), ex);
+ this.complete(ChunkLoadTask.createEmptyHolder());
+ }
+
+ if (this.checkCancelled()) {
+ return;
+ }
+
+ try {
+ chunkHolder = ChunkSerializer.loadChunk(this.world,
+ chunkManager.structureManager, chunkManager.getVillagePlace(), chunkPos,
+ chunkData.chunkData, true);
+ } catch (final Throwable ex) {
+ PaperFileIOThread.LOGGER.error("Could not de-serialize chunk data for task: " + this.toString(), ex);
+ this.complete(ChunkLoadTask.createEmptyHolder());
+ return;
+ }
+
+ this.complete(chunkHolder);
+ }
+ }
+
+ private void complete(final ChunkSerializer.InProgressChunkHolder holder) {
+ this.hasCompleted = true;
+ holder.poiData = this.chunkData == null ? null : this.chunkData.poiData;
+
+ this.taskManager.chunkLoadTasks.compute(Long.valueOf(IOUtil.getCoordinateKey(this.chunkX, this.chunkZ)), (final Long keyInMap, final ChunkLoadTask valueInMap) -> {
+ if (valueInMap != ChunkLoadTask.this) {
+ throw new IllegalStateException("Expected this task to be scheduled, but another was! Other: " + valueInMap + ", current: " + ChunkLoadTask.this);
+ }
+ if (valueInMap.cancelled) {
+ return null;
+ }
+ try {
+ ChunkLoadTask.this.onComplete.accept(holder);
+ } catch (final Throwable thr) {
+ PaperFileIOThread.LOGGER.error("Failed to complete chunk data for task: " + this.toString(), thr);
+ }
+ return null;
+ });
+ }
+}
diff --git a/src/main/java/com/destroystokyo/paper/io/chunk/ChunkSaveTask.java b/src/main/java/com/destroystokyo/paper/io/chunk/ChunkSaveTask.java
new file mode 100644
index 0000000000000000000000000000000000000000..69ebbfa171385c46a84d1a0d241d168a8c2af145
--- /dev/null
+++ b/src/main/java/com/destroystokyo/paper/io/chunk/ChunkSaveTask.java
@@ -0,0 +1,111 @@
+package com.destroystokyo.paper.io.chunk;
+
+import co.aikar.timings.Timing;
+import com.destroystokyo.paper.io.PaperFileIOThread;
+import com.destroystokyo.paper.io.IOUtil;
+import com.destroystokyo.paper.io.PrioritizedTaskQueue;
+
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.atomic.AtomicInteger;
+import net.minecraft.nbt.CompoundTag;
+import net.minecraft.server.level.ServerLevel;
+import net.minecraft.world.level.chunk.ChunkAccess;
+import net.minecraft.world.level.chunk.storage.ChunkSerializer;
+
+public final class ChunkSaveTask extends ChunkTask {
+
+ public final ChunkSerializer.AsyncSaveData asyncSaveData;
+ public final ChunkAccess chunk;
+ public final CompletableFuture<CompoundTag> onComplete = new CompletableFuture<>();
+
+ private final AtomicInteger attemptedPriority;
+
+ public ChunkSaveTask(final ServerLevel world, final int chunkX, final int chunkZ, final int priority,
+ final ChunkTaskManager taskManager, final ChunkSerializer.AsyncSaveData asyncSaveData,
+ final ChunkAccess chunk) {
+ super(world, chunkX, chunkZ, priority, taskManager);
+ this.chunk = chunk;
+ this.asyncSaveData = asyncSaveData;
+ this.attemptedPriority = new AtomicInteger(priority);
+ }
+
+ @Override
+ public void run() {
+ // can be executed asynchronously or synchronously
+ final CompoundTag compound;
+
+ try (Timing ignored = this.world.timings.chunkUnloadDataSave.startTimingIfSync()) {
+ compound = ChunkSerializer.saveChunk(this.world, this.chunk, this.asyncSaveData);
+ } catch (final Throwable ex) {
+ // has a plugin modified something it should not have and made us CME?
+ PaperFileIOThread.LOGGER.error("Failed to serialize unloading chunk data for task: " + this.toString() + ", falling back to a synchronous execution", ex);
+
+ // Note: We add to the server thread queue here since this is what the server will drain tasks from
+ // when waiting for chunks
+ ChunkTaskManager.queueChunkWaitTask(() -> {
+ try (Timing ignored = this.world.timings.chunkUnloadDataSave.startTiming()) {
+ CompoundTag data = PaperFileIOThread.FAILURE_VALUE;
+
+ try {
+ data = ChunkSerializer.saveChunk(this.world, this.chunk, this.asyncSaveData);
+ PaperFileIOThread.LOGGER.info("Successfully serialized chunk data for task: " + this.toString() + " synchronously");
+ } catch (final Throwable ex1) {
+ PaperFileIOThread.LOGGER.fatal("Failed to synchronously serialize unloading chunk data for task: " + this.toString() + "! Chunk data will be lost", ex1);
+ }
+
+ ChunkSaveTask.this.complete(data);
+ }
+ });
+
+ return; // the main thread will now complete the data
+ }
+
+ this.complete(compound);
+ }
+
+ @Override
+ public boolean raisePriority(final int priority) {
+ if (!PrioritizedTaskQueue.validPriority(priority)) {
+ throw new IllegalStateException("Invalid priority: " + priority);
+ }
+
+ // we know priority is valid here
+ for (int curr = this.attemptedPriority.get();;) {
+ if (curr <= priority) {
+ break; // curr is higher/same priority
+ }
+ if (this.attemptedPriority.compareAndSet(curr, priority)) {
+ break;
+ }
+ curr = this.attemptedPriority.get();
+ }
+
+ return super.raisePriority(priority);
+ }
+
+ @Override
+ public boolean updatePriority(final int priority) {
+ if (!PrioritizedTaskQueue.validPriority(priority)) {
+ throw new IllegalStateException("Invalid priority: " + priority);
+ }
+ this.attemptedPriority.set(priority);
+ return super.updatePriority(priority);
+ }
+
+ private void complete(final CompoundTag compound) {
+ try {
+ this.onComplete.complete(compound);
+ } catch (final Throwable thr) {
+ PaperFileIOThread.LOGGER.error("Failed to complete chunk data for task: " + this.toString(), thr);
+ }
+ if (compound != PaperFileIOThread.FAILURE_VALUE) {
+ PaperFileIOThread.Holder.INSTANCE.scheduleSave(this.world, this.chunkX, this.chunkZ, null, compound, this.attemptedPriority.get());
+ }
+ this.taskManager.chunkSaveTasks.compute(Long.valueOf(IOUtil.getCoordinateKey(this.chunkX, this.chunkZ)), (final Long keyInMap, final ChunkSaveTask valueInMap) -> {
+ if (valueInMap != ChunkSaveTask.this) {
+ throw new IllegalStateException("Expected this task to be scheduled, but another was! Other: " + valueInMap + ", this: " + ChunkSaveTask.this);
+ }
+ return null;
+ });
+ }
+}
diff --git a/src/main/java/com/destroystokyo/paper/io/chunk/ChunkTask.java b/src/main/java/com/destroystokyo/paper/io/chunk/ChunkTask.java
new file mode 100644
index 0000000000000000000000000000000000000000..058fb5a41565e6ce2acbd1f4d071a1b8be449f5d
--- /dev/null
+++ b/src/main/java/com/destroystokyo/paper/io/chunk/ChunkTask.java
@@ -0,0 +1,40 @@
+package com.destroystokyo.paper.io.chunk;
+
+import com.destroystokyo.paper.io.PaperFileIOThread;
+import com.destroystokyo.paper.io.PrioritizedTaskQueue;
+import net.minecraft.server.level.ServerLevel;
+
+abstract class ChunkTask extends PrioritizedTaskQueue.PrioritizedTask implements Runnable {
+
+ public final ServerLevel world;
+ public final int chunkX;
+ public final int chunkZ;
+ public final ChunkTaskManager taskManager;
+
+ public ChunkTask(final ServerLevel world, final int chunkX, final int chunkZ, final int priority,
+ final ChunkTaskManager taskManager) {
+ super(priority);
+ this.world = world;
+ this.chunkX = chunkX;
+ this.chunkZ = chunkZ;
+ this.taskManager = taskManager;
+ }
+
+ @Override
+ public String toString() {
+ return "Chunk task: class:" + this.getClass().getName() + ", for world '" + this.world.getWorld().getName() +
+ "', (" + this.chunkX + "," + this.chunkZ + "), hashcode:" + this.hashCode() + ", priority: " + this.getPriority();
+ }
+
+ @Override
+ public boolean raisePriority(final int priority) {
+ PaperFileIOThread.Holder.INSTANCE.bumpPriority(this.world, this.chunkX, this.chunkZ, priority);
+ return super.raisePriority(priority);
+ }
+
+ @Override
+ public boolean updatePriority(final int priority) {
+ PaperFileIOThread.Holder.INSTANCE.setPriority(this.world, this.chunkX, this.chunkZ, priority);
+ return super.updatePriority(priority);
+ }
+}
diff --git a/src/main/java/com/destroystokyo/paper/io/chunk/ChunkTaskManager.java b/src/main/java/com/destroystokyo/paper/io/chunk/ChunkTaskManager.java
new file mode 100644
2021-06-13 04:16:10 +02:00
index 0000000000000000000000000000000000000000..18ae2e2b339d357fbe0f6f2b18bc14c0dfe4c222
2021-06-11 14:02:28 +02:00
--- /dev/null
+++ b/src/main/java/com/destroystokyo/paper/io/chunk/ChunkTaskManager.java
@@ -0,0 +1,513 @@
+package com.destroystokyo.paper.io.chunk;
+
+import com.destroystokyo.paper.io.PaperFileIOThread;
+import com.destroystokyo.paper.io.IOUtil;
+import com.destroystokyo.paper.io.PrioritizedTaskQueue;
+import com.destroystokyo.paper.io.QueueExecutorThread;
+import net.minecraft.nbt.CompoundTag;
+import net.minecraft.server.MinecraftServer;
+import net.minecraft.server.level.ChunkHolder;
+import net.minecraft.server.level.ServerChunkCache;
+import net.minecraft.server.level.ServerLevel;
+import net.minecraft.util.thread.BlockableEventLoop;
+import net.minecraft.world.level.chunk.ChunkAccess;
+import net.minecraft.world.level.chunk.ChunkStatus;
+import net.minecraft.world.level.chunk.storage.ChunkSerializer;
+import org.apache.commons.lang.StringUtils;
+import org.apache.logging.log4j.Level;
+import org.bukkit.Bukkit;
+import org.spigotmc.AsyncCatcher;
+
+import java.util.ArrayDeque;
+import java.util.HashSet;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.function.Consumer;
+
+public final class ChunkTaskManager {
+
+ private final QueueExecutorThread<ChunkTask>[] workers;
+ private final ServerLevel world;
+
+ private final PrioritizedTaskQueue<ChunkTask> queue;
+ private final boolean perWorldQueue;
+
+ final ConcurrentHashMap<Long, ChunkLoadTask> chunkLoadTasks = new ConcurrentHashMap<>(64, 0.5f);
+ final ConcurrentHashMap<Long, ChunkSaveTask> chunkSaveTasks = new ConcurrentHashMap<>(64, 0.5f);
+
+ private final PrioritizedTaskQueue<ChunkTask> chunkTasks = new PrioritizedTaskQueue<>(); // used if async chunks are disabled in config
+
+ protected static QueueExecutorThread<ChunkTask>[] globalWorkers;
+ protected static QueueExecutorThread<ChunkTask> globalUrgentWorker;
+ protected static PrioritizedTaskQueue<ChunkTask> globalQueue;
+ protected static PrioritizedTaskQueue<ChunkTask> globalUrgentQueue;
+
+ protected static final ConcurrentLinkedQueue<Runnable> CHUNK_WAIT_QUEUE = new ConcurrentLinkedQueue<>();
+
+ public static final ArrayDeque<ChunkInfo> WAITING_CHUNKS = new ArrayDeque<>(); // stack
+
+ private static final class ChunkInfo {
+
+ public final int chunkX;
+ public final int chunkZ;
+ public final ServerLevel world;
+
+ public ChunkInfo(final int chunkX, final int chunkZ, final ServerLevel world) {
+ this.chunkX = chunkX;
+ this.chunkZ = chunkZ;
+ this.world = world;
+ }
+
+ @Override
+ public String toString() {
+ return "[( " + this.chunkX + "," + this.chunkZ + ") in '" + this.world.getWorld().getName() + "']";
+ }
+ }
+
+ public static void pushChunkWait(final ServerLevel world, final int chunkX, final int chunkZ) {
+ synchronized (WAITING_CHUNKS) {
+ WAITING_CHUNKS.push(new ChunkInfo(chunkX, chunkZ, world));
+ }
+ }
+
+ public static void popChunkWait() {
+ synchronized (WAITING_CHUNKS) {
+ WAITING_CHUNKS.pop();
+ }
+ }
+
+ private static ChunkInfo[] getChunkInfos() {
+ ChunkInfo[] chunks;
+ synchronized (WAITING_CHUNKS) {
+ chunks = WAITING_CHUNKS.toArray(new ChunkInfo[0]);
+ }
+ return chunks;
+ }
+
+ public static void dumpAllChunkLoadInfo() {
+ ChunkInfo[] chunks = getChunkInfos();
+ if (chunks.length > 0) {
+ PaperFileIOThread.LOGGER.log(Level.ERROR, "Chunk wait task info below: ");
+
+ for (final ChunkInfo chunkInfo : chunks) {
+ final long key = IOUtil.getCoordinateKey(chunkInfo.chunkX, chunkInfo.chunkZ);
+ final ChunkLoadTask loadTask = chunkInfo.world.asyncChunkTaskManager.chunkLoadTasks.get(key);
+ final ChunkSaveTask saveTask = chunkInfo.world.asyncChunkTaskManager.chunkSaveTasks.get(key);
+
+ PaperFileIOThread.LOGGER.log(Level.ERROR, chunkInfo.chunkX + "," + chunkInfo.chunkZ + " in '" + chunkInfo.world.getWorld().getName() + ":");
+ PaperFileIOThread.LOGGER.log(Level.ERROR, "Load Task - " + (loadTask == null ? "none" : loadTask.toString()));
+ PaperFileIOThread.LOGGER.log(Level.ERROR, "Save Task - " + (saveTask == null ? "none" : saveTask.toString()));
+ // log current status of chunk to indicate whether we're waiting on generation or loading
+ ChunkHolder chunkHolder = chunkInfo.world.getChunkSource().chunkMap.getVisibleChunkIfPresent(key);
+
+ dumpChunkInfo(new HashSet<>(), chunkHolder, chunkInfo.chunkX, chunkInfo.chunkZ);
+ }
+ }
+ }
+
+ static void dumpChunkInfo(Set<ChunkHolder> seenChunks, ChunkHolder chunkHolder, int x, int z) {
+ dumpChunkInfo(seenChunks, chunkHolder, x, z, 0, 1);
+ }
+
+ static void dumpChunkInfo(Set<ChunkHolder> seenChunks, ChunkHolder chunkHolder, int x, int z, int indent, int maxDepth) {
+ if (seenChunks.contains(chunkHolder)) {
+ return;
+ }
+ if (indent > maxDepth) {
+ return;
+ }
+ seenChunks.add(chunkHolder);
+ String indentStr = StringUtils.repeat(" ", indent);
+ if (chunkHolder == null) {
+ PaperFileIOThread.LOGGER.log(Level.ERROR, indentStr + "Chunk Holder - null for (" + x +"," + z +")");
+ } else {
2021-06-13 04:16:10 +02:00
+ ChunkAccess chunk = chunkHolder.getLastAvailable();
2021-06-11 14:02:28 +02:00
+ ChunkStatus holderStatus = chunkHolder.getChunkHolderStatus();
+ PaperFileIOThread.LOGGER.log(Level.ERROR, indentStr + "Chunk Holder - non-null");
+ PaperFileIOThread.LOGGER.log(Level.ERROR, indentStr + "Chunk Status - " + ((chunk == null) ? "null chunk" : chunk.getStatus().toString()));
+ PaperFileIOThread.LOGGER.log(Level.ERROR, indentStr + "Chunk Ticket Status - " + ChunkHolder.getStatus(chunkHolder.getTicketLevel()));
+ PaperFileIOThread.LOGGER.log(Level.ERROR, indentStr + "Chunk Holder Status - " + ((holderStatus == null) ? "null" : holderStatus.toString()));
+ }
+ }
+
+ public static void initGlobalLoadThreads(int threads) {
+ if (threads <= 0 || globalWorkers != null) {
+ return;
+ }
+
+ globalWorkers = new QueueExecutorThread[threads];
+ globalQueue = new PrioritizedTaskQueue<>();
+ globalUrgentQueue = new PrioritizedTaskQueue<>();
+
+ for (int i = 0; i < threads; ++i) {
+ globalWorkers[i] = new QueueExecutorThread<>(globalQueue, (long)0.10e6); //0.1ms
+ globalWorkers[i].setName("Paper Async Chunk Task Thread #" + i);
+ globalWorkers[i].setPriority(Thread.NORM_PRIORITY - 1);
+ globalWorkers[i].setUncaughtExceptionHandler((final Thread thread, final Throwable throwable) -> {
+ PaperFileIOThread.LOGGER.fatal("Thread '" + thread.getName() + "' threw an uncaught exception!", throwable);
+ });
+
+ globalWorkers[i].start();
+ }
+
+ globalUrgentWorker = new QueueExecutorThread<>(globalUrgentQueue, (long)0.10e6); //0.1ms
+ globalUrgentWorker.setName("Paper Async Chunk Urgent Task Thread");
+ globalUrgentWorker.setPriority(Thread.NORM_PRIORITY+1);
+ globalUrgentWorker.setUncaughtExceptionHandler((final Thread thread, final Throwable throwable) -> {
+ PaperFileIOThread.LOGGER.fatal("Thread '" + thread.getName() + "' threw an uncaught exception!", throwable);
+ });
+
+ globalUrgentWorker.start();
+ }
+
+ /**
+ * Creates this chunk task manager to operate off the specified number of threads. If the specified number of threads is
+ * less-than or equal to 0, then this chunk task manager will operate off of the world's chunk task queue.
+ * @param world Specified world.
+ * @param threads Specified number of threads.
+ * @see ServerChunkCache#mainThreadProcessor
+ */
+ public ChunkTaskManager(final ServerLevel world, final int threads) {
+ this.world = world;
+ this.workers = threads <= 0 ? null : new QueueExecutorThread[threads];
+ this.queue = new PrioritizedTaskQueue<>();
+ this.perWorldQueue = true;
+
+ for (int i = 0; i < threads; ++i) {
+ this.workers[i] = new QueueExecutorThread<>(this.queue, (long)0.10e6); //0.1ms
+ this.workers[i].setName("Async chunk loader thread #" + i + " for world: " + world.getWorld().getName());
+ this.workers[i].setPriority(Thread.NORM_PRIORITY - 1);
+ this.workers[i].setUncaughtExceptionHandler((final Thread thread, final Throwable throwable) -> {
+ PaperFileIOThread.LOGGER.fatal("Thread '" + thread.getName() + "' threw an uncaught exception!", throwable);
+ });
+
+ this.workers[i].start();
+ }
+ }
+
+ /**
+ * Creates the chunk task manager to work from the global workers. When {@link #close(boolean)} is invoked,
+ * the global queue is not shutdown. If the global workers is configured to be disabled or use 0 threads, then
+ * this chunk task manager will operate off of the world's chunk task queue.
+ * @param world The world that this task manager is responsible for
+ * @see ServerChunkCache#mainThreadProcessor
+ */
+ public ChunkTaskManager(final ServerLevel world) {
+ this.world = world;
+ this.workers = globalWorkers;
+ this.queue = globalQueue;
+ this.perWorldQueue = false;
+ }
+
+ public boolean pollNextChunkTask() {
+ final ChunkTask task = this.chunkTasks.poll();
+
+ if (task != null) {
+ task.run();
+ return true;
+ }
+ return false;
+ }
+
+ /**
+ * Polls and runs the next available chunk wait queue task. This is to be used when the server is waiting on a chunk queue.
+ * (per-world can cause issues if all the worker threads are blocked waiting for a response from the main thread)
+ */
+ public static boolean pollChunkWaitQueue() {
+ final Runnable run = CHUNK_WAIT_QUEUE.poll();
+ if (run != null) {
+ run.run();
+ return true;
+ }
+ return false;
+ }
+
+ /**
+ * Queues a chunk wait task. Note that this will execute out of order with respect to tasks scheduled on a world's
+ * chunk task queue, since this is the global chunk wait queue.
+ */
+ public static void queueChunkWaitTask(final Runnable runnable) {
+ CHUNK_WAIT_QUEUE.add(runnable);
+ }
+
+ private static void drainChunkWaitQueue() {
+ Runnable run;
+ while ((run = CHUNK_WAIT_QUEUE.poll()) != null) {
+ run.run();
+ }
+ }
+
+ /**
+ * The exact same as {@link #scheduleChunkLoad(int, int, int, Consumer, boolean)}, except that the chunk data is provided as
+ * the {@code data} parameter.
+ */
+ public ChunkLoadTask scheduleChunkLoad(final int chunkX, final int chunkZ, final int priority,
+ final Consumer<ChunkSerializer.InProgressChunkHolder> onComplete,
+ final boolean intendingToBlock, final CompletableFuture<CompoundTag> dataFuture) {
+ final ServerLevel world = this.world;
+
+ return this.chunkLoadTasks.compute(Long.valueOf(IOUtil.getCoordinateKey(chunkX, chunkZ)), (final Long keyInMap, final ChunkLoadTask valueInMap) -> {
+ if (valueInMap != null) {
+ if (!valueInMap.cancelled) {
+ throw new IllegalStateException("Double scheduling chunk load for task: " + valueInMap.toString());
+ }
+ valueInMap.cancelled = false;
+ valueInMap.onComplete = onComplete;
+ return valueInMap;
+ }
+
+ final ChunkLoadTask ret = new ChunkLoadTask(world, chunkX, chunkZ, priority, ChunkTaskManager.this, onComplete);
+
+ dataFuture.thenAccept((final CompoundTag data) -> {
+ final boolean failed = data == PaperFileIOThread.FAILURE_VALUE;
+ PaperFileIOThread.Holder.INSTANCE.loadChunkDataAsync(world, chunkX, chunkZ, priority, (final PaperFileIOThread.ChunkData chunkData) -> {
+ ret.chunkData = chunkData;
+ if (!failed) {
+ chunkData.chunkData = data;
+ }
+ ChunkTaskManager.this.internalSchedule(ret); // only schedule to the worker threads here
+ }, true, failed, intendingToBlock); // read data off disk if the future fails
+ });
+
+ return ret;
+ });
+ }
+
+ public void cancelChunkLoad(final int chunkX, final int chunkZ) {
+ this.chunkLoadTasks.compute(IOUtil.getCoordinateKey(chunkX, chunkZ), (final Long keyInMap, final ChunkLoadTask valueInMap) -> {
+ if (valueInMap == null) {
+ return null;
+ }
+
+ if (valueInMap.cancelled) {
+ PaperFileIOThread.LOGGER.warn("Task " + valueInMap.toString() + " is already cancelled!");
+ }
+ valueInMap.cancelled = true;
+ if (valueInMap.cancel()) {
+ return null;
+ }
+
+ return valueInMap;
+ });
+ }
+
+ /**
+ * Schedules an asynchronous chunk load for the specified coordinates. The onComplete parameter may be invoked asynchronously
+ * on a worker thread or on the world's chunk executor queue. As such the code that is executed for the parameter should be
+ * carefully chosen.
+ * @param chunkX Chunk's x coordinate
+ * @param chunkZ Chunk's z coordinate
+ * @param priority Priority for this task
+ * @param onComplete The consumer to invoke with the {@link ChunkSerializer.InProgressChunkHolder} object once this task is complete
+ * @param intendingToBlock Whether the caller is intending to block on this task completing (this is a performance tune, and has no adverse side-effects)
+ * @return The {@link ChunkLoadTask} associated with
+ */
+ public ChunkLoadTask scheduleChunkLoad(final int chunkX, final int chunkZ, final int priority,
+ final Consumer<ChunkSerializer.InProgressChunkHolder> onComplete,
+ final boolean intendingToBlock) {
+ final ServerLevel world = this.world;
+
+ return this.chunkLoadTasks.compute(Long.valueOf(IOUtil.getCoordinateKey(chunkX, chunkZ)), (final Long keyInMap, final ChunkLoadTask valueInMap) -> {
+ if (valueInMap != null) {
+ if (!valueInMap.cancelled) {
+ throw new IllegalStateException("Double scheduling chunk load for task: " + valueInMap.toString());
+ }
+ valueInMap.cancelled = false;
+ valueInMap.onComplete = onComplete;
+ return valueInMap;
+ }
+
+ final ChunkLoadTask ret = new ChunkLoadTask(world, chunkX, chunkZ, priority, ChunkTaskManager.this, onComplete);
+
+ PaperFileIOThread.Holder.INSTANCE.loadChunkDataAsync(world, chunkX, chunkZ, priority, (final PaperFileIOThread.ChunkData chunkData) -> {
+ ret.chunkData = chunkData;
+ ChunkTaskManager.this.internalSchedule(ret); // only schedule to the worker threads here
+ }, true, true, intendingToBlock);
+
+ return ret;
+ });
+ }
+
+ /**
+ * Schedules an async save for the specified chunk. The chunk, at the beginning of this call, must be completely unloaded
+ * from the world.
+ * @param chunkX Chunk's x coordinate
+ * @param chunkZ Chunk's z coordinate
+ * @param priority Priority for this task
+ * @param asyncSaveData Async save data. See {@link ChunkSerializer#getAsyncSaveData(ServerLevel, ChunkAccess)}
+ * @param chunk Chunk to save
+ * @return The {@link ChunkSaveTask} associated with the save task.
+ */
+ public ChunkSaveTask scheduleChunkSave(final int chunkX, final int chunkZ, final int priority,
+ final ChunkSerializer.AsyncSaveData asyncSaveData,
+ final ChunkAccess chunk) {
+ AsyncCatcher.catchOp("chunk save schedule");
+
+ final ServerLevel world = this.world;
+
+ return this.chunkSaveTasks.compute(Long.valueOf(IOUtil.getCoordinateKey(chunkX, chunkZ)), (final Long keyInMap, final ChunkSaveTask valueInMap) -> {
+ if (valueInMap != null) {
+ throw new IllegalStateException("Double scheduling chunk save for task: " + valueInMap.toString());
+ }
+
+ final ChunkSaveTask ret = new ChunkSaveTask(world, chunkX, chunkZ, priority, ChunkTaskManager.this, asyncSaveData, chunk);
+
+ ChunkTaskManager.this.internalSchedule(ret);
+
+ return ret;
+ });
+ }
+
+ /**
+ * Returns a completable future which will be completed with the <b>un-copied</b> chunk data for an in progress async save.
+ * Returns {@code null} if no save is in progress.
+ * @param chunkX Chunk's x coordinate
+ * @param chunkZ Chunk's z coordinate
+ */
+ public CompletableFuture<CompoundTag> getChunkSaveFuture(final int chunkX, final int chunkZ) {
+ final ChunkSaveTask chunkSaveTask = this.chunkSaveTasks.get(Long.valueOf(IOUtil.getCoordinateKey(chunkX, chunkZ)));
+ if (chunkSaveTask == null) {
+ return null;
+ }
+ return chunkSaveTask.onComplete;
+ }
+
+ /**
+ * Returns the chunk object being used to serialize data async for an unloaded chunk. Note that modifying this chunk
+ * is not safe to do as another thread is handling its save. The chunk is also not loaded into the world.
+ * @param chunkX Chunk's x coordinate
+ * @param chunkZ Chunk's z coordinate
+ * @return Chunk object for an in-progress async save, or {@code null} if no save is in progress
+ */
+ public ChunkAccess getChunkInSaveProgress(final int chunkX, final int chunkZ) {
+ final ChunkSaveTask chunkSaveTask = this.chunkSaveTasks.get(Long.valueOf(IOUtil.getCoordinateKey(chunkX, chunkZ)));
+ if (chunkSaveTask == null) {
+ return null;
+ }
+ return chunkSaveTask.chunk;
+ }
+
+ public void flush() {
+ // flush here since we schedule tasks on the IO thread that can schedule tasks here
+ drainChunkWaitQueue();
+ PaperFileIOThread.Holder.INSTANCE.flush();
+ drainChunkWaitQueue();
+
+ if (this.workers == null) {
+ if (Bukkit.isPrimaryThread() || MinecraftServer.getServer().hasStopped()) {
+ ((BlockableEventLoop<Runnable>)this.world.getChunkSource().mainThreadProcessor).runAllTasks();
+ } else {
+ CompletableFuture<Void> wait = new CompletableFuture<>();
+ MinecraftServer.getServer().scheduleOnMain(() -> {
+ ((BlockableEventLoop<Runnable>)this.world.getChunkSource().mainThreadProcessor).runAllTasks();
+ });
+ wait.join();
+ }
+ } else {
+ for (final QueueExecutorThread<ChunkTask> worker : this.workers) {
+ worker.flush();
+ }
+ }
+ if (globalUrgentWorker != null) globalUrgentWorker.flush();
+
+ // flush again since tasks we execute async saves
+ drainChunkWaitQueue();
+ PaperFileIOThread.Holder.INSTANCE.flush();
+ }
+
+ public void close(final boolean wait) {
+ // flush here since we schedule tasks on the IO thread that can schedule tasks to this task manager
+ // we do this regardless of the wait param since after we invoke close no tasks can be queued
+ PaperFileIOThread.Holder.INSTANCE.flush();
+
+ if (this.workers == null) {
+ if (wait) {
+ this.flush();
+ }
+ return;
+ }
+
+ if (this.workers != globalWorkers) {
+ for (final QueueExecutorThread<ChunkTask> worker : this.workers) {
+ worker.close(false, this.perWorldQueue);
+ }
+ }
+
+ if (wait) {
+ this.flush();
+ }
+ }
+
+ public void raisePriority(final int chunkX, final int chunkZ, final int priority) {
+ final Long chunkKey = Long.valueOf(IOUtil.getCoordinateKey(chunkX, chunkZ));
+
+ ChunkTask chunkSaveTask = this.chunkSaveTasks.get(chunkKey);
+ if (chunkSaveTask != null) {
+ // don't bump save into urgent queue
+ raiseTaskPriority(chunkSaveTask, priority != PrioritizedTaskQueue.HIGHEST_PRIORITY ? priority : PrioritizedTaskQueue.HIGH_PRIORITY);
+ }
+
+ ChunkLoadTask chunkLoadTask = this.chunkLoadTasks.get(chunkKey);
+ if (chunkLoadTask != null) {
+ raiseTaskPriority(chunkLoadTask, priority);
+ }
+ }
+
+ private void raiseTaskPriority(ChunkTask task, int priority) {
+ final boolean raised = task.raisePriority(priority);
+ if (task.isScheduled() && raised && this.workers != null) {
+ // only notify if we're in queue to be executed
+ if (priority == PrioritizedTaskQueue.HIGHEST_PRIORITY) {
+ // was in another queue but became urgent later, add to urgent queue and the previous
+ // queue will just have to ignore this task if it has already been started.
+ // Ultimately, we now have 2 potential queues that can pull it out whoever gets it first
+ // but the urgent queue has dedicated thread(s) so it's likely to win....
+ globalUrgentQueue.add(task);
+ this.internalScheduleNotifyUrgent();
+ } else {
+ this.internalScheduleNotify();
+ }
+ }
+ }
+
+ protected void internalSchedule(final ChunkTask task) {
+ if (this.workers == null) {
+ this.chunkTasks.add(task);
+ return;
+ }
+
+ // It's important we order the task to be executed before notifying. Avoid a race condition where the worker thread
+ // wakes up and goes to sleep before we actually schedule (or it's just about to sleep)
+ if (task.getPriority() == PrioritizedTaskQueue.HIGHEST_PRIORITY) {
+ globalUrgentQueue.add(task);
+ this.internalScheduleNotifyUrgent();
+ } else {
+ this.queue.add(task);
+ this.internalScheduleNotify();
+ }
+
+ }
+
+ protected void internalScheduleNotify() {
+ if (this.workers == null) {
+ return;
+ }
+ for (final QueueExecutorThread<ChunkTask> worker : this.workers) {
+ if (worker.notifyTasks()) {
+ // break here since we only want to wake up one worker for scheduling one task
+ break;
+ }
+ }
+ }
+
+
+ protected void internalScheduleNotifyUrgent() {
+ if (globalUrgentWorker == null) {
+ return;
+ }
+ globalUrgentWorker.notifyTasks();
+ }
+
+}
diff --git a/src/main/java/net/minecraft/network/protocol/game/ServerboundCommandSuggestionPacket.java b/src/main/java/net/minecraft/network/protocol/game/ServerboundCommandSuggestionPacket.java
2021-06-13 04:16:10 +02:00
index a5e438a834826161c52ca9db57d234d9ff80a591..b8bc1b9b8e8a33df90a963f9f9769292bf595642 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/net/minecraft/network/protocol/game/ServerboundCommandSuggestionPacket.java
+++ b/src/main/java/net/minecraft/network/protocol/game/ServerboundCommandSuggestionPacket.java
@@ -14,7 +14,7 @@ public class ServerboundCommandSuggestionPacket implements Packet<ServerGamePack
2021-06-13 04:16:10 +02:00
public ServerboundCommandSuggestionPacket(FriendlyByteBuf buf) {
2021-06-11 14:02:28 +02:00
this.id = buf.readVarInt();
- this.command = buf.readUtf(32500);
+ this.command = buf.readUtf(2048);
}
@Override
diff --git a/src/main/java/net/minecraft/server/MCUtil.java b/src/main/java/net/minecraft/server/MCUtil.java
2021-06-13 04:16:10 +02:00
index 966f0951832040c38fdd2caa58f7eae372aa0f59..49fd3486a6c595749f33bbe1c1bec0454e4725c5 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/net/minecraft/server/MCUtil.java
+++ b/src/main/java/net/minecraft/server/MCUtil.java
2021-06-13 04:16:10 +02:00
@@ -500,4 +500,8 @@ public final class MCUtil {
return null;
2021-06-11 14:02:28 +02:00
}
}
+
2021-06-13 04:16:10 +02:00
+ public static int getTicketLevelFor(net.minecraft.world.level.chunk.ChunkStatus status) {
+ return net.minecraft.server.level.ChunkMap.MAX_VIEW_DISTANCE + net.minecraft.world.level.chunk.ChunkStatus.getDistance(status);
2021-06-11 14:02:28 +02:00
+ }
}
diff --git a/src/main/java/net/minecraft/server/Main.java b/src/main/java/net/minecraft/server/Main.java
2021-06-13 04:16:10 +02:00
index fb0b3c5770f66cc3590f5ac4e690a33cb6179be3..7ce854edba32ffcafaa5268d4bb2822a5233e40b 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/net/minecraft/server/Main.java
+++ b/src/main/java/net/minecraft/server/Main.java
2021-06-13 04:16:10 +02:00
@@ -217,6 +217,7 @@ public class Main {
2021-06-11 14:02:28 +02:00
convertable_conversionsession.a((IRegistryCustom) iregistrycustom_dimension, (SaveData) object);
*/
2021-06-13 04:16:10 +02:00
+ Class.forName(net.minecraft.world.entity.npc.VillagerTrades.class.getName());// Paper - load this sync so it won't fail later async
2021-06-11 14:02:28 +02:00
final DedicatedServer dedicatedserver = (DedicatedServer) MinecraftServer.spin((thread) -> {
DedicatedServer dedicatedserver1 = new DedicatedServer(optionset, datapackconfiguration1, thread, iregistrycustom_dimension, convertable_conversionsession, resourcepackrepository, datapackresources, null, dedicatedserversettings, DataFixers.getDataFixer(), minecraftsessionservice, gameprofilerepository, usercache, LoggerChunkProgressListener::new);
diff --git a/src/main/java/net/minecraft/server/MinecraftServer.java b/src/main/java/net/minecraft/server/MinecraftServer.java
index 7b760a65cad1f7dd5adb3a05b8b5ed7d0f49c0a9..00920e574ee1f104781d1f776d88ec9cd45eda93 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/net/minecraft/server/MinecraftServer.java
+++ b/src/main/java/net/minecraft/server/MinecraftServer.java
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -1000,7 +1000,7 @@ public abstract class MinecraftServer extends ReentrantBlockableEventLoop<TickTa
2021-06-13 04:16:10 +02:00
this.getProfileCache().save(false); // Paper
2021-06-11 14:02:28 +02:00
}
// Spigot end
-
+ com.destroystokyo.paper.io.PaperFileIOThread.Holder.INSTANCE.close(true, true); // Paper
}
public String getLocalIp() {
diff --git a/src/main/java/net/minecraft/server/level/ChunkHolder.java b/src/main/java/net/minecraft/server/level/ChunkHolder.java
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
index bbf2dee8cdd2d8239d9230b1a7fff4f07c2edaf8..ffa3dc07ec566464ce10abe793de61efa9673fe9 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/net/minecraft/server/level/ChunkHolder.java
+++ b/src/main/java/net/minecraft/server/level/ChunkHolder.java
2021-06-17 23:39:36 +02:00
@@ -151,6 +151,18 @@ public class ChunkHolder {
2021-06-13 04:16:10 +02:00
return chunkstatus;
}
2021-06-11 14:02:28 +02:00
}
2021-06-13 04:16:10 +02:00
+ return null;
+ }
2021-06-11 14:02:28 +02:00
+
+ public ChunkStatus getChunkHolderStatus() {
2021-06-13 04:16:10 +02:00
+ for (ChunkStatus curr = ChunkStatus.FULL, next = curr.getParent(); curr != next; curr = next, next = next.getParent()) {
2021-06-11 14:02:28 +02:00
+ CompletableFuture<Either<ChunkAccess, ChunkHolder.ChunkLoadingFailure>> future = this.getFutureIfPresentUnchecked(curr);
+ Either<ChunkAccess, ChunkHolder.ChunkLoadingFailure> either = future.getNow(null);
+ if (either == null || !either.left().isPresent()) {
+ continue;
+ }
+ return curr;
+ }
2021-06-13 04:16:10 +02:00
return null;
}
2021-06-17 23:39:36 +02:00
@@ -375,7 +387,7 @@ public class ChunkHolder {
2021-06-13 04:16:10 +02:00
ChunkStatus chunkstatus = ChunkHolder.getStatus(this.oldTicketLevel);
ChunkStatus chunkstatus1 = ChunkHolder.getStatus(this.ticketLevel);
2021-06-11 14:02:28 +02:00
boolean flag = this.oldTicketLevel <= ChunkMap.MAX_CHUNK_DISTANCE;
- boolean flag1 = this.ticketLevel <= ChunkMap.MAX_CHUNK_DISTANCE;
+ boolean flag1 = this.ticketLevel <= ChunkMap.MAX_CHUNK_DISTANCE; // Paper - diff on change: (flag1 = new ticket level is in loadable range)
2021-06-13 04:16:10 +02:00
ChunkHolder.FullChunkStatus playerchunk_state = ChunkHolder.getFullChunkStatus(this.oldTicketLevel);
ChunkHolder.FullChunkStatus playerchunk_state1 = ChunkHolder.getFullChunkStatus(this.ticketLevel);
2021-06-11 14:02:28 +02:00
// CraftBukkit start
diff --git a/src/main/java/net/minecraft/server/level/ChunkMap.java b/src/main/java/net/minecraft/server/level/ChunkMap.java
index 75719f5ba225db613f1669785da15b70159b7ab6..c70d12b6af98562faaddef8a6d77d1f2f45c8d44 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/net/minecraft/server/level/ChunkMap.java
+++ b/src/main/java/net/minecraft/server/level/ChunkMap.java
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -447,6 +447,7 @@ public class ChunkMap extends ChunkStorage implements ChunkHolder.PlayerProvider
2021-06-11 14:02:28 +02:00
public void close() throws IOException {
try {
this.queueSorter.close();
+ this.level.asyncChunkTaskManager.close(true); // Paper - Required since we're closing regionfiles in the next line
this.poiManager.close();
} finally {
super.close();
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -483,7 +484,8 @@ public class ChunkMap extends ChunkStorage implements ChunkHolder.PlayerProvider
2021-06-11 14:02:28 +02:00
this.processUnloads(() -> {
return true;
});
- this.flushWorker();
+ this.level.asyncChunkTaskManager.flush(); // Paper - flush to preserve behavior compat with pre-async behaviour
+// this.i(); // Paper - nuke IOWorker
} else {
this.visibleChunkMap.values().stream().filter(ChunkHolder::wasAccessibleSinceLastSave).forEach((playerchunk) -> {
2021-07-07 08:52:40 +02:00
ChunkAccess ichunkaccess = (ChunkAccess) playerchunk.getChunkToSave().getNow(null); // CraftBukkit - decompile error
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -498,16 +500,20 @@ public class ChunkMap extends ChunkStorage implements ChunkHolder.PlayerProvider
2021-06-11 14:02:28 +02:00
}
- private static final double UNLOAD_QUEUE_RESIZE_FACTOR = 0.96; // Spigot
+ private static final double UNLOAD_QUEUE_RESIZE_FACTOR = 0.90; // Spigot // Paper - unload more
protected void tick(BooleanSupplier shouldKeepTicking) {
ProfilerFiller gameprofilerfiller = this.level.getProfiler();
+ try (Timing ignored = this.level.timings.poiUnload.startTiming()) { // Paper
gameprofilerfiller.push("poi");
this.poiManager.tick(shouldKeepTicking);
+ } // Paper
gameprofilerfiller.popPush("chunk_unload");
if (!this.level.noSave()) {
+ try (Timing ignored = this.level.timings.chunkUnload.startTiming()) { // Paper
this.processUnloads(shouldKeepTicking);
+ }// Paper
}
gameprofilerfiller.pop();
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -528,12 +534,13 @@ public class ChunkMap extends ChunkStorage implements ChunkHolder.PlayerProvider
2021-06-11 14:02:28 +02:00
if (playerchunk != null) {
this.pendingUnloads.put(j, playerchunk);
this.modified = true;
+ this.scheduleUnload(j, playerchunk); // Paper - Move up - don't leak chunks
// Spigot start
if (!shouldKeepTicking.getAsBoolean() && this.toDrop.size() <= targetSize && activityAccountant.activityTimeIsExhausted()) {
break;
}
// Spigot end
- this.scheduleUnload(j, playerchunk);
+ //this.a(j, playerchunk); // Paper - move up because spigot did a dumb
}
}
activityAccountant.endActivity(); // Spigot
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -547,6 +554,46 @@ public class ChunkMap extends ChunkStorage implements ChunkHolder.PlayerProvider
2021-06-11 14:02:28 +02:00
}
+ // Paper start - async chunk save for unload
+ // Note: This is very unsafe to call if the chunk is still in use.
2021-06-13 04:16:10 +02:00
+ // This is also modeled after PlayerChunkMap#save(IChunkAccess, boolean), with the intentional difference being
2021-06-11 14:02:28 +02:00
+ // serializing the chunk is left to a worker thread.
+ private void asyncSave(ChunkAccess chunk) {
+ ChunkPos chunkPos = chunk.getPos();
+ CompoundTag poiData;
+ try (Timing ignored = this.level.timings.chunkUnloadPOISerialization.startTiming()) {
+ poiData = this.getVillagePlace().getData(chunk.getPos());
+ }
+
+ com.destroystokyo.paper.io.PaperFileIOThread.Holder.INSTANCE.scheduleSave(this.level, chunkPos.x, chunkPos.z,
+ poiData, null, com.destroystokyo.paper.io.PrioritizedTaskQueue.NORMAL_PRIORITY);
2021-06-11 14:02:28 +02:00
+
+ if (!chunk.isUnsaved()) {
+ return;
+ }
+
+ ChunkStatus chunkstatus = chunk.getStatus();
+
2021-06-13 04:16:10 +02:00
+ // Copied from PlayerChunkMap#save(IChunkAccess, boolean)
2021-06-11 14:02:28 +02:00
+ if (chunkstatus.getChunkType() != ChunkStatus.ChunkType.LEVELCHUNK) {
2021-06-13 04:16:10 +02:00
+ // Paper start - Optimize save by using status cache
+ if (chunkstatus == ChunkStatus.EMPTY && chunk.getAllStarts().values().stream().noneMatch(StructureStart::isValid)) {
+ return;
2021-06-11 14:02:28 +02:00
+ }
+ }
+
+ ChunkSerializer.AsyncSaveData asyncSaveData;
+ try (Timing ignored = this.level.timings.chunkUnloadPrepareSave.startTiming()) {
+ asyncSaveData = ChunkSerializer.getAsyncSaveData(this.level, chunk);
+ }
+
+ this.level.asyncChunkTaskManager.scheduleChunkSave(chunkPos.x, chunkPos.z, com.destroystokyo.paper.io.PrioritizedTaskQueue.NORMAL_PRIORITY,
2021-06-11 14:02:28 +02:00
+ asyncSaveData, chunk);
+
+ chunk.setUnsaved(false);
+ }
+ // Paper end
+
2021-06-13 04:16:10 +02:00
private void scheduleUnload(long pos, ChunkHolder holder) {
CompletableFuture<ChunkAccess> completablefuture = holder.getChunkToSave();
2021-06-11 14:02:28 +02:00
Consumer<ChunkAccess> consumer = (ichunkaccess) -> { // CraftBukkit - decompile error
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -566,7 +613,16 @@ public class ChunkMap extends ChunkStorage implements ChunkHolder.PlayerProvider
2021-06-11 14:02:28 +02:00
((LevelChunk) ichunkaccess).setLoaded(false);
}
- this.save(ichunkaccess);
2021-06-13 04:16:10 +02:00
+ // Paper start - async chunk saving
2021-06-11 14:02:28 +02:00
+ try {
2021-06-13 04:16:10 +02:00
+ this.asyncSave(ichunkaccess);
+ } catch (ThreadDeath ex) {
+ throw ex; // bye
2021-06-11 14:02:28 +02:00
+ } catch (Throwable ex) {
+ LOGGER.fatal("Failed to prepare async save, attempting synchronous save", ex);
+ this.save(ichunkaccess);
+ }
2021-06-13 04:16:10 +02:00
+ // Paper end - async chunk saving
if (this.entitiesInLevel.remove(pos) && ichunkaccess instanceof LevelChunk) {
LevelChunk chunk = (LevelChunk) ichunkaccess;
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -631,19 +687,23 @@ public class ChunkMap extends ChunkStorage implements ChunkHolder.PlayerProvider
2021-06-11 14:02:28 +02:00
}
private CompletableFuture<Either<ChunkAccess, ChunkHolder.ChunkLoadingFailure>> scheduleChunkLoad(ChunkPos pos) {
- return CompletableFuture.supplyAsync(() -> {
+ // Paper start - Async chunk io
+ final java.util.function.BiFunction<ChunkSerializer.InProgressChunkHolder, Throwable, Either<ChunkAccess, ChunkHolder.ChunkLoadingFailure>> syncLoadComplete = (chunkHolder, ioThrowable) -> {
try (Timing ignored = this.level.timings.chunkLoad.startTimingIfSync()) { // Paper
this.level.getProfiler().incrementCounter("chunkLoad");
- CompoundTag nbttagcompound; // Paper
- try (Timing ignored2 = this.level.timings.chunkIO.startTimingIfSync()) { // Paper start - timings
- nbttagcompound = this.readChunk(pos);
- } // Paper end
+ // Paper start
+ if (ioThrowable != null) {
+ com.destroystokyo.paper.util.SneakyThrow.sneaky(ioThrowable);
+ }
- if (nbttagcompound != null) {try (Timing ignored2 = this.level.timings.chunkLoadLevelTimer.startTimingIfSync()) { // Paper start - timings
- boolean flag = nbttagcompound.contains("Level", 10) && nbttagcompound.getCompound("Level").contains("Status", 8);
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
+ this.getVillagePlace().loadInData(pos, chunkHolder.poiData);
+ chunkHolder.tasks.forEach(Runnable::run);
+ // Paper end
2021-06-11 14:02:28 +02:00
- if (flag) {
- ProtoChunk protochunk = ChunkSerializer.read(this.level, this.structureManager, this.poiManager, pos, nbttagcompound);
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
+ if (chunkHolder.protoChunk != null) {try (Timing ignored2 = this.level.timings.chunkLoadLevelTimer.startTimingIfSync()) { // Paper start - timings // Paper - chunk is created async
+
2021-06-11 14:02:28 +02:00
+ if (true) {
+ ProtoChunk protochunk = chunkHolder.protoChunk;
this.markPosition(pos, protochunk.getStatus().getChunkType());
2021-06-13 04:16:10 +02:00
return Either.left(protochunk);
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -666,7 +726,32 @@ public class ChunkMap extends ChunkStorage implements ChunkHolder.PlayerProvider
2021-06-11 14:02:28 +02:00
this.markPositionReplaceable(pos);
2021-06-13 04:16:10 +02:00
return Either.left(new ProtoChunk(pos, UpgradeData.EMPTY, this.level));
2021-06-11 14:02:28 +02:00
- }, this.mainThreadExecutor);
+ // Paper start - Async chunk io
+ };
+ CompletableFuture<Either<ChunkAccess, ChunkHolder.ChunkLoadingFailure>> ret = new CompletableFuture<>();
+
+ Consumer<ChunkSerializer.InProgressChunkHolder> chunkHolderConsumer = (ChunkSerializer.InProgressChunkHolder holder) -> {
+ // Go into the chunk load queue and not server task queue so we can be popped out even faster.
+ com.destroystokyo.paper.io.chunk.ChunkTaskManager.queueChunkWaitTask(() -> {
+ try {
+ ret.complete(syncLoadComplete.apply(holder, null));
+ } catch (Exception e) {
+ ret.completeExceptionally(e);
+ }
+ });
+ };
+
+ CompletableFuture<CompoundTag> chunkSaveFuture = this.level.asyncChunkTaskManager.getChunkSaveFuture(pos.x, pos.z);
+ if (chunkSaveFuture != null) {
+ this.level.asyncChunkTaskManager.scheduleChunkLoad(pos.x, pos.z,
+ com.destroystokyo.paper.io.PrioritizedTaskQueue.HIGH_PRIORITY, chunkHolderConsumer, false, chunkSaveFuture);
+ this.level.asyncChunkTaskManager.raisePriority(pos.x, pos.z, com.destroystokyo.paper.io.PrioritizedTaskQueue.HIGH_PRIORITY);
+ } else {
+ this.level.asyncChunkTaskManager.scheduleChunkLoad(pos.x, pos.z,
+ com.destroystokyo.paper.io.PrioritizedTaskQueue.NORMAL_PRIORITY, chunkHolderConsumer, false);
+ }
+ return ret;
+ // Paper end
}
private void markPositionReplaceable(ChunkPos chunkcoordintpair) {
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -848,6 +933,7 @@ public class ChunkMap extends ChunkStorage implements ChunkHolder.PlayerProvider
2021-06-11 14:02:28 +02:00
}
public boolean save(ChunkAccess chunk) {
+ try (co.aikar.timings.Timing ignored = this.level.timings.chunkSave.startTiming()) { // Paper
this.poiManager.flush(chunk.getPos());
if (!chunk.isUnsaved()) {
return false;
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -859,7 +945,7 @@ public class ChunkMap extends ChunkStorage implements ChunkHolder.PlayerProvider
2021-06-11 14:02:28 +02:00
ChunkStatus chunkstatus = chunk.getStatus();
if (chunkstatus.getChunkType() != ChunkStatus.ChunkType.LEVELCHUNK) {
2021-06-13 04:16:10 +02:00
- if (this.isExistingChunkFull(chunkcoordintpair)) {
+ if (false && this.isExistingChunkFull(chunkcoordintpair)) { // Paper
2021-06-11 14:02:28 +02:00
return false;
}
2021-06-13 04:16:10 +02:00
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -869,9 +955,16 @@ public class ChunkMap extends ChunkStorage implements ChunkHolder.PlayerProvider
2021-06-11 14:02:28 +02:00
}
this.level.getProfiler().incrementCounter("chunkSave");
- CompoundTag nbttagcompound = ChunkSerializer.write(this.level, chunk);
+ CompoundTag nbttagcompound;
+ try (co.aikar.timings.Timing ignored1 = this.level.timings.chunkSaveDataSerialization.startTiming()) { // Paper
+ nbttagcompound = ChunkSerializer.write(this.level, chunk);
+ } // Paper
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
+
2021-06-11 14:02:28 +02:00
- this.write(chunkcoordintpair, nbttagcompound);
+ // Paper start - async chunk io
+ com.destroystokyo.paper.io.PaperFileIOThread.Holder.INSTANCE.scheduleSave(this.level, chunkcoordintpair.x, chunkcoordintpair.z,
+ null, nbttagcompound, com.destroystokyo.paper.io.PrioritizedTaskQueue.NORMAL_PRIORITY);
+ // Paper end - async chunk io
this.markPosition(chunkcoordintpair, chunkstatus.getChunkType());
return true;
} catch (Exception exception) {
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -880,6 +973,7 @@ public class ChunkMap extends ChunkStorage implements ChunkHolder.PlayerProvider
2021-06-11 14:02:28 +02:00
return false;
}
}
+ } // Paper
}
private boolean isExistingChunkFull(ChunkPos chunkcoordintpair) {
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -1007,6 +1101,35 @@ public class ChunkMap extends ChunkStorage implements ChunkHolder.PlayerProvider
2021-06-11 14:02:28 +02:00
}
}
+ // Paper start - Asynchronous chunk io
+ @Nullable
+ @Override
+ public CompoundTag read(ChunkPos chunkcoordintpair) throws IOException {
+ if (Thread.currentThread() != com.destroystokyo.paper.io.PaperFileIOThread.Holder.INSTANCE) {
+ CompoundTag ret = com.destroystokyo.paper.io.PaperFileIOThread.Holder.INSTANCE
+ .loadChunkDataAsyncFuture(this.level, chunkcoordintpair.x, chunkcoordintpair.z, com.destroystokyo.paper.io.IOUtil.getPriorityForCurrentThread(),
+ false, true, true).join().chunkData;
+
+ if (ret == com.destroystokyo.paper.io.PaperFileIOThread.FAILURE_VALUE) {
+ throw new IOException("See logs for further detail");
+ }
+ return ret;
+ }
+ return super.read(chunkcoordintpair);
+ }
+
+ @Override
+ public void write(ChunkPos chunkcoordintpair, CompoundTag nbttagcompound) throws IOException {
+ if (Thread.currentThread() != com.destroystokyo.paper.io.PaperFileIOThread.Holder.INSTANCE) {
+ com.destroystokyo.paper.io.PaperFileIOThread.Holder.INSTANCE.scheduleSave(
+ this.level, chunkcoordintpair.x, chunkcoordintpair.z, null, nbttagcompound,
+ com.destroystokyo.paper.io.IOUtil.getPriorityForCurrentThread());
+ return;
+ }
+ super.write(chunkcoordintpair, nbttagcompound);
+ }
+ // Paper end
+
@Nullable
2021-06-13 04:16:10 +02:00
private CompoundTag readChunk(ChunkPos pos) throws IOException {
2021-06-11 14:02:28 +02:00
CompoundTag nbttagcompound = this.read(pos);
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -1360,6 +1483,7 @@ public class ChunkMap extends ChunkStorage implements ChunkHolder.PlayerProvider
2021-06-11 14:02:28 +02:00
}
2021-06-17 21:52:26 +02:00
+ @Deprecated public PoiManager getVillagePlace() { return this.getPoiManager(); } // Paper - OBFHELPER
2021-06-11 14:02:28 +02:00
protected PoiManager getPoiManager() {
return this.poiManager;
}
diff --git a/src/main/java/net/minecraft/server/level/ServerChunkCache.java b/src/main/java/net/minecraft/server/level/ServerChunkCache.java
index 1f3fe980e71c13b5e7852349cba1cb0e4aa42dcd..30b88f0e8ad40438af8a7d5bec1471a60ddd977b 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/net/minecraft/server/level/ServerChunkCache.java
+++ b/src/main/java/net/minecraft/server/level/ServerChunkCache.java
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -481,10 +481,128 @@ public class ServerChunkCache extends ChunkSource {
2021-06-13 04:16:10 +02:00
return ret;
2021-06-11 14:02:28 +02:00
}
2021-06-13 04:16:10 +02:00
// Paper end
+ // Paper start - async chunk io
+ public ChunkAccess getChunkAtImmediately(int x, int z) {
+ ChunkHolder holder = this.chunkMap.getVisibleChunkIfPresent(ChunkPos.asLong(x, z));
+ if (holder == null) {
+ return null;
+ }
+
+ return holder.getLastAvailable();
+ }
2021-06-11 14:02:28 +02:00
+
+ private long asyncLoadSeqCounter;
+
+ public CompletableFuture<Either<ChunkAccess, ChunkHolder.ChunkLoadingFailure>> getChunkAtAsynchronously(int x, int z, boolean gen, boolean isUrgent) {
+ if (Thread.currentThread() != this.mainThread) {
+ CompletableFuture<Either<ChunkAccess, ChunkHolder.ChunkLoadingFailure>> future = new CompletableFuture<Either<ChunkAccess, ChunkHolder.ChunkLoadingFailure>>();
+ this.mainThreadProcessor.execute(() -> {
+ this.getChunkAtAsynchronously(x, z, gen, isUrgent).whenComplete((chunk, ex) -> {
+ if (ex != null) {
+ future.completeExceptionally(ex);
+ } else {
+ future.complete(chunk);
+ }
+ });
+ });
+ return future;
+ }
+
+ long k = ChunkPos.asLong(x, z);
+ ChunkPos chunkPos = new ChunkPos(x, z);
+
+ ChunkAccess ichunkaccess;
+
+ // try cache
+ for (int l = 0; l < 4; ++l) {
+ if (k == this.lastChunkPos[l] && ChunkStatus.FULL == this.lastChunkStatus[l]) {
+ ichunkaccess = this.lastChunk[l];
+ if (ichunkaccess != null) { // CraftBukkit - the chunk can become accessible in the meantime TODO for non-null chunks it might also make sense to check that the chunk's state hasn't changed in the meantime
+
+ // move to first in cache
+
+ for (int i1 = 3; i1 > 0; --i1) {
+ this.lastChunkPos[i1] = this.lastChunkPos[i1 - 1];
+ this.lastChunkStatus[i1] = this.lastChunkStatus[i1 - 1];
+ this.lastChunk[i1] = this.lastChunk[i1 - 1];
+ }
+
+ this.lastChunkPos[0] = k;
+ this.lastChunkStatus[0] = ChunkStatus.FULL;
+ this.lastChunk[0] = ichunkaccess;
+
+ return CompletableFuture.completedFuture(Either.left(ichunkaccess));
+ }
+ }
+ }
+
+ if (gen) {
+ return this.bringToFullStatusAsync(x, z, chunkPos, isUrgent);
+ }
+
+ ChunkAccess current = this.getChunkAtImmediately(x, z); // we want to bypass ticket restrictions
+ if (current != null) {
2021-06-13 04:16:10 +02:00
+ if (!(current instanceof net.minecraft.world.level.chunk.ImposterProtoChunk) && !(current instanceof LevelChunk)) {
2021-06-11 14:02:28 +02:00
+ return CompletableFuture.completedFuture(ChunkHolder.UNLOADED_CHUNK);
+ }
+ // we know the chunk is at full status here (either in read-only mode or the real thing)
+ return this.bringToFullStatusAsync(x, z, chunkPos, isUrgent);
+ }
+
+ // here we don't know what status it is and we're not supposed to generate
+ // so we asynchronously load empty status
+ return this.bringToStatusAsync(x, z, chunkPos, ChunkStatus.EMPTY, isUrgent).thenCompose((either) -> {
+ ChunkAccess chunk = either.left().orElse(null);
2021-06-13 04:16:10 +02:00
+ if (!(chunk instanceof net.minecraft.world.level.chunk.ImposterProtoChunk) && !(chunk instanceof LevelChunk)) {
2021-06-11 14:02:28 +02:00
+ // the chunk on disk was not a full status chunk
+ return CompletableFuture.completedFuture(ChunkHolder.UNLOADED_CHUNK);
+ }
2021-06-13 04:16:10 +02:00
+ // bring to full status if required
2021-06-11 14:02:28 +02:00
+ return this.bringToFullStatusAsync(x, z, chunkPos, isUrgent);
+ });
+ }
+
+ private CompletableFuture<Either<ChunkAccess, ChunkHolder.ChunkLoadingFailure>> bringToFullStatusAsync(int x, int z, ChunkPos chunkPos, boolean isUrgent) {
+ return this.bringToStatusAsync(x, z, chunkPos, ChunkStatus.FULL, isUrgent);
+ }
+
+ private CompletableFuture<Either<ChunkAccess, ChunkHolder.ChunkLoadingFailure>> bringToStatusAsync(int x, int z, ChunkPos chunkPos, ChunkStatus status, boolean isUrgent) {
+ CompletableFuture<Either<ChunkAccess, ChunkHolder.ChunkLoadingFailure>> future = this.getChunkFutureMainThread(x, z, status, true, isUrgent);
+ Long identifier = Long.valueOf(this.asyncLoadSeqCounter++);
2021-06-13 04:16:10 +02:00
+ int ticketLevel = net.minecraft.server.MCUtil.getTicketLevelFor(status);
2021-06-11 14:02:28 +02:00
+ this.addTicketAtLevel(TicketType.ASYNC_LOAD, chunkPos, ticketLevel, identifier);
+
+ return future.thenComposeAsync((Either<ChunkAccess, ChunkHolder.ChunkLoadingFailure> either) -> {
+ // either left -> success
+ // either right -> failure
+
+ this.removeTicketAtLevel(TicketType.ASYNC_LOAD, chunkPos, ticketLevel, identifier);
+ this.addTicketAtLevel(TicketType.UNKNOWN, chunkPos, ticketLevel, chunkPos); // allow unloading
+
+ Optional<ChunkHolder.ChunkLoadingFailure> failure = either.right();
+
+ if (failure.isPresent()) {
+ // failure
+ throw new IllegalStateException("Chunk failed to load: " + failure.get().toString());
+ }
+
+ return CompletableFuture.completedFuture(either);
+ }, this.mainThreadProcessor);
+ }
+
+ public <T> void addTicketAtLevel(TicketType<T> ticketType, ChunkPos chunkPos, int ticketLevel, T identifier) {
+ this.distanceManager.addTicketAtLevel(ticketType, chunkPos, ticketLevel, identifier);
+ }
+
+ public <T> void removeTicketAtLevel(TicketType<T> ticketType, ChunkPos chunkPos, int ticketLevel, T identifier) {
+ this.distanceManager.removeTicketAtLevel(ticketType, chunkPos, ticketLevel, identifier);
+ }
2021-06-13 04:16:10 +02:00
+ // Paper end - async chunk io
2021-06-11 14:02:28 +02:00
@Nullable
@Override
public ChunkAccess getChunk(int x, int z, ChunkStatus leastStatus, boolean create) {
+ final int x1 = x; final int z1 = z; // Paper - conflict on variable change
if (Thread.currentThread() != this.mainThread) {
return (ChunkAccess) CompletableFuture.supplyAsync(() -> {
return this.getChunk(x, z, leastStatus, create);
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -507,13 +625,18 @@ public class ServerChunkCache extends ChunkSource {
2021-06-11 14:02:28 +02:00
}
gameprofilerfiller.incrementCounter("getChunkCacheMiss");
- CompletableFuture<Either<ChunkAccess, ChunkHolder.ChunkLoadingFailure>> completablefuture = this.getChunkFutureMainThread(x, z, leastStatus, create);
+ CompletableFuture<Either<ChunkAccess, ChunkHolder.ChunkLoadingFailure>> completablefuture = this.getChunkFutureMainThread(x, z, leastStatus, create, true); // Paper
2021-06-13 04:16:10 +02:00
ServerChunkCache.MainThreadExecutor chunkproviderserver_a = this.mainThreadProcessor;
2021-06-11 14:02:28 +02:00
2021-06-13 04:16:10 +02:00
Objects.requireNonNull(completablefuture);
2021-06-11 14:02:28 +02:00
if (!completablefuture.isDone()) { // Paper
+ // Paper start - async chunk io/loading
+ this.level.asyncChunkTaskManager.raisePriority(x1, z1, com.destroystokyo.paper.io.PrioritizedTaskQueue.HIGHEST_PRIORITY);
+ com.destroystokyo.paper.io.chunk.ChunkTaskManager.pushChunkWait(this.level, x1, z1);
+ // Paper end
this.level.timings.syncChunkLoad.startTiming(); // Paper
2021-06-13 04:16:10 +02:00
chunkproviderserver_a.managedBlock(completablefuture::isDone);
2021-06-11 14:02:28 +02:00
+ com.destroystokyo.paper.io.chunk.ChunkTaskManager.popChunkWait(); // Paper - async chunk debug
this.level.timings.syncChunkLoad.stopTiming(); // Paper
} // Paper
ichunkaccess = (ChunkAccess) ((Either) completablefuture.join()).map((ichunkaccess1) -> {
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -600,6 +723,11 @@ public class ServerChunkCache extends ChunkSource {
2021-06-11 14:02:28 +02:00
}
2021-06-13 04:16:10 +02:00
private CompletableFuture<Either<ChunkAccess, ChunkHolder.ChunkLoadingFailure>> getChunkFutureMainThread(int i, int j, ChunkStatus chunkstatus, boolean flag) {
2021-06-11 14:02:28 +02:00
+ // Paper start - add isUrgent - old sig left in place for dirty nms plugins
2021-06-13 04:16:10 +02:00
+ return getChunkFutureMainThread(i, j, chunkstatus, flag, false);
2021-06-11 14:02:28 +02:00
+ }
+ private CompletableFuture<Either<ChunkAccess, ChunkHolder.ChunkLoadingFailure>> getChunkFutureMainThread(int i, int j, ChunkStatus chunkstatus, boolean flag, boolean isUrgent) {
+ // Paper end
2021-06-13 04:16:10 +02:00
ChunkPos chunkcoordintpair = new ChunkPos(i, j);
2021-06-11 14:02:28 +02:00
long k = chunkcoordintpair.toLong();
2021-06-13 04:16:10 +02:00
int l = 33 + ChunkStatus.getDistance(chunkstatus);
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -997,11 +1125,12 @@ public class ServerChunkCache extends ChunkSource {
2021-06-11 14:02:28 +02:00
// CraftBukkit start - process pending Chunk loadCallback() and unloadCallback() after each run task
2021-06-13 04:16:10 +02:00
public boolean pollTask() {
2021-06-11 14:02:28 +02:00
try {
+ boolean execChunkTask = com.destroystokyo.paper.io.chunk.ChunkTaskManager.pollChunkWaitQueue() || ServerChunkCache.this.level.asyncChunkTaskManager.pollNextChunkTask(); // Paper
if (ServerChunkCache.this.runDistanceManagerUpdates()) {
return true;
} else {
ServerChunkCache.this.lightEngine.tryScheduleUpdate();
- return super.pollTask();
+ return super.pollTask() || execChunkTask; // Paper
}
} finally {
chunkMap.callbackExecutor.run();
diff --git a/src/main/java/net/minecraft/server/level/ServerLevel.java b/src/main/java/net/minecraft/server/level/ServerLevel.java
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
index 1d95dae881fd89056e2ad86a8f33aecf90cf8a3b..769753441eea00296d818b7a741551fd3034cb6a 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/net/minecraft/server/level/ServerLevel.java
+++ b/src/main/java/net/minecraft/server/level/ServerLevel.java
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -295,6 +295,78 @@ public class ServerLevel extends Level implements WorldGenLevel {
}
}
2021-06-11 14:02:28 +02:00
}
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
+
2021-06-11 14:02:28 +02:00
+ // Paper start - Asynchronous IO
+ public final com.destroystokyo.paper.io.PaperFileIOThread.ChunkDataController poiDataController = new com.destroystokyo.paper.io.PaperFileIOThread.ChunkDataController() {
+ @Override
2021-06-13 04:16:10 +02:00
+ public void writeData(int x, int z, net.minecraft.nbt.CompoundTag compound) throws java.io.IOException {
2021-06-11 14:02:28 +02:00
+ ServerLevel.this.getChunkSource().chunkMap.getVillagePlace().write(new ChunkPos(x, z), compound);
+ }
+
+ @Override
2021-06-13 04:16:10 +02:00
+ public net.minecraft.nbt.CompoundTag readData(int x, int z) throws java.io.IOException {
2021-06-11 14:02:28 +02:00
+ return ServerLevel.this.getChunkSource().chunkMap.getVillagePlace().read(new ChunkPos(x, z));
+ }
+
+ @Override
2021-06-13 04:16:10 +02:00
+ public <T> T computeForRegionFile(int chunkX, int chunkZ, java.util.function.Function<net.minecraft.world.level.chunk.storage.RegionFile, T> function) {
2021-06-11 14:02:28 +02:00
+ synchronized (ServerLevel.this.getChunkSource().chunkMap.getVillagePlace()) {
2021-06-13 04:16:10 +02:00
+ net.minecraft.world.level.chunk.storage.RegionFile file;
2021-06-11 14:02:28 +02:00
+
+ try {
+ file = ServerLevel.this.getChunkSource().chunkMap.getVillagePlace().getFile(new ChunkPos(chunkX, chunkZ), false);
+ } catch (java.io.IOException ex) {
+ throw new RuntimeException(ex);
+ }
+
+ return function.apply(file);
+ }
+ }
+
+ @Override
2021-06-13 04:16:10 +02:00
+ public <T> T computeForRegionFileIfLoaded(int chunkX, int chunkZ, java.util.function.Function<net.minecraft.world.level.chunk.storage.RegionFile, T> function) {
2021-06-11 14:02:28 +02:00
+ synchronized (ServerLevel.this.getChunkSource().chunkMap.getVillagePlace()) {
2021-06-13 04:16:10 +02:00
+ net.minecraft.world.level.chunk.storage.RegionFile file = ServerLevel.this.getChunkSource().chunkMap.getVillagePlace().getRegionFileIfLoaded(new ChunkPos(chunkX, chunkZ));
2021-06-11 14:02:28 +02:00
+ return function.apply(file);
+ }
+ }
+ };
+
+ public final com.destroystokyo.paper.io.PaperFileIOThread.ChunkDataController chunkDataController = new com.destroystokyo.paper.io.PaperFileIOThread.ChunkDataController() {
+ @Override
2021-06-13 04:16:10 +02:00
+ public void writeData(int x, int z, net.minecraft.nbt.CompoundTag compound) throws java.io.IOException {
2021-06-11 14:02:28 +02:00
+ ServerLevel.this.getChunkSource().chunkMap.write(new ChunkPos(x, z), compound);
+ }
+
+ @Override
2021-06-13 04:16:10 +02:00
+ public net.minecraft.nbt.CompoundTag readData(int x, int z) throws java.io.IOException {
2021-06-11 14:02:28 +02:00
+ return ServerLevel.this.getChunkSource().chunkMap.read(new ChunkPos(x, z));
+ }
+
+ @Override
2021-06-13 04:16:10 +02:00
+ public <T> T computeForRegionFile(int chunkX, int chunkZ, java.util.function.Function<net.minecraft.world.level.chunk.storage.RegionFile, T> function) {
2021-06-11 14:02:28 +02:00
+ synchronized (ServerLevel.this.getChunkSource().chunkMap) {
2021-06-13 04:16:10 +02:00
+ net.minecraft.world.level.chunk.storage.RegionFile file;
2021-06-11 14:02:28 +02:00
+
+ try {
+ file = ServerLevel.this.getChunkSource().chunkMap.regionFileCache.getFile(new ChunkPos(chunkX, chunkZ), false);
+ } catch (java.io.IOException ex) {
+ throw new RuntimeException(ex);
+ }
+
+ return function.apply(file);
+ }
+ }
+
+ @Override
2021-06-13 04:16:10 +02:00
+ public <T> T computeForRegionFileIfLoaded(int chunkX, int chunkZ, java.util.function.Function<net.minecraft.world.level.chunk.storage.RegionFile, T> function) {
2021-06-11 14:02:28 +02:00
+ synchronized (ServerLevel.this.getChunkSource().chunkMap) {
2021-06-13 04:16:10 +02:00
+ net.minecraft.world.level.chunk.storage.RegionFile file = ServerLevel.this.getChunkSource().chunkMap.regionFileCache.getRegionFileIfLoaded(new ChunkPos(chunkX, chunkZ));
2021-06-11 14:02:28 +02:00
+ return function.apply(file);
+ }
+ }
+ };
+ public final com.destroystokyo.paper.io.chunk.ChunkTaskManager asyncChunkTaskManager;
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
// Paper end
2021-06-11 14:02:28 +02:00
// Add env and gen to constructor, WorldData -> WorldDataServer
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
@@ -368,6 +440,8 @@ public class ServerLevel extends Level implements WorldGenLevel {
2021-06-13 04:16:10 +02:00
this.sleepStatus = new SleepStatus();
2021-06-11 14:02:28 +02:00
this.getCraftServer().addWorld(this.getWorld()); // CraftBukkit
+
+ this.asyncChunkTaskManager = new com.destroystokyo.paper.io.chunk.ChunkTaskManager(this); // Paper
}
public void setWeatherParameters(int clearDuration, int rainDuration, boolean raining, boolean thundering) {
2021-06-11 14:02:28 +02:00
diff --git a/src/main/java/net/minecraft/server/level/TicketType.java b/src/main/java/net/minecraft/server/level/TicketType.java
2021-06-13 04:16:10 +02:00
index 0d536d72ac918fbd403397ff369d10143ee9c204..be677d437d17b74c6188ce1bd5fc6fdc228fd92f 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/net/minecraft/server/level/TicketType.java
+++ b/src/main/java/net/minecraft/server/level/TicketType.java
2021-06-13 04:16:10 +02:00
@@ -8,6 +8,7 @@ import net.minecraft.world.level.ChunkPos;
public class TicketType<T> {
2021-06-11 14:02:28 +02:00
public static final TicketType<Long> FUTURE_AWAIT = create("future_await", Long::compareTo); // Paper
+ public static final TicketType<Long> ASYNC_LOAD = create("async_load", Long::compareTo); // Paper
2021-06-13 04:16:10 +02:00
private final String name;
private final Comparator<T> comparator;
2021-06-11 14:02:28 +02:00
diff --git a/src/main/java/net/minecraft/server/network/ServerGamePacketListenerImpl.java b/src/main/java/net/minecraft/server/network/ServerGamePacketListenerImpl.java
2021-07-07 08:52:40 +02:00
index 265d147de39e306fac27913b2dc3899e82c63796..912171f8b3280a6c45311104216e04d661daff8a 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/net/minecraft/server/network/ServerGamePacketListenerImpl.java
+++ b/src/main/java/net/minecraft/server/network/ServerGamePacketListenerImpl.java
2021-07-07 08:52:40 +02:00
@@ -711,6 +711,13 @@ public class ServerGamePacketListenerImpl implements ServerPlayerConnection, Ser
2021-06-11 14:02:28 +02:00
server.scheduleOnMain(() -> this.disconnect(new TranslatableComponent("disconnect.spam", new Object[0]))); // Paper
return;
}
+ // Paper start
+ String str = packet.getCommand(); int index = -1;
+ if (str.length() > 64 && ((index = str.indexOf(' ')) == -1 || index >= 64)) {
+ server.scheduleOnMain(() -> this.disconnect(new TranslatableComponent("disconnect.spam", new Object[0]))); // Paper
+ return;
+ }
+ // Paper end
// CraftBukkit end
StringReader stringreader = new StringReader(packet.getCommand());
diff --git a/src/main/java/net/minecraft/world/entity/ai/village/poi/PoiManager.java b/src/main/java/net/minecraft/world/entity/ai/village/poi/PoiManager.java
index 3ca8b13744b406c3e563747f0cb69647c94103df..7b1d2748328ffc1447bcacd1316f2c6fdbaf92b0 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/net/minecraft/world/entity/ai/village/poi/PoiManager.java
+++ b/src/main/java/net/minecraft/world/entity/ai/village/poi/PoiManager.java
2021-06-13 04:16:10 +02:00
@@ -37,9 +37,11 @@ public class PoiManager extends SectionStorage<PoiSection> {
public static final int VILLAGE_SECTION_SIZE = 1;
private final PoiManager.DistanceTracker distanceTracker;
2021-06-11 14:02:28 +02:00
private final LongSet loadedChunks = new LongOpenHashSet();
2021-06-13 04:16:10 +02:00
+ private final net.minecraft.server.level.ServerLevel world; // Paper
2021-06-11 14:02:28 +02:00
2021-06-13 04:16:10 +02:00
public PoiManager(File directory, DataFixer dataFixer, boolean dsync, LevelHeightAccessor world) {
super(directory, PoiSection::codec, PoiSection::new, dataFixer, DataFixTypes.POI_CHUNK, dsync, world);
+ this.world = (net.minecraft.server.level.ServerLevel)world; // Paper
this.distanceTracker = new PoiManager.DistanceTracker();
2021-06-11 14:02:28 +02:00
}
2021-06-13 04:16:10 +02:00
@@ -172,7 +174,18 @@ public class PoiManager extends SectionStorage<PoiSection> {
2021-06-11 14:02:28 +02:00
@Override
public void tick(BooleanSupplier shouldKeepTicking) {
- super.tick(shouldKeepTicking);
+ // Paper start - async chunk io
2021-06-13 04:16:10 +02:00
+ while (!this.dirty.isEmpty() && shouldKeepTicking.getAsBoolean()) {
+ ChunkPos chunkcoordintpair = SectionPos.of(this.dirty.firstLong()).chunk();
2021-06-11 14:02:28 +02:00
+
2021-06-13 04:16:10 +02:00
+ net.minecraft.nbt.CompoundTag data;
+ try (co.aikar.timings.Timing ignored1 = this.world.timings.poiSaveDataSerialization.startTiming()) {
+ data = this.getData(chunkcoordintpair);
2021-06-11 14:02:28 +02:00
+ }
2021-06-13 04:16:10 +02:00
+ com.destroystokyo.paper.io.PaperFileIOThread.Holder.INSTANCE.scheduleSave(this.world,
+ chunkcoordintpair.x, chunkcoordintpair.z, data, null, com.destroystokyo.paper.io.PrioritizedTaskQueue.NORMAL_PRIORITY);
2021-06-11 14:02:28 +02:00
+ }
+ // Paper end
this.distanceTracker.runAllUpdates();
}
2021-06-13 04:16:10 +02:00
@@ -265,6 +278,35 @@ public class PoiManager extends SectionStorage<PoiSection> {
2021-06-11 14:02:28 +02:00
}
}
+ // Paper start - Asynchronous chunk io
+ @javax.annotation.Nullable
+ @Override
2021-06-13 04:16:10 +02:00
+ public net.minecraft.nbt.CompoundTag read(ChunkPos chunkcoordintpair) throws java.io.IOException {
2021-06-11 14:02:28 +02:00
+ if (this.world != null && Thread.currentThread() != com.destroystokyo.paper.io.PaperFileIOThread.Holder.INSTANCE) {
2021-06-13 04:16:10 +02:00
+ net.minecraft.nbt.CompoundTag ret = com.destroystokyo.paper.io.PaperFileIOThread.Holder.INSTANCE
2021-06-11 14:02:28 +02:00
+ .loadChunkDataAsyncFuture(this.world, chunkcoordintpair.x, chunkcoordintpair.z, com.destroystokyo.paper.io.IOUtil.getPriorityForCurrentThread(),
+ true, false, true).join().poiData;
+
+ if (ret == com.destroystokyo.paper.io.PaperFileIOThread.FAILURE_VALUE) {
+ throw new java.io.IOException("See logs for further detail");
+ }
+ return ret;
+ }
+ return super.read(chunkcoordintpair);
+ }
+
+ @Override
2021-06-13 04:16:10 +02:00
+ public void write(ChunkPos chunkcoordintpair, net.minecraft.nbt.CompoundTag nbttagcompound) throws java.io.IOException {
2021-06-11 14:02:28 +02:00
+ if (this.world != null && Thread.currentThread() != com.destroystokyo.paper.io.PaperFileIOThread.Holder.INSTANCE) {
+ com.destroystokyo.paper.io.PaperFileIOThread.Holder.INSTANCE.scheduleSave(
+ this.world, chunkcoordintpair.x, chunkcoordintpair.z, nbttagcompound, null,
+ com.destroystokyo.paper.io.IOUtil.getPriorityForCurrentThread());
+ return;
+ }
+ super.write(chunkcoordintpair, nbttagcompound);
+ }
+ // Paper end
+
public static enum Occupancy {
2021-06-13 04:16:10 +02:00
HAS_SPACE(PoiRecord::hasSpace),
IS_OCCUPIED(PoiRecord::isOccupied),
2021-06-11 14:02:28 +02:00
diff --git a/src/main/java/net/minecraft/world/level/TickNextTickData.java b/src/main/java/net/minecraft/world/level/TickNextTickData.java
2021-06-13 04:16:10 +02:00
index 3af31dc2c82c11ee78d497c5777615c17cb13c7a..3b8c04f6ffd7e6c197465aa1caf633ba92529472 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/net/minecraft/world/level/TickNextTickData.java
+++ b/src/main/java/net/minecraft/world/level/TickNextTickData.java
2021-06-13 04:16:10 +02:00
@@ -4,7 +4,7 @@ import java.util.Comparator;
import net.minecraft.core.BlockPos;
2021-06-11 14:02:28 +02:00
public class TickNextTickData<T> {
- private static long counter;
+ private static final java.util.concurrent.atomic.AtomicLong COUNTER = new java.util.concurrent.atomic.AtomicLong(); // Paper - async chunk loading
private final T type;
public final BlockPos pos;
public final long triggerTick;
2021-06-13 04:16:10 +02:00
@@ -16,7 +16,7 @@ public class TickNextTickData<T> {
2021-06-11 14:02:28 +02:00
}
public TickNextTickData(BlockPos pos, T t, long time, TickPriority priority) {
2021-06-13 04:16:10 +02:00
- this.c = (long)(counter++);
+ this.c = (TickNextTickData.COUNTER.getAndIncrement()); // Paper - async chunk loading
2021-06-11 14:02:28 +02:00
this.pos = pos.immutable();
this.type = t;
this.triggerTick = time;
diff --git a/src/main/java/net/minecraft/world/level/chunk/storage/ChunkSerializer.java b/src/main/java/net/minecraft/world/level/chunk/storage/ChunkSerializer.java
index 03190535999d30aea0428631ae576b18f5d10eb7..b8bdd69bda00ba0eb72c161d3b49101b8221ac41 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/net/minecraft/world/level/chunk/storage/ChunkSerializer.java
+++ b/src/main/java/net/minecraft/world/level/chunk/storage/ChunkSerializer.java
@@ -67,7 +67,30 @@ public class ChunkSerializer {
2021-06-11 14:02:28 +02:00
2021-06-13 04:16:10 +02:00
public ChunkSerializer() {}
2021-06-11 14:02:28 +02:00
+ // Paper start
+ public static final class InProgressChunkHolder {
+
+ public final ProtoChunk protoChunk;
2021-06-13 04:16:10 +02:00
+ public final java.util.ArrayDeque<Runnable> tasks;
2021-06-11 14:02:28 +02:00
+
+ public CompoundTag poiData;
+
2021-06-13 04:16:10 +02:00
+ public InProgressChunkHolder(final ProtoChunk protoChunk, final java.util.ArrayDeque<Runnable> tasks) {
2021-06-11 14:02:28 +02:00
+ this.protoChunk = protoChunk;
+ this.tasks = tasks;
+ }
+ }
2021-06-13 04:16:10 +02:00
+ // Paper end
2021-06-11 14:02:28 +02:00
+
2021-06-13 04:16:10 +02:00
public static ProtoChunk read(ServerLevel world, StructureManager structureManager, PoiManager poiStorage, ChunkPos pos, CompoundTag nbt) {
+ // Paper start - add variant for async calls
+ InProgressChunkHolder holder = loadChunk(world, structureManager, poiStorage, pos, nbt, true);
2021-06-11 14:02:28 +02:00
+ holder.tasks.forEach(Runnable::run);
+ return holder.protoChunk;
+ }
2021-06-13 04:16:10 +02:00
+ public static InProgressChunkHolder loadChunk(ServerLevel world, StructureManager structureManager, PoiManager poiStorage, ChunkPos pos, CompoundTag nbt, boolean distinguish) {
+ java.util.ArrayDeque<Runnable> tasksToExecuteOnMain = new java.util.ArrayDeque<>();
2021-06-11 14:02:28 +02:00
+ // Paper end
2021-06-13 04:16:10 +02:00
ChunkGenerator chunkgenerator = world.getChunkSource().getGenerator();
2021-06-11 14:02:28 +02:00
BiomeSource worldchunkmanager = chunkgenerator.getBiomeSource();
2021-06-13 04:16:10 +02:00
CompoundTag nbttagcompound1 = nbt.getCompound("Level");
@@ -94,7 +117,9 @@ public class ChunkSerializer {
2021-06-11 14:02:28 +02:00
LevelLightEngine lightengine = chunkproviderserver.getLightEngine();
if (flag) {
+ tasksToExecuteOnMain.add(() -> { // Paper - delay this task since we're executing off-main
2021-06-13 04:16:10 +02:00
lightengine.retainData(pos, true);
2021-06-11 14:02:28 +02:00
+ }); // Paper - delay this task since we're executing off-main
}
2021-06-13 04:16:10 +02:00
for (int j = 0; j < nbttaglist.size(); ++j) {
@@ -110,16 +135,28 @@ public class ChunkSerializer {
2021-06-13 04:16:10 +02:00
achunksection[world.getSectionIndexFromSectionY(b0)] = chunksection;
2021-06-11 14:02:28 +02:00
}
+ tasksToExecuteOnMain.add(() -> { // Paper - delay this task since we're executing off-main
2021-06-13 04:16:10 +02:00
poiStorage.checkConsistencyWithBlocks(pos, chunksection);
2021-06-11 14:02:28 +02:00
+ }); // Paper - delay this task since we're executing off-main
}
if (flag) {
if (nbttagcompound2.contains("BlockLight", 7)) {
- lightengine.queueSectionData(LightLayer.BLOCK, SectionPos.of(pos, b0), new DataLayer(nbttagcompound2.getByteArray("BlockLight")), true);
+ // Paper start - delay this task since we're executing off-main
+ DataLayer blockLight = new DataLayer(nbttagcompound2.getByteArray("BlockLight"));
+ tasksToExecuteOnMain.add(() -> {
2021-06-13 04:16:10 +02:00
+ lightengine.queueSectionData(LightLayer.BLOCK, SectionPos.of(chunkcoordintpair1, b0), blockLight, true);
2021-06-11 14:02:28 +02:00
+ });
+ // Paper end - delay this task since we're executing off-main
}
2021-06-13 04:16:10 +02:00
if (flag1 && nbttagcompound2.contains("SkyLight", 7)) {
2021-06-11 14:02:28 +02:00
- lightengine.queueSectionData(LightLayer.SKY, SectionPos.of(pos, b0), new DataLayer(nbttagcompound2.getByteArray("SkyLight")), true);
+ // Paper start - delay this task since we're executing off-main
+ DataLayer skyLight = new DataLayer(nbttagcompound2.getByteArray("SkyLight"));
+ tasksToExecuteOnMain.add(() -> {
2021-06-13 04:16:10 +02:00
+ lightengine.queueSectionData(LightLayer.SKY, SectionPos.of(chunkcoordintpair1, b0), skyLight, true);
2021-06-11 14:02:28 +02:00
+ });
+ // Paper end - delay this task since we're executing off-main
}
}
}
@@ -233,7 +270,7 @@ public class ChunkSerializer {
2021-06-11 14:02:28 +02:00
}
if (chunkstatus_type == ChunkStatus.ChunkType.LEVELCHUNK) {
- return new ImposterProtoChunk((LevelChunk) object);
+ return new InProgressChunkHolder(new ImposterProtoChunk((LevelChunk) object), tasksToExecuteOnMain); // Paper - Async chunk loading
} else {
ProtoChunk protochunk1 = (ProtoChunk) object;
@@ -272,11 +309,92 @@ public class ChunkSerializer {
2021-06-11 14:02:28 +02:00
protochunk1.setCarvingMask(worldgenstage_features, BitSet.valueOf(nbttagcompound5.getByteArray(s1)));
}
- return protochunk1;
+ return new InProgressChunkHolder(protochunk1, tasksToExecuteOnMain); // Paper - Async chunk loading
+ }
+ }
+
2021-06-11 14:02:28 +02:00
+ // Paper start - async chunk save for unload
+ public static final class AsyncSaveData {
+ public final DataLayer[] blockLight;
2021-06-11 14:02:28 +02:00
+ public final DataLayer[] skyLight;
+
+ public final ListTag blockTickList; // non-null if we had to go to the server's tick list
+ public final ListTag fluidTickList; // non-null if we had to go to the server's tick list
+ public final ListTag blockEntities;
2021-06-11 14:02:28 +02:00
+
+ public final long worldTime;
+
+ public AsyncSaveData(DataLayer[] blockLight, DataLayer[] skyLight,
+ ListTag blockTickList, ListTag fluidTickList, ListTag blockEntities, long worldTime) {
2021-06-11 14:02:28 +02:00
+ this.blockLight = blockLight;
+ this.skyLight = skyLight;
+ this.blockTickList = blockTickList;
+ this.fluidTickList = fluidTickList;
+ this.blockEntities = blockEntities;
2021-06-11 14:02:28 +02:00
+ this.worldTime = worldTime;
+ }
+ }
+
+ // must be called sync
+ public static AsyncSaveData getAsyncSaveData(ServerLevel world, ChunkAccess chunk) {
+ org.spigotmc.AsyncCatcher.catchOp("preparation of chunk data for async save");
+ ChunkPos chunkPos = chunk.getPos();
+
+ ThreadedLevelLightEngine lightenginethreaded = world.getChunkSource().getLightEngine();
+
+ DataLayer[] blockLight = new DataLayer[lightenginethreaded.getMaxLightSection() - lightenginethreaded.getMinLightSection()];
+ DataLayer[] skyLight = new DataLayer[lightenginethreaded.getMaxLightSection() - lightenginethreaded.getMinLightSection()];
2021-06-11 14:02:28 +02:00
+
+ for (int i = lightenginethreaded.getMinLightSection(); i < lightenginethreaded.getMaxLightSection(); ++i) {
2021-06-11 14:02:28 +02:00
+ DataLayer blockArray = lightenginethreaded.getLayerListener(LightLayer.BLOCK).getDataLayerData(SectionPos.of(chunkPos, i));
+ DataLayer skyArray = lightenginethreaded.getLayerListener(LightLayer.SKY).getDataLayerData(SectionPos.of(chunkPos, i));
+
+ // copy data for safety
+ if (blockArray != null) {
+ blockArray = blockArray.copy();
+ }
+ if (skyArray != null) {
+ skyArray = skyArray.copy();
+ }
+
+ blockLight[i - lightenginethreaded.getMinLightSection()] = blockArray;
+ skyLight[i - lightenginethreaded.getMinLightSection()] = skyArray;
2021-06-11 14:02:28 +02:00
+ }
+
+ TickList<Block> blockTickList = chunk.getBlockTicks();
+
+ ListTag blockTickListSerialized;
+ if (blockTickList instanceof ProtoTickList || blockTickList instanceof ChunkTickList) {
+ blockTickListSerialized = null;
+ } else {
+ blockTickListSerialized = world.getBlockTicks().save(chunkPos);
+ }
2021-06-11 14:02:28 +02:00
+
+ TickList<Fluid> fluidTickList = chunk.getLiquidTicks();
+
+ ListTag fluidTickListSerialized;
+ if (fluidTickList instanceof ProtoTickList || fluidTickList instanceof ChunkTickList) {
+ fluidTickListSerialized = null;
+ } else {
+ fluidTickListSerialized = world.getLiquidTicks().save(chunkPos);
+ }
+
+ ListTag blockEntitiesSerialized = new ListTag();
+ for (final BlockPos blockPos : chunk.getBlockEntitiesPos()) {
+ final CompoundTag blockEntityNbt = chunk.getBlockEntityNbtForSaving(blockPos);
+ if (blockEntityNbt != null) {
+ blockEntitiesSerialized.add(blockEntityNbt);
+ }
}
+
+ return new AsyncSaveData(blockLight, skyLight, blockTickListSerialized, fluidTickListSerialized, blockEntitiesSerialized, world.getGameTime());
}
2021-06-11 14:02:28 +02:00
public static CompoundTag write(ServerLevel world, ChunkAccess chunk) {
+ return saveChunk(world, chunk, null);
+ }
2021-06-13 04:16:10 +02:00
+ public static CompoundTag saveChunk(ServerLevel world, ChunkAccess chunk, AsyncSaveData asyncsavedata) {
2021-06-11 14:02:28 +02:00
+ // Paper end
2021-06-13 04:16:10 +02:00
ChunkPos chunkcoordintpair = chunk.getPos();
2021-06-11 14:02:28 +02:00
CompoundTag nbttagcompound = new CompoundTag();
CompoundTag nbttagcompound1 = new CompoundTag();
@@ -285,7 +403,7 @@ public class ChunkSerializer {
2021-06-11 14:02:28 +02:00
nbttagcompound.put("Level", nbttagcompound1);
nbttagcompound1.putInt("xPos", chunkcoordintpair.x);
nbttagcompound1.putInt("zPos", chunkcoordintpair.z);
- nbttagcompound1.putLong("LastUpdate", world.getGameTime());
2021-06-13 04:16:10 +02:00
+ nbttagcompound1.putLong("LastUpdate", asyncsavedata != null ? asyncsavedata.worldTime : world.getGameTime()); // Paper - async chunk unloading
nbttagcompound1.putLong("InhabitedTime", chunk.getInhabitedTime());
nbttagcompound1.putString("Status", chunk.getStatus().getName());
UpgradeData chunkconverter = chunk.getUpgradeData();
@@ -304,9 +422,17 @@ public class ChunkSerializer {
2021-06-11 14:02:28 +02:00
LevelChunkSection chunksection = (LevelChunkSection) Arrays.stream(achunksection).filter((chunksection1) -> {
2021-06-13 04:16:10 +02:00
return chunksection1 != null && SectionPos.blockToSectionCoord(chunksection1.bottomBlockY()) == finalI; // CraftBukkit - decompile errors
2021-06-11 14:02:28 +02:00
}).findFirst().orElse(LevelChunk.EMPTY_SECTION);
- DataLayer nibblearray = lightenginethreaded.getLayerListener(LightLayer.BLOCK).getDataLayerData(SectionPos.of(chunkcoordintpair, i));
- DataLayer nibblearray1 = lightenginethreaded.getLayerListener(LightLayer.SKY).getDataLayerData(SectionPos.of(chunkcoordintpair, i));
-
+ // Paper start - async chunk save for unload
+ DataLayer nibblearray; // block light
+ DataLayer nibblearray1; // sky light
+ if (asyncsavedata == null) {
+ nibblearray = lightenginethreaded.getLayerListener(LightLayer.BLOCK).getDataLayerData(SectionPos.of(chunkcoordintpair, i)); /// Paper - diff on method change (see getAsyncSaveData)
+ nibblearray1 = lightenginethreaded.getLayerListener(LightLayer.SKY).getDataLayerData(SectionPos.of(chunkcoordintpair, i)); // Paper - diff on method change (see getAsyncSaveData)
+ } else {
+ nibblearray = asyncsavedata.blockLight[i - lightenginethreaded.getMinLightSection()];
+ nibblearray1 = asyncsavedata.skyLight[i - lightenginethreaded.getMinLightSection()];
2021-06-11 14:02:28 +02:00
+ }
+ // Paper end
if (chunksection != LevelChunk.EMPTY_SECTION || nibblearray != null || nibblearray1 != null) {
2021-06-13 04:16:10 +02:00
CompoundTag nbttagcompound2 = new CompoundTag();
2021-06-11 14:02:28 +02:00
@@ -338,8 +464,17 @@ public class ChunkSerializer {
nbttagcompound1.putIntArray("Biomes", biomestorage.writeBiomes());
}
- ListTag nbttaglist1 = new ListTag();
- Iterator iterator = chunk.getBlockEntitiesPos().iterator();
+ // Paper start
+ ListTag nbttaglist1;
+ Iterator<BlockPos> iterator;
+ if (asyncsavedata != null) {
+ nbttaglist1 = asyncsavedata.blockEntities;
+ iterator = java.util.Collections.emptyIterator();
+ } else {
+ nbttaglist1 = new ListTag();
+ iterator = chunk.getBlockEntitiesPos().iterator();
+ }
+ // Paper end
CompoundTag nbttagcompound3;
@@ -382,6 +517,10 @@ public class ChunkSerializer {
2021-06-11 14:02:28 +02:00
nbttagcompound1.put("ToBeTicked", ((ProtoTickList) ticklist).save());
} else if (ticklist instanceof ChunkTickList) {
nbttagcompound1.put("TileTicks", ((ChunkTickList) ticklist).save());
+ // Paper start - async chunk save for unload
+ } else if (asyncsavedata != null) {
+ nbttagcompound1.put("TileTicks", asyncsavedata.blockTickList);
+ // Paper end
} else {
2021-06-13 04:16:10 +02:00
nbttagcompound1.put("TileTicks", world.getBlockTicks().save(chunkcoordintpair));
2021-06-11 14:02:28 +02:00
}
@@ -392,6 +531,10 @@ public class ChunkSerializer {
2021-06-11 14:02:28 +02:00
nbttagcompound1.put("LiquidsToBeTicked", ((ProtoTickList) ticklist1).save());
} else if (ticklist1 instanceof ChunkTickList) {
nbttagcompound1.put("LiquidTicks", ((ChunkTickList) ticklist1).save());
+ // Paper start - async chunk save for unload
+ } else if (asyncsavedata != null) {
+ nbttagcompound1.put("LiquidTicks", asyncsavedata.fluidTickList);
+ // Paper end
} else {
2021-06-13 04:16:10 +02:00
nbttagcompound1.put("LiquidTicks", world.getLiquidTicks().save(chunkcoordintpair));
2021-06-11 14:02:28 +02:00
}
diff --git a/src/main/java/net/minecraft/world/level/chunk/storage/ChunkStorage.java b/src/main/java/net/minecraft/world/level/chunk/storage/ChunkStorage.java
2021-07-07 08:52:40 +02:00
index 8121470cdd94bde4d84f70882f3bdb3dc067cf15..6f13c7adce7d4b3d170045ea5ef2a841d34ae7b0 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/net/minecraft/world/level/chunk/storage/ChunkStorage.java
+++ b/src/main/java/net/minecraft/world/level/chunk/storage/ChunkStorage.java
2021-06-13 04:16:10 +02:00
@@ -21,27 +21,38 @@ import net.minecraft.world.level.storage.DimensionDataStorage;
2021-06-11 14:02:28 +02:00
public class ChunkStorage implements AutoCloseable {
2021-06-13 04:16:10 +02:00
- private final IOWorker worker;
+ // Paper - nuke IO worker
2021-06-11 14:02:28 +02:00
protected final DataFixer fixerUpper;
@Nullable
- private LegacyStructureDataHandler legacyStructureHandler;
2021-06-13 04:16:10 +02:00
+ // Paper start - async chunk loading
+ private volatile LegacyStructureDataHandler legacyStructureHandler;
2021-06-11 14:02:28 +02:00
+ private final Object persistentDataLock = new Object(); // Paper
+ public final RegionFileStorage regionFileCache;
2021-06-13 04:16:10 +02:00
+ // Paper end - async chunk loading
2021-06-11 14:02:28 +02:00
2021-06-13 04:16:10 +02:00
public ChunkStorage(File directory, DataFixer dataFixer, boolean dsync) {
this.fixerUpper = dataFixer;
- this.worker = new IOWorker(directory, dsync, "chunk");
+ // Paper start - async chunk io
+ // remove IO worker
+ this.regionFileCache = new RegionFileStorage(directory, dsync); // Paper - nuke IOWorker
+ // Paper end - async chunk io
2021-06-11 14:02:28 +02:00
}
// CraftBukkit start
private boolean check(ServerChunkCache cps, int x, int z) throws IOException {
ChunkPos pos = new ChunkPos(x, z);
if (cps != null) {
- com.google.common.base.Preconditions.checkState(org.bukkit.Bukkit.isPrimaryThread(), "primary thread");
- if (cps.hasChunk(x, z)) {
+ //com.google.common.base.Preconditions.checkState(org.bukkit.Bukkit.isPrimaryThread(), "primary thread"); // Paper - this function is now MT-Safe
+ if (cps.getChunkAtIfCachedImmediately(x, z) != null) { // Paper - isLoaded is a ticket level check, not a chunk loaded check!
return true;
}
}
2021-06-13 04:16:10 +02:00
- CompoundTag nbt = this.read(pos);
+ // Paper start - prioritize
+ CompoundTag nbt = cps == null ? read(pos) :
+ com.destroystokyo.paper.io.PaperFileIOThread.Holder.INSTANCE.loadChunkData((ServerLevel)cps.getLevel(), x, z,
+ com.destroystokyo.paper.io.PrioritizedTaskQueue.HIGHER_PRIORITY, false, true).chunkData;
+ // Paper end
if (nbt != null) {
CompoundTag level = nbt.getCompound("Level");
if (level.getBoolean("TerrainPopulated")) {
@@ -77,11 +88,13 @@ public class ChunkStorage implements AutoCloseable {
2021-06-11 14:02:28 +02:00
if (i < 1493) {
nbttagcompound = NbtUtils.update(this.fixerUpper, DataFixTypes.CHUNK, nbttagcompound, i, 1493);
if (nbttagcompound.getCompound("Level").getBoolean("hasLegacyStructureData")) {
+ synchronized (this.persistentDataLock) { // Paper - Async chunk loading
if (this.legacyStructureHandler == null) {
this.legacyStructureHandler = LegacyStructureDataHandler.getLegacyStructureHandler(resourcekey, (DimensionDataStorage) supplier.get());
}
nbttagcompound = this.legacyStructureHandler.updateFromLegacy(nbttagcompound);
+ } // Paper - Async chunk loading
}
}
2021-06-13 04:16:10 +02:00
@@ -99,22 +112,26 @@ public class ChunkStorage implements AutoCloseable {
2021-06-11 14:02:28 +02:00
@Nullable
2021-06-13 04:16:10 +02:00
public CompoundTag read(ChunkPos chunkPos) throws IOException {
- return this.worker.load(chunkPos);
+ return this.regionFileCache.read(chunkPos); // Paper - async chunk io
2021-06-11 14:02:28 +02:00
}
2021-06-13 04:16:10 +02:00
- public void write(ChunkPos chunkPos, CompoundTag nbt) {
- this.worker.store(chunkPos, nbt);
+ // Paper start - async chunk io
+ public void write(ChunkPos chunkPos, CompoundTag nbt) throws IOException {
+ this.regionFileCache.write(chunkPos, nbt);
+ // Paper end - Async chunk loading
2021-06-11 14:02:28 +02:00
if (this.legacyStructureHandler != null) {
+ synchronized (this.persistentDataLock) { // Paper - Async chunk loading
2021-06-13 04:16:10 +02:00
this.legacyStructureHandler.removeIndex(chunkPos.toLong());
+ } // Paper - Async chunk loading
2021-06-11 14:02:28 +02:00
}
2021-06-13 04:16:10 +02:00
}
public void flushWorker() {
2021-07-07 08:52:40 +02:00
- this.worker.synchronize(true).join();
2021-06-13 04:16:10 +02:00
+ com.destroystokyo.paper.io.PaperFileIOThread.Holder.INSTANCE.flush(); // Paper - nuke IO worker
2021-06-11 14:02:28 +02:00
}
public void close() throws IOException {
- this.worker.close();
2021-06-13 04:16:10 +02:00
+ this.regionFileCache.close(); // Paper - nuke IO worker
2021-06-11 14:02:28 +02:00
}
}
diff --git a/src/main/java/net/minecraft/world/level/chunk/storage/RegionFile.java b/src/main/java/net/minecraft/world/level/chunk/storage/RegionFile.java
2021-06-17 23:39:36 +02:00
index 84327c8fe1dc62c7b99950261a344042b4456616..298b5abbc792dd33be38acbd1c572c9778c4d2d2 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/net/minecraft/world/level/chunk/storage/RegionFile.java
+++ b/src/main/java/net/minecraft/world/level/chunk/storage/RegionFile.java
2021-06-13 04:16:10 +02:00
@@ -47,6 +47,7 @@ public class RegionFile implements AutoCloseable {
private final IntBuffer timestamps;
@VisibleForTesting
2021-06-11 14:02:28 +02:00
protected final RegionBitmap usedSectors;
+ public final java.util.concurrent.locks.ReentrantLock fileLock = new java.util.concurrent.locks.ReentrantLock(true); // Paper
2021-06-13 04:16:10 +02:00
public RegionFile(File file, File directory, boolean dsync) throws IOException {
this(file.toPath(), directory.toPath(), RegionFileVersion.VERSION_DEFLATE, dsync);
2021-06-17 23:39:36 +02:00
@@ -231,7 +232,7 @@ public class RegionFile implements AutoCloseable {
2021-06-11 14:02:28 +02:00
return (byteCount + 4096 - 1) / 4096;
}
- public boolean doesChunkExist(ChunkPos pos) {
+ public synchronized boolean doesChunkExist(ChunkPos pos) { // Paper - synchronized
int i = this.getOffset(pos);
if (i == 0) {
2021-06-17 23:39:36 +02:00
@@ -398,6 +399,11 @@ public class RegionFile implements AutoCloseable {
2021-06-11 14:02:28 +02:00
}
public void close() throws IOException {
+ // Paper start - Prevent regionfiles from being closed during use
+ this.fileLock.lock();
+ synchronized (this) {
+ try {
+ // Paper end
try {
this.padToFullSector();
2021-06-13 04:16:10 +02:00
} finally {
2021-06-17 23:39:36 +02:00
@@ -407,6 +413,10 @@ public class RegionFile implements AutoCloseable {
2021-06-11 14:02:28 +02:00
this.file.close();
}
}
+ } finally { // Paper start - Prevent regionfiles from being closed during use
+ this.fileLock.unlock();
+ }
+ } // Paper end
}
diff --git a/src/main/java/net/minecraft/world/level/chunk/storage/RegionFileStorage.java b/src/main/java/net/minecraft/world/level/chunk/storage/RegionFileStorage.java
2021-06-16 00:24:12 +02:00
index f973792c8ee4523eb4efdf31d0a97cb3358e1b3b..ebb1a050beab9530942c4498335f084c89faef06 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/net/minecraft/world/level/chunk/storage/RegionFileStorage.java
+++ b/src/main/java/net/minecraft/world/level/chunk/storage/RegionFileStorage.java
2021-06-16 00:24:12 +02:00
@@ -28,11 +28,32 @@ public class RegionFileStorage implements AutoCloseable {
2021-06-13 04:16:10 +02:00
this.sync = dsync;
2021-06-11 14:02:28 +02:00
}
2021-06-13 04:16:10 +02:00
- private RegionFile getFile(ChunkPos chunkcoordintpair, boolean existingOnly) throws IOException { // CraftBukkit
+ // Paper start
+ public synchronized RegionFile getRegionFileIfLoaded(ChunkPos chunkcoordintpair) {
+ return this.regionCache.getAndMoveToFirst(ChunkPos.asLong(chunkcoordintpair.getRegionX(), chunkcoordintpair.getRegionZ()));
+ }
+
+ public synchronized boolean chunkExists(ChunkPos pos) throws IOException {
+ RegionFile regionfile = getFile(pos, true);
+
+ return regionfile != null ? regionfile.hasChunk(pos) : false;
+ }
+
2021-06-16 00:24:12 +02:00
+ public synchronized RegionFile getFile(ChunkPos chunkcoordintpair, boolean existingOnly) throws IOException { // CraftBukkit
2021-06-11 14:02:28 +02:00
+ return this.getFile(chunkcoordintpair, existingOnly, false);
+ }
+ public synchronized RegionFile getFile(ChunkPos chunkcoordintpair, boolean existingOnly, boolean lock) throws IOException {
+ // Paper end
long i = ChunkPos.asLong(chunkcoordintpair.getRegionX(), chunkcoordintpair.getRegionZ());
RegionFile regionfile = (RegionFile) this.regionCache.getAndMoveToFirst(i);
if (regionfile != null) {
+ // Paper start
+ if (lock) {
+ // must be in this synchronized block
+ regionfile.fileLock.lock();
+ }
+ // Paper end
return regionfile;
} else {
if (this.regionCache.size() >= com.destroystokyo.paper.PaperConfig.regionFileCacheSize) { // Paper - configurable
2021-06-16 00:24:12 +02:00
@@ -50,6 +71,12 @@ public class RegionFileStorage implements AutoCloseable {
2021-06-13 04:16:10 +02:00
RegionFile regionfile1 = new RegionFile(file1, this.folder, this.sync);
2021-06-11 14:02:28 +02:00
this.regionCache.putAndMoveToFirst(i, regionfile1);
+ // Paper start
+ if (lock) {
+ // must be in this synchronized block
+ regionfile1.fileLock.lock();
+ }
+ // Paper end
return regionfile1;
}
}
2021-06-16 00:24:12 +02:00
@@ -57,11 +84,12 @@ public class RegionFileStorage implements AutoCloseable {
2021-06-11 14:02:28 +02:00
@Nullable
public CompoundTag read(ChunkPos pos) throws IOException {
// CraftBukkit start - SPIGOT-5680: There's no good reason to preemptively create files on read, save that for writing
- RegionFile regionfile = this.getFile(pos, true);
+ RegionFile regionfile = this.getFile(pos, true, true); // Paper
if (regionfile == null) {
return null;
}
// CraftBukkit end
+ try { // Paper
DataInputStream datainputstream = regionfile.getChunkDataInputStream(pos);
2021-06-13 04:16:10 +02:00
CompoundTag nbttagcompound;
2021-06-16 00:24:12 +02:00
@@ -98,10 +126,14 @@ public class RegionFileStorage implements AutoCloseable {
2021-06-11 14:02:28 +02:00
}
return nbttagcompound;
+ } finally { // Paper start
+ regionfile.fileLock.unlock();
+ } // Paper end
}
2021-06-13 04:16:10 +02:00
protected void write(ChunkPos pos, @Nullable CompoundTag nbt) throws IOException {
2021-06-11 14:02:28 +02:00
- RegionFile regionfile = this.getFile(pos, false); // CraftBukkit
+ RegionFile regionfile = this.getFile(pos, false, true); // CraftBukkit // Paper
+ try { // Paper
int attempts = 0; Exception laste = null; while (attempts++ < 5) { try { // Paper
2021-06-13 04:16:10 +02:00
if (nbt == null) {
2021-06-16 00:24:12 +02:00
@@ -140,9 +172,12 @@ public class RegionFileStorage implements AutoCloseable {
2021-06-11 14:02:28 +02:00
MinecraftServer.LOGGER.error("Failed to save chunk", laste);
}
// Paper end
+ } finally { // Paper start
+ regionfile.fileLock.unlock();
+ } // Paper end
}
- public void close() throws IOException {
+ public synchronized void close() throws IOException { // Paper -> synchronized
ExceptionCollector<IOException> exceptionsuppressor = new ExceptionCollector<>();
ObjectIterator objectiterator = this.regionCache.values().iterator();
2021-06-16 00:24:12 +02:00
@@ -159,7 +194,7 @@ public class RegionFileStorage implements AutoCloseable {
2021-06-13 04:16:10 +02:00
exceptionsuppressor.throwIfPresent();
2021-06-11 14:02:28 +02:00
}
2021-06-13 04:16:10 +02:00
- public void flush() throws IOException {
+ public synchronized void flush() throws IOException { // Paper - synchronize
ObjectIterator objectiterator = this.regionCache.values().iterator();
while (objectiterator.hasNext()) {
2021-06-11 14:02:28 +02:00
diff --git a/src/main/java/net/minecraft/world/level/chunk/storage/SectionStorage.java b/src/main/java/net/minecraft/world/level/chunk/storage/SectionStorage.java
2021-06-16 00:24:12 +02:00
index e5e138fb23d03eb63e547e74d3e14ec9d96d8107..90f7b06bd2c558be35c4577044fa033e1fb5cc22 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/net/minecraft/world/level/chunk/storage/SectionStorage.java
+++ b/src/main/java/net/minecraft/world/level/chunk/storage/SectionStorage.java
2021-06-16 00:24:12 +02:00
@@ -30,10 +30,10 @@ import net.minecraft.world.level.LevelHeightAccessor;
2021-06-11 14:02:28 +02:00
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
-public class SectionStorage<R> implements AutoCloseable {
+public class SectionStorage<R> extends RegionFileStorage implements AutoCloseable { // Paper - nuke IOWorker
private static final Logger LOGGER = LogManager.getLogger();
2021-06-13 04:16:10 +02:00
private static final String SECTIONS_TAG = "Sections";
2021-06-11 14:02:28 +02:00
- private final IOWorker worker;
2021-06-13 04:16:10 +02:00
+ // Paper - remove mojang I/O thread
private final Long2ObjectMap<Optional<R>> storage = new Long2ObjectOpenHashMap<>();
2021-06-16 00:24:12 +02:00
public final LongLinkedOpenHashSet dirty = new LongLinkedOpenHashSet();
2021-06-11 14:02:28 +02:00
private final Function<Runnable, Codec<R>> codec;
2021-06-13 04:16:10 +02:00
@@ -43,12 +43,13 @@ public class SectionStorage<R> implements AutoCloseable {
protected final LevelHeightAccessor levelHeightAccessor;
2021-06-11 14:02:28 +02:00
2021-06-13 04:16:10 +02:00
public SectionStorage(File directory, Function<Runnable, Codec<R>> codecFactory, Function<Runnable, R> factory, DataFixer dataFixer, DataFixTypes dataFixTypes, boolean dsync, LevelHeightAccessor world) {
+ super(directory, dsync); // Paper - nuke IOWorker
2021-06-11 14:02:28 +02:00
this.codec = codecFactory;
this.factory = factory;
2021-06-13 04:16:10 +02:00
this.fixerUpper = dataFixer;
this.type = dataFixTypes;
this.levelHeightAccessor = world;
- this.worker = new IOWorker(directory, dsync, directory.getName());
+ // Paper - remove mojang I/O thread
2021-06-11 14:02:28 +02:00
}
protected void tick(BooleanSupplier shouldKeepTicking) {
2021-06-13 04:16:10 +02:00
@@ -106,13 +107,18 @@ public class SectionStorage<R> implements AutoCloseable {
2021-06-11 14:02:28 +02:00
}
2021-06-13 04:16:10 +02:00
private void readColumn(ChunkPos chunkPos) {
- this.readColumn(chunkPos, NbtOps.INSTANCE, this.tryRead(chunkPos));
+ // Paper start - expose function to load in data
+ this.loadInData(chunkPos, this.tryRead(chunkPos));
2021-06-11 14:02:28 +02:00
+ }
+ public void loadInData(ChunkPos chunkPos, CompoundTag compound) {
+ this.readColumn(chunkPos, NbtOps.INSTANCE, compound);
2021-06-13 04:16:10 +02:00
+ // Paper end - expose function to load in data
2021-06-11 14:02:28 +02:00
}
@Nullable
private CompoundTag tryRead(ChunkPos pos) {
try {
- return this.worker.load(pos);
+ return this.read(pos); // Paper - nuke IOWorker
2021-06-13 04:16:10 +02:00
} catch (IOException var3) {
LOGGER.error("Error reading chunk {} data from disk", pos, var3);
2021-06-11 14:02:28 +02:00
return null;
2021-06-13 04:16:10 +02:00
@@ -156,13 +162,26 @@ public class SectionStorage<R> implements AutoCloseable {
Dynamic<Tag> dynamic = this.writeColumn(chunkPos, NbtOps.INSTANCE);
Tag tag = dynamic.getValue();
if (tag instanceof CompoundTag) {
- this.worker.store(chunkPos, (CompoundTag)tag);
+ try { this.write(chunkPos, (CompoundTag)tag); } catch (IOException ioexception) { SectionStorage.LOGGER.error("Error writing data to disk", ioexception); } // Paper - nuke IOWorker
2021-06-11 14:02:28 +02:00
} else {
2021-06-13 04:16:10 +02:00
LOGGER.error("Expected compound tag, got {}", (Object)tag);
2021-06-11 14:02:28 +02:00
}
}
+ // Paper start - internal get data function, copied from above
+ private CompoundTag getDataInternal(ChunkPos chunkcoordintpair) {
+ Dynamic<Tag> dynamic = this.writeColumn(chunkcoordintpair, NbtOps.INSTANCE);
+ Tag nbtbase = (Tag) dynamic.getValue();
+
+ if (nbtbase instanceof CompoundTag) {
+ return (CompoundTag)nbtbase;
+ } else {
+ SectionStorage.LOGGER.error("Expected compound tag, got {}", nbtbase);
+ }
+ return null;
+ }
+ // Paper end
2021-06-13 04:16:10 +02:00
private <T> Dynamic<T> writeColumn(ChunkPos chunkPos, DynamicOps<T> dynamicOps) {
2021-06-11 14:02:28 +02:00
Map<T, T> map = Maps.newHashMap();
2021-06-13 04:16:10 +02:00
@@ -219,6 +238,23 @@ public class SectionStorage<R> implements AutoCloseable {
2021-06-11 14:02:28 +02:00
2021-06-13 04:16:10 +02:00
@Override
public void close() throws IOException {
2021-06-11 14:02:28 +02:00
- this.worker.close();
2021-06-13 04:16:10 +02:00
+ //this.worker.close(); // Paper - nuke I/O worker
+ }
2021-06-11 14:02:28 +02:00
+
+ // Paper start - get data function
+ public CompoundTag getData(ChunkPos chunkcoordintpair) {
+ // Note: Copied from above
2021-06-13 04:16:10 +02:00
+ // This is checking if the data needs to be written, then it builds it later in getDataInternal(ChunkCoordIntPair)
2021-06-11 14:02:28 +02:00
+ if (!this.dirty.isEmpty()) {
2021-06-13 04:16:10 +02:00
+ for (int i = this.levelHeightAccessor.getMinSection(); i < this.levelHeightAccessor.getMaxSection(); ++i) {
2021-06-11 14:02:28 +02:00
+ long j = SectionPos.of(chunkcoordintpair, i).asLong();
+
+ if (this.dirty.contains(j)) {
+ return this.getDataInternal(chunkcoordintpair);
+ }
+ }
+ }
+ return null;
}
+ // Paper end
}
diff --git a/src/main/java/org/bukkit/craftbukkit/CraftWorld.java b/src/main/java/org/bukkit/craftbukkit/CraftWorld.java
index 74825cd20ae25bbaa1f69ecb208a1c01c1213083..e43135fc10cb2d024295cbe89daf7127fec4898c 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/org/bukkit/craftbukkit/CraftWorld.java
+++ b/src/main/java/org/bukkit/craftbukkit/CraftWorld.java
@@ -1786,6 +1786,34 @@ public class CraftWorld extends CraftRegionAccessor implements World {
2021-06-11 14:02:28 +02:00
public DragonBattle getEnderDragonBattle() {
2021-06-13 04:16:10 +02:00
return (this.getHandle().dragonFight() == null) ? null : new CraftDragonBattle(this.getHandle().dragonFight());
2021-06-11 14:02:28 +02:00
}
+ // Paper start
+ @Override
2021-06-13 04:16:10 +02:00
+ public java.util.concurrent.CompletableFuture<Chunk> getChunkAtAsync(int x, int z, boolean gen, boolean urgent) {
2021-06-11 14:02:28 +02:00
+ if (Bukkit.isPrimaryThread()) {
+ net.minecraft.world.level.chunk.LevelChunk immediate = this.world.getChunkSource().getChunkAtIfLoadedImmediately(x, z);
+ if (immediate != null) {
2021-06-13 04:16:10 +02:00
+ return java.util.concurrent.CompletableFuture.completedFuture(immediate.getBukkitChunk());
2021-06-11 14:02:28 +02:00
+ }
+ } else {
2021-06-13 04:16:10 +02:00
+ java.util.concurrent.CompletableFuture<Chunk> future = new java.util.concurrent.CompletableFuture<Chunk>();
2021-06-11 14:02:28 +02:00
+ world.getServer().execute(() -> {
+ getChunkAtAsync(x, z, gen, urgent).whenComplete((chunk, err) -> {
+ if (err != null) {
+ future.completeExceptionally(err);
+ } else {
+ future.complete(chunk);
+ }
+ });
+ });
+ return future;
+ }
+
+ return this.world.getChunkSource().getChunkAtAsynchronously(x, z, gen, urgent).thenComposeAsync((either) -> {
+ net.minecraft.world.level.chunk.LevelChunk chunk = (net.minecraft.world.level.chunk.LevelChunk) either.left().orElse(null);
2021-06-13 04:16:10 +02:00
+ return java.util.concurrent.CompletableFuture.completedFuture(chunk == null ? null : chunk.getBukkitChunk());
2021-06-11 14:02:28 +02:00
+ }, net.minecraft.server.MinecraftServer.getServer());
+ }
+ // Paper end
// Spigot start
@Override
diff --git a/src/main/java/org/bukkit/craftbukkit/entity/CraftEntity.java b/src/main/java/org/bukkit/craftbukkit/entity/CraftEntity.java
index 95bc2136eaa118f28c7ec2d8b557ed0773f5f9ad..2ef8ffdf8c957aa91caa3b628cfeb0276989bc75 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/org/bukkit/craftbukkit/entity/CraftEntity.java
+++ b/src/main/java/org/bukkit/craftbukkit/entity/CraftEntity.java
@@ -15,6 +15,7 @@ import net.minecraft.network.chat.Component;
import net.minecraft.server.level.ChunkMap;
import net.minecraft.server.level.ServerLevel;
2021-06-11 14:02:28 +02:00
import net.minecraft.server.level.ServerPlayer;
+import net.minecraft.server.level.TicketType;
import net.minecraft.world.damagesource.DamageSource;
import net.minecraft.world.entity.AreaEffectCloud;
import net.minecraft.world.entity.Entity;
@@ -519,6 +520,37 @@ public abstract class CraftEntity implements org.bukkit.entity.Entity {
2021-06-13 04:16:10 +02:00
this.entity.setYHeadRot(yaw);
2021-06-11 14:02:28 +02:00
}
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
+ // Paper start
+ @Override
+ public java.util.concurrent.CompletableFuture<Boolean> teleportAsync(Location location, TeleportCause cause) {
+ Preconditions.checkArgument(location != null, "location");
+ location.checkFinite();
+ Location locationClone = location.clone(); // clone so we don't need to worry about mutations after this call.
+
+ net.minecraft.server.level.ServerLevel world = ((CraftWorld)locationClone.getWorld()).getHandle();
+ java.util.concurrent.CompletableFuture<Boolean> ret = new java.util.concurrent.CompletableFuture<>();
+
+ world.loadChunksForMoveAsync(getHandle().getBoundingBoxAt(locationClone.getX(), locationClone.getY(), locationClone.getZ()), location.getX(), location.getZ(), (list) -> {
+ net.minecraft.server.level.ServerChunkCache chunkProviderServer = world.getChunkSource();
+ for (net.minecraft.world.level.chunk.ChunkAccess chunk : list) {
+ chunkProviderServer.addTicketAtLevel(net.minecraft.server.level.TicketType.POST_TELEPORT, chunk.getPos(), 33, CraftEntity.this.getEntityId());
2021-06-11 14:02:28 +02:00
+ }
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
+ net.minecraft.server.MinecraftServer.getServer().scheduleOnMain(() -> {
+ try {
+ ret.complete(CraftEntity.this.teleport(locationClone, cause) ? Boolean.TRUE : Boolean.FALSE);
+ } catch (Throwable throwable) {
+ if (throwable instanceof ThreadDeath) {
+ throw (ThreadDeath)throwable;
+ }
+ ret.completeExceptionally(throwable);
+ }
+ });
2021-06-11 14:02:28 +02:00
+ });
Merge tuinity (#6413) This PR contains all of Tuinity's patches. Very notable ones are: - Highly optimised collisions - Optimised entity lookups by bounding box (Mojang made regressions in 1.17, this brings it back to 1.16) - Starlight https://github.com/PaperMC/Starlight - Rewritten dataconverter system https://github.com/PaperMC/DataConverter - Random block ticking optimisation (wrongly dropped from Paper 1.17) - Chunk ticking optimisations - Anything else I've forgotten in the 60 or so patches If you are a previous Tuinity user, your config will not migrate. You must do it yourself. The config options have simply been moved into paper.yml, so it will be an easy migration. However, please note that the chunk loading options in tuinity.yml are NOT compatible with the options in paper.yml. * Port tuinity, initial patchset * Update gradle to 7.2 jmp said it fixes rebuildpatches not working for me. it fucking better * Completely clean apply * Remove tuinity config, add per player api patch * Remove paper reobf mappings patch * Properly update gradlew * Force clean rebuild * Mark fixups Comments and ATs still need to be done * grep -r "Tuinity" * Fixup * Ensure gameprofile lastaccess is written only under the state lock * update URL for dataconverter * Only clean rebuild tuinity patches might fix merge conflicts * Use UTF-8 for gradlew * Clean rb patches again * Convert block ids used as item ids Neither the converters of pre 1.13 nor DFU handled these cases, as by the time they were written the game at the time didn't consider these ids valid - they would be air. Because of this, some worlds have logspam since only DataConverter (not DFU or legacy converters) will warn when an invalid id has been seen. While quite a few do need to now be considered as air, quite a lot do not. So it makes sense to add conversion for these items, instead of simply suppressing or ignoring the logs. I've now added id -> string conversion for all block ids that could be used as items that existed in the game before 1.7.10 (I have no interest in tracking down the exact version block ids stopped working) that were on https://minecraft-ids.grahamedgecombe.com/ Items that did not directly convert to new items will be instead converted to air: stems, wheat crops, piston head, tripwire wire block * Fix LightPopulated parsing in V1466 The DFU code was checking if the number existed, not if it didn't exist. I misread the original code. * Always parse protochunk light sources unless it is marked as non-lit Chunks not marked as lit will always go through the light engine, so they should always have their block sources parsed. * Update custom names to JSON for players Missed this fix from CB, as it was inside the DataFixers class. I decided to double check all of the CB changes again: DataFixers.java was the only area I missed, as I had inspected all datafixer diffs and implemented them all into DataConverter. I also checked Bootstrap.java again, and re-evaluated their changes. I had previously done this, but determined that they were all bad. The changes to make standing_sign block map to oak_sign block in V1450 is bad, because that's not the item id V1450 accepts. Only in 1.14 did oak_sign even exist, and as expected there is a converter to rename all existing sign items/blocks. The fix to register the portal block under id 1440 is useless, as the flattenning logic will default to the lowest registered id - which is the exact blockstate that CB registers into 1440. So it just doesn't do anything. The extra item ids in the id -> string converter are already added, but I found this from EMC originally. The change for the spawn egg id 23 -> Arrow is just wrong, that id DOES correspond to TippedArrow, NOT Arrow. As expected, the spawn egg already has a dedicated mapping for Arrow, which is id 10 - which was Arrow's entity id. I also ported a fix for the cooked_fished id update. This doesn't really matter since there is already a dataconverter to fix this, but the game didn't accept cooked_fished at the time. So I see no harm. * Review all converters and walkers - Refactor V99 to have helper methods for defining entity/tile entity types - Automatically namespace all ids that should be namespaced. While vanilla never saved non-namespaced data for things that are namespaced, plugins/users might have. - Synchronised the identity ensure map in HelperBlockFlatteningV1450 - Code style consistency - Add missing log warning in V102 for ITEM_NAME type conversion - Use getBoolean instead of getByte - Use ConverterAbstractEntityRename for V143 TippedArrow -> Arrow rename, as it will affect ENTITY_NAME type - Always set isVillager to false in V502 for Zombie - Register V808's converter under subversion 1 like DFU - Register a breakpoint for V1.17.1. In the future, all final versions of major releases will have a breakpoint so that the work required to determine if a converter needs a breakpoint is minimal - Validate that a dataconverter is only registered for a version that is registered - ConverterFlattenTileEntity is actually ConverterFlattenEntity It even registered the converters under TILE_ENTITY, instead of ENTITY. - Fix id comparison in V1492 STRUCTURE_FEATURE renamer - Use ConverterAbstractStatsRename for V1510 stats renamer At the time I had written that class, the abstract renamer didn't exist. - Ensure OwnerUUID is at least set to empty string in V1904 if the ocelot is converted to a cat (this is likely so that it retains a collar) - Use generic read/write for Records in V1946 Records is actually a list, not a map. So reading map was invalid. * Always set light to zero when propagating decrease This fixes an almost infinite loop where light values would be spam queued on a very small subset on blocks. This also likely fixes the memory issues people were seeing. * re-organize patches * Apply and fix conflicts * Revert some patches getChunkAt retains chunks so that plugins don't spam loads revert mc-4 fix will remain unless issues pop up * Shuffle iterated chunks if per player is not enabled Can help with some mob spawning stacking up at locations * Make per player default, migrate all configs * Adjust comments in fixups * Rework config for player chunk loader Old config is not compatible. Move all configs to be under `settings` in paper.yml The player chunk loader has been modified to less aggressively load chunks, but to send chunks at higher rates compared to tuinity. There are new config entries to tune this behavior. * Add back old constructor to CompressionEncoder/Decoder (fixes Tuinity #358) * Raise chunk loading default limits * Reduce worldgen thread workers for lower core count cpus * Raise limits for chunk loading config Also place it under `chunk-loading` * Disable max chunk send rate by default * Fix conflicts and rebuild patches * Drop default send rate again Appears to be still causing problems for no known reason * Raise chunk send limits to 100 per player While a low limit fixes ping issues for some people, most people do not suffer from this issue and thus should not suffer from an extremely slow load-in rate. * Rebase part 1 Autosquash the fixups * Move not implemented up * Fixup mc-dev fixes Missed this one * Rebase per player viewdistance api into the original api patch * Remove old light engine patch part 1 The prioritisation must be kept from it, so that part has been rebased into the priority patch. Part 2 will deal with rebasing all of the patches _after_ * Rebase remaining patches for old light patch removal * Remove other mid tick patch * Remove Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch Replaced by `Do not copy visible chunks` * Revert AT for Vec3i setX/Y/Z The class is immutable. set should not be exposed * Remove old IntegerUtil class * Replace old CraftChunk#getEntities patch * Remove import for SWMRNibbleArray in ChunkAccess * Finished merge checklist * Remove ensureTickThread impl in urgency patch Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com> Co-authored-by: Jason Penilla <11360596+jpenilla@users.noreply.github.com>
2021-08-31 13:02:11 +02:00
+
+ return ret;
2021-06-11 14:02:28 +02:00
+ }
+ // Paper end
+
@Override
public boolean teleport(Location location) {
2021-06-13 04:16:10 +02:00
return this.teleport(location, TeleportCause.PLUGIN);
2021-06-11 14:02:28 +02:00
diff --git a/src/main/java/org/spigotmc/WatchdogThread.java b/src/main/java/org/spigotmc/WatchdogThread.java
2021-06-13 04:16:10 +02:00
index a01a266a25e3267c94c20f8597b4b596efe20faa..1ffb208094f521883ef0e23baf5fb29380b14273 100644
2021-06-11 14:02:28 +02:00
--- a/src/main/java/org/spigotmc/WatchdogThread.java
+++ b/src/main/java/org/spigotmc/WatchdogThread.java
2021-06-13 04:16:10 +02:00
@@ -117,6 +117,7 @@ public class WatchdogThread extends Thread
2021-06-11 14:02:28 +02:00
// Paper end - Different message for short timeout
log.log( Level.SEVERE, "------------------------------" );
log.log( Level.SEVERE, "Server thread dump (Look for plugins here before reporting to Paper!):" ); // Paper
2021-06-13 04:16:10 +02:00
+ com.destroystokyo.paper.io.chunk.ChunkTaskManager.dumpAllChunkLoadInfo(); // Paper
WatchdogThread.dumpThread( ManagementFactory.getThreadMXBean().getThreadInfo( MinecraftServer.getServer().serverThread.getId(), Integer.MAX_VALUE ), log );
2021-06-11 14:02:28 +02:00
log.log( Level.SEVERE, "------------------------------" );
//