Paper/patches/server/0760-Only-write-chunk-data-to-disk-if-it-serializes-witho.patch

98 lines
4.4 KiB
Diff
Raw Normal View History

From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Spottedleaf <Spottedleaf@users.noreply.github.com>
Date: Sun, 19 Dec 2021 09:13:41 -0800
Subject: [PATCH] Only write chunk data to disk if it serializes without
throwing
This ensures at least a valid version of the chunk exists
on disk, even if outdated
diff --git a/src/main/java/net/minecraft/world/level/chunk/storage/RegionFile.java b/src/main/java/net/minecraft/world/level/chunk/storage/RegionFile.java
2022-12-07 17:46:46 +01:00
index 9bae47f99336c377beb72c4b50b7f01cb4db15da..dcfe090c269d4cbcc2eb1b6f85392848bb34656c 100644
--- a/src/main/java/net/minecraft/world/level/chunk/storage/RegionFile.java
+++ b/src/main/java/net/minecraft/world/level/chunk/storage/RegionFile.java
@@ -1004,6 +1004,9 @@ public class RegionFile implements AutoCloseable {
}
}
+
+ public static final int MAX_CHUNK_SIZE = 500 * 1024 * 1024; // Paper - don't write garbage data to disk if writing serialization fails
+
// Paper end
private class ChunkBuffer extends ByteArrayOutputStream {
@@ -1019,6 +1022,24 @@ public class RegionFile implements AutoCloseable {
this.pos = chunkcoordintpair;
}
+ // Paper start - don't write garbage data to disk if writing serialization fails
+ @Override
+ public void write(final int b) {
+ if (this.count > MAX_CHUNK_SIZE) {
+ throw new RegionFileStorage.RegionFileSizeException("Region file too large: " + this.count);
+ }
+ super.write(b);
+ }
+
+ @Override
+ public void write(final byte[] b, final int off, final int len) {
+ if (this.count + len > MAX_CHUNK_SIZE) {
+ throw new RegionFileStorage.RegionFileSizeException("Region file too large: " + (this.count + len));
+ }
+ super.write(b, off, len);
+ }
+ // Paper end
+
public void close() throws IOException {
ByteBuffer bytebuffer = ByteBuffer.wrap(this.buf, 0, this.count);
diff --git a/src/main/java/net/minecraft/world/level/chunk/storage/RegionFileStorage.java b/src/main/java/net/minecraft/world/level/chunk/storage/RegionFileStorage.java
2022-12-07 22:35:34 +01:00
index d5b501bcb33a785ef0301bf6bb8d396b9d4fbb6a..bd502ca721de0cab438d995efa00ad0554c0d2fe 100644
--- a/src/main/java/net/minecraft/world/level/chunk/storage/RegionFileStorage.java
+++ b/src/main/java/net/minecraft/world/level/chunk/storage/RegionFileStorage.java
Rewrite chunk system (#8177) Patch documentation to come Issues with the old system that are fixed now: - World generation does not scale with cpu cores effectively. - Relies on the main thread for scheduling and maintaining chunk state, dropping chunk load/generate rates at lower tps. - Unreliable prioritisation of chunk gen/load calls that block the main thread. - Shutdown logic is utterly unreliable, as it has to wait for all chunks to unload - is it guaranteed that the chunk system is in a state on shutdown that it can reliably do this? Watchdog shutdown also typically failed due to thread checks, which is now resolved. - Saving of data is not unified (i.e can save chunk data without saving entity data, poses problems for desync if shutdown is really abnormal. - Entities are not loaded with chunks. This caused quite a bit of headache for Chunk#getEntities API, but now the new chunk system loads entities with chunks so that they are ready whenever the chunk loads in. Effectively brings the behavior back to 1.16 era, but still storing entities in their own separate regionfiles. The above list is not complete. The patch documentation will complete it. New chunk system hard relies on starlight and dataconverter, and most importantly the new concurrent utilities in ConcurrentUtil. Some of the old async chunk i/o interface (i.e the old file io thread reroutes _some_ calls to the new file io thread) is kept for plugin compat reasons. It will be removed in the next major version of minecraft. The old legacy chunk system patches have been moved to the removed folder in case we need them again.
2022-09-26 10:02:51 +02:00
@@ -302,10 +302,17 @@ public class RegionFileStorage implements AutoCloseable {
NbtIo.write(nbt, (DataOutput) dataoutputstream);
regionfile.setStatus(pos.x, pos.z, ChunkSerializer.getStatus(nbt)); // Paper - cache status on disk
regionfile.setOversized(pos.x, pos.z, false); // Paper - We don't do this anymore, mojang stores differently, but clear old meta flag if it exists to get rid of our own meta file once last oversized is gone
+ dataoutputstream.close(); // Paper - only write if successful
+ // Paper start - don't write garbage data to disk if writing serialization fails
+ } catch (RegionFileSizeException e) {
+ attempts = 5; // Don't retry
+ regionfile.clear(pos);
+ throw e;
+ // Paper end - don't write garbage data to disk if writing serialization fails
} catch (Throwable throwable) {
if (dataoutputstream != null) {
try {
- dataoutputstream.close();
+ //dataoutputstream.close(); // Paper - don't write garbage data to disk if writing serialization fails
} catch (Throwable throwable1) {
throwable.addSuppressed(throwable1);
}
Rewrite chunk system (#8177) Patch documentation to come Issues with the old system that are fixed now: - World generation does not scale with cpu cores effectively. - Relies on the main thread for scheduling and maintaining chunk state, dropping chunk load/generate rates at lower tps. - Unreliable prioritisation of chunk gen/load calls that block the main thread. - Shutdown logic is utterly unreliable, as it has to wait for all chunks to unload - is it guaranteed that the chunk system is in a state on shutdown that it can reliably do this? Watchdog shutdown also typically failed due to thread checks, which is now resolved. - Saving of data is not unified (i.e can save chunk data without saving entity data, poses problems for desync if shutdown is really abnormal. - Entities are not loaded with chunks. This caused quite a bit of headache for Chunk#getEntities API, but now the new chunk system loads entities with chunks so that they are ready whenever the chunk loads in. Effectively brings the behavior back to 1.16 era, but still storing entities in their own separate regionfiles. The above list is not complete. The patch documentation will complete it. New chunk system hard relies on starlight and dataconverter, and most importantly the new concurrent utilities in ConcurrentUtil. Some of the old async chunk i/o interface (i.e the old file io thread reroutes _some_ calls to the new file io thread) is kept for plugin compat reasons. It will be removed in the next major version of minecraft. The old legacy chunk system patches have been moved to the removed folder in case we need them again.
2022-09-26 10:02:51 +02:00
@@ -313,10 +320,7 @@ public class RegionFileStorage implements AutoCloseable {
throw throwable;
}
-
- if (dataoutputstream != null) {
- dataoutputstream.close();
- }
+ // Paper - move into try block to only write if successfully serialized
}
// Paper start
return;
Rewrite chunk system (#8177) Patch documentation to come Issues with the old system that are fixed now: - World generation does not scale with cpu cores effectively. - Relies on the main thread for scheduling and maintaining chunk state, dropping chunk load/generate rates at lower tps. - Unreliable prioritisation of chunk gen/load calls that block the main thread. - Shutdown logic is utterly unreliable, as it has to wait for all chunks to unload - is it guaranteed that the chunk system is in a state on shutdown that it can reliably do this? Watchdog shutdown also typically failed due to thread checks, which is now resolved. - Saving of data is not unified (i.e can save chunk data without saving entity data, poses problems for desync if shutdown is really abnormal. - Entities are not loaded with chunks. This caused quite a bit of headache for Chunk#getEntities API, but now the new chunk system loads entities with chunks so that they are ready whenever the chunk loads in. Effectively brings the behavior back to 1.16 era, but still storing entities in their own separate regionfiles. The above list is not complete. The patch documentation will complete it. New chunk system hard relies on starlight and dataconverter, and most importantly the new concurrent utilities in ConcurrentUtil. Some of the old async chunk i/o interface (i.e the old file io thread reroutes _some_ calls to the new file io thread) is kept for plugin compat reasons. It will be removed in the next major version of minecraft. The old legacy chunk system patches have been moved to the removed folder in case we need them again.
2022-09-26 10:02:51 +02:00
@@ -362,4 +366,13 @@ public class RegionFileStorage implements AutoCloseable {
}
}
+
+ // Paper start
+ public static final class RegionFileSizeException extends RuntimeException {
+
+ public RegionFileSizeException(String message) {
+ super(message);
+ }
+ }
+ // Paper end
}