Upstream
/
Yatopia
mirror of https://github.com/YatopiaMC/Yatopia.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

New network system (#340) 

* New network system

* Revert code readability

* Change comment style

* Bring the patch to standarts

* Fix astei's nickname

Sorry

* Remove explicit logging

* Adapt FastThreadLocalThread

* Add new network optimisation and cleanup useless import

* Port krypton

* Minimize diff

* Some bug fixe

* Fix typo

* Fix mail adress

Co-authored-by: Hugo Planque <hookwood@pop-os.localdomain>
Co-authored-by: Ivan Pekov <ivan@mrivanplays.com>
Co-authored-by: Zoe <duplexsys@protonmail.com>
This commit is contained in:
Hugo Planque 2021-01-23 19:57:49 +01:00 committed by GitHub
parent ebf26c2f82
commit b27d77ca49
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 863 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
Licensing/LICENSE.md
View File

@ -1,6 +1,7 @@
The project without the build tools (Everything that is not a .patch file or in the buildSrc folder) are licensed under the MIT license found [here](MIT.md).<br>
All files in the buildSrc folder are licensed under MIT found [here](../buildSrc/license.txt)<br>
All files in the buildSrc folder are licensed under MIT found [here](../buildSrc/license.txt).<br>
All patches (.patch files) marked with "lithium" are licensed under LGPL3 found [here](https://github.com/jellysquid3/lithium-fabric/blob/1.16.x/dev/LICENSE.txt).<br>
All patches (.patch files) marked with "hydrogen" are licensed under LGPL3 found [here](https://github.com/jellysquid3/hydrogen-fabric/blob/1.16.x/LICENSE.txt).
All patches (.patch files) marked with "hydrogen" are licensed under LGPL3 found [here](https://github.com/jellysquid3/hydrogen-fabric/blob/1.16.x/LICENSE.txt).<br>
All patches (.patch files) marked with "krypton" are licensed under MIT found [here](https://github.com/astei/krypton/blob/master/LICENSE).<br>
All other patches (.patch files) included in this repo are licensed under the MIT license found [here](MIT.md).<br>
See [EMC](https://github.com/starlis/empirecraft/blob/master/README.md), [Akarin](https://github.com/Akarin-project/Akarin/blob/1.16.3/LICENSE.md), [Purpur](https://github.com/pl3xgaming/Purpur/blob/ver/1.16.5/LICENSE), [AirplaneLite](https://github.com/Technove/AirplaneLite/blob/master/PATCHES-LICENSE), [Origami](https://github.com/Minebench/Origami/blob/1.16/PATCHES-LICENSE), and [Tuinity](https://github.com/Spottedleaf/Tuinity/blob/master/PATCHES-LICENSE) for the license of patches automatically pulled during upstream updates.

2
PATCHES.md
View File

@ -242,6 +242,7 @@ This is an overview over all patches that are currently used.
| server | Multi-Threaded Server Ticking Vanilla | Spottedleaf | |
| server | Multi-Threaded ticking CraftBukkit | Spottedleaf | |
| server | Name craft scheduler threads according to the plugin using | Spottedleaf | |
| server | New Network System | Hugo Planque | Ivan Pekov |
| server | Nuke streams off BlockPosition | Ivan Pekov | |
| server | Nuke streams off SectionPosition | Ivan Pekov | |
| server | Optimise EntityInsentient#checkDespawn | Spottedleaf | |
@ -291,6 +292,7 @@ This is an overview over all patches that are currently used.
| server | Players should not cram to death | William Blake Galbreath | |
| server | Populator seed controls | Spottedleaf | |
| server | Port hydrogen | JellySquid | |
| server | Port krypton | Andrew Steinborn | Hugo Planque |
| server | Prevent light queue overfill when no players are online | Spottedleaf | |
| server | Prevent long map entry creation in light engine | Spottedleaf | |
| server | Prevent unload() calls removing tickets for sync loads | Spottedleaf | |

1
README.md
View File

@ -19,6 +19,7 @@ Yatopia combines the code from many [Paper](https://github.com/PaperMC/Paper) fo
* [Purpur](https://github.com/pl3xgaming/Purpur)
* [AirplaneLite](https://github.com/Technove/AirplaneLite)
* [Hydrogen](https://github.com/jellysquid3/hydrogen-fabric)
* [Krypton](https://github.com/astei/krypton)
## Try it out

307
patches/server/0066-New-Network-System.patch Normal file
View File

@ -0,0 +1,307 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Hugo Planque <hookwood01@gmail.com>
Date: Mon, 18 Jan 2021 11:27:08 +0100
Subject: [PATCH] New Network System
Co-authored-by: Ivan Pekov <ivan@mrivanplays.com>
diff --git a/pom.xml b/pom.xml
index 8af1a91102c5cc4c230f622e6629e46e95f17d44..9b7ed0de1054285dadff6aefc95c7207079504a6 100644
--- a/pom.xml
+++ b/pom.xml
@@ -53,9 +53,17 @@
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
- <version>4.1.50.Final</version>
+ <version>4.1.58.Final</version> <!-- Yatopia -->
</dependency>
<!-- Tuinity end - fix compile issue (cannot see new api) by moving netty include BEFORE server jar -->
+ <!-- Yatopia start - Add IOUring beta support -->
+ <dependency>
+ <groupId>io.netty.incubator</groupId>
+ <artifactId>netty-incubator-transport-native-io_uring</artifactId>
+ <version>0.0.3.Final</version>
+ <classifier>linux-x86_64</classifier>
+ </dependency>
+ <!-- Yatopia end -->
<dependency>
<groupId>io.papermc</groupId>
<artifactId>minecraft-server</artifactId>
diff --git a/src/main/java/net/minecraft/server/MinecraftServer.java b/src/main/java/net/minecraft/server/MinecraftServer.java
index 87cf9cd88d1fb5ae70d19e5618ebfb67d281304a..a1c2bea7c93433434b4e4dfd0bb4b9620657c40d 100644
--- a/src/main/java/net/minecraft/server/MinecraftServer.java
+++ b/src/main/java/net/minecraft/server/MinecraftServer.java
@@ -1740,6 +1740,7 @@ public abstract class MinecraftServer extends IAsyncTaskHandlerReentrant<TickTas
return true;
}
+ public final boolean isUsingNativeTransport() { return l(); } // Yatopia - OBFHELPER
public abstract boolean l();
public boolean getPVP() {
diff --git a/src/main/java/net/minecraft/server/ServerConnection.java b/src/main/java/net/minecraft/server/ServerConnection.java
index 0668d383db1f3a81d1053954d72678c7ac5aecec..f20be527bec58bad8e4a5bb7bb887949cb6d1d99 100644
--- a/src/main/java/net/minecraft/server/ServerConnection.java
+++ b/src/main/java/net/minecraft/server/ServerConnection.java
@@ -11,6 +11,7 @@ import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelOption;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.epoll.Epoll;
+import io.netty.channel.epoll.EpollChannelOption; // Yatopia
import io.netty.channel.epoll.EpollEventLoopGroup;
import io.netty.channel.epoll.EpollServerSocketChannel;
import io.netty.channel.nio.NioEventLoopGroup;
@@ -26,16 +27,20 @@ import java.util.List;
import javax.annotation.Nullable;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
+import org.yatopiamc.yatopia.server.network.NetworkType; // Yatopia
public class ServerConnection {
private static final Logger LOGGER = LogManager.getLogger();
+ /* // Yatopia Start - New network system - Remove unused fields
public static final LazyInitVar<NioEventLoopGroup> a = new LazyInitVar<>(() -> {
return new NioEventLoopGroup(0, (new ThreadFactoryBuilder()).setNameFormat("Netty Server IO #%d").setDaemon(true).build());
});
public static final LazyInitVar<EpollEventLoopGroup> b = new LazyInitVar<>(() -> {
return new EpollEventLoopGroup(0, (new ThreadFactoryBuilder()).setNameFormat("Netty Epoll Server IO #%d").setDaemon(true).build());
});
+ */
+ // Yatopia end
private final MinecraftServer e;
public volatile boolean c;
private final List<ChannelFuture> listeningChannels = Collections.synchronizedList(Lists.newArrayList());
@@ -52,15 +57,29 @@ public class ServerConnection {
}
// Paper end
+ // Yatopia start - New network system
+ private final NetworkType networkType;
+ private final EventLoopGroup boss, worker;
+ // Yatopia end
+
public ServerConnection(MinecraftServer minecraftserver) {
this.e = minecraftserver;
this.c = true;
+
+ // Yatopia start - New network system
+ this.networkType = NetworkType.bestType(minecraftserver);
+ this.boss = networkType.createEventLoopGroup(NetworkType.LoopGroupType.BOSS);
+ this.worker = networkType.createEventLoopGroup(NetworkType.LoopGroupType.WORKER);
+ // Yatopia end
}
public void a(@Nullable InetAddress inetaddress, int i) throws IOException {
+ /* // Yatopia start - New network system
List list = this.listeningChannels;
+ */
synchronized (this.listeningChannels) {
+ /*
Class oclass;
LazyInitVar lazyinitvar;
@@ -73,16 +92,25 @@ public class ServerConnection {
lazyinitvar = ServerConnection.a;
ServerConnection.LOGGER.info("Using default channel type");
}
+ */
+
// Tuinity start - indicate Velocity natives in use
ServerConnection.LOGGER.info("Tuinity: Using " + com.velocitypowered.natives.util.Natives.compress.getLoadedVariant() + " compression from Velocity.");
ServerConnection.LOGGER.info("Tuinity: Using " + com.velocitypowered.natives.util.Natives.cipher.getLoadedVariant() + " cipher from Velocity.");
// Tuinity end
- this.listeningChannels.add(((ServerBootstrap) ((ServerBootstrap) (new ServerBootstrap()).channel(oclass)).childHandler(new ChannelInitializer<Channel>() {
+ ServerConnection.LOGGER.info("Yatopia: Using " + networkType.getName() + " network type.");
+ ServerBootstrap serverBootstrap = new ServerBootstrap();
+
+ if (org.yatopiamc.yatopia.server.YatopiaConfig.tcpFastOpen && networkType == NetworkType.EPOLL) {
+ serverBootstrap.option(EpollChannelOption.TCP_FASTOPEN, 3); // Will improve the connection with some speed
+ }
+ this.listeningChannels.add((((serverBootstrap).channelFactory(networkType.getServerSocketChannelFactory())).childHandler(new ChannelInitializer<Channel>() {
protected void initChannel(Channel channel) throws Exception {
try {
channel.config().setOption(ChannelOption.TCP_NODELAY, true);
+ channel.config().setOption(ChannelOption.IP_TOS, 0x18); // Change the priority of the packet to immediate
} catch (ChannelException channelexception) {
;
}
@@ -97,7 +125,8 @@ public class ServerConnection {
channel.pipeline().addLast("packet_handler", (ChannelHandler) object);
((NetworkManager) object).setPacketListener(new HandshakeListener(ServerConnection.this.e, (NetworkManager) object));
}
- }).group((EventLoopGroup) lazyinitvar.a()).localAddress(inetaddress, i)).option(ChannelOption.AUTO_READ, false).bind().syncUninterruptibly()); // CraftBukkit
+ }).group(boss, worker).localAddress(inetaddress, i)).option(ChannelOption.AUTO_READ, false).bind().syncUninterruptibly()); // CraftBukkit
+ // Yatopia end
}
}
diff --git a/src/main/java/org/yatopiamc/yatopia/server/YatopiaConfig.java b/src/main/java/org/yatopiamc/yatopia/server/YatopiaConfig.java
index 35f212c2ac43ebea6ce9c4a333738c7a869ebc18..c4d0dabd408c7a943dafd6ca89b598cb4afb440e 100644
--- a/src/main/java/org/yatopiamc/yatopia/server/YatopiaConfig.java
+++ b/src/main/java/org/yatopiamc/yatopia/server/YatopiaConfig.java
@@ -303,4 +303,11 @@ public class YatopiaConfig {
logPlayerLoginLoc = getBoolean("settings.log-player-login-location", logPlayerLoginLoc);
}
+ public static boolean ioUringBeta = false;
+ public static boolean tcpFastOpen = false;
+ private static void newNetworkSystem() {
+ ioUringBeta = getBoolean("network.io-uring", ioUringBeta);
+ tcpFastOpen = getBoolean("network.ftcp-fastopen", tcpFastOpen);
+ }
+
}
diff --git a/src/main/java/org/yatopiamc/yatopia/server/network/NettyThreadFactory.java b/src/main/java/org/yatopiamc/yatopia/server/network/NettyThreadFactory.java
new file mode 100644
index 0000000000000000000000000000000000000000..3e74e23f3cc44b7547d9f8575b411059e0e65449
--- /dev/null
+++ b/src/main/java/org/yatopiamc/yatopia/server/network/NettyThreadFactory.java
@@ -0,0 +1,27 @@
+package org.yatopiamc.yatopia.server.network;
+
+import io.netty.util.concurrent.FastThreadLocalThread;
+import java.util.concurrent.ThreadFactory;
+import java.util.concurrent.atomic.AtomicInteger;
+
+/**
+ * Based off of Velocity's VelocityNettyThreadFactory
+ */
+public class NettyThreadFactory implements ThreadFactory {
+
+ private final AtomicInteger threadNumber = new AtomicInteger();
+ private final String networkTypeName, loopGroupTypeName;
+
+ public NettyThreadFactory(String networkTypeName, String loopGroupTypeName) {
+ this.networkTypeName = networkTypeName;
+ this.loopGroupTypeName = loopGroupTypeName;
+ }
+
+ @Override
+ public Thread newThread(Runnable r) {
+ return new FastThreadLocalThread(
+ r,
+ "Netty " + networkTypeName + " " + loopGroupTypeName + " #" + threadNumber.getAndIncrement()
+ );
+ }
+}
diff --git a/src/main/java/org/yatopiamc/yatopia/server/network/NetworkType.java b/src/main/java/org/yatopiamc/yatopia/server/network/NetworkType.java
new file mode 100644
index 0000000000000000000000000000000000000000..6b9d788dfef2c51111e9f2129a04fce7754c51ba
--- /dev/null
+++ b/src/main/java/org/yatopiamc/yatopia/server/network/NetworkType.java
@@ -0,0 +1,110 @@
+package org.yatopiamc.yatopia.server.network;
+
+import io.netty.channel.ChannelFactory;
+import io.netty.channel.EventLoopGroup;
+import io.netty.channel.epoll.Epoll;
+import io.netty.channel.epoll.EpollEventLoopGroup;
+import io.netty.channel.epoll.EpollServerSocketChannel;
+import io.netty.channel.kqueue.KQueue;
+import io.netty.channel.kqueue.KQueueEventLoopGroup;
+import io.netty.channel.kqueue.KQueueServerSocketChannel;
+import io.netty.channel.nio.NioEventLoopGroup;
+import io.netty.channel.socket.ServerSocketChannel;
+import io.netty.channel.socket.nio.NioServerSocketChannel;
+import io.netty.incubator.channel.uring.IOUring;
+import io.netty.incubator.channel.uring.IOUringEventLoopGroup;
+import io.netty.incubator.channel.uring.IOUringServerSocketChannel;
+
+import java.util.concurrent.ThreadFactory;
+import java.util.function.BiFunction;
+
+import net.minecraft.server.MinecraftServer;
+import org.spigotmc.SpigotConfig;
+import org.yatopiamc.yatopia.server.YatopiaConfig;
+
+/**
+ * Based off of Velocity's TransportType
+ */
+public enum NetworkType {
+ NIO("NIO", NioServerSocketChannel::new, (name, type) -> new NioEventLoopGroup(8, createThreadFactory(name, type))),
+
+ EPOLL("Epoll", EpollServerSocketChannel::new, (name, type) -> new EpollEventLoopGroup(8, createThreadFactory(name, type))),
+
+ IOURING("IOUring", IOUringServerSocketChannel::new, (name, type) -> new IOUringEventLoopGroup(8, createThreadFactory(name, type))),
+
+ KQUEUE("KQueue", KQueueServerSocketChannel::new, (name, type) -> new KQueueEventLoopGroup(8, createThreadFactory(name, type)));
+
+ public final String name;
+ public final ChannelFactory<? extends ServerSocketChannel> serverSocketChannelFactory;
+ public final BiFunction<String, LoopGroupType, EventLoopGroup> eventLoopGroupFactory;
+
+ NetworkType(final String name,
+ final ChannelFactory<? extends ServerSocketChannel> serverSocketChannelFactory,
+ final BiFunction<String, LoopGroupType, EventLoopGroup> eventLoopGroupFactory) {
+ this.name = name;
+ this.serverSocketChannelFactory = serverSocketChannelFactory;
+ this.eventLoopGroupFactory = eventLoopGroupFactory;
+ }
+
+ public String getName() {
+ return name;
+ }
+
+ public ChannelFactory<? extends ServerSocketChannel> getServerSocketChannelFactory() {
+ return serverSocketChannelFactory;
+ }
+
+ @Override
+ public String toString() {
+ return this.name;
+ }
+
+ public EventLoopGroup createEventLoopGroup(final LoopGroupType type) {
+ return this.eventLoopGroupFactory.apply(this.name, type);
+ }
+
+ private static ThreadFactory createThreadFactory(final String name, final LoopGroupType type) {
+ return new NettyThreadFactory(name, type.toString());
+ }
+
+ public static NetworkType bestType(MinecraftServer minecraftServer) {
+ if (!minecraftServer.isUsingNativeTransport()) {
+ return NIO;
+ }
+ // Actually, there is a decompression problem with zlib from bungeecord that makes
+ // IOUring not available on spigot server with bungeecord
+ // https://github.com/netty/netty-incubator-transport-io_uring/issues/40
+ // Looks like iouring send very small compressed packet and trigger PacketDecompressor
+ if (!SpigotConfig.bungee && YatopiaConfig.ioUringBeta && MinecraftServer.getServer().ax() < 0) {
+ if (IOUring.isAvailable()) {
+ return IOURING;
+ }
+ }
+
+ if (Epoll.isAvailable()) {
+ return EPOLL;
+ }
+
+ if (KQueue.isAvailable()) {
+ return KQUEUE;
+ }
+
+ return NIO;
+ }
+
+ public enum LoopGroupType {
+ BOSS("Boss"),
+ WORKER("Worker");
+
+ private final String name;
+
+ LoopGroupType(final String name) {
+ this.name = name;
+ }
+
+ @Override
+ public String toString() {
+ return this.name;
+ }
+ }
+}

0
patches/server/0066-Port-hydrogen.patch → patches/server/0067-Port-hydrogen.patch
View File

550
patches/server/0068-Port-krypton.patch Normal file
View File

@ -0,0 +1,550 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Andrew Steinborn <git@steinborn.me>
Date: Thu, 21 Jan 2021 00:40:24 +0100
Subject: [PATCH] Port krypton
Co-authored-by: Hugo Planque <hookwood01@gmail.com>
diff --git a/src/main/java/me/steinborn/krypton/mod/shared/network/VarintByteDecoder.java b/src/main/java/me/steinborn/krypton/mod/shared/network/VarintByteDecoder.java
new file mode 100644
index 0000000000000000000000000000000000000000..cdf5a3b1f7ec27171f3825f89cfb8d398fb3fd79
--- /dev/null
+++ b/src/main/java/me/steinborn/krypton/mod/shared/network/VarintByteDecoder.java
@@ -0,0 +1,47 @@
+package me.steinborn.krypton.mod.shared.network;
+
+import io.netty.util.ByteProcessor;
+
+public class VarintByteDecoder implements ByteProcessor {
+ private int readVarint;
+ private int bytesRead;
+ private DecodeResult result = DecodeResult.TOO_SHORT;
+
+ @Override
+ public boolean process(byte k) {
+ readVarint |= (k & 0x7F) << bytesRead++ * 7;
+ if (bytesRead > 3) {
+ result = DecodeResult.TOO_BIG;
+ return false;
+ }
+ if ((k & 0x80) != 128) {
+ result = DecodeResult.SUCCESS;
+ return false;
+ }
+ return true;
+ }
+
+ public int readVarint() {
+ return readVarint;
+ }
+
+ public int varintBytes() {
+ return bytesRead;
+ }
+
+ public DecodeResult getResult() {
+ return result;
+ }
+
+ public void reset() {
+ readVarint = 0;
+ bytesRead = 0;
+ result = DecodeResult.TOO_SHORT;
+ }
+
+ public enum DecodeResult {
+ SUCCESS,
+ TOO_SHORT,
+ TOO_BIG
+ }
+}
\ No newline at end of file
diff --git a/src/main/java/me/steinborn/krypton/mod/shared/network/compression/MinecraftCompressDecoder.java b/src/main/java/me/steinborn/krypton/mod/shared/network/compression/MinecraftCompressDecoder.java
new file mode 100644
index 0000000000000000000000000000000000000000..e9a51c71e136be14ebe8a240c4b21205079fc71b
--- /dev/null
+++ b/src/main/java/me/steinborn/krypton/mod/shared/network/compression/MinecraftCompressDecoder.java
@@ -0,0 +1,68 @@
+package me.steinborn.krypton.mod.shared.network.compression;
+
+import com.velocitypowered.natives.compression.VelocityCompressor;
+import io.netty.buffer.ByteBuf;
+import io.netty.channel.ChannelHandlerContext;
+import io.netty.handler.codec.CorruptedFrameException;
+import io.netty.handler.codec.MessageToMessageDecoder;
+import net.minecraft.server.PacketDataSerializer;
+
+import java.util.List;
+
+import static com.velocitypowered.natives.util.MoreByteBufUtils.ensureCompatible;
+import static com.velocitypowered.natives.util.MoreByteBufUtils.preferredBuffer;
+
+public class MinecraftCompressDecoder extends MessageToMessageDecoder<ByteBuf> {
+
+ private static final int VANILLA_MAXIMUM_UNCOMPRESSED_SIZE = 2 * 1024 * 1024; // 2MiB
+ private static final int HARD_MAXIMUM_UNCOMPRESSED_SIZE = 16 * 1024 * 1024; // 16MiB
+
+ private static final int UNCOMPRESSED_CAP =
+ Boolean.getBoolean("velocity.increased-compression-cap")
+ ? HARD_MAXIMUM_UNCOMPRESSED_SIZE : VANILLA_MAXIMUM_UNCOMPRESSED_SIZE;
+
+ private final int threshold;
+ private final VelocityCompressor compressor;
+
+ public MinecraftCompressDecoder(int threshold, VelocityCompressor compressor) {
+ this.threshold = threshold;
+ this.compressor = compressor;
+ }
+
+ @Override
+ protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {
+ PacketDataSerializer wrappedBuf = new PacketDataSerializer(in);
+ int claimedUncompressedSize = wrappedBuf.readVarInt();
+ if (claimedUncompressedSize == 0) {
+ // This message is not compressed.
+ out.add(in.retainedSlice());
+ return;
+ }
+
+ if (claimedUncompressedSize < threshold) {
+ throw new CorruptedFrameException("Uncompressed size " + claimedUncompressedSize + " is less than"
+ + " threshold " + threshold);
+ }
+ if (claimedUncompressedSize > UNCOMPRESSED_CAP) {
+ throw new CorruptedFrameException("Uncompressed size " + claimedUncompressedSize + " exceeds hard " +
+ "threshold of " + UNCOMPRESSED_CAP);
+ }
+
+ ByteBuf compatibleIn = ensureCompatible(ctx.alloc(), compressor, in);
+ ByteBuf uncompressed = preferredBuffer(ctx.alloc(), compressor, claimedUncompressedSize);
+ try {
+ compressor.inflate(compatibleIn, uncompressed, claimedUncompressedSize);
+ out.add(uncompressed);
+ } catch (Exception e) {
+ uncompressed.release();
+ throw e;
+ } finally {
+ compatibleIn.release();
+ }
+ }
+
+ @Override
+ public void handlerRemoved(ChannelHandlerContext ctx) throws Exception {
+ compressor.close();
+ }
+}
\ No newline at end of file
diff --git a/src/main/java/me/steinborn/krypton/mod/shared/network/compression/MinecraftCompressEncoder.java b/src/main/java/me/steinborn/krypton/mod/shared/network/compression/MinecraftCompressEncoder.java
new file mode 100644
index 0000000000000000000000000000000000000000..1114e1db59476353ad609a519b71479438db7b0c
--- /dev/null
+++ b/src/main/java/me/steinborn/krypton/mod/shared/network/compression/MinecraftCompressEncoder.java
@@ -0,0 +1,59 @@
+package me.steinborn.krypton.mod.shared.network.compression;
+
+
+import com.velocitypowered.natives.compression.VelocityCompressor;
+import com.velocitypowered.natives.util.MoreByteBufUtils;
+import io.netty.buffer.ByteBuf;
+import io.netty.channel.ChannelHandlerContext;
+import io.netty.handler.codec.MessageToByteEncoder;
+import net.minecraft.server.PacketDataSerializer;
+
+public class MinecraftCompressEncoder extends MessageToByteEncoder<ByteBuf> {
+
+ private final int threshold;
+ private final VelocityCompressor compressor;
+
+ public MinecraftCompressEncoder(int threshold, VelocityCompressor compressor) {
+ this.threshold = threshold;
+ this.compressor = compressor;
+ }
+
+ @Override
+ protected void encode(ChannelHandlerContext ctx, ByteBuf msg, ByteBuf out) throws Exception {
+ PacketDataSerializer wrappedBuf = new PacketDataSerializer(out);
+ int uncompressed = msg.readableBytes();
+ if (uncompressed < threshold) {
+ // Under the threshold, there is nothing to do.
+ wrappedBuf.writeVarInt(0);
+ out.writeBytes(msg);
+ } else {
+ wrappedBuf.writeVarInt(uncompressed);
+ ByteBuf compatibleIn = MoreByteBufUtils.ensureCompatible(ctx.alloc(), compressor, msg);
+ try {
+ compressor.deflate(compatibleIn, out);
+ } finally {
+ compatibleIn.release();
+ }
+ }
+ }
+
+ @Override
+ protected ByteBuf allocateBuffer(ChannelHandlerContext ctx, ByteBuf msg, boolean preferDirect)
+ throws Exception {
+ // We allocate bytes to be compressed plus 1 byte. This covers two cases:
+ //
+ // - Compression
+ // According to https://github.com/ebiggers/libdeflate/blob/master/libdeflate.h#L103,
+ // if the data compresses well (and we do not have some pathological case) then the maximum
+ // size the compressed size will ever be is the input size minus one.
+ // - Uncompressed
+ // This is fairly obvious - we will then have one more than the uncompressed size.
+ int initialBufferSize = msg.readableBytes() + 1;
+ return MoreByteBufUtils.preferredBuffer(ctx.alloc(), compressor, initialBufferSize);
+ }
+
+ @Override
+ public void handlerRemoved(ChannelHandlerContext ctx) throws Exception {
+ compressor.close();
+ }
+}
\ No newline at end of file
diff --git a/src/main/java/me/steinborn/krypton/mod/shared/network/pipeline/MinecraftCipherDecoder.java b/src/main/java/me/steinborn/krypton/mod/shared/network/pipeline/MinecraftCipherDecoder.java
new file mode 100644
index 0000000000000000000000000000000000000000..2612c350446b172629b8030602ed812fa69f24a4
--- /dev/null
+++ b/src/main/java/me/steinborn/krypton/mod/shared/network/pipeline/MinecraftCipherDecoder.java
@@ -0,0 +1,36 @@
+package me.steinborn.krypton.mod.shared.network.pipeline;
+
+import com.google.common.base.Preconditions;
+import com.velocitypowered.natives.encryption.VelocityCipher;
+import com.velocitypowered.natives.util.MoreByteBufUtils;
+import io.netty.buffer.ByteBuf;
+import io.netty.channel.ChannelHandlerContext;
+import io.netty.handler.codec.MessageToMessageDecoder;
+
+import java.util.List;
+
+public class MinecraftCipherDecoder extends MessageToMessageDecoder<ByteBuf> {
+
+ private final VelocityCipher cipher;
+
+ public MinecraftCipherDecoder(VelocityCipher cipher) {
+ this.cipher = Preconditions.checkNotNull(cipher, "cipher");
+ }
+
+ @Override
+ protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {
+ ByteBuf compatible = MoreByteBufUtils.ensureCompatible(ctx.alloc(), cipher, in).slice();
+ try {
+ cipher.process(compatible);
+ out.add(compatible);
+ } catch (Exception e) {
+ compatible.release(); // compatible will never be used if we throw an exception
+ throw e;
+ }
+ }
+
+ @Override
+ public void handlerRemoved(ChannelHandlerContext ctx) throws Exception {
+ cipher.close();
+ }
+}
diff --git a/src/main/java/me/steinborn/krypton/mod/shared/network/pipeline/MinecraftCipherEncoder.java b/src/main/java/me/steinborn/krypton/mod/shared/network/pipeline/MinecraftCipherEncoder.java
new file mode 100644
index 0000000000000000000000000000000000000000..068b0d51daf11045a5054cddd53897ecdd117a37
--- /dev/null
+++ b/src/main/java/me/steinborn/krypton/mod/shared/network/pipeline/MinecraftCipherEncoder.java
@@ -0,0 +1,36 @@
+package me.steinborn.krypton.mod.shared.network.pipeline;
+
+import com.google.common.base.Preconditions;
+import com.velocitypowered.natives.encryption.VelocityCipher;
+import com.velocitypowered.natives.util.MoreByteBufUtils;
+import io.netty.buffer.ByteBuf;
+import io.netty.channel.ChannelHandlerContext;
+import io.netty.handler.codec.MessageToMessageEncoder;
+
+import java.util.List;
+
+public class MinecraftCipherEncoder extends MessageToMessageEncoder<ByteBuf> {
+
+ private final VelocityCipher cipher;
+
+ public MinecraftCipherEncoder(VelocityCipher cipher) {
+ this.cipher = Preconditions.checkNotNull(cipher, "cipher");
+ }
+
+ @Override
+ protected void encode(ChannelHandlerContext ctx, ByteBuf msg, List<Object> out) throws Exception {
+ ByteBuf compatible = MoreByteBufUtils.ensureCompatible(ctx.alloc(), cipher, msg);
+ try {
+ cipher.process(compatible);
+ out.add(compatible);
+ } catch (Exception e) {
+ compatible.release(); // compatible will never be used if we throw an exception
+ throw e;
+ }
+ }
+
+ @Override
+ public void handlerRemoved(ChannelHandlerContext ctx) throws Exception {
+ cipher.close();
+ }
+}
\ No newline at end of file
diff --git a/src/main/java/net/minecraft/server/LegacyPingHandler.java b/src/main/java/net/minecraft/server/LegacyPingHandler.java
index 4a49fe4cc600e2b70963302ddae0c4479849f3f5..3abc3869b8012f060e1997822ffdb321f4884929 100644
--- a/src/main/java/net/minecraft/server/LegacyPingHandler.java
+++ b/src/main/java/net/minecraft/server/LegacyPingHandler.java
@@ -23,6 +23,11 @@ public class LegacyPingHandler extends ChannelInboundHandlerAdapter {
}
public void channelRead(ChannelHandlerContext channelhandlercontext, Object object) throws Exception {
+ // Yatopia start - New network system
+ if (!channelhandlercontext.channel().isActive()) {
+ ((ByteBuf) object).clear();
+ return;
+ } // Yatopia end
ByteBuf bytebuf = (ByteBuf) object;
// Paper start - Make legacy ping handler more reliable
diff --git a/src/main/java/net/minecraft/server/LoginListener.java b/src/main/java/net/minecraft/server/LoginListener.java
index 847122f76f6d951b24b22c86276140e02aaf37d6..174197f41bedab6a45e96323adff4f3f6238cef2 100644
--- a/src/main/java/net/minecraft/server/LoginListener.java
+++ b/src/main/java/net/minecraft/server/LoginListener.java
@@ -8,6 +8,7 @@ import java.math.BigInteger;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.SocketAddress;
+import java.security.GeneralSecurityException; // Yatopia
import java.security.PrivateKey;
import java.util.Arrays;
import java.util.Random;
@@ -256,8 +257,8 @@ public class LoginListener implements PacketLoginInListener {
s = (new BigInteger(MinecraftEncryption.a("", this.server.getKeyPair().getPublic(), this.loginKey))).toString(16);
this.g = LoginListener.EnumProtocolState.AUTHENTICATING;
- this.networkManager.a(this.loginKey); // Tuinity
- } catch (CryptographyException cryptographyexception) {
+ this.networkManager.setupEncryption(this.loginKey); // Tuinity // Yatopia - New network system
+ } catch (CryptographyException | GeneralSecurityException cryptographyexception) { // Yatopia
throw new IllegalStateException("Protocol error", cryptographyexception);
}
diff --git a/src/main/java/net/minecraft/server/NetworkManager.java b/src/main/java/net/minecraft/server/NetworkManager.java
index 08227ab446d6332af76491a063653f7f13f43560..f2a8c317eb8a20a9f9decc36421f98750aa374e3 100644
--- a/src/main/java/net/minecraft/server/NetworkManager.java
+++ b/src/main/java/net/minecraft/server/NetworkManager.java
@@ -21,6 +21,17 @@ import java.net.SocketAddress;
import java.util.Queue;
import javax.annotation.Nullable;
import javax.crypto.Cipher;
+// Yatopia start
+import javax.crypto.SecretKey;
+import me.steinborn.krypton.mod.shared.network.compression.MinecraftCompressDecoder;
+import me.steinborn.krypton.mod.shared.network.compression.MinecraftCompressEncoder;
+import me.steinborn.krypton.mod.shared.network.pipeline.MinecraftCipherDecoder;
+import me.steinborn.krypton.mod.shared.network.pipeline.MinecraftCipherEncoder;
+import com.velocitypowered.natives.compression.VelocityCompressor;
+import com.velocitypowered.natives.encryption.VelocityCipher;
+import com.velocitypowered.natives.util.Natives;
+import java.security.GeneralSecurityException;
+// Yatopia end
import org.apache.commons.lang3.Validate;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
@@ -107,6 +118,17 @@ public class NetworkManager extends SimpleChannelInboundHandler<Packet<?>> {
}
// Tuinity end - allow controlled flushing
+ // Yatopia start
+ public void setupEncryption(SecretKey key) throws GeneralSecurityException {
+ VelocityCipher decryption = Natives.cipher.get().forDecryption(key);
+ VelocityCipher encryption = Natives.cipher.get().forEncryption(key);
+
+ this.n = true;
+ this.channel.pipeline().addBefore("splitter", "decrypt", new MinecraftCipherDecoder(decryption));
+ this.channel.pipeline().addBefore("prepender", "encrypt", new MinecraftCipherEncoder(encryption));
+ }
+ // Yatopia end
+
public NetworkManager(EnumProtocolDirection enumprotocoldirection) {
this.h = enumprotocoldirection;
}
@@ -623,6 +645,19 @@ public class NetworkManager extends SimpleChannelInboundHandler<Packet<?>> {
}
public void setCompressionLevel(int i) {
+ // Yatopia start - New network system
+ if (i >= 0) {
+ VelocityCompressor compressor = Natives.compress.get().create(4);
+ MinecraftCompressEncoder encoder = new MinecraftCompressEncoder(i, compressor);
+ MinecraftCompressDecoder decoder = new MinecraftCompressDecoder(i, compressor);
+
+ channel.pipeline().addBefore("decoder", "decompress", decoder);
+ channel.pipeline().addBefore("encoder", "compress", encoder);
+ } else {
+ this.channel.pipeline().remove("decompress");
+ this.channel.pipeline().remove("compress");
+ }
+ /*
if (i >= 0) {
if (this.channel.pipeline().get("decompress") instanceof PacketDecompressor) {
((PacketDecompressor) this.channel.pipeline().get("decompress")).a(i);
@@ -644,6 +679,7 @@ public class NetworkManager extends SimpleChannelInboundHandler<Packet<?>> {
this.channel.pipeline().remove("compress");
}
}
+ */ // Yatopia end
}
diff --git a/src/main/java/net/minecraft/server/PacketDataSerializer.java b/src/main/java/net/minecraft/server/PacketDataSerializer.java
index f43193c1090238f2241b878120247d1b3d0d4e57..7dc31ee3211a895993c522a7155a0d8641fd442c 100644
--- a/src/main/java/net/minecraft/server/PacketDataSerializer.java
+++ b/src/main/java/net/minecraft/server/PacketDataSerializer.java
@@ -7,6 +7,7 @@ import io.netty.buffer.ByteBuf;
import io.netty.buffer.ByteBufAllocator;
import io.netty.buffer.ByteBufInputStream;
import io.netty.buffer.ByteBufOutputStream;
+import io.netty.buffer.ByteBufUtil; // Yatopia
import io.netty.handler.codec.DecoderException;
import io.netty.handler.codec.EncoderException;
import io.netty.util.ByteProcessor;
@@ -30,7 +31,7 @@ import org.bukkit.craftbukkit.inventory.CraftItemStack; // CraftBukkit
public class PacketDataSerializer extends ByteBuf {
- private final ByteBuf a;
+ private final ByteBuf a; private final ByteBuf getParent() { return a; } // Yatopia - OBFHELPER
public PacketDataSerializer(ByteBuf bytebuf) {
this.a = bytebuf;
@@ -210,6 +211,7 @@ public class PacketDataSerializer extends ByteBuf {
return new UUID(this.readLong(), this.readLong());
}
+ public PacketDataSerializer writeVarInt(int i){ return d(i); } // Yatopia - OBFHELPER
public PacketDataSerializer d(int i) {
while ((i & -128) != 0) {
this.writeByte(i & 127 | 128);
@@ -358,6 +360,8 @@ public class PacketDataSerializer extends ByteBuf {
}
public PacketDataSerializer a(String s, int i) {
+ // Yatopia start - New network system
+ /*
byte[] abyte = s.getBytes(StandardCharsets.UTF_8);
if (abyte.length > i) {
@@ -367,6 +371,16 @@ public class PacketDataSerializer extends ByteBuf {
this.writeBytes(abyte);
return this;
}
+ */
+ int utf8Bytes = ByteBufUtil.utf8Bytes(s);
+ if (utf8Bytes > i) {
+ throw new EncoderException("String too big (was " + utf8Bytes + " bytes encoded, max " + i + ")");
+ } else {
+ this.writeVarInt(utf8Bytes);
+ this.writeCharSequence(s, StandardCharsets.UTF_8);
+ return new PacketDataSerializer(getParent());
+ }
+ // Yatopia end
}
public MinecraftKey p() {
diff --git a/src/main/java/net/minecraft/server/PacketSplitter.java b/src/main/java/net/minecraft/server/PacketSplitter.java
index 2aaa8770edfd8acc6861c23176e405863858b275..b1e1aa95b5d7a1555428327f2e7fd0eafc679dc1 100644
--- a/src/main/java/net/minecraft/server/PacketSplitter.java
+++ b/src/main/java/net/minecraft/server/PacketSplitter.java
@@ -5,14 +5,20 @@ import io.netty.buffer.Unpooled;
import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.ByteToMessageDecoder;
import io.netty.handler.codec.CorruptedFrameException;
+// Yatopia start
+import io.netty.handler.codec.DecoderException;
+import me.steinborn.krypton.mod.shared.network.VarintByteDecoder;
+// Yatopia end
import java.util.List;
public class PacketSplitter extends ByteToMessageDecoder {
+ private final VarintByteDecoder reader = new VarintByteDecoder(); // Yatopia
private final byte[] lenBuf = new byte[3]; // Paper
public PacketSplitter() {}
protected void decode(ChannelHandlerContext channelhandlercontext, ByteBuf bytebuf, List<Object> list) throws Exception {
+ /* // Yatopia start - New network system
// Paper start - if channel is not active just discard the packet
if (!channelhandlercontext.channel().isActive()) {
bytebuf.skipBytes(bytebuf.readableBytes());
@@ -53,5 +59,38 @@ public class PacketSplitter extends ByteToMessageDecoder {
}
throw new CorruptedFrameException("length wider than 21-bit");
+ */
+ if (!channelhandlercontext.channel().isActive()) {
+ bytebuf.clear();
+ return;
+ }
+
+ reader.reset();
+
+ int varintEnd = bytebuf.forEachByte(reader);
+ if (varintEnd == -1) {
+ // We tried to go beyond the end of the buffer. This is probably a good sign that the
+ // buffer was too short to hold a proper varint.
+ return;
+ }
+
+ if (reader.getResult() == VarintByteDecoder.DecodeResult.SUCCESS) {
+ int readLen = reader.readVarint();
+ if (readLen < 0) {
+ throw new DecoderException("Bad packet length");
+ } else if (readLen == 0) {
+ // skip over the empty packet and ignore it
+ bytebuf.readerIndex(varintEnd + 1);
+ } else {
+ int minimumRead = reader.varintBytes() + readLen;
+ if (bytebuf.isReadable(minimumRead)) {
+ list.add(bytebuf.retainedSlice(varintEnd + 1, readLen));
+ bytebuf.skipBytes(minimumRead);
+ }
+ }
+ } else if (reader.getResult() == VarintByteDecoder.DecodeResult.TOO_BIG) {
+ throw new DecoderException("Varint too big");
+ }
+ // Yatopia end
}
}
diff --git a/src/main/java/org/bukkit/craftbukkit/Main.java b/src/main/java/org/bukkit/craftbukkit/Main.java
index 756be0886856aabc31e436f82948d3d069f66fef..c1dd04b57e8ffeb343b0efdec36d6fb36c16884f 100644
--- a/src/main/java/org/bukkit/craftbukkit/Main.java
+++ b/src/main/java/org/bukkit/craftbukkit/Main.java
@@ -10,6 +10,7 @@ import java.util.List;
import java.util.concurrent.TimeUnit;
import java.util.logging.Level;
import java.util.logging.Logger;
+import io.netty.util.ResourceLeakDetector; // Yatopia
import joptsimple.OptionParser;
import joptsimple.OptionSet;
import net.minecrell.terminalconsole.TerminalConsoleAppender; // Paper
@@ -289,6 +290,24 @@ public class Main {
// Paper End
}
}
+ // Yatopia start - New network system
+ // By default, Netty allocates 16MiB arenas for the PooledByteBufAllocator. This is too much
+ // memory for Minecraft, which imposes a maximum packet size of 2MiB! We'll use 4MiB as a more
+ // sane default.
+ //
+ // Note: io.netty.allocator.pageSize << io.netty.allocator.maxOrder is the formula used to
+ // compute the chunk size. We lower maxOrder from its default of 11 to 9. (We also use a null
+ // check, so that the user is free to choose another setting if need be.)
+ if (System.getProperty("io.netty.allocator.maxOrder") == null) {
+ System.setProperty("io.netty.allocator.maxOrder", "9");
+ }
+
+ // Disable the resource leak detector by default as it reduces performance. Allow the user to
+ // override this if desired.
+ if (System.getProperty("io.netty.leakDetection.level") == null) {
+ ResourceLeakDetector.setLevel(ResourceLeakDetector.Level.DISABLED);
+ }
+ // Yatopia end
// Paper start - Log Java and OS versioning to help with debugging plugin issues
java.lang.management.RuntimeMXBean runtimeMX = java.lang.management.ManagementFactory.getRuntimeMXBean();