Paper/paper-api-generator/src/main/java/io/papermc/generator/rewriter/parser/LineParser.java

214 lines
8.7 KiB
Java
Raw Normal View History

improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
package io.papermc.generator.rewriter.parser;
2024-04-22 18:40:27 +02:00
import com.google.common.base.Preconditions;
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
import io.papermc.generator.rewriter.context.ImportCollector;
2024-04-22 18:40:27 +02:00
import io.papermc.generator.rewriter.parser.closure.AbstractClosure;
import io.papermc.generator.rewriter.parser.closure.Closure;
import io.papermc.generator.rewriter.parser.closure.ClosureType;
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
import io.papermc.generator.rewriter.parser.step.IterativeStep;
import io.papermc.generator.rewriter.parser.step.StepManager;
2024-04-22 18:40:27 +02:00
import io.papermc.generator.rewriter.parser.step.model.AnnotationSkipSteps;
import io.papermc.generator.rewriter.parser.step.model.ImportStatementSteps;
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
import org.jetbrains.annotations.ApiStatus;
2024-04-22 18:40:27 +02:00
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
import java.util.List;
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
@ApiStatus.Internal
public class LineParser {
private final StepManager stepManager = new StepManager();
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
2024-04-22 18:40:27 +02:00
private Closure nearestClosure;
2024-04-22 18:40:27 +02:00
@Nullable
public Closure getNearestClosure() {
return this.nearestClosure;
}
2024-04-22 18:40:27 +02:00
// internal use only or when nearestClosure = null
// doesn't support leaf closure char escape
public boolean tryAdvanceStartClosure(@NotNull ClosureType type, @NotNull StringReader line) {
if (line.trySkipString(type.start)) { // closure has been consumed
Closure previousNearestClosure = this.nearestClosure;
this.nearestClosure = Closure.create(type);
if (previousNearestClosure != null) {
if (ClosureType.LEAFS.contains(previousNearestClosure.getType())) {
throw new ParserException("Nested closure in a leaf closure is not allowed", line);
}
2024-04-22 18:40:27 +02:00
((AbstractClosure) this.nearestClosure).setParent(previousNearestClosure);
}
2024-04-22 18:40:27 +02:00
this.nearestClosure.onStart(line);
return true;
}
2024-04-22 18:40:27 +02:00
return false;
}
// for all closure, leaf closure type should use the other similar method after this one if possible
// ignoreNestedClosureTypes order matter here
public ClosureAdvanceResult tryAdvanceEndClosure(@NotNull Closure closure, @NotNull StringReader line) {
Preconditions.checkState(this.nearestClosure != null && this.nearestClosure.hasUpperClosure(closure), "Need to be in an upper closure of " + closure + " to find its end identifier");
boolean directClosureFound = this.nearestClosure == closure;
boolean canSearchEndClosure = this.nearestClosure == null || directClosureFound;
char previousChar = '\0';
if (line.getCursor() >= 1) {
previousChar = line.peek(-1);
}
2024-04-22 18:40:27 +02:00
ClosureType type = closure.getType();
if (canSearchEndClosure && line.trySkipString(type.end)) { // closure has been consumed
// skip escape closed closure
if (type.end.length() == 1 && type.start.equals(type.end) && ClosureType.ALLOW_ESCAPE.contains(type)) {
if (previousChar == '\\') {
return ClosureAdvanceResult.skip();
}
2024-04-22 18:40:27 +02:00
}
this.nearestClosure.onEnd(line);
if (this.nearestClosure.parent() != null) {
this.nearestClosure = this.nearestClosure.parent();
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
} else {
2024-04-22 18:40:27 +02:00
this.nearestClosure = null;
}
return ClosureAdvanceResult.find(directClosureFound);
}
return ClosureAdvanceResult.NONE;
}
public boolean trySkipNestedClosures(@NotNull Closure inClosure, @NotNull StringReader line, @NotNull List<ClosureType> computedTypes) {
boolean directClosureFound = this.nearestClosure == inClosure;
boolean isLeaf = this.nearestClosure != null && ClosureType.LEAFS.contains(this.nearestClosure.getType());
if (this.nearestClosure != null && !directClosureFound) {
final boolean advanced;
if (isLeaf) {
advanced = this.tryAdvanceEndClosure(this.nearestClosure.getType(), line) != ClosureTypeAdvanceResult.IGNORED;
} else {
advanced = this.tryAdvanceEndClosure(this.nearestClosure, line).advanced();
}
if (advanced) {
return true;
}
}
if (this.nearestClosure == null || !isLeaf) { // leaf take the priority and doesn't allow any other nested type
for (ClosureType type : computedTypes) {
if (this.tryAdvanceStartClosure(type, line)) {
return true;
}
2024-04-22 18:40:27 +02:00
}
}
return false;
}
// only valid for leaf closure type
public ClosureTypeAdvanceResult tryAdvanceEndClosure(@NotNull ClosureType type, @NotNull StringReader line) {
Preconditions.checkArgument(ClosureType.LEAFS.contains(type), "Only leaf closure can be advanced using its type only, for other, use the closure equivalent method to take in account nested closure");
Preconditions.checkState(this.nearestClosure != null && this.nearestClosure.getType() == type, "Need an direct upper closure of " + type);
2024-04-22 18:40:27 +02:00
char previousChar = '\0';
if (line.getCursor() >= 1) {
previousChar = line.peek(-1);
}
if (line.trySkipString(type.end)) { // closure has been consumed
// skip escape closed closure
if (type.end.length() == 1 && type.start.equals(type.end) && ClosureType.ALLOW_ESCAPE.contains(type)) {
if (previousChar == '\\') {
return ClosureTypeAdvanceResult.SKIPPED;
}
}
2024-04-22 18:40:27 +02:00
this.nearestClosure.onEnd(line);
if (this.nearestClosure.parent() != null) {
this.nearestClosure = this.nearestClosure.parent();
} else {
this.nearestClosure = null;
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
}
2024-04-22 18:40:27 +02:00
return ClosureTypeAdvanceResult.CHANGED;
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
}
2024-04-22 18:40:27 +02:00
return ClosureTypeAdvanceResult.IGNORED;
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
}
2024-04-22 18:40:27 +02:00
public boolean skipComment(@NotNull StringReader line) {
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
int previousCursor = line.getCursor();
2024-04-22 18:40:27 +02:00
if ((this.nearestClosure != null && this.nearestClosure.getType() == ClosureType.COMMENT) ||
this.tryAdvanceStartClosure(ClosureType.COMMENT, line)) { // open comment?
ClosureTypeAdvanceResult result;
while ((result = this.tryAdvanceEndClosure(ClosureType.COMMENT, line)) != ClosureTypeAdvanceResult.CHANGED && line.canRead()) { // closed comment?
if (result == ClosureTypeAdvanceResult.IGNORED) {
line.skip();
}
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
}
return line.getCursor() > previousCursor;
}
return false;
}
2024-04-22 18:40:27 +02:00
public boolean skipCommentOrWhitespace(@NotNull StringReader line) {
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
boolean skipped = false;
while (this.skipComment(line) || line.skipWhitespace() > 0) {
skipped = true;
}
return skipped;
}
2024-04-22 18:40:27 +02:00
public boolean trySkipCommentOrWhitespaceUntil(@NotNull StringReader line, char terminator) {
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
int previousCursor = line.getCursor();
boolean skipped = this.skipCommentOrWhitespace(line);
if (skipped && line.canRead() && line.peek() != terminator) {
line.setCursor(previousCursor);
skipped = false;
}
return skipped;
}
2024-04-22 18:40:27 +02:00
public boolean peekSingleLineComment(@NotNull StringReader line) {
return line.canRead(2) && line.peek() == '/' && line.peek(1) == '/';
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
}
2024-04-22 18:40:27 +02:00
public boolean consumeImports(@NotNull StringReader line, @NotNull ImportCollector collector) {
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
outerLoop:
while (line.canRead()) {
IterativeStep step;
while ((step = this.stepManager.getSteps().poll()) != null) {
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
step.run(line, this);
if (!line.canRead()) {
break outerLoop;
}
}
if (this.skipCommentOrWhitespace(line)) {
continue;
}
if (this.peekSingleLineComment(line)) {
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
// check single line comment only after multi line to avoid ignoring the end of multi line comment starting with // on the newline
break;
}
// not commented
char c = line.peek();
2024-04-22 18:40:27 +02:00
if (AnnotationSkipSteps.canStart(c)) { // handle annotation with param to avoid open curly bracket that occur in array argument
this.stepManager.enqueue(new AnnotationSkipSteps());
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
continue;
} else if (c == '{') {
return true;
2024-04-22 18:40:27 +02:00
} else if (ImportStatementSteps.canStart(line)) {
this.stepManager.enqueue(new ImportStatementSteps(collector));
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
continue;
}
line.skip();
}
return false;
}
2024-04-22 18:40:27 +02:00
@NotNull
public StepManager getSteps() {
return this.stepManager;
improve source parsing to be more accurate The previous way had several flaw and was not really future proof. This new system is based on an iterative string reader similar to how brigadier parse arguments which simplify a lot of things tracking the cursor internally. The comments and javadoc are now properly skipped to prevent commented line to be counted as an import. The first scope detection to stop the parser once all imports have been processed now detect the first '{' char uncommented. Previously it only worked for api gen and was really restrictive since it detected the following pattern: <class type> <class name> as a workaround. The annotations are also skipped to prevent curly bracket that could be in it to be counted too (this include the arguments as well). Type name extended on multi line are also supported and checked at runtime as defined in the JLS: https://docs.oracle.com/javase/specs/jls/se17/html/jls-3.html#jls-3.8 For fully qualified name, each part separated by a dot are checked. Keyword are also checked to prevent @interface to be considered as a regular annotation when it's more a class definition. Whitespace and comments in between type name are stripped to collect the right import in the end. Annotation between the package name and the class name are not supported but is not needed for import and annotation since they are forbidden by the compiler. Multi line stuff is handled using a list of tasks to process in the right order for the imports or annotations with two types of task: repeat and basic. This new system is now used in the marker comment detection and in the old generated code test. There's also a lot of tests to make sure no regression happen in the future that can be enabled by inluding the tag 'parser' (disabled in CI runner by default). Additionnaly marker stub are not anymore allowed by default while parsing import, annotation and metadata stuff but can be reenabled with the system property 'paper.generator.rewriter.searchMarkerInMetadata' for experimental/testing purpose. Short type name resolution (based on the collected imports) has been redone to supports static star import and inner class type referenced either implicitly or via import.
2024-04-11 18:50:09 +02:00
}
}