Architectural Decision Records
This page attempts to cover how the project has progressed to where it is today, decisions made along the way that have shaped the development to better or worse.
This page was first written on 2018-08-23, so decisions made before that are post fact.
Last Modified: 2019-10-20
1.0.0 - 1.6.2
How the project began - December 2016
Project began on December 1st as a first Java project for Rsl.
The plugin would display information about a single player combined from different plugins in one command. No data was stored on Plan.
The plugin also calculated averages for different values with
OfflinePlayerlist.
2.0.0 (28th December 2016 - 16th January 2017)
Goals:
- Display statistics on a website
- Host a webserver on the plugin
- Start recording own data
📌 Chart Visualization: Google Chart API
Until 2.4.0
Problem: Gamemode piechart and online player graph needed to be added.
Solution: Google Chart API.Due to lack of knowledge of JavaScript programming language, an old Java library was chosen to perform data visualization. This library was already deprecated, but the term was not understood at the time. This API used URL parameters to source data for image graphs.
Impact: 13 days after release visualization stopped working due to URL size growing past the limitations.
📌 Web Backend: Java ServerSocket
Until 3.5.2
Problem: Host a Java webserver with small file size
Solution: Java ServerSocketImpact: Due to improper closing of resources the website functioned extremely poorly, and required two refreshes to load the page, leading to swap of server code later.
📌 Server side rendering
Until 5.0 - Still partially present
Problem: Serve HTML from webserver
Solution: Server side Java html creationUse html "rendered" to a String from data held in an object cache each time a request was made on the server.
Html files are first read from the jar, then placeholders are replaced with values.Impact: Html generation is dependent on the server running on the plugin, or files generated by the plugin. Client side implementations will take a lot more effort in the future.
Technical debt: Large amount of code is dedicated to server side rendering.
📌 Database support: SQLite & MySQL
Problem: Storage for gathered data
Solution: SQL databases, used Fé-plugin as example.Impact: SQLite and MySQL are great first choice for a database. Asynchronous execution is required. SQLite has limitations in table alterations, which lead to some version transition bugs.
📌 Cache: All gathered data
Until 3.0.0
Problem: Can't remember
Solution: Cache that stores player data until they log outImpact: Data loss on server shutdown, not so good use of memory
2.1.0 - 2.3.1 (17th - 30th January 2017)
Development goal: Bugfixes & Improvements
📌 Backups: SQLite
Problem: Database needs to be backed up
Solution: Write code that creates new SQLite database as backupImpact: SQLite has limitations in table alterations, leading to version transition bugs in the future.
2.4.0 - 2.6.2 (31st Jan - 22nd February 2017)
Goals:
- Make the page more visually appealing
- Provide more data
📌 Data: Sessions
Impact: Riddled with issues related to not understanding how to calculate things in programming, and how to save sessions when the server goes down (all now fixed), these small objects turned out to be a great addition that is now heavily relied on the plugin. Possible drawback from them is the sheer number of them could cause performance impacts in the future.
📌 Security by obscurity: Randomized code to access pages
Until 3.5.2
Problem: Users do not want their players to view the server page
Solution: Place a randomly generated code in the addressImpact: Not a very good solution. Eventually removed after introduction actual user authentication.
📌 Javascript: Single page scroller
Problem: Everything does not fit on one page
Solution: Navigation scrolls the page horizontally while rest is hiddenImpact: Single page to load for multiple tabs of information - less scrolling. Causes loading times to increase.
2.7.0 - 2.8.4 (23rd Feb - 15th March 2017)
📌 Chart Visualization: ChartJS
Until 3.6.0
Problem: Google Chart Java library does not work (URL length problems)
Solution: ChartJSDue to old visualization running out of character space in URL specification, the graphs stopped working, so a new graphing library was needed. After some searching ChartJS found to be a viable option. The script was served via CDN and all graph data was replaced into the html files via placeholders.
Impact: While amazing compared to the old graphs (and very good quality), ChartJS started to lag the websites around 10k points in the graphs. This lead to point reduction algorithm later.
- 📌 Icons: Font Awesome 4
📌 Html: Cards
Problem: Everything does not fit on one column
Solution: Split tabs in 2 with css float property (Later replaced by Bootstrap grid)
2.9.0 - 3.0.2 (16th Mar - 9th May 2017)
📌 Data: Store on event
Problem: Data loss on server shutdown
Solution: Store data as soon as it is possibleA vital change to data gathering was made, where when an event fired, the data gained from that was saved to the database as soon as possible.
Impact: Data loss reduced
📌 Chart Visualization: plotly.js (World Map)
Until 3.6.0
Problem: ChartJS does not support world maps
Solution: plotly.jsImpact: Due to poor documentation, the map is limited to a 300x300 pixel square, which leads to it's replacement
3.1.0 - 3.5.5 (10th May - 26nd July 2017)
📌 Display Data of Plugins via direct Java calls
Until 5.0
Problem: Addition of data from Plugins to the website
Solution: PluginData APIImpact: While great for displaying data of other plugins, this leads to requirement for Bungee - Server connections to obtain data. This dependency on server specific plugin data turns out to be a very difficult dependency to remove.
The API was eventually replaced with DataExtension API that stores the data in the database.
3.6.0 (26th Jul - 1st August 2017)
📌 Web Backend: sun.java.HttpServer
Problem: Java ServerSocket does not directly support HTTPS
Solution: sun.java.HttpServerImpact: Higher performance, HTTPS & more reliable web server, that can use multiple threads & offers high customizability via code. Because this relies on sun.java code, the webserver may one day stop working if Oracle decides to remove this code from the JRE. The Webserver uses
HTTP 1.1implementation so speed improvements from newer specifications are not possible without change of backend.
📌 User Authentication: Basic Access Authentication
Problem: Authenticating users
Solution: Basic Access AuthenticationImpact: Easy to implement, but limited to HTTPS connections.
📌 Web Backend: HTTPS with Java Keystores
Problem: Basic Access Authentication needs HTTPS
Solution: sun.java.HttpServerImpact: HTTPS implementation requires Java Keystore to be used, which is difficult due to bad tutorials. Situation is later improved with PKCS12 support.
📌 Chart Visualization: HighCharts
Problem: ChartJS starts lagging around 10k points in line graph
Solution: HighCharts found via bStatsImpact: Very good performance compared to ChartJS and Plotly. Free for non-commercial projects, which prevents Plan from becoming premium unless a license fee is paid.
3.6.2 - 3.6.4 (2nd - 18th August 2017)
📌 Cache: Guava
Until 4.1.5
Impact: 1.7.X servers become unsupported because Guava 10 is not shaded into Plan
📌 Locale uses Enum for Keys
Until 4.4.2
Impact: Due to default values being absent from the locale keys, it becomes difficult to add new messages.
4.0.0 (19th Aug - 1st October 2017)
Goal: Support BungeeCord networks
📌 Plugin Framework: Abstract Plugin Framework
Problem: Platforms perform same things with different classes
Solution: Abstraction library for different platformsImpact: Hard to change code in the abstraction framework if Plan doesn't work because of it.
📌 Schema change: Based on server table
Problem: Old schema is not compatible with multiple servers in same database
Solution: Change schemaImpact: Multiple servers are supported, more join operations in queries. Most of old data has to be discarded due to changed schema
🔒 Dev decision: No changes that jepardize release cycle
Problem: JFrog Artifactory in use becomes unavailable and builds start failing
Solution: Do not allow 3rd party repositories for critical dependenciesDo not accept unreliable changes to vital parts of the release cycle, such as addition of Jenkins, snapshot repositories or PRs that remove all files at some point in their commits
- 📌 MySQL Connection pooling: DHCP2
📌 Generic Processing Queue
Problem: BukkitScheduler is not available on Bungee
Solution: Java implementation for async execution of runnablesImpact: Reduced reliance on BukkitScheduler, Increased runtime complexity
📌 Proxy<->Server Connection System
Until 5.0
Problem: Server pages can not be rendered on Bungee
Solution: HTTP based page sharing protocolIn order to provide all server pages of the bukkit servers, a way to get the pages to bungee and requests to bukkit was required. (Since the Server page rendering part was dependent on Bukkit at the time.) Plugin messaging channel seemed unreliable due to player online requirement, so decision was made to use the in place WebServer instead.
Impact: Increased set-up complexity, difficult to remove later
4.0.1 - 4.0.7 (2nd Oct - 13 November 2017)
Goal: Bugfixes to network functionality
4.1.0 (14 Nov - 2nd December 2017)
Goal: Improve visual aspect of the website.
📌 Template: Admin BSB
Until 5.0
Problem: CSS is a lot of work
Solution: A ready made template: Admin BSBImpact: New features such as searchable player table. Increases server page load time. More professional look brings more users, Lock in to Bootstrap 3.
4.1.1 - 4.1.7 (3rd Dec 2017 - 2nd March 2018)
📌 Transfer pages via MySQL
Problem: Proxy<->Server connection system is unreliable
Solution: Transfer pages via MySQLImpact: MySQL database size on disk goes through the roof, connection reliance is still there because requests need to be transferred in some way between the servers. Ultimately reverted, but setting transfer is kept around.
- 📌 Removed Google Guava usage
📌 Refactor plugin to use static getters
Until 4.5.0
Problem: Plugin is difficult to maintain
Solution: Refactor based on ideas from Kingdoms source codeThis decision came from exposure to Kingdoms API, which uses
getInstance()methods that seemed convenient.
Implementation on Plan does not work, because on Kindoms the modules are singletons, on Plan they are not (Testing multiple systems interacting is not possible with Singletons).Impact: Increased usage of static everywhere and this lead to a lot of code smells further down the line. Testing is difficult.
4.2.0 (2nd May - 7th April 2018)
Goal: Support Sponge
4.3.0 - 4.3.2 (3rd Apr - 31st May 2018)
Goal: Comply with GDPR (Anonymize IPs)
4.4.0 - 4.4.4 (1st June - 23rd August 2018)
Goal: Make it easier to develop Plan
📌 Data: DataStore object model for Analysis
Until 5.0
Problem: Analysis of data is difficult to extend
Solution: New objects that work on Key - Supplier model, objects don't know what they contain.Impact: Addition of new analysis is easy using Mutators. Memory usage is increased a lot. The change is not as useful as it first seemed as it was not really needed. - It was not possible to reuse some of the code when rewriting analysis again due to the memory consumption issues.
📌 Move Inspect page generation to BungeeCord
Problem: Proxy<->Server connection reliance
Solution: Move inspect page to BungeeCordImpact: BungeeCord servers start crashing due to a hidden bug in java UrlConnection timeouts being set with System properties. This takes a month to hunt down, and a lot of architectural changes are made as attempted fixes.
This bug was hidden before, because Bungee made less connections per day.
📌 Locale contains default values
Problem: It is difficult to add new locale messages
Solution: Add default values to Locale keysImpact: Easier to update locale.
📌 MySQL Connection pooling: HikariCP
Problem: Bungee is crashing due to thread starvation/socket leaks Solution: Swap MySQL connection pool
Impact: Performance increase, no impact on the bug
📌 Proxy<->Server Connections: Apache HttpClient
Until 5.0
Problem: Bungee is crashing due to thread starvation/socket leaks
Solution: Swap HTTP client in connection systemImpact: Better API, no impact on the bug. Bug later fixed with two system properties
📌 Database patching
Problem: Database schema changes are tracked by a number
Solution: Dedicaded code that checks if the database has newest schemaImpact: More reliable database changes from version to version, allows patching after a patch fails.
4.5.0 (11 August 2018 - 27 October 2018)
Goal: Remove static getInstance methods (Code is too tightly coupled)
📌 Dependency Injection: Dagger
Problem: It is difficult to keep track of dependencies
Solution: DaggerImpact: Code will be easier to maintain because class dependencies are clear. Changes revealed many points where Single Responsibility Principle is not followed, which will lead to further refactoring & easier to understand code down the line. Harder to onboard new developers.
- 🔒 Move to LGPLv3 license from proprietary license
4.5.1 - 4.5.2 (28 october 2018 - 2 December 2018)
📌 Split Project into modules
Problem: Different logging libraries in Sponge and Bukkit needs Plan to split in two artifacts to be deployed
Solution: Split project into platform based modules to properly relocate dependencies based on platform.Impact: All platforms are now packaged into a single jar. Maven does not like Dagger with multi-module builds and testing becomes extremely tedious because
mvn clean packagehas to be run before every test run.
4.6.0 - 4.6.2 (3 December 2018 - 4 February 2019)
📌 Build System: Maven -> Gradle
Problem: Maven does not like Dagger with multi-module builds and testing is extremely tedious
Solution: Change to GradleImpact: Improved IDE compatibility and testing before each commit is now a viable workflow again.
- 🔒 Write more tests, especially when reproducing issues.
4.7.0 - 4.7.2 (4 February 2019 - 10 March 2019)
- 📌 Refactored Database to use Transactions
- 📌 Prevent queries with a lock before patch transactions are finished
Impact: Increased database reliability and reduced exceptions from unpatched database schema. Code for queries now easier to manage.
4.8.0 - 4.8.6 (10 March 2019 - 30 June 2019)
📌 Replace PluginData API with DataExtension API
Problem: Proxy<->Server connection system reliance
Solution: DataExtension API that stores plugin data in DatabaseImpact: Writing extensions no longer requires Proxy<->Server connection. API is more verbose and takes longer to get a grasp to. Adding support to new plugins with the API is easier. Dynamic generation is more difficult.
5.0 (15 June 2019 - ongoing)
📌 Partial client side rendering (JSON+jQuery)
Problem: High reliance on server side rendering
Solution: Serve data with JSONImpact: Most queries required a rewrite.
📌 Template: SBAdmin 2
Problem: Site uses Bootstrap 3
Solution: Change to SBAdmin 2 template
📌 Removal of proxy<->server connection system
Problem: Connection system causes large overhead on networks
Solution: Remove the connection systemLarge prepwork had to be done before the removal, removing all dependencies of the system, such as PluginData API.
Impact: Easier istallation
📌 Restructuring packages according to data flow
Problem: Some classes are in packages that don't make sense
Solution: Split packages based on data flowHere is a draft of the structure proposal https://github.com/plan-player-analytics/Plan/issues/1021#issuecomment-526286750
Impact: Other plugins using internal classes in Plan break, might delay some people from updating. Data classes can be split by domain and gathering and delivery objects can be separated.
Reality: Some classes from the old structure are still used in packages of the "wrong" domain, such as
SessionandTaskSystem