Skip to content
This repository was archived by the owner on Jul 28, 2021. It is now read-only.

[v1.1.0] #7

Merged
merged 24 commits into from
Jun 16, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
4521d87
Improve file write handling.
ShindouMihou May 23, 2021
e7de376
Implement update checker scheduler (only if config allows).
ShindouMihou May 23, 2021
9fe3c7f
Preloading, Aggregation, Configurable buffer and text size.
ShindouMihou May 23, 2021
e408652
More safer write method.
ShindouMihou May 23, 2021
5e7b5dd
Shutdown hook that saves the files.
ShindouMihou May 23, 2021
98350de
Update README.md
ShindouMihou May 23, 2021
75adee1
Optimize Imports (IntelliJ)
ShindouMihou May 23, 2021
de558dd
Add customizable heartbeat interval.
ShindouMihou May 23, 2021
ee7b05c
Bump pom.xml to 1.0.5
ShindouMihou May 23, 2021
9cd13c6
Fix something in README.md
ShindouMihou May 23, 2021
1c282df
Version 1.2 - automatically compress all old files to GZIP.
ShindouMihou May 24, 2021
50f956e
Remove an unnecessary log.
ShindouMihou May 24, 2021
e33e49e
Fix issue with UpdateListener only accepting STRING as values.
ShindouMihou May 24, 2021
85d034b
Add support for console commands.
ShindouMihou May 24, 2021
8694b82
FIX All overloaded methods should be placed next to each other.
ShindouMihou May 24, 2021
dd656f4
Bump to 1.0.7
ShindouMihou May 24, 2021
e54c8a2
Merge remote-tracking branch 'origin/master' into master
ShindouMihou May 24, 2021
2fc0dba
Add fastutil, caffeine and logback-classic (important for speed chang…
ShindouMihou Jun 3, 2021
b2d2e1d
[v1.1.0-DEV]
ShindouMihou Jun 3, 2021
d662428
Bump to v1.1.0
ShindouMihou Jun 3, 2021
50c461f
Merge branch 'master' into development
ShindouMihou Jun 3, 2021
ed120c7
Enforce Authorization header, Fixes malfunctioning write and more thr…
ShindouMihou Jun 11, 2021
61ecee3
Fixes?
ShindouMihou Jun 11, 2021
7e0d76b
Fixes some critical issue with any write methods not replying to requ…
ShindouMihou Jun 16, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 40 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,12 +90,15 @@ If you want to quickly get up and running with your application then feel free t
Sending requests is also straightforward with the database, everything is on JSON format.
Also, everything is sent via webhooks which means the address to use is something like: `ws://127.0.0.1:5995`

## Notice
All requests towards RoseDB v1.1.0 must have the Authorization header which is validated at connection, this is not something
that will be backward compatible since we want to enforce this as soon as possible.

**GET REQUESTS**

To send a GET request, you can do:
```json
{
"authorization": "Authorization here",
"method": "get",
"database": "Database here",
"collection": "Collection Here",
Expand All @@ -107,7 +110,7 @@ To send a GET request, you can do:
It should reply with:
```json
{
"response": "{//entire json data here}",
"response": "json",
"kode": 1,
"replyTo": "Unique identifier from request"
}
Expand All @@ -120,7 +123,6 @@ To send a AGGREGATE request, there are two ways:
* Collection aggregation sample request:
```json
{
"authorization": "Authorization here",
"method":"aggregate",
"database":"rose_db",
"collection":"test",
Expand All @@ -131,7 +133,6 @@ To send a AGGREGATE request, there are two ways:
* Database aggregation sample request:
```json
{
"authorization": "Authorization here",
"method":"aggregate",
"database":"rose_db",
"unique":"Unique Identifier Here"
Expand Down Expand Up @@ -174,7 +175,6 @@ To send a AGGREGATE request, there are two ways:
* UPDATE and ADD are both the same except UPDATE uses `"method":"update"`
```json
{
"authorization": "Authorization here",
"method": "add",
"database": "Database here",
"collection": "Collection Here",
Expand All @@ -187,7 +187,6 @@ To send a AGGREGATE request, there are two ways:
Update also allows you to update (and add) values and keys, for example:
```json
{
"authorization": "ca72b368-0c4a-4a73-9e4c-9e22474b359c",
"method": "update",
"database": "rose_db",
"collection": "mana",
Expand Down Expand Up @@ -220,7 +219,6 @@ It should reply with something like:
To send a DELETE request, you can do:
```json
{
"authorization": "Authorization here",
"method": "delete",
"database": "Database here",
"collection": "Collection Here",
Expand All @@ -241,7 +239,6 @@ It should reply with:
Similar to UPDATE requests, you can also delete multiple or single keys from the data, for example:
```json
{
"authorization": "ca72b368-0c4a-4a73-9e4c-9e22474b359c",
"method": "delete",
"database": "rose_db",
"collection": "mana",
Expand Down Expand Up @@ -272,7 +269,6 @@ There are two ways for DROP requests, one is for dropping collections and the ot
Example of a collection drop
```json
{
"authorization": "8a4b93a0-a6d8-4403-a44f-5cff82a537e5",
"method": "drop",
"database": "rose_db",
"collection": "Mana",
Expand All @@ -292,7 +288,6 @@ It should reply with:
Example of a database drop:
```json
{
"authorization": "8a4b93a0-a6d8-4403-a44f-5cff82a537e5",
"method": "drop",
"database": "rose_db",
"unique": "Unique identifier to receive from callback."
Expand All @@ -308,6 +303,41 @@ The expected response should be
}
```

**Revert Request**
You can revert an add or update request by simply sending a revert request to the server which looks like:
```
{
"method": "revert",
"database": "rose_db",
"collection": "mana",
"idenitifer": "revert_test",
"unique": "5"
}
```

The response of this request will be the last version of the file.
```
{
"response": "{\"test\":1}",
"kode": 1,
"replyTo": "Unique ID here."
}
```

You can also disable this feature via the config for maybe a little performance boost but I doubt it would add much, disabling the feature
would then give out this reply to any attempt to revert:
```
{
"response": "This method is unsupported by the server.",
"kode": 0,
"replyTo": "Unique ID here"
}
```

Limitations of revert:
* The versions are saved on application-level cache and is dumped when the application is closed normally and not abruptly.
* Each item is limited to one version and will be overriden each time an *ADD* or *UPDATE* request is sent that will override the item.

## Image Examples
* Collection Drop: ![collection drop](https://media.discordapp.net/attachments/731377154817916939/845257934480343040/unknown.png)
* Database Drop: ![database drop](https://media.discordapp.net/attachments/731377154817916939/845257852083109928/unknown.png)
Expand Down
17 changes: 11 additions & 6 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

<groupId>pw.mihou</groupId>
<artifactId>RoseDB</artifactId>
<version>1.0.7</version>
<version>1.1.0</version>
<build>
<plugins>
<plugin>
Expand Down Expand Up @@ -72,13 +72,18 @@
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.8.0-beta4</version>
<artifactId>slf4j-api</artifactId>
<version>1.7.30</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.8.0-beta4</version>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>1.2.3</version>
</dependency>
<dependency>
<groupId>com.github.ben-manes.caffeine</groupId>
<artifactId>caffeine</artifactId>
<version>3.0.2</version>
</dependency>
</dependencies>

Expand Down
72 changes: 50 additions & 22 deletions src/main/java/pw/mihou/rosedb/RoseDB.java
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
package pw.mihou.rosedb;

import ch.qos.logback.classic.Level;
import ch.qos.logback.classic.Logger;
import org.json.JSONException;
import org.json.JSONObject;
import org.slf4j.LoggerFactory;
import pw.mihou.rosedb.connections.RoseServer;
import pw.mihou.rosedb.enums.Levels;
import pw.mihou.rosedb.io.FileHandler;
import pw.mihou.rosedb.io.Scheduler;
import pw.mihou.rosedb.utility.ColorPalette;
import pw.mihou.rosedb.utility.Terminal;
import pw.mihou.rosedb.utility.UpdateChecker;
import java.io.File;
Expand All @@ -24,16 +28,37 @@ public class RoseDB {
public static boolean preload;
public static int buffer;
public static int heartbeat;
public static boolean versioning;
public static int size;

public static void main(String[] args) throws URISyntaxException {
System.out.println(" ______ ______ ______ ______ _____ ______ \n" +
"/\\ == \\/\\ __ \\/\\ ___\\/\\ ___\\/\\ __-./\\ == \\ \n" +
"\\ \\ __<\\ \\ \\/\\ \\ \\___ \\ \\ __\\\\ \\ \\/\\ \\ \\ __< \n" +
" \\ \\_\\ \\_\\ \\_____\\/\\_____\\ \\_____\\ \\____-\\ \\_____\\ \n" +
" \\/_/ /_/\\/_____/\\/_____/\\/_____/\\/____/ \\/_____/");

Terminal.setLoggingLevel(Levels.ERROR);
String vanity = "\tr ______ ______ ______ ______ _____ ______ \n" +
"\tb/\\ == \\/\\ __ \\/\\ ___\\/\\ ___\\/\\ __-./\\ == \\ \n" +
"\ty\\ \\ __<\\ \\ \\/\\ \\ \\___ \\ \\ __\\\\ \\ \\/\\ \\ \\ __< \n" +
"\tc \\ \\_\\ \\_\\ \\_____\\/\\_____\\ \\_____\\ \\____-\\ \\_____\\ \n" +
"\tr \\/_/ /_/\\/_____/\\/_____/\\/_____/\\/____/ \\/_____/n";
vanity = vanity.replaceAll("r", ColorPalette.ANSI_RED)
.replaceAll("g", ColorPalette.ANSI_GREEN)
.replaceAll("b", ColorPalette.ANSI_BLUE)
.replaceAll("y", ColorPalette.ANSI_YELLOW)
.replaceAll("c", ColorPalette.ANSI_CYAN)
.replaceAll("n", ColorPalette.ANSI_RESET);
System.out.println(vanity);
System.out.printf("Version: %s, Build: %s, Configuration Version: %s, Creator: %s\n", UpdateChecker.VERSION, UpdateChecker.BUILD, UpdateChecker.CONFIG_VERSION, "Shindou Mihou");

((Logger) LoggerFactory.getLogger("io.javalin.Javalin")).setLevel(Level.ERROR);
((Logger) LoggerFactory.getLogger("org.eclipse.jetty.util.log")).setLevel(Level.ERROR);
System.setProperty("org.eclipse.jetty.util.log.class", "org.eclipse.jetty.util.log.StdErrLog");
System.setProperty("org.eclipse.jetty.LEVEL", "OFF");
System.setProperty("org.eclipse.jetty.util.log.announce", "false");
((Logger) LoggerFactory.getLogger("io.javalin.Javalin")).setLevel(Level.ERROR);
((Logger) LoggerFactory.getLogger("org.eclipse.jetty.util.log")).setLevel(Level.ERROR);
((Logger) LoggerFactory.getLogger("org.eclipse.jetty")).setLevel(Level.ERROR);
((Logger) LoggerFactory.getLogger("o.e.jetty.io")).setLevel(Level.ERROR);
((Logger) LoggerFactory.getLogger("o.e.j.w.common")).setLevel(Level.ERROR);


Terminal.setLoggingLevel(Level.ERROR);
if (!new File("config.json").exists()) {
FileHandler.writeToFile("config.json", new JSONObject()
.put("directory", new File(RoseDB.class.getProtectionDomain().getCodeSource().getLocation().toURI().getPath()).getParentFile().getPath() + File.separator + "Database" + File.separator)
Expand All @@ -45,6 +70,7 @@ public static void main(String[] args) throws URISyntaxException {
.put("preload", true)
.put("maxTextMessageBufferSizeMB", 5)
.put("maxTextMessageSizeMB", 5)
.put("versioning", true)
.put("heartbeatIntervalSeconds", 30)
.put("configVersion", UpdateChecker.CONFIG_VERSION)
.toString()).join();
Expand Down Expand Up @@ -79,6 +105,7 @@ public static void main(String[] args) throws URISyntaxException {
buffer = config.getInt("maxTextMessageBufferSizeMB");
size = config.getInt("maxTextMessageSizeMB");
heartbeat = config.getInt("heartbeatIntervalSeconds");
versioning = config.getBoolean("versioning");

if(heartbeat > 300 || heartbeat < 25){
Terminal.log(Levels.ERROR, "Minimum heartbeat interval should be at 25 seconds to prevent overloading clients.");
Expand All @@ -98,7 +125,7 @@ public static void main(String[] args) throws URISyntaxException {
boolean mkdirs = new File(directory).mkdirs();

if(!mkdirs){
Terminal.log(Levels.ERROR, "We couldn't create the folders on " + directory);
Terminal.log(Levels.ERROR, "We couldn't create the folders on {}", directory);
}
}

Expand All @@ -113,32 +140,32 @@ public static void main(String[] args) throws URISyntaxException {

FileHandler.setDirectory(directory);
RoseServer.run(port);

} else {
Terminal.log(Levels.ERROR, "Rose cannot write on read or read on " + directory);
Terminal.log(Levels.ERROR, "Rose cannot write on read or read on {}", directory);
}
} catch (JSONException e){
Terminal.log(Levels.ERROR, e.getMessage());
Terminal.log(Levels.ERROR, "An error from this side is caused by a misconfiguration of config.json, please fix your config.json.");
} catch (ArithmeticException e){
Terminal.log(Levels.ERROR, e.getMessage());
Terminal.log(Levels.ERROR, "If this exception was thrown at the start, please check your config.json whether everything meets 32 bit integer limit.");
} catch (JSONException | ArithmeticException e){
Terminal.log(Levels.ERROR, "An error occurred, if this is sent from startup, " +
"please check config.json otherwise please send an issue at https://github.com/ShindouMihou/RoseDB/issues." +
"\nAvailable Processors (cores): {}, Free memory (MB): {}, Maximum Memory allocated to JVM (MB): {}, JDK Version: {}, JDK Vendor: {}, OS: {}, Architecture: {}, OS Version: {}" +
".\n\nError: {}",
Runtime.getRuntime().availableProcessors(), (Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()) / (1000 * 1000),
Runtime.getRuntime().totalMemory(), System.getProperty("java.vm.version"), System.getProperty("java.vm.vendor"),
System.getProperty("os.name"), System.getProperty("os.arch"), System.getProperty("os.version"), e.getMessage());
}

}

private static void startUpdateChecker(){
Scheduler.schedule(() -> {
boolean update = UpdateChecker.check();
if(update){
if(UpdateChecker.check()){
Terminal.log(Levels.INFO, "There is a newer version of RoseDB available, please update on https://github.com/ShindouMihou/RoseDB/releases");
}
}, 0, 12, TimeUnit.HOURS);
}

public static Levels rootLevel(String configValue) {
return configValue.equalsIgnoreCase("DEBUG") ? Levels.DEBUG : (configValue.equalsIgnoreCase("INFO") ? Levels.INFO :
(configValue.equalsIgnoreCase("ERROR") ? Levels.ERROR : Levels.WARNING));
public static Level rootLevel(String configValue) {
return configValue.equalsIgnoreCase("DEBUG") ? Level.DEBUG : (configValue.equalsIgnoreCase("INFO") ? Level.INFO :
(configValue.equalsIgnoreCase("ERROR") ? Level.ERROR : Level.WARN));
}

private static JSONObject updateConfig(JSONObject original) throws URISyntaxException, JSONException {
Expand All @@ -148,8 +175,9 @@ private static JSONObject updateConfig(JSONObject original) throws URISyntaxExce
.put("authorization", Optional.ofNullable(original.getString("authorization")).orElse(UUID.randomUUID().toString()))
.put("loggingLevel", Optional.ofNullable(original.getString("loggingLevel")).orElse("INFO"))
.put("cores", original.isNull("cores") ? 1 : original.getInt("cores"))
.put("updateChecker", original.isNull("updateChecker") ? true : original.getBoolean("updateChecker"))
.put("updateChecker", original.isNull("updateChecker") || original.getBoolean("updateChecker"))
.put("preload", true)
.put("versioning", true)
.put("maxTextMessageBufferSizeMB", original.isNull("maxTextMessageBufferSizeMB") ? 5 : original.getInt("maxTextMessageBufferSizeMB"))
.put("maxTextMessageSizeMB", original.isNull("maxTextMessageSizeMB") ? 5 : original.getInt("maxTextMessageSizeMB"))
.put("heartbeatIntervalSeconds", original.isNull("heartbeatIntervalSeconds") ? 30 : original.getInt("heartbeatIntervalSeconds"))
Expand Down
Loading