Increased API Timeout to 60s and further minor improvements (#850)

* feat: increased http request timeouts from 30s to 60s

* feat: increased http request timeouts from 30s to 60s

* feat: added supoprt status for different database implementations

* chore: some clenaups

* chore: some clenaups

* feat: sqlite became default now

* chore: change a message

* chore: review changes

README.md

@@ -83,7 +83,7 @@ you don't want to connect to the database while running the node. Furthermore, t
 __Sqlite__ is just like H2 a file based (embedded) database. Signum Node builds up the entire database out of the box and does not require any further set up.
 This makes Sqlite an interesting choice especially for less-technical users who just want to start and/or run a local (not publicly accessible) node. Choose this, if you want to run just a local node without public exposure. 
 Furthermore, the resulting database file is easily shareable, such others can use a snapshot and sync from there.
-Sqlite is still experimental, and considered as a substitute for H2.
+Sqlite is considered as a replacement for H2.
 
 __MariaDB__ and __PostgreSQL__ on the other hand require an additional set-up. It is the better choice for publicly accessible nodes, 
 as they are considered more stable, especially under higher load. 
@@ -93,15 +93,15 @@ The performance hit for MariaDB and PostgreSQL is related to the TCP/IP connecti
 Due to that model concurrent access is possible, i.e. one can run an additional service against the same database, which is not possible with H2, as the file gets locked.
 
 |            | Stability | Speed | Setup | Backup | Concurrency | Purpose                          |
-|------------|-----------|------|-------|--------|-------------|----------------------------------|
-| H2         | - (1)     | ⭐⭐⭐   | ⭐⭐⭐    | ⭐⭐⭐       | ❌           | Local Node                       |  
-| Sqlite     | ⭐⭐ (2)    | ⭐⭐   | ⭐⭐⭐    | ⭐⭐⭐       | ✅ (3)       | Local Node                       |  
-| MariaDB    | ⭐⭐        | ⭐    | ⭐     | ⭐       | ✅           | Public Node, Additional Services |  
-| PostgreSQL | ⭐⭐ (2)    | ⭐    | ⭐     | ⭐       | ✅           | Public Node, Additional Services |  
+|------------|----------|-------|-------|--------|-------------|----------------------------------|
+| ~~H2~~     | - (1)    | ⭐⭐⭐   | ⭐⭐⭐    | ⭐⭐⭐       | ❌           | Local Node [DEPRECATED]          |  
+| Sqlite     | ⭐⭐   | ⭐⭐*   | ⭐⭐⭐    | ⭐⭐⭐       | ✅ (3)       | Local Node                       |  
+| MariaDB    | ⭐⭐       | ⭐     | ⭐     | ⭐       | ✅           | Public Node, Additional Services |  
+| PostgreSQL | ⭐⭐ (2)   | ⭐     | ⭐     | ⭐       | ✅           | Public Node, Additional Services |  
 
-> (1) H2 - even with updated Version 2 - has proven to be unstable and causing database issues - mostly File I/O exceptions, and data inconsistencies. 
+> (1) DEPRECATED - H2 - even with updated Version 2 - has proven to be unstable and causing database issues - mostly File I/O exceptions, OOM and data inconsistencies. It will be removed with 3.9, so move to sqlite
 
-> (2) PostgreSQL and Sqlite support is still experimental. So, stability needs to be proven over time, but in general Postgres itself is as least stable/reliable as MariaDB.
+> (2) PostgreSQL support is still experimental. So, stability needs to be proven over time, but in general Postgres itself is as least stable/reliable as MariaDB.
 
 > (3) Sqlite supports concurrent reading. Writing is still limited to one writer at a time.
 ---- 
@@ -146,9 +146,9 @@ Note: the example ID above `8952122635653861124` is the [SNA](https://www.sna.si
 
 The cash-back is paid from block `1,029,000.`
 
-**H2/MariaDB**
+**SQLite/MariaDB**
 
-By default Signum Node is using H2 (file based) as database. 
+By default Signum Node is using SQLite (file based) as database. 
 If you like to use MariaDB you will need to adjust your `conf/node.properties`:
 
 ```properties

conf/mainnet/node.h2.properties

@@ -1,10 +1,21 @@
+#################################################################################################
+#
+#       H2 is deprecated....Please stop using H2 (in mainnet) and move to Sqlite for local nodes
+#       For public nodes go with MariaDB or Postgres (as of 3.8.4 still experimental)
+#
+#
+#       This file is kept only for backwards compatibility
+#
+#################################################################################################
+
+
 #### API SERVER ####
 API.Listen = 0.0.0.0
 API.allowed = *
 
 #### DATABASE ####
 # Reduces the database file size dramatically, if on... but needs time on shutdown
-# Db.H2.DefragOnShutdown is deprecated and now Db.Optimize
+# Db.H2. DefragOnShutdown is deprecated and now Db.Optimize
 DB.Optimize = on
 # Database connection JDBC url
 DB.Url=jdbc:h2:file:/db/signum;DB_CLOSE_ON_EXIT=FALSE

conf/node-default.properties

@@ -27,19 +27,36 @@
 
 # node.cashBackId = 8952122635653861124
 
-#### DATABASE BACKEND SELECTION ####
+#### DATABASE SECTION ####
 
-## We are using the H2 file based as default with no username/password.
-# DB.Url=jdbc:h2:file:./db/signum-v2;DB_CLOSE_ON_EXIT=FALSE
+## Maximum allowed connections by Database connection pool
+# DB.Connections = 30
 
-## If you want to use Sqlite:
+## If a database, i.e. SQLITE, supports optimization features, like SHUTDOWN DEFRAG, VACUUM etc. This might require some additional shutdown or startup time (depending on the database).
+## This can help improve performance and reduce the size of the database file on disk.
+## This is "on" by default but can cost some time while starting/shutting down.
+# DB.Optimize = off
+
+## If you want to use Sqlite (recommended for local/non-public nodes):
 # DB.Url=jdbc:sqlite:file:./db/signum.sqlite.db
 
-## If you want to use MariaDB:
+## Sqlite Journaling Mode
+## https://www.sqlite.org/pragma.html#pragma_journal_mode
+## Possible Values are: DELETE,TRUNCATE,PERSIST,WAL (default, recommended)
+## WAL may occupy up to twice more disc space than others while running the node, but allows read concurrency and usually better performance (see more here: https://www.sqlite.org/wal.html)
+## It's highly recommended to use WAL mode during syncing, to dramatically reduce I/O operations and though faster sync times.
+## Info: MEMORY journal mode is not supported
+# DB.SqliteJournalMode = WAL
 
+## If you want to use MariaDB (recommended for public nodes):
 # DB.Url=jdbc:mariadb://localhost:3306/signum
-# DB.Username=signum_user
-# DB.Password=S!gnumP@$S
+# DB.Username=signumnode
+# DB.Password=s1gn00m_n0d3
+
+## If you want to use Postgres (Experimental - considered alternative for MariaDB):
+# DB.Url=jdbc:postgresql://localhost:5432/signum?sslMode=disabled
+# DB.Username=signumnode
+# DB.Password=s1gn00m_n0d3
 
 #### PEER 2 PEER NETWORKING ####
 
@@ -260,7 +277,7 @@
 
 ## Idle timeout for http/json API request connections, milliseconds.
 
-# API.ServerIdleTimeout = 30000
+# API.ServerIdleTimeout = 60000
 
 ## Directory with html and javascript files for the new client UI, and admin tools utilizing
 ## the http/json API.
@@ -288,28 +305,7 @@
 
 #### DATABASE ####
 
-## Number of concurrent connections to the Database
-
-# DB.Connections = 30
-
-## Make H2 database defrag and compact when shutting down.
-## This is "on" by default but can cost some time while shutting down.
-
-# @deprecated - Use Db.Optimize
-# Db.H2.DefragOnShutdown = on
-
-## If a database, i.e. H2 or SQLITE, supports optimization features, like SHUTDOWN DEFRAG, VACUUM etc. This might require some additional shutdown or startup time (depending on the database).
-## This can help improve performance and reduce the size of the database file on disk.
-## This is "on" by default but can cost some time while starting/shutting down.
-# DB.Optimize = off
 
-## Sqlite Journaling Mode
-## https://www.sqlite.org/pragma.html#pragma_journal_mode
-## Possible Values are: DELETE,TRUNCATE,PERSIST,WAL (default, recommended)
-## WAL may occupy up to twice more disc space than others while running the node, but allows read concurrency and usually better performance (see more here: https://www.sqlite.org/wal.html)
-## It's highly recommended to use WAL mode during syncing, to dramatically reduce I/O operations and though faster sync times.
-## Info: MEMORY journal mode is not supported
-# DB.SqliteJournalMode = WAL
 
 ## Enable trimming of derived objects tables.
 

conf/testnet/node.h2.properties

@@ -1,3 +1,13 @@
+#################################################################################################
+#
+#       H2 is deprecated....Please stop using H2 and move to Sqlite for local nodes
+#       For public nodes go with MariaDB or Postgres (as of 3.8.4 still experimental)
+#
+#
+#       This file is kept only for backwards compatibility
+#
+#################################################################################################
+
 node.network = signum.net.TestnetNetwork
 
 #### API SERVER ####

src/brs/db/sql/Db.java

@@ -251,8 +251,8 @@ private static String getDatabaseVersion() {
       Record record = queryVersion.fetchOne();
       if (record != null) {
         version = record.get(0, String.class);
-        if (!databaseInstance.isStable()) {
-          version += " (EXPERIMENTAL)";
+        if (databaseInstance.getSupportStatus() != DatabaseInstance.SupportStatus.STABLE) {
+          version += " (" + databaseInstance.getSupportStatus().toString() + ")";
         }
       }
     } catch (Exception e) {

src/brs/db/sql/dialects/DatabaseInstance.java

@@ -4,7 +4,14 @@
 import com.zaxxer.hikari.HikariDataSource;
 import org.jooq.SQLDialect;
 
+
+
 public interface DatabaseInstance {
+  enum SupportStatus {
+    STABLE,
+    DEPRECATED,
+    EXPERIMENTAL
+  }
   void onStartup();
   void onShutdown();
   HikariConfig getConfig();
@@ -13,5 +20,5 @@ public interface DatabaseInstance {
   String getMigrationClassPath();
   String getDatabaseVersionSQLScript();
   SQLDialect getDialect();
-  boolean isStable();
+  SupportStatus getSupportStatus();
 }

src/brs/db/sql/dialects/DatabaseInstanceBaseImpl.java

@@ -11,6 +11,7 @@
 import java.sql.SQLException;
 import java.sql.Statement;
 
+
 public abstract class DatabaseInstanceBaseImpl implements DatabaseInstance {
   private static final Logger logger = LoggerFactory.getLogger(DatabaseInstanceBaseImpl.class);
 

src/brs/db/sql/dialects/DatabaseInstanceH2.java

@@ -44,20 +44,19 @@ protected void onShutdownImpl() {
 
   @Override
   protected void onStartupImpl() {
-    logger.warn("H2 shows certain instabilities and is not recommended for use in production, i.e. public nodes anymore.");
-    logger.warn("We recommend to use MariaDB instead. At the an experimental SQLite alternative is being provided.");
+    logger.warn("### DEPRECATION NOTICE ###");
+    logger.warn("H2 shows instabilities causing database corruptions and is not recommended for use in MainNet anymore.");
+    logger.warn("We recommend to use SQLite as a stable alternative");
+    logger.warn("--------------------------");
+    logger.warn("### H2 SUPPORT WILL BE REMOVED IN VERSION 3.9 ###");
+    logger.warn("--------------------------");
   }
 
   @Override
   public SQLDialect getDialect() {
     return SQLDialect.H2;
   }
 
-
-  @Override
-  public void onStartup() {
-  }
-
   @Override
   public String getMigrationSqlScriptPath() {
     return "classpath:/db/migration_h2_v2";
@@ -69,7 +68,7 @@ public String getDatabaseVersionSQLScript() {
   }
 
   @Override
-  public boolean isStable() {
-    return true;
+  public SupportStatus getSupportStatus() {
+    return SupportStatus.DEPRECATED;
   }
 }

src/brs/db/sql/dialects/DatabaseInstanceMariaDb.java

@@ -54,7 +54,7 @@ public String getDatabaseVersionSQLScript() {
   }
 
   @Override
-  public boolean isStable() {
-    return true;
+  public SupportStatus getSupportStatus() {
+    return SupportStatus.STABLE;
   }
 }

src/brs/db/sql/dialects/DatabaseInstancePostgres.java

@@ -41,7 +41,7 @@ public String getDatabaseVersionSQLScript() {
   }
 
   @Override
-  public boolean isStable() {
-    return false;
+  public SupportStatus getSupportStatus() {
+    return SupportStatus.EXPERIMENTAL;
   }
 }

src/brs/db/sql/dialects/DatabaseInstanceSqlite.java

@@ -115,7 +115,6 @@ protected void onStartupImpl() {
 
   }
 
-
   @Override
   public String getMigrationSqlScriptPath() {
     return "classpath:/db/migration_sqlite";
@@ -127,7 +126,7 @@ public String getDatabaseVersionSQLScript() {
   }
 
   @Override
-  public boolean isStable() {
-    return false;
+  public SupportStatus getSupportStatus() {
+    return SupportStatus.STABLE;
   }
 }

src/brs/props/Props.java

@@ -109,8 +109,7 @@ public class Props {
 
   // DB options
   public static final Prop<Boolean> DB_SKIP_CHECK  = new Prop<>("DB.SkipCheck", false);
-  // TODO: change this when SQLITE turns out to be stable
-  public static final Prop<String> DB_URL          = new Prop<>("DB.Url", "jdbc:h2:file:./db/signum-v2;DB_CLOSE_ON_EXIT=FALSE");
+  public static final Prop<String> DB_URL          = new Prop<>("DB.Url", "jdbc:sqlite:file:./db/signum.sqlite.db");
   public static final Prop<String> DB_USERNAME     = new Prop<>("DB.Username", "");
   public static final Prop<String> DB_PASSWORD     = new Prop<>("DB.Password", "");
   public static final Prop<Integer> DB_CONNECTIONS  = new Prop<>("DB.Connections", 30);
@@ -179,7 +178,7 @@ public class Props {
   public static final Prop<String> API_SSL_KEY_STORE_PATH     = new Prop<>("API.SSL_keyStorePath", "keystore");
   public static final Prop<String> API_SSL_KEY_STORE_PASSWORD = new Prop<>("API.SSL_keyStorePassword", "password");
   public static final Prop<String> API_SSL_LETSENCRYPT_PATH = new Prop<>("API.SSL_letsencryptPath", "");
-  public static final Prop<Integer> API_SERVER_IDLE_TIMEOUT = new Prop<>("API.ServerIdleTimeout", 30000);
+  public static final Prop<Integer> API_SERVER_IDLE_TIMEOUT = new Prop<>("API.ServerIdleTimeout", 60_000);
   public static final Prop<Boolean> API_SERVER_ENFORCE_POST = new Prop<>("API.ServerEnforcePOST", true);
   public static final Prop<String> API_ALLOWED_ORIGINS = new Prop<>("API.AllowedOrigins", "*");
 

src/signum/net/MockNetwork.java

@@ -29,10 +29,9 @@ public MockNetwork() {
     setProperty(Props.PK2_BLOCK_HEIGHT, "0");
     setProperty(Props.SMART_ALIASES_HEIGHT, "0");
     setProperty(Props.DEV_NEXT_FORK_BLOCK_HEIGHT, "0");
-    
+
     setProperty(Props.BRS_CHECKPOINT_HEIGHT, "-1");
     setProperty(Props.BRS_CHECKPOINT_HASH, "");
-
-    setProperty(Props.DB_URL, "jdbc:h2:file:./db/signum-mock;DB_CLOSE_ON_EXIT=FALSE");
+    setProperty(Props.DB_URL, "jdbc:sqlite:file:./db/signum.sqlite-mock.db");
   }
 }

src/signum/net/TestnetNetwork.java

@@ -64,8 +64,6 @@ public TestnetNetwork() {
 
     setProperty(Props.P2P_USE_PEERS_DB, "false");
     setProperty(Props.P2P_SAVE_PEERS, "false");
-
-    setProperty(Props.DB_URL, "jdbc:h2:file:./db/signum-testnet-v2;DB_CLOSE_ON_EXIT=FALSE");
-
+    setProperty(Props.DB_URL, "jdbc:sqlite:file:./db/signum.sqlite-testnet.db");
   }
 }