tigase.sh install-schema (#822)

Activities

wojciech.kapcia@tigase.net opened 8 years ago

Due Date
2017-05-30

This is a follow-up to #4941#note-23 - in addition to upgrade-schema and setting-up schema via web-up we need a way to setup schema from commandline (like DBSchemaLoader was doing) - you could probably could re-use existing, recent commandline parser

Andrzej Wójcik (Tigase) commented 8 years ago

Feature implemented as a new task install-schema. I've reused code from tigase-http-api setup module and schema loaders to create this task.

wojciech.kapcia@tigase.net commented 8 years ago

I think that it should handle following cases:

we have defined data-source in init.properties and skip parameters - load schema there with defaults;
we explicitly define parameters (db type, components, location, etc) - it should load schema to the defined location
we decide to use interactive mode - it should ask for the all possible entries, and as defaults use what is defined in init.properties - if available.

Problems summary:

lack of any control where to install schema - neither via init.properties nor via parameters
doubled logs (this also applies to web-installer - looks like logger is defined twice)

Details

Currently it doesn't seem to work intuitively/following available information:

If we have defined the repositories in init.properties therefore simple install-schema should install it to the defined locations - doesn't seem to work:

wojtek@atlantiscity.local ~/dev/tigase/tigase-server/server $ ./scripts/tigase.sh  install-schema  etc/tigase.conf
Upgrading database schema:
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
java.lang.IllegalArgumentException: You are missing arguments! Missing are: [dbType]
	at tigase.util.ui.console.ParameterParser.parseArgs(ParameterParser.java:452)
	at tigase.db.util.SchemaManager.execute(SchemaManager.java:144)
	at tigase.db.util.SchemaManager.main(SchemaManager.java:117)

wojtek@atlantiscity.local ~/dev/tigase/tigase-server/server $ ./scripts/tigase.sh  install-schema  etc/tigase.conf -T derby
Upgrading database schema: -T derby
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
LogLevel: CONFIG
2017-05-16 11:51:33.640 [main]             DBSchemaLoader.init()                   CONFIG:   Parameters: tigase.db.util.DBSchemaLoader$Parameters@334b392d
may 16, 2017 11:51:33 AM tigase.util.DNSResolverDefault <init>
ADVERTENCIA: Resolving default host name: localhost took: 10
may 16, 2017 11:51:34 AM tigase.sys.TigaseRuntime <init>
INFORMACIÓN: Using PS Eden Space memory pool for reporting eden memory usage.
may 16, 2017 11:51:34 AM tigase.sys.TigaseRuntime <init>
INFORMACIÓN: Using PS Survivor Space memory pool for reporting survivor memory usage.
may 16, 2017 11:51:34 AM tigase.sys.TigaseRuntime <init>
INFORMACIÓN: Using PS Old Gen memory pool for reporting (old) memory usage.
may 16, 2017 11:51:34 AM tigase.db.util.SchemaManager loadSchemas
INFORMACIÓN: found 1 data sources to upgrade...
may 16, 2017 11:51:34 AM tigase.db.util.SchemaManager loadSchemas
INFORMACIÓN: begining upgrade...
LogLevel: CONFIG
2017-05-16 11:51:34.791 [main]             DBSchemaLoader.init()                   CONFIG:   Parameters: tigase.db.util.DBSchemaLoader$Parameters@751686aa
2017-05-16 11:51:34.791 [main]             DBSchemaLoader.init()                   CONFIG:   Parameters: tigase.db.util.DBSchemaLoader$Parameters@751686aa
2017-05-16 11:51:34.791 [main]             DBSchemaLoader.validateDBConnection()   INFO:     Validating DBConnection, URI: jdbc:derby:tigasedb;create=true
2017-05-16 11:51:34.791 [main]             DBSchemaLoader.validateDBConnection()   INFO:     Validating DBConnection, URI: jdbc:derby:tigasedb;create=true
2017-05-16 11:51:35.125 [main]             DBSchemaLoader.validateDBConnection()   CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@559e3f67, jTDS 1.3.1, com.mysql.jdbc.Driver@436a563f, com.mysql.fabric.jdbc.FabricMySQLDriver@6475e778, org.postgresql.Driver@2a54c92e]
2017-05-16 11:51:35.125 [main]             DBSchemaLoader.validateDBConnection()   CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@559e3f67, jTDS 1.3.1, com.mysql.jdbc.Driver@436a563f, com.mysql.fabric.jdbc.FabricMySQLDriver@6475e778, org.postgresql.Driver@2a54c92e]
2017-05-16 11:51:35.126 [main]             DBSchemaLoader.validateDBConnection()   INFO:     Connection OK
2017-05-16 11:51:35.126 [main]             DBSchemaLoader.validateDBConnection()   INFO:     Connection OK
2017-05-16 11:51:35.127 [main]             DBSchemaLoader.validateDBExists()       INFO:     Validating whether DB Exists, URI: jdbc:derby:tigasedb;create=true
2017-05-16 11:51:35.127 [main]             DBSchemaLoader.validateDBExists()       INFO:     Validating whether DB Exists, URI: jdbc:derby:tigasedb;create=true
2017-05-16 11:51:35.130 [main]             DBSchemaLoader.withConnection()         CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@559e3f67, jTDS 1.3.1, com.mysql.jdbc.Driver@436a563f, com.mysql.fabric.jdbc.FabricMySQLDriver@6475e778, org.postgresql.Driver@2a54c92e]
2017-05-16 11:51:35.130 [main]             DBSchemaLoader.withConnection()         CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@559e3f67, jTDS 1.3.1, com.mysql.jdbc.Driver@436a563f, com.mysql.fabric.jdbc.FabricMySQLDriver@6475e778, org.postgresql.Driver@2a54c92e]
2017-05-16 11:51:35.130 [main]             DBSchemaLoader.lambda$validateDBExists$2()  INFO: Exists OK
2017-05-16 11:51:35.130 [main]             DBSchemaLoader.lambda$validateDBExists$2()  INFO: Exists OK
2017-05-16 11:51:35.133 [main]             DBSchemaLoader.loadSchemaFile()         INFO:     Loading schema from file(s): database/derby-schema-7-2.sql, URI: jdbc:derby:tigasedb;create=true
2017-05-16 11:51:35.133 [main]             DBSchemaLoader.loadSchemaFile()         INFO:     Loading schema from file(s): database/derby-schema-7-2.sql, URI: jdbc:derby:tigasedb;create=true
2017-05-16 11:51:35.135 [main]             DBSchemaLoader.withConnection()         CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@559e3f67, jTDS 1.3.1, com.mysql.jdbc.Driver@436a563f, com.mysql.fabric.jdbc.FabricMySQLDriver@6475e778, org.postgresql.Driver@2a54c92e]
2017-05-16 11:51:35.135 [main]             DBSchemaLoader.withConnection()         CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@559e3f67, jTDS 1.3.1, com.mysql.jdbc.Driver@436a563f, com.mysql.fabric.jdbc.FabricMySQLDriver@6475e778, org.postgresql.Driver@2a54c92e]
2017-05-16 11:51:35.295 [main]             DBSchemaLoader.lambda$loadSchemaFile$6()  SEVERE: Failed to execute query: create table tig_users ( uid BIGINT GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, user_id varchar(2049) NOT NULL, user_pw varchar(255) default NULL, acc_create_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP, last_login timestamp, last_logout timestamp, online_status int default 0, failed_logins int default 0, account_status int default 1 )
2017-05-16 11:51:35.295 [main]             DBSchemaLoader.lambda$loadSchemaFile$6()  SEVERE: Failed to execute query: create table tig_users ( uid BIGINT GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, user_id varchar(2049) NOT NULL, user_pw varchar(255) default NULL, acc_create_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP, last_login timestamp, last_logout timestamp, online_status int default 0, failed_logins int default 0, account_status int default 1 )
2017-05-16 11:51:35.296 [main]             DBSchemaLoader.withConnection()         SEVERE:


=====
Failure: Table/View 'TIG_USERS' already exists in Schema 'APP'.
=====


2017-05-16 11:51:35.296 [main]             DBSchemaLoader.withConnection()         SEVERE:


=====
Failure: Table/View 'TIG_USERS' already exists in Schema 'APP'.
=====


2017-05-16 11:51:35.297 [main]             DBSchemaLoader.addXmppAdminAccount()    WARNING:  Database schema is invalid
2017-05-16 11:51:35.297 [main]             DBSchemaLoader.addXmppAdminAccount()    WARNING:  Database schema is invalid
2017-05-16 11:51:35.298 [main]             DBSchemaLoader.postInstallation()       WARNING:  Database schema is invalid
2017-05-16 11:51:35.298 [main]             DBSchemaLoader.postInstallation()       WARNING:  Database schema is invalid
2017-05-16 11:51:35.298 [main]             DBSchemaLoader.shutdownDerby()          INFO:     Validating DBConnection, URI: jdbc:derby:tigasedb;create=true
2017-05-16 11:51:35.298 [main]             DBSchemaLoader.shutdownDerby()          INFO:     Validating DBConnection, URI: jdbc:derby:tigasedb;create=true
2017-05-16 11:51:35.315 [main]             DBSchemaLoader.withConnection()         SEVERE:


=====
Failure: Database 'tigasedb' shutdown.
=====


2017-05-16 11:51:35.315 [main]             DBSchemaLoader.withConnection()         SEVERE:


=====
Failure: Database 'tigasedb' shutdown.
=====


may 16, 2017 11:51:35 AM tigase.db.util.SchemaManager loadSchemas
INFORMACIÓN: schema upgrade finished!




  =============================================================================
  	Schema installation finished

  Data source: default with uri jdbc:derby:tigasedb;create=true
  	Checking connection to database	ok
  	Checking if database exists	ok
  	Loading schema: Tigase XMPP Server (Core), version: 7.2.0	error
  		Message: Loading schema from file(s): database/derby-schema-7-2.sql, URI: jdbc:derby:tigasedb;create=true
  		         DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@559e3f67, jTDS 1.3.1, com.mysql.jdbc.Driver@436a563f, com.mysql.fabric.jdbc.FabricMySQLDriver@6475e778, org.postgresql.Driver@2a54c92e]
  		         Failed to execute query: create table tig_users ( uid BIGINT GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY, user_id varchar(2049) NOT NULL, user_pw varchar(255) default NULL, acc_create_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP, last_login timestamp, last_logout timestamp, online_status int default 0, failed_logins int default 0, account_status int default 1 )



  		         =====
  		         Failure: Table/View 'TIG_USERS' already exists in Schema 'APP'.
  		         =====
  	Adding XMPP admin accounts	error
  		Message: Database schema is invalid
  	Post installation action	error
  		Message: Database schema is invalid

  Example init.properties configuration file:

  --debug = server
  --virt-hosts = localhost
  'config-type' = 'default'
  dataSource () {
      default () {
          uri = 'jdbc:derby:tigasedb;create=true'
      }
  }
  =============================================================================

dataSource () {
    default () {
        uri = 'jdbc:derby:tigasedb2;create=true'
    }
}

help doesn't list supported options so it's impossible to know what to do/to enter:

wojtek@atlantiscity.local ~/dev/tigase/tigase-server/server $ ./scripts/tigase.sh  install-schema  etc/tigase.conf --help
Upgrading database schema: --help
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Usage:
	$ java -cp "path_to_binary.jar" some.package.Class [task] [options]
		if the option defines default then <value> is optional

Tasks:
	upgrade-schema	-	Upgrade schema of databases specified in your config file
	install-schema	-	Install schema to database
	destroy-schema	-	Destroy database and schemas

Options:

	--help,  (optional)
		Print the help

	-I, --interactive,  (optional)
		Enable interactive mode, which will result in prompting for missing parameters

	-C value, --components=value,  (optional)
		List of enabled components identifiers to load schema for (default: amp,bosh,pubsub,c2s,monitor,ws2s,s2s,eventbus,http,muc,message-archive)

	-T value, --dbType=value,
		Database server type, possible values: [derby, jdbc, mysql, postgresql, sqlserver]

in interactive mode it asks only about components and dbtype - it's impossible to specify credentials/location/name:

wojtek@atlantiscity.local ~/dev/tigase/tigase-server/server $ ./scripts/tigase.sh  install-schema  etc/tigase.conf -I
Upgrading database schema: -I
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Using interactive mode for: [help, components, dbType], skipped options will use default if available
components [amp,bosh,pubsub,c2s,monitor,ws2s,s2s,eventbus,http,muc,message-archive]:
dbType: derby
LogLevel: CONFIG
2017-05-16 12:02:00.462 [main]             DBSchemaLoader.init()                   CONFIG:   Parameters: tigase.db.util.DBSchemaLoader$Parameters@334b392d
may 16, 2017 12:02:00 PM tigase.util.DNSResolverDefault <init>
ADVERTENCIA: Resolving default host name: localhost took: 15
may 16, 2017 12:02:01 PM tigase.sys.TigaseRuntime <init>
INFORMACIÓN: Using PS Eden Space memory pool for reporting eden memory usage.
may 16, 2017 12:02:01 PM tigase.sys.TigaseRuntime <init>
INFORMACIÓN: Using PS Survivor Space memory pool for reporting survivor memory usage.
may 16, 2017 12:02:01 PM tigase.sys.TigaseRuntime <init>
INFORMACIÓN: Using PS Old Gen memory pool for reporting (old) memory usage.
may 16, 2017 12:02:01 PM tigase.db.util.SchemaManager loadSchemas
INFORMACIÓN: found 1 data sources to upgrade...
may 16, 2017 12:02:01 PM tigase.db.util.SchemaManager loadSchemas
INFORMACIÓN: begining upgrade...
^C

Currently it seems it's impossible to specify where you wan't to install the schema - it seems to always use the defaults. Previously you could specify where you wan't to install the schema:

wojtek@atlantiscity.local ~/dev/tmps/tigase-server-dists/tigase-issue #7.2.0-SNAPSHOT-b4803 $ java -cp "jars/*" tigase.db.util.DBSchemaLoader
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Usage:
	$ java -cp "path_to_binary.jar" some.package.Class [options]
		if the option defines default then <value> is optional

Options:

	--help,  (optional)
		Print the help

	-I, --interactive,  (optional)
		Enable interactive mode, which will result in prompting for missing parameters

	-T value, --dbType=value,
		Database server type, possible values: [derby, mysql, postgresql, sqlserver] (default: mysql)

	-V value, --schemaVersion=value,
		Intended version of the schema to be loaded, possible values: [4, 5, 5-1, 7-1, 7-2] (default: 7-2)

	-D value, --dbName=value,
		Name of the database that will be created and to which schema will be loaded (default: tigasedb)

	-H value, --dbHostname=value,
		Address of the database instance (default: localhost)

	-U value, --dbUser=value,
		Name of the user that will be created specifically to access Tigase XMPP Server (default: tigase_user)

	-P value, --dbPass=value,
		Password of the user that will be created specifically to access Tigase XMPP Server (default: tigase_pass)

	-R value, --rootUser=value,
		Database root account username used to create tigase user and database (default: root)

	-A value, --rootPass=value,
		Database root account password used to create tigase user and database (default: root)

	-F value, --file=value,  (optional)
		Comma separated list of SQL files that will be processed

	-Q value, --query=value,  (optional)
		Custom query to be executed

	-L value, --logLevel=value,  (optional)
		Java Logger level during loading process (default: CONFIG)

	-S, --useSSL,  (optional)
		Enable SSL support for database connection (if database supports it) (default: false)

	-J value, --adminJID=value,  (optional)
		Comma separated list of administrator JID(s)

	-N value, --adminJIDpass=value,  (optional)
		Password that will be used for the entered JID(s) - one for all configured administrators

	--getURI,  (optional)
		Generate database URI (default: false)

	--ignoreMissingFiles=value,  (optional)
		Force ignoring missing files errors (default: false)

wojtek@atlantiscity.local ~/dev/tmps/tigase-server-dists/tigase-issue #7.2.0-SNAPSHOT-b4803 $

wojciech.kapcia@tigase.net commented 8 years ago

Scratch that... a bit.

OK, it's possible to specify all parameters but one has to use -T to specify database and -I mode - could be handy to mention it somewhere.

It could be also helpful to allow specifying the parameter list as well as giving information what those parameters are.

Previous 3 points still applies as suggestions.

wojciech.kapcia@tigase.net commented 8 years ago

Would be nice - proper list of parameters in the log:

2017-05-16 12:26:44.629 [main]             DBSchemaLoader.init()                   CONFIG:   Parameters: tigase.db.util.DBSchemaLoader$Parameters@334b392d

Andrzej Wójcik (Tigase) commented 8 years ago

I've improved an output of --help parameter as we discussed. Rest is working fine on newest build. I was not able to replicate duplicate log entries, most likely it works fine on a proper build of Tigase XMPP Server.

I still need to fix the output of parameters.

Andrzej Wójcik (Tigase) commented 8 years ago

Fixed presentation of parameters passed to SchemaLoader

wojciech.kapcia@tigase.net commented 7 years ago

I've tried basic form (with supposed detection of the parameters from config file?) and with following in @etc/init.properties@:

'dataSource' {
    'default' {
        'uri' = 'jdbc:mysql://localhost:3306/tigasedb?user=tigase&password=tigase&useUnicode=true&characterEncoding=UTF-8&autoCreateUser=true&useSSL=false'
    }
}

'muc' (active: true) {}
'pubsub' (active: true) {}

  	Schema installation finished

  Data source: default with uri jdbc:mysql://localhost/tigasedb?user=tigase_user&password=tigase_pass&useSSL=false
  	Checking connection to database	ok
  	Checking if database exists	ok
  	Loading schema: Tigase XMPP Server (Core), version: 7.2.0	ok
  	Adding XMPP admin accounts	warning
  		Message: Error: No admin users entered
  	Post installation action	ok

wojciech.kapcia@tigase.net commented 7 years ago

What's more, "components" parameter is way off:

it lists all components (even if they don't have schema), for example "bosh"
providing some components (eg.: "muc,pubsub") results in really weird suggested config

  Example init.properties configuration file:

  --debug = server
  --virt-hosts = localhost
  'config-type' = 'default'
  dataSource () {
      default () {
          uri = 'jdbc:mysql://localhost/tigasedb?user=tigase_user&password=tigase_pass&useSSL=false'
      }
  }
  amp (active: false) {}
  bosh (active: false) {}
  c2s (active: false) {}
  eventbus (active: false) {}
  http (active: false) {}
  'message-archive' (active: false) {}
  monitor (active: false) {}
  s2s (active: false) {}
  ws2s (active: false) {}

Andrzej Wójcik (Tigase) commented 7 years ago

This is the same as a component list which is used by setup. I cannot guess which components have their own schema, I need to generate config with them and then check if any of loaded classes implements data source aware interface.

In fact, you should pass all components which you expect to use, not only components for which you want to load schema, as a user is not aware which components have their own schema. A user should pass components which he needs and we should take care of the rest.

Suggested config is OK. It applied all thing you asked it for. You wanted PubSub and MUC (which are enabled by default - skipped in config), and you have not specified that you want c2s,bosh,ws2s - as a result, those components are disabled in a config.

I think that you expected this task to install schema for particular components and that's it, while it is more powerful and could be used almost as a replacement for a web based setup.

In my opinion, this task works ok but should be properly described in the documentation.

wojciech.kapcia@tigase.net commented 7 years ago

OK, there is a (now correctly, we discussed it previously but it slipped my mind) detailed description of the tasks:

upgrade-schema - Upgrade schema of databases specified in your config file - it's not possible to specify parameters

install-schema - Install schema to database - it requires specifying database parameters where schema will be installed (config file will be ignored)

destroy-schema - Destroy database and schemas (DANGEROUS)

Hence only first/third option detects configuration, and install-schema requires all information.

Andrzej Wójcik wrote:

This is the same as a component list which is used by setup. I cannot guess which components have their own schema, I need to generate config with them and then check if any of loaded classes implements data source aware interface.

In fact, you should pass all components which you expect to use, not only components for which you want to load schema, as a user is not aware which components have their own schema. A user should pass components which he needs and we should take care of the rest.

Suggested config is OK. It applied all thing you asked it for. You wanted PubSub and MUC (which are enabled by default - skipped in config), and you have not specified that you want c2s,bosh,ws2s - as a result, those components are disabled in a config.

I think that you expected this task to install schema for particular components and that's it, while it is more powerful and could be used almost as a replacement for a web based setup.

In my opinion, this task works ok but should be properly described in the documentation.

The task is called install-schema (or @upgrade-schema@) hence I would imagine that anyone running it would expect it to handle schema-related operations (and wouldn't care much that it is more powerful). What's more, current description reads:

-C value, --components=value, (optional)

  List of enabled components identifiers to load schema for (default: amp,bosh,pubsub,c2s,monitor,ws2s,s2s,eventbus,http,muc,message-archive)

which informs, that schema will be loaded for the listed components. In addition users tends to avoid details and when faced with "Example init.properties configuration file:" they will use it resulting in confusion when server fails to operates correctly.

I would say, that it would make more sense (while running from console), in case of using -C components option, to:

load basic schema
load schema for configured components
prepare example confing which would include database configuration and configuration related to enabling listed components, leaving out rest (i.e. not assuming that lack of them equals disabling them).

At any rate, it doesn't seem to work for me:

wojtek@atlantiscity.local ~/dev/tigase/tigase-server/server $ ./scripts/tigase.sh install-schema etc/my_tigase.conf -T mysql -D tigasedbtest -U tigasetest -P tigasetest -C "muc,pubsub"

LogLevel: CONFIG
2017-05-30 13:33:30.615 [main]             DBSchemaLoader.init()                   CONFIG:   Parameters: [ingoreMissingFiles: false, logLevel: CONFIG, adminPassword: null, admins: null, dbRootPass: root, dbRootUser: root, dbType: mysql, dbName: tigasedbtest, dbHostname: localhost, dbUser: tigasetest, dbPass: tigasetest, useSSL: false, file: null, query: null]
may 30, 2017 1:33:30 PM tigase.util.DNSResolverDefault <init>
ADVERTENCIA: Resolving default host name: localhost took: 15
may 30, 2017 1:33:31 PM tigase.db.util.SchemaManager loadSchemas
INFORMACIÓN: found 1 data sources to upgrade...
may 30, 2017 1:33:31 PM tigase.db.util.SchemaManager loadSchemas
INFORMACIÓN: begining upgrade...
LogLevel: CONFIG
2017-05-30 13:33:31.542 [main]             DBSchemaLoader.init()                   CONFIG:   Parameters: [ingoreMissingFiles: false, logLevel: CONFIG, adminPassword: null, admins: null, dbRootPass: root, dbRootUser: root, dbType: mysql, dbName: tigasedbtest, dbHostname: localhost, dbUser: tigasetest, dbPass: tigasetest, useSSL: false, file: null, query: null]
2017-05-30 13:33:31.542 [main]             DBSchemaLoader.init()                   CONFIG:   Parameters: [ingoreMissingFiles: false, logLevel: CONFIG, adminPassword: null, admins: null, dbRootPass: root, dbRootUser: root, dbType: mysql, dbName: tigasedbtest, dbHostname: localhost, dbUser: tigasetest, dbPass: tigasetest, useSSL: false, file: null, query: null]
2017-05-30 13:33:31.543 [main]             DBSchemaLoader.validateDBConnection()   INFO:     Validating DBConnection, URI: jdbc:mysql://localhost/?user=root&password=root&useSSL=false
2017-05-30 13:33:31.543 [main]             DBSchemaLoader.validateDBConnection()   INFO:     Validating DBConnection, URI: jdbc:mysql://localhost/?user=root&password=root&useSSL=false
2017-05-30 13:33:32.006 [main]             DBSchemaLoader.validateDBConnection()   CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@bb50de6, jTDS 1.3.1, com.mysql.jdbc.Driver@702b656a, com.mysql.fabric.jdbc.FabricMySQLDriver@5c41b8d8, org.postgresql.Driver@218f2f51]
2017-05-30 13:33:32.006 [main]             DBSchemaLoader.validateDBConnection()   CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@bb50de6, jTDS 1.3.1, com.mysql.jdbc.Driver@702b656a, com.mysql.fabric.jdbc.FabricMySQLDriver@5c41b8d8, org.postgresql.Driver@218f2f51]
2017-05-30 13:33:32.008 [main]             DBSchemaLoader.validateDBConnection()   INFO:     Connection OK
2017-05-30 13:33:32.008 [main]             DBSchemaLoader.validateDBConnection()   INFO:     Connection OK
2017-05-30 13:33:32.009 [main]             DBSchemaLoader.validateDBExists()       INFO:     Validating whether DB Exists, URI: jdbc:mysql://localhost/tigasedbtest?user=tigasetest&password=tigasetest&useSSL=false
2017-05-30 13:33:32.009 [main]             DBSchemaLoader.validateDBExists()       INFO:     Validating whether DB Exists, URI: jdbc:mysql://localhost/tigasedbtest?user=tigasetest&password=tigasetest&useSSL=false
2017-05-30 13:33:32.031 [main]             DBSchemaLoader.withConnection()         CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@bb50de6, jTDS 1.3.1, com.mysql.jdbc.Driver@702b656a, com.mysql.fabric.jdbc.FabricMySQLDriver@5c41b8d8, org.postgresql.Driver@218f2f51]
2017-05-30 13:33:32.031 [main]             DBSchemaLoader.withConnection()         CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@bb50de6, jTDS 1.3.1, com.mysql.jdbc.Driver@702b656a, com.mysql.fabric.jdbc.FabricMySQLDriver@5c41b8d8, org.postgresql.Driver@218f2f51]
2017-05-30 13:33:32.076 [main]             DBSchemaLoader.lambda$null$3()          INFO:      OK
2017-05-30 13:33:32.076 [main]             DBSchemaLoader.lambda$null$3()          INFO:      OK
2017-05-30 13:33:32.078 [main]             DBSchemaLoader.loadSchemaFile()         INFO:     Loading schema from file(s): database/mysql-schema-7-2.sql, URI: jdbc:mysql://localhost/tigasedbtest?user=root&password=root&useSSL=false
2017-05-30 13:33:32.078 [main]             DBSchemaLoader.loadSchemaFile()         INFO:     Loading schema from file(s): database/mysql-schema-7-2.sql, URI: jdbc:mysql://localhost/tigasedbtest?user=root&password=root&useSSL=false
2017-05-30 13:33:32.086 [main]             DBSchemaLoader.withConnection()         CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@bb50de6, jTDS 1.3.1, com.mysql.jdbc.Driver@702b656a, com.mysql.fabric.jdbc.FabricMySQLDriver@5c41b8d8, org.postgresql.Driver@218f2f51]
2017-05-30 13:33:32.086 [main]             DBSchemaLoader.withConnection()         CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@bb50de6, jTDS 1.3.1, com.mysql.jdbc.Driver@702b656a, com.mysql.fabric.jdbc.FabricMySQLDriver@5c41b8d8, org.postgresql.Driver@218f2f51]
2017-05-30 13:33:32.413 [main]             DBSchemaLoader.lambda$loadSchemaFile$6()  INFO:    completed OK
2017-05-30 13:33:32.413 [main]             DBSchemaLoader.lambda$loadSchemaFile$6()  INFO:    completed OK
2017-05-30 13:33:32.414 [main]             DBSchemaLoader.addXmppAdminAccount()    WARNING:  Error: No admin users entered
2017-05-30 13:33:32.414 [main]             DBSchemaLoader.addXmppAdminAccount()    WARNING:  Error: No admin users entered
2017-05-30 13:33:32.415 [main]             DBSchemaLoader.postInstallation()       INFO:     Post Installation, URI: jdbc:mysql://localhost/tigasedbtest?user=root&password=root&useSSL=false
2017-05-30 13:33:32.415 [main]             DBSchemaLoader.postInstallation()       INFO:     Post Installation, URI: jdbc:mysql://localhost/tigasedbtest?user=root&password=root&useSSL=false
2017-05-30 13:33:32.420 [main]             DBSchemaLoader.withConnection()         CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@bb50de6, jTDS 1.3.1, com.mysql.jdbc.Driver@702b656a, com.mysql.fabric.jdbc.FabricMySQLDriver@5c41b8d8, org.postgresql.Driver@218f2f51]
2017-05-30 13:33:32.420 [main]             DBSchemaLoader.withConnection()         CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@bb50de6, jTDS 1.3.1, com.mysql.jdbc.Driver@702b656a, com.mysql.fabric.jdbc.FabricMySQLDriver@5c41b8d8, org.postgresql.Driver@218f2f51]
2017-05-30 13:33:32.420 [main]             DBSchemaLoader.lambda$postInstallation$5()  INFO: Finalizing...
2017-05-30 13:33:32.420 [main]             DBSchemaLoader.lambda$postInstallation$5()  INFO: Finalizing...
2017-05-30 13:33:32.421 [main]             DBSchemaLoader.lambda$postInstallation$5()  INFO:  completed OK
2017-05-30 13:33:32.421 [main]             DBSchemaLoader.lambda$postInstallation$5()  INFO:  completed OK
may 30, 2017 1:33:32 PM tigase.db.util.SchemaManager loadSchemas
INFORMACIÓN: schema upgrade finished!




  =============================================================================
  	Schema installation finished

  Data source: default with uri jdbc:mysql://localhost/tigasedbtest?user=tigasetest&password=tigasetest&useSSL=false
  	Checking connection to database	ok
  	Checking if database exists	ok
  	Loading schema: Tigase XMPP Server (Core), version: 7.2.0	ok
  	Adding XMPP admin accounts	warning
  		Message: Error: No admin users entered
  	Post installation action	ok

  Example init.properties configuration file:

  --debug = server
  --virt-hosts = localhost
  'config-type' = 'default'
  dataSource () {
      default () {
          uri = 'jdbc:mysql://localhost/tigasedbtest?user=tigasetest&password=tigasetest&useSSL=false'
      }
  }
  amp (active: false) {}
  bosh (active: false) {}
  c2s (active: false) {}
  eventbus (active: false) {}
  http (active: false) {}
  'message-archive' (active: false) {}
  monitor (active: false) {}
  s2s (active: false) {}
  ws2s (active: false) {}
  =============================================================================

…

wojtek@atlantiscity.local ~/dev/tigase/tigase-server/server $ mysql -u root -p
Enter password:
…
mysql> use tigasedbtest
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+--------------------------+
| Tables_in_tigasedbtest   |
+--------------------------+
| tig_broadcast_jids       |
| tig_broadcast_messages   |
| tig_broadcast_recipients |
| tig_cluster_nodes        |
| tig_nodes                |
| tig_offline_messages     |
| tig_pairs                |
| tig_users                |
+--------------------------+
8 rows in set (0.00 sec)

Andrzej Wójcik (Tigase) commented 7 years ago

Changes to be made:

Value of -C should be a list of changes of active components

Now if you will pass names of components, then only this components will be loaded. Instead, we should use a list of changes, ie. +pubsub,-muc to enable pubsub and disable muc. With this entry, other components state will be same as it would be without this entry.

Generated configuration file should contain entries for active components, ie. if user specifies that pubsub should be enabled, proper entry should be added to config file even though pubsub is enabled by default.
Fix issue with duplicated log entries during execution of command

To do:

Check if changes do not have negative impact on web-based setup utility.

Andrzej Wójcik (Tigase) commented 7 years ago

I've modified behavior of install-schema task, modified generated config file and fixed the issue with duplicated entries in a log file (or in a console).

wojciech.kapcia@tigase.net commented 7 years ago

wojtek@atlantiscity.local ~/dev/tmps/tigase-server-dists/tigase-issue #7.2.0-SNAPSHOT-b4841 $ ./scripts/tigase.sh install-schema etc/tigase.conf -T mysql -D tigaseinstall222 -C "-muc,-pubsub"

LogLevel: CONFIG
2017-05-31 12:40:46.364 [main]             DBSchemaLoader.init()                   CONFIG:   Parameters: [ingoreMissingFiles: false, logLevel: CONFIG, adminPassword: null, admins: null, dbRootPass: root, dbRootUser: root, dbType: mysql, dbName: tigaseinstall222, dbHostname: localhost, dbUser: tigase_user, dbPass: tigase_pass, useSSL: false, file: null, query: null]
may 31, 2017 12:40:46 PM tigase.util.DNSResolverDefault <init>
ADVERTENCIA: Resolving default host name: localhost took: 75
may 31, 2017 12:40:47 PM tigase.db.util.SchemaManager loadSchemas
INFORMACIÓN: found 1 data sources to upgrade...
may 31, 2017 12:40:47 PM tigase.db.util.SchemaManager loadSchemas
INFORMACIÓN: begining upgrade...
LogLevel: CONFIG
2017-05-31 12:40:47.573 [main]             DBSchemaLoader.init()                   CONFIG:   Parameters: [ingoreMissingFiles: false, logLevel: CONFIG, adminPassword: null, admins: null, dbRootPass: root, dbRootUser: root, dbType: mysql, dbName: tigaseinstall222, dbHostname: localhost, dbUser: tigase_user, dbPass: tigase_pass, useSSL: false, file: null, query: null]
2017-05-31 12:40:47.574 [main]             DBSchemaLoader.validateDBConnection()   INFO:     Validating DBConnection, URI: jdbc:mysql://localhost/?user=root&password=root&useSSL=false
2017-05-31 12:40:48.073 [main]             DBSchemaLoader.validateDBConnection()   CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@4ede035e, jTDS 1.3.1, com.mysql.jdbc.Driver@20f63ddc, com.mysql.fabric.jdbc.FabricMySQLDriver@3578a45b, org.postgresql.Driver@26aee0a6]
2017-05-31 12:40:48.075 [main]             DBSchemaLoader.validateDBConnection()   INFO:     Connection OK
2017-05-31 12:40:48.076 [main]             DBSchemaLoader.validateDBExists()       INFO:     Validating whether DB Exists, URI: jdbc:mysql://localhost/tigaseinstall222?user=tigase_user&password=tigase_pass&useSSL=false
2017-05-31 12:40:48.099 [main]             DBSchemaLoader.withConnection()         CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@4ede035e, jTDS 1.3.1, com.mysql.jdbc.Driver@20f63ddc, com.mysql.fabric.jdbc.FabricMySQLDriver@3578a45b, org.postgresql.Driver@26aee0a6]
2017-05-31 12:40:48.149 [main]             DBSchemaLoader.lambda$null$240()        INFO:      OK
2017-05-31 12:40:48.152 [main]             DBSchemaLoader.loadSchemaFile()         INFO:     Loading schema from file(s): database/mysql-schema-7-2.sql, URI: jdbc:mysql://localhost/tigaseinstall222?user=root&password=root&useSSL=false
2017-05-31 12:40:48.161 [main]             DBSchemaLoader.withConnection()         CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@4ede035e, jTDS 1.3.1, com.mysql.jdbc.Driver@20f63ddc, com.mysql.fabric.jdbc.FabricMySQLDriver@3578a45b, org.postgresql.Driver@26aee0a6]
2017-05-31 12:40:48.470 [main]             DBSchemaLoader.lambda$loadSchemaFile$243()  INFO:  completed OK
2017-05-31 12:40:48.471 [main]             DBSchemaLoader.loadSchemaFile()         INFO:     Loading schema from file(s): database/mysql-message-archiving-schema-1.3.0.sql, URI: jdbc:mysql://localhost/tigaseinstall222?user=root&password=root&useSSL=false
2017-05-31 12:40:48.476 [main]             DBSchemaLoader.withConnection()         CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@4ede035e, jTDS 1.3.1, com.mysql.jdbc.Driver@20f63ddc, com.mysql.fabric.jdbc.FabricMySQLDriver@3578a45b, org.postgresql.Driver@26aee0a6]
2017-05-31 12:40:48.701 [main]             DBSchemaLoader.lambda$loadSchemaFile$243()  INFO:  completed OK
2017-05-31 12:40:48.702 [main]             DBSchemaLoader.loadSchemaFile()         INFO:     Loading schema from file(s): database/mysql-muc-schema-2.5.0.sql, URI: jdbc:mysql://localhost/tigaseinstall222?user=root&password=root&useSSL=false
2017-05-31 12:40:48.707 [main]             DBSchemaLoader.withConnection()         CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@4ede035e, jTDS 1.3.1, com.mysql.jdbc.Driver@20f63ddc, com.mysql.fabric.jdbc.FabricMySQLDriver@3578a45b, org.postgresql.Driver@26aee0a6]
2017-05-31 12:40:48.756 [main]             DBSchemaLoader.lambda$loadSchemaFile$243()  INFO:  completed OK
2017-05-31 12:40:48.757 [main]             DBSchemaLoader.loadSchemaFile()         INFO:     Loading schema from file(s): database/mysql-pubsub-schema-3.3.0.sql, URI: jdbc:mysql://localhost/tigaseinstall222?user=root&password=root&useSSL=false
2017-05-31 12:40:48.761 [main]             DBSchemaLoader.withConnection()         CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@4ede035e, jTDS 1.3.1, com.mysql.jdbc.Driver@20f63ddc, com.mysql.fabric.jdbc.FabricMySQLDriver@3578a45b, org.postgresql.Driver@26aee0a6]
2017-05-31 12:40:49.018 [main]             DBSchemaLoader.lambda$loadSchemaFile$243()  INFO:  completed OK
2017-05-31 12:40:49.019 [main]             DBSchemaLoader.addXmppAdminAccount()    WARNING:  Error: No admin users entered
2017-05-31 12:40:49.019 [main]             DBSchemaLoader.postInstallation()       INFO:     Post Installation, URI: jdbc:mysql://localhost/tigaseinstall222?user=root&password=root&useSSL=false
2017-05-31 12:40:49.025 [main]             DBSchemaLoader.withConnection()         CONFIG:   DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@4ede035e, jTDS 1.3.1, com.mysql.jdbc.Driver@20f63ddc, com.mysql.fabric.jdbc.FabricMySQLDriver@3578a45b, org.postgresql.Driver@26aee0a6]
2017-05-31 12:40:49.025 [main]             DBSchemaLoader.lambda$postInstallation$242()  INFO: Finalizing...
2017-05-31 12:40:49.026 [main]             DBSchemaLoader.lambda$postInstallation$242()  INFO:  completed OK
may 31, 2017 12:40:49 PM tigase.db.util.SchemaManager loadSchemas
INFORMACIÓN: schema upgrade finished!




  =============================================================================
  	Schema installation finished

  Data source: default with uri jdbc:mysql://localhost/tigaseinstall222?user=tigase_user&password=tigase_pass&useSSL=false
  	Checking connection to database	ok
  	Checking if database exists	ok
  	Loading schema: Tigase XMPP Server (Core), version: 7.2.0	ok
  	Loading schema: Tigase Message Archiving Component, version: 1.3.0	ok
  	Loading schema: Tigase MUC Component, version: 2.5.0	ok
  	Loading schema: Tigase PubSub Component, version: 3.3.0	ok
  	Adding XMPP admin accounts	warning
  		Message: Error: No admin users entered
  	Post installation action	ok

  Example init.properties configuration file:

  --debug = server
  --virt-hosts = localhost
  'config-type' = 'default'
  dataSource () {
      default () {
          uri = 'jdbc:mysql://localhost/tigaseinstall222?user=tigase_user&password=tigase_pass&useSSL=false'
      }
  }
  amp () {}
  bosh () {}
  c2s () {}
  eventbus () {}
  http () {}
  'message-archive' () {}
  monitor () {}
  muc () {}
  pubsub () {}
  s2s () {}
  ws2s () {}
  =============================================================================

The problem stems from the bad parsing of the elements and considering "-muc,-pubsbu" as next parameter instead of value. Following (different format) works:

--components="muc,-pubsub"

Andrzej Wójcik (Tigase) commented 7 years ago

Fixed parameters parsing issue

wojciech.kapcia@tigase.net commented 7 years ago

Works good now.

Type	New Feature
Priority	Normal
Assignee	wojciech.kapcia@tigase.net
RedmineID	5484
Version	tigase-server-8.0.0
Spent time	67h 30m

Issue Votes (0)

Watchers (0)

Reference

tigase/_server/server-core#822