Projects tigase _server server-core Issues #858
tigase upgrade-schema runs into error on conversion (#858)
Daniel Wisnewski opened 7 years ago

Attempting to upgrade v7.1.0 to v7.2.0 to replicate instructions from Eric on upgrading.

Setup and ran fresh copy of v7.1.0 with all components active, made small number of users and short message history to populate tables.

Shut down tigase v7.1.0, copy config files, and ran v7.2.0 upgrade-config. This worked OK, ran update-schema and entered user/password. Following error appears:

  =============================================================================
  	Schema upgrade finished
  
  Data source: default with uri jdbc:mysql://localhost/tigasedb?user=tigase&password=tigase12
  	Checking connection to database	ok
  	Checking if database exists	ok
  	Loading schema: Tigase XMPP Server (Core), version: 7.2.0	error
  		Message: Loading schema from file(s): database/mysql-schema-7-2.sql, URI: jdbc:mysql://localhost/tigasedb?user=root&password=root
  		         DriverManager (available drivers): [org.apache.derby.jdbc.AutoloadedDriver@4b20ca2b, jTDS 1.3.1, com.mysql.jdbc.Driver@4c398c80, com.mysql.fabric.jdbc.FabricMySQLDriver@607fbe09, org.postgresql.Driver@57fd91c9]
  		         Failed to execute query: create procedure TigUpgradeMsgHistory() begin if exists (select 1 from information_schema.tables where table_schema = database() and table_name = 'msg_history') then alter table msg_history rename tig_offline_messages; else create table if not exists tig_offline_messages ( msg_id bigint unsigned not null auto_increment, ts timestamp(6) default current_timestamp(6), expired timestamp null default null, sender varchar(2049), sender_sha1 char(128), receiver varchar(2049) not null, receiver_sha1 char(128), msg_type int not null default 0, message mediumtext character set utf8mb4 collate utf8mb4_unicode_ci not null, primary key (msg_id), key tig_offline_messages_expired_index (expired), key tig_offline_messages_receiver_sha1_index (receiver_sha1), key tig_offline_messages_receiver_sha1_sender_sha1_index (receiver_sha1, sender_sha1) ) ENGINE=InnoDB default character set utf8 ROW_FORMAT=DYNAMIC; end if; if exists (select 1 from information_schema.columns where table_schema = database() and table_name = 'tig_offline_messages' and column_name = 'expired') then alter table tig_offline_messages modify expired timestamp(6) null default null; end if; if exists (select 1 from information_schema.columns where table_schema = database() and table_name = 'tig_offline_messages' and column_name = 'ts') then alter table tig_offline_messages modify ts timestamp(6) default current_timestamp(6); end if; if exists (select 1 from information_schema.columns where table_schema = database() and table_name = 'tig_offline_messages' and column_name = 'message') then alter table tig_offline_messages modify message mediumtext character set utf8mb4 collate utf8mb4_unicode_ci not null; end if; if not exists (select 1 from information_schema.columns where table_schema = database() and table_name = 'tig_offline_messages' and column_name = 'receiver') then alter table tig_offline_messages add receiver varchar(2049) character set utf8, add receiver_sha1 char(128), add sender varchar(2049) character set utf8, add sender_sha1 char(128), add key tig_offline_messages_expired_index (expired), add key tig_offline_messages_receiver_sha1_index (receiver_sha1), add key tig_offline_messages_receiver_sha1_sender_sha1_index (receiver_sha1, sender_sha1), drop index expired, drop index sender_uid, drop index receiver_uid; end if; if exists (select 1 from information_schema.columns where table_schema = database() and table_name = 'user_jid') and exists (select 1 from information_schema.columns where table_schema = database() and table_name = 'tig_offline_messages' and column_name = 'receiver_uid') then update tig_offline_messages set receiver = (select jid from user_jid where jid_id = receiver_uid), sender = (select jid from user_jid where jid_id = sender_uid) where receiver is null; end if; update tig_offline_messages set receiver_sha1 = sha1(lower(receiver)), sender_sha1 = sha1(lower(sender)) where receiver_sha1 is null; alter table tig_offline_messages modify receiver varchar(2049) character set utf8 not null, modify receiver_sha1 char(128) not null; if exists (select 1 from information_schema.columns where table_schema = database() and table_name = 'tig_offline_messages' and column_name = 'receiver_uid') then alter table tig_offline_messages drop column sender_uid, drop column receiver_uid; end if; if not exists (select 1 from information_schema.statistics where table_schema = database() and table_name = 'tig_offline_messages' and index_name = 'PRIMARY') then alter table tig_offline_messages add primary key (msg_id); end if; if exists (select 1 from information_schema.tables where table_schema = database() and table_name = 'broadcast_msgs') then alter table broadcast_msgs rename tig_broadcast_messages; alter table tig_broadcast_messages modify expired timestamp(6) not null, modify msg mediumtext character set utf8mb4 collate utf8mb4_unicode_ci not null; else create table if not exists tig_broadcast_messages ( id varchar(128) not null, expired timestamp(6) not null, msg mediumtext character set utf8mb4 collate utf8mb4_unicode_ci not null, primary key (id) ); end if; create table if not exists tig_broadcast_jids ( jid_id bigint unsigned not null auto_increment, jid varchar(2049) not null, jid_sha1 char(128) not null, primary key (jid_id) ); if not exists (select 1 from information_schema.statistics where table_schema = database() and table_name = 'tig_broadcast_jids' and index_name = 'tig_broadcast_jids_jid_sha1') then create index tig_broadcast_jids_jid_sha1 on tig_broadcast_jids (jid_sha1); end if; if exists (select 1 from information_schema.tables where table_schema = database() and table_name = 'user_jid') and exists (select 1 from information_schema.tables where table_schema = database() and table_name = 'public.broadcast_msgs_recipients') then insert into tig_broadcast_jids (jid, jid_sha1) select u.jid, sha1(lower(u.jid)) from user_jid u inner join broadcast_msgs_recipients b on u.jid_id = b.jid_id where not exists (select 1 from tig_broadcast_jids bj where bj.jid = u.jid); end if; create table if not exists tig_broadcast_recipients ( msg_id varchar(128) not null references tig_broadcast_messages(id), jid_id bigint not null references tig_broadcast_jids(jid_id), primary key (msg_id, jid_id) ); if exists (select 1 from information_schema.tables where table_schema = database() and table_name = 'user_jid') and exists (select 1 from information_schema.tables where table_schema = database() and table_name = 'public.broadcast_msgs_recipients') then insert into tig_broadcast_recipients (msg_id, jid_id) select x.msg_id, x.jid_id from (select bmr.msg_id, bj.jid_id from broadcast_msgs_recipients bmr inner join user_jid uj on uj.jid_id = bmr.jid_id inner join tig_broadcast_jids bj on bj.jid_sha1 = sha1(lower(uj.jid)) ) x where not exists (select 1 from tig_broadcast_recipients br where br.msg_id = x.msg_id and br.jid_id = x.jid_id); end if; if exists (select 1 from information_schema.tables where table_schema = database() and table_name = 'public.broadcast_msgs_recipients') then drop table broadcast_msgs_recipients; end if; if exists (select 1 from information_schema.tables where table_schema = database() and table_name = 'user_jid') then drop table user_jid; end if; end 
  		         
  		         
  		         
  		         =====
  		         Failure: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '(6) default current_timestamp(6), expired timestamp null default null, sender va' at line 1
  		         =====
  	Adding XMPP admin accounts	error
  		Message: Database schema is invalid
  	Post installation action	error
  		Message: Database schema is invalid
  =============================================================================

Wojciech, assigning to you since I think you wrote this? Looks like it could be a formatting issue?

MA was enabled for v7.1.0.

Andrzej Wójcik (Tigase) commented 7 years ago

%Daniel What version of MySQL server are you running?

I'm asking this as error:

Failure: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax
to use near '(6) default current_timestamp(6), expired timestamp null default null, sender va' at line 1

was being reported when MySQL version was too old and did not support for the extended precision of timestamp fields.

Daniel Wisnewski commented 7 years ago

Ah thanks, Wojciech Andrzej! I thought I had v5.7 but turns out it was an older version. After upgrading mysql, conversion was successful. Rejecting issue due to user error.

wojciech.kapcia@tigase.net commented 7 years ago

nitpicking: it was Andrzej that helped you :-)

Artur Hefczyc commented 7 years ago

%andrzej.wojcik , %wojtek would it be possible to have some code inside the Tigase which checks the DB version and prints an error if version is not supported? I guess it is definitely possible, my question is rather how much work would it need to have it.

wojciech.kapcia@tigase.net commented 7 years ago

%kobit It is planned (#3582) for this version but has to wait for it's turn (however in the end it may be solved differently - we are still discussing with Andrzej best possible solution)

Artur Hefczyc commented 7 years ago

Super!

Andrzej Wójcik (Tigase) commented 7 years ago

%kobit %wojtek I think that you two are talking about two different things here.

Wojciech is mentioning verification of supported database schema version, while Artur was asking about retrieving and verifying database software version.

Artur, yes it would be possible but I would not suggest going this way. Database version does not have a standardized format, so we would need to create a parser for each database and then compare values. However, ie. our MySQL schema works with MariaDB and adding this check will make it more difficult to run on MariaDB. Also some hosting or cloud providers are adding something to version string returned by their branded version of database, which would throw errors in this cases as unsupported.

Artur Hefczyc commented 7 years ago

Ok, I understand. Thank you for explanation Andrzej. I agree with you, it does not make much sense to detect versions of all possible databases. Instead we will have something like recommended versions of supported databases.

wojciech.kapcia@tigase.net commented 7 years ago

Artur Hefczyc wrote:

Instead we will have something like recommended versions of supported databases.

%kobit

We already do http://docs.tigase.org/tigase-server/snapshot/Administration_Guide/html_chunk/databasemgnt.html#_recommended_database_versions

issue 1 of 1
Type
Bug
Priority
Normal
Assignee
RedmineID
5735
Version
tigase-server-8.0.0
Spent time
11h 15m
Issue Votes (0)
Watchers (0)
Reference
tigase/_server/server-core#858
Please wait...
Page is in error, reload to recover