Difference between revisions of "ContextRestore Bundle"

(Created page with '= Context Restore Documentation = == Introduction == This document contains descriptions about the context restore bundle. This bundle is a solution for restoring one single co...')
 
 
(12 intermediate revisions by 3 users not shown)
Line 4: Line 4:
  
 
This document contains descriptions about the context restore bundle. This
 
This document contains descriptions about the context restore bundle. This
bundle is a solution for restoring one single context out of a mysql dump from a
+
bundle is a solution for restoring one single context out of a mysql dump from
complete Open-Xchange database. In a distributed setup, it is possible to use
+
a complete Open-Xchange database. In a distributed setup, it is possible to use
 
several files containing the dumps of different databases. The tool described
 
several files containing the dumps of different databases. The tool described
within this document does fetch the needed informations out of these files.
+
within this document does fetch the needed information out of these files.
Architecture
 
The context restore bundle is designed as an admin bundle so it gets installed
 
into the OSGi environment of the OX admin daemon.
 
Deliverables
 
Package description
 
The package open-xchange-admin-plugin-contextrestore contains the bundle
 
itself and the corresponding configuration files. The bundle will get installed as a
 
directory below /opt/open-xchange/bundles. The configuration file can be
 
found in:
 
/opt/open-xchange/etc/admindaemon/plugin/contextrestore.properties.
 
  
 
== Limitations ==
 
== Limitations ==
For the restore process, the tool requires some information to be available in the
 
database. One is the availability of the old database pool id. The tool will dig out
 
the pool id the context used before from the mysql dump, and will try to get a
 
connection to the corresponding database through this pool id. If the pool id
 
does not exist any more, an error is thrown.
 
The same applies to the filestore_id.
 
Requirements
 
For correct operation, it is required that the mysql dumps for this tool represent
 
a consistent state of all databases the Open-Xchange server has under control.
 
Furthermore this consistent state needs also to include the backup of the
 
filestore.
 
  
== Installation ==
+
You must not delete the last context from within a database schema.
 +
If you just have ONE context left in your system, do NOT delete it.
 +
That is because the deletion of the last contexts leads into deletion of the entire schema.
 +
In that case, you have to restore a complete database dump. restorecontext will not work in that case.
  
After the package has been installed, a new shell script named restorecontext
+
== Requirements ==
is to be found under /opt/open-xchange/sbin. The admin daemon needs to be
+
 
restarted to make the new functionality available. The functionality is accessible
+
For correct operation, it is required that the mysql dumps for this tool
via RMI, CLT and SOAP.
+
represent a consistent state of all databases the Open-Xchange server has under
Usage
+
control.  Furthermore this consistent state needs also to include the backup of
A complete database backup has to be done before the new function can be
+
the filestore.
used. The Tool described within this document requires a full SQL dump of the
+
 
databases involved. This can be done by executing the command mysqldump -
+
{{InstallPlugin|pluginname=open-xchange-admin-plugin-contextrestore|sopath=stable|version=7.x}}
all-databases -–single-transaction, on all database machines registered in
+
 
Open-Xchange as well as the configuration Database. These two parameters are
+
After the package has been installed, a new shell script named
very important. The first one lets the dump contain every database on the
+
<tt>/opt/open-xchange/sbin/restorecontext</tt> The server needs to be
specified database host, and the second is used for a consistent backup. Instead
+
restarted to make the new functionality available. The functionality is
of –-all-databases you can also use –-database with the right database(s), if
+
accessible via RMI, CLT and SOAP.
none of the switches are given as direct argument then mysql will left out
+
 
essential data in the dump. For a full backup the switch --all-databases is
+
== Usage ==
recommended.
+
 
 +
A complete database backup has to be done before the new function can be used.
 +
The Tool described within this document requires a full SQL dump of the
 +
databases involved. This can be done by executing the command  
 +
 
 +
$ mysqldump --all-databases --single-transaction --hex-blob
 +
 
 +
on all database machines registered in Open-Xchange as well as the
 +
configuration Database. These three parameters are very important. The first one
 +
lets the dump contain every database on the specified database host, the
 +
second is used for a consistent backup and the third one is used to let all binary data
 +
appear as hex in the dump. Instead of <tt>–-all-databases</tt> you can also
 +
use <tt>–-database</tt> with the right database(s), if none of the switches are given as
 +
direct argument then mysql will left out essential data in the dump. For a full
 +
backup the switch <tt>--all-databases</tt> is recommended.
  
 
The usage of the command line tool is self-explanatory as it confirms to the
 
The usage of the command line tool is self-explanatory as it confirms to the
standard unix behavior. --help will show a short help with the available
+
standard unix behavior. <tt>--help</tt> will show a short help with the available
 
parameters.
 
parameters.
 +
 
The main arguments of the tool are different database dumps (from the configdb
 
The main arguments of the tool are different database dumps (from the configdb
 
and the distributed user data databases). On success the tool returns the path to
 
and the distributed user data databases). On success the tool returns the path to
Line 59: Line 56:
 
afterwards in a separate process which is not part of the Open-Xchange system.
 
afterwards in a separate process which is not part of the Open-Xchange system.
 
An example of a restore would look like this:
 
An example of a restore would look like this:
restorecontext -A <username> -P <password> -c 7777 -f
+
 
<complete
+
$ restorecontext -A oxadminmaster -P secret -c 1 \
path to configdb dump>,<complete path to first user data database
+
    -f /path/to/configdb.sql,/path/to/userdb.sql,/path/to/userdb1.sql,...
dump>[,<complete path to the n-th user data database dump>]
+
 
file:///tmp/filestore2/7777_ctx_store
+
'''Note:''' restorecontext will only restore data from the database, not from the filestore. During the restore the context and the filestore content will be deleted, so make a backup first or if you deleted an context by accident you have to restore the filestore content after restorecontext.
Note that the parts in <> have to be replaced by the parameters for your setup
+
 
 +
 
 +
=== Old dump without --hex-blob ===
 +
 
 +
If you have done a dump according to a former version of this document the dump contains the
 +
direct binary data which can't be understood by the restore tool. You will get errors like
 +
"Data truncation: Data too long for column 'uuid' at row 1".
 +
 
 +
'''ATTENTION: The steps below need to be done on a NEW MySQL instance NOT the productive one.'''
 +
 
 +
To get the correct version of the dump you need to apply the dump to a clean MySQL database.
 +
If the dump was created with <tt>--all-databases</tt> you can just pipe the contents into mysql:
 +
<tt>mysql < mydump.txt</tt>. If you have different dump for each database you have to create
 +
the corresponding database in MySQL first. The name of the database to be created can be found at
 +
the beginning of the dump file
 +
<code>
 +
-- Host: localhost    Database: configdb
 +
</code>
 +
So for the example simply issue <tt>create database configdb</tt> and apply the dump directly to
 +
this database: <tt>mysql configdb < mydump.txt</tt> (pay attention to give the name of the database
 +
to mysql in this case).
 +
Afterwards just create the mysqldump from this database with the correct --hex-blob parameter.

Latest revision as of 10:42, 5 October 2016

Context Restore Documentation

Introduction

This document contains descriptions about the context restore bundle. This bundle is a solution for restoring one single context out of a mysql dump from a complete Open-Xchange database. In a distributed setup, it is possible to use several files containing the dumps of different databases. The tool described within this document does fetch the needed information out of these files.

Limitations

You must not delete the last context from within a database schema. If you just have ONE context left in your system, do NOT delete it. That is because the deletion of the last contexts leads into deletion of the entire schema. In that case, you have to restore a complete database dump. restorecontext will not work in that case.

Requirements

For correct operation, it is required that the mysql dumps for this tool represent a consistent state of all databases the Open-Xchange server has under control. Furthermore this consistent state needs also to include the backup of the filestore.

Install on OX 7.x

Debian GNU/Linux 8.0

Add the following entry to /etc/apt/sources.list.d/open-xchange.list if not already present:

deb https://software.open-xchange.com/OX6/stable/DebianJessie/ /
# if you have a valid maintenance subscription, please uncomment the 
# following and add the ldb account data to the url so that the most recent
# packages get installed
# deb https://[CUSTOMERID:PASSWORD]@software.open-xchange.com/OX6/stable/updates/DebianJessie/ /

and run

$ apt-get update
$ apt-get install open-xchange-admin-plugin-contextrestore

Debian GNU/Linux 9.0

Add the following entry to /etc/apt/sources.list.d/open-xchange.list if not already present:

deb https://software.open-xchange.com/OX6/stable/DebianStretch/ /
# if you have a valid maintenance subscription, please uncomment the 
# following and add the ldb account data to the url so that the most recent
# packages get installed
# deb https://[CUSTOMERID:PASSWORD]@software.open-xchange.com/OX6/stable/updates/DebianStretch/ /

and run

$ apt-get update
$ apt-get install open-xchange-admin-plugin-contextrestore

SUSE Linux Enterprise Server 12

Add the package repository using zypper if not already present:

$ zypper ar https://software.open-xchange.com/OX6/stable/SLE_12 ox

If you have a valid maintenance subscription, please run the following command and add the ldb account data to the url so that the most recent packages get installed:

$ zypper ar https://[CUSTOMERID:PASSWORD]@software.open-xchange.com/OX6/stable/updates/SLES11 ox-updates

and run

$ zypper ref
$ zypper in open-xchange-admin-plugin-contextrestore

RedHat Enterprise Linux 6

Start a console and create a software repository file if not already present:

$ vim /etc/yum.repos.d/ox.repo

[ox]
name=Open-Xchange
baseurl=https://software.open-xchange.com/OX6/stable/RHEL6/
gpgkey=https://software.open-xchange.com/oxbuildkey.pub
enabled=1
gpgcheck=1
metadata_expire=0m
# if you have a valid maintenance subscription, please uncomment the 
# following and add the ldb account data to the url so that the most recent
# packages get installed
# [ox-updates]
# name=Open-Xchange Updates
# baseurl=https://[CUSTOMERID:PASSWORD]@software.open-xchange.com/OX6/stable/updates/RHEL6/
# gpgkey=https://software.open-xchange.com/oxbuildkey.pub
# enabled=1
# gpgcheck=1
# metadata_expire=0m

and run

$ yum update
$ yum install open-xchange-admin-plugin-contextrestore

RedHat Enterprise Linux 7

Start a console and create a software repository file if not already present:

$ vim /etc/yum.repos.d/ox.repo

[ox]
name=Open-Xchange
baseurl=https://software.open-xchange.com/OX6/stable/RHEL7/
gpgkey=https://software.open-xchange.com/oxbuildkey.pub
enabled=1
gpgcheck=1
metadata_expire=0m
# if you have a valid maintenance subscription, please uncomment the 
# following and add the ldb account data to the url so that the most recent
# packages get installed
# [ox-updates]
# name=Open-Xchange Updates
# baseurl=https://[CUSTOMERID:PASSWORD]@software.open-xchange.com/OX6/stable/updates/RHEL7/
# gpgkey=https://software.open-xchange.com/oxbuildkey.pub
# enabled=1
# gpgcheck=1
# metadata_expire=0m

and run

$ yum update
$ yum install open-xchange-admin-plugin-contextrestore

CentOS 6

Start a console and create a software repository file if not already present:

$ vim /etc/yum.repos.d/ox.repo

[ox]
name=Open-Xchange
baseurl=https://software.open-xchange.com/OX6/stable/RHEL6/
gpgkey=https://software.open-xchange.com/oxbuildkey.pub
enabled=1
gpgcheck=1
metadata_expire=0m
# if you have a valid maintenance subscription, please uncomment the 
# following and add the ldb account data to the url so that the most recent
# packages get installed
# [ox-updates]
# name=Open-Xchange Updates
# baseurl=https://[CUSTOMERID:PASSWORD]@software.open-xchange.com/OX6/stable/updates/RHEL6/
# gpgkey=https://software.open-xchange.com/oxbuildkey.pub
# enabled=1
# gpgcheck=1
# metadata_expire=0m

and run

$ yum update
$ yum install open-xchange-admin-plugin-contextrestore

CentOS 7

Start a console and create a software repository file if not already present:

$ vim /etc/yum.repos.d/ox.repo

[ox]
name=Open-Xchange
baseurl=https://software.open-xchange.com/OX6/stable/RHEL7/
gpgkey=https://software.open-xchange.com/oxbuildkey.pub
enabled=1
gpgcheck=1
metadata_expire=0m
# if you have a valid maintenance subscription, please uncomment the 
# following and add the ldb account data to the url so that the most recent
# packages get installed
# [ox-updates]
# name=Open-Xchange Updates
# baseurl=https://[CUSTOMERID:PASSWORD]@software.open-xchange.com/OX6/stable/updates/RHEL7/
# gpgkey=https://software.open-xchange.com/oxbuildkey.pub
# enabled=1
# gpgcheck=1
# metadata_expire=0m

and run

$ yum update
$ yum install open-xchange-admin-plugin-contextrestore

After the package has been installed, a new shell script named /opt/open-xchange/sbin/restorecontext The server needs to be restarted to make the new functionality available. The functionality is accessible via RMI, CLT and SOAP.

Usage

A complete database backup has to be done before the new function can be used. The Tool described within this document requires a full SQL dump of the databases involved. This can be done by executing the command

$ mysqldump --all-databases --single-transaction --hex-blob

on all database machines registered in Open-Xchange as well as the configuration Database. These three parameters are very important. The first one lets the dump contain every database on the specified database host, the second is used for a consistent backup and the third one is used to let all binary data appear as hex in the dump. Instead of –-all-databases you can also use –-database with the right database(s), if none of the switches are given as direct argument then mysql will left out essential data in the dump. For a full backup the switch --all-databases is recommended.

The usage of the command line tool is self-explanatory as it confirms to the standard unix behavior. --help will show a short help with the available parameters.

The main arguments of the tool are different database dumps (from the configdb and the distributed user data databases). On success the tool returns the path to the filestore of the given context. The files below this path have to be restored afterwards in a separate process which is not part of the Open-Xchange system. An example of a restore would look like this:

$ restorecontext -A oxadminmaster -P secret -c 1 \
   -f /path/to/configdb.sql,/path/to/userdb.sql,/path/to/userdb1.sql,...

Note: restorecontext will only restore data from the database, not from the filestore. During the restore the context and the filestore content will be deleted, so make a backup first or if you deleted an context by accident you have to restore the filestore content after restorecontext.


Old dump without --hex-blob

If you have done a dump according to a former version of this document the dump contains the direct binary data which can't be understood by the restore tool. You will get errors like "Data truncation: Data too long for column 'uuid' at row 1".

ATTENTION: The steps below need to be done on a NEW MySQL instance NOT the productive one.

To get the correct version of the dump you need to apply the dump to a clean MySQL database. If the dump was created with --all-databases you can just pipe the contents into mysql: mysql < mydump.txt. If you have different dump for each database you have to create the corresponding database in MySQL first. The name of the database to be created can be found at the beginning of the dump file -- Host: localhost Database: configdb So for the example simply issue create database configdb and apply the dump directly to this database: mysql configdb < mydump.txt (pay attention to give the name of the database to mysql in this case). Afterwards just create the mysqldump from this database with the correct --hex-blob parameter.