AppSuite:OX Guard Upgrade OSGI
Upgrading OX Guard from 2.2.x to 2.4.0
Introduction
With OX Guard 2.4.0 the first step was made and the product was moved into the OSGi stack. Although OX Guard now has a tighter integration with the core that does not mean that it lost its micro-service character. OX Guard still relies on the support API to accomplish its tasks; tasks like capability check, HTML sanitising and mail address resolving. The database calls however were completely eliminated (along with their support API) and now the core's OSGi services are used for that purpose. The same applies to the configuration.
In the next few sections it is described what was changed and the necessary steps to upgrade from 2.2.x to 2.4.0. For a fresh installation refer to the main OX Guard article.
Changes
This section covers the changes introduced with OX Guard 2.4.0.
Configuration Structure
Alongside with the OSGi-fication of the product, a bit of clean up took place, regarding the configuration files and their structure.
The most obvious change was to get rid of the separate /opt/open-xchange/guard directory that the entire configuration was residing in. The contents of this directory are now spread accross the appropriate folders in /opt/open-xchange.
Templates
The templates (originally located under /opt/open-xchange/guard/templates) were moved to the templates directory of the open-xchange installation, i.e. to /opt/open-xchange/templates/guard.
Translations
The translations (originally located under /opt/open-xchange/guard/l10n) were moved to the i18n directory of the open-xchange installation, i.e. to /opt/open-xchange/i18n. The translations formerly had a prefix main-*_*.po (i.e. main-de_DE.po) should now start with guard- (i.e. guard-de_DE.po) to be able to differentiate between middleware translations and those provided by OX Guard. Even the languages.xml and those starting with templates-* should be located within the above mentioned i18n directory.
Properties
guard.properties
Prior to 2.4.0, there were two guard.properties files, one for the core groupware which contained properties that were used to enable the OX Guard capability to various modules as well as some extra functionality, and one for OX Guard itself, containing all properties for that product. Since the /opt/open-xchange/guard directory was deemed obsolete and subject to be removed and to avoid the name clashing of the two property files, they had to be renamed. A merge was not option, simply because they were provided and used by different packages. Therefore, the properties file guard.properties (under /opt/open-xchange/etc/) which accomodated the properties for the AppSuite Guard bundle is now renamed to guard-api.properties.
The OX Guard guard.properties file (under /opt/open-xchange/guard/etc) which housed all the OX Guard properties, is now renamed and moved under /opt/open-xchange/guard/etc/guard-core.properties.
private_dns_key and public_dns_key
Those two property files were not needed anymore, thus they were marked as obsolete and removed with OX Guard 2.4.0.
Farewell Jetty, welcome Grizzly
With the big step OX Guard took and moved into the OSGi stack, Jetty was rendered obsolete and had to be removed as well. OX Guard now uses Grizzly to register its servlets.
Apache
The first change in the Apache configuration is the port number of the balancer directive regarding OX Guard. Since Jetty was removed and Grizzly is in charge now, the port was adjusted to reflect that change.
The second change is the order of the ProxyPass locations. The OX Guard one, had to be moved forward because it was getting masked and overwritten by the AppSuite ones.
Guest Reader moved to a separate package
In order to allow more flexibility on distributed deployments, the "Guest Reader" component has moved to a separate package that is being installed on the frontend server. The UI is then provided by Apache rather than Jetty.
Because of this, the location of the Guest reader URL will change. By default, the reader package will be installed under /var/www/guard/reader
So, in most setups, the reader URL would change from
domain/appsuite/api/oxguard/reader/reader.html?....
to
domain/guard/reader/reader.html?....
To handle this change, the easiest thing to do is change the externalReaderPath configuration in the new guard-core.proprties file from the old location to the new
com.openexchange.guard.externalReaderPath=domain/guard/reader/reader.html
Assuming that there are already several Guest emails out there with the old Jetty URL, Guard OSGI will automatically redirect all requests from the old location to the new
Command Line Tool
The command line tool also got a "lifting" and was brought to the standards of the rest of the command line tools of the OX family. The CLT now includes optional JMX authentication. Furthermore, the functionality of initialising the OX Guard installation was removed from the command line tool. It is now being done automatically upon server start up. The rest of the functionality is still there, though.
usage: guard [-i [-d /custom/path]] [-t john.doe@example.com] [-u
joan.doe@ox.io] [-e john.doe@example.com] [-n
joan.doe@example.com] [-j jmxUser] [-w jmxPassword] [-x
jmx.host] [-o <JMX_PORT>] [-p <RMI_PORT>] [--responsetimeout
<TIMEOUT>]
-d,--directory <arg> Full path of the 'oxguardpass' file
(defaults to current directory). This flag
should be used in conjunction with the
'--init' switch.
e.g. guard --init --directory
/opt/open-xchange/etc
-e,--reset <arg> Resets the specified e-mail address and
sends a new password to the user
e.g. guard --reset john@somewhere.com
-h,--help Prints a help text
-i,--init Initialise guard.
-j,--jmx-user <arg> JMX user
-n,--remove-pin <arg> Removes the PIN for the specified user
e.g. guard --remove-pin john@somewhere.com
-o,--jmx-port <arg> JMX port (default: '9999')
-p,--port <arg> The optional RMI port (default:1099)
--responsetimeout <arg> The optional response timeout in seconds
when reading data from server (default: 0s;
infinite)
-s,--server <arg> The optional RMI server (default: localhost)
-t,--test <arg> Test the specified e-mail address against
the MailResolver
e.g. guard --test john@somewhere.com
-u,--upgrade <arg> Upgrades the specified guest account to an
OX account
e.g. guard --upgrade john@somewhere.com
-w,--jmx-password <arg> JMX password
-x,--jmx-host <arg> JMX host (default: 'localhost')
Command line tool for OX Guard
Logging
The /opt/open-xchange/guard/etc/logback.xml file for OX Guard was completely removed. Now logging is being configured globally in /opt/open-xchange/etc/logback.xml. All OX Guard events that were previously logged in /var/log/open-xchange/guard/guard.log are now being logged in the log file as the groupware, namely in /var/log/open-xchange/open-xchange.log.0.
The SMTP relevant section is no longer present, but it can be configured separately by adding it again to the previous mentioned logback.xml file.
<!-- Configure SMTP Triggered emails here. Once done, uncomment out appender under root below -->
<appender name="SMTP" class="ch.qos.logback.classic.net.SMTPAppender">
<smtpHost>smtphost</smtpHost>
<to>user@admin</to>
<from>notify@admin</from>
<subject>Guard - %marker - %msg%n </subject>
<layout class="ch.qos.logback.classic.html.HTMLLayout">
<pattern>%date%level%logger{24}%msg</pattern>
</layout>
<evaluator class="ch.qos.logback.classic.boolex.JaninoEventEvaluator">
<expression>
marker != null && (marker.contains("NOTIFY_MAJOR") || marker.contains("NOTIFY_MINOR"))
</expression>
</evaluator>
</appender>
The following tag has to be added to the root level in order for this to work.
<appender-ref ref="SMTP" />
For example:
<root level="INFO">
<appender-ref ref="ASYNC"/>
<appender-ref ref="SMTP" />
</root>
Upgrading from 2.2.x to 2.4.0
There are a few things to consider when upgrading from a version 2.2.x to 2.4.0. In this section we will cover that upgrade path.
The first thing is that the file storage implementations are now being kept into two separate packages, i.e. open-xchange-guard-file-storage and open-xchange-guard-s3-storage for the file storage and S3 storage support respectively. This means that one of the packages has to be manually installed after the upgrade.
The second thing is that the open-xchange-guard system service is now gone. Since, OX Guard is intergraded into the OSGi stack, it will start automatically with the backend, i.e. when the open-xchange system service starts.
With those two things in mind, OX Guard can be upgraded as described here. Again, after the upgrade is complete there is no need to restart the open-xchange-guard system service; neither the open-xchange system service should be restarted just yet. There are a few things to do before having a fully functional OX Guard backend.
The Guest Reader component has moved to the open-xchange-guard-reader package, which needs to be installed on the frontend server next to App Suite UI.
Step 1 - Enhance repository sources
If you currently do not have an Open-Xchange middleware installed you have to adapt your Open-Xchange repository file to be able to solve required dependencies to the middleware packages. Add the following to the file that already should contain an entry for the OX Guard repository.
Debian 7.x
deb https://software.open-xchange.com/products/appsuite/stable/backend/DebianWheezy /
Debian 8.x
deb https://software.open-xchange.com/products/appsuite/stable/backend/DebianJessie /
RedHat Enterprise Linux 6 or CentOS 6
[ox-backend]
name=Open-Xchange-backend
baseurl=https://software.open-xchange.com/products/appsuite/stable/backend/RHEL6/
gpgkey=https://software.open-xchange.com/oxbuildkey.pub
enabled=1
gpgcheck=1
metadata_expire=0m
Redhat Enterprise Linux 7 or CentOS 7
[ox-backend]
name=Open-Xchange-backend
baseurl=https://software.open-xchange.com/products/appsuite/stable/backend/RHEL7/
gpgkey=https://software.open-xchange.com/oxbuildkey.pub
enabled=1
gpgcheck=1
metadata_expire=0m
SUSE Linux Enterprise Server 11.x
$ zypper ar https://software.open-xchange.com/products/appsuite/stable/backend/SLES11 ox-backend
SUSE Linux Enterprise Server 12.x
$ zypper ar https://software.open-xchange.com/products/appsuite/stable/backend/SLE_12 ox-backend
After adding new repositories retrieve new lists of packages by executing your desired package managers update call.
Step 2 - Update OX Guard
Upgrade the existing installation by executing your desired package mangers upgrade call, for instance apt-get dist-upgrade, yum upgrade and so on.
If you are running an OX Guard-only node (because of infelicitous set dependencies) you have to install an implementation of the package open-xchange-authentication and open-xchange-spamhandler which will never be used but checked for. We suggest to install open-xchange-authentication-database and open-xchange-spamhandler-default. Installing those packages as well as chosing a open-xchange-guard-storage implementation will install all required middleware dependencies.
After the update /etc/init.d/open-xchange-guard shouldn't be available any more but OX Guard will start when starting the open-xchange system service.
Step 3 - Choose and install your storage
According to your previous installation, you will have to manually install the relevant open-xchange-guard-storage package. Of course, you can switch to the one or the other storage if you wish, but be aware that no data will be migrated. You will have to do this manually.
File Storage Support
If you want the file storage support, and depending on your Linux distribution, you can install it with the following command:
Debian 7.x/8.x
apt-get install open-xchange-guard-file-storage
RedHat Enterprise Linux 6.x/7.x or CentOS 6.x/7.x
yum install open-xchange-guard-file-storage
SUSE Linux Enterprise Server 11.x/12.x
zypper in open-xchange-guard-file-storage
S3 Storage Support
If you want the S3 storage support, and depending on your Linux distribution, you can install it with the following command:
Debian 7.x/8.x
apt-get install open-xchange-guard-s3-storage
RedHat Enterprise Linux 6.x/7.x or CentOS 6.x/7.x
yum install open-xchange-guard-s3-storage
SUSE Linux Enterprise Server 11.x/12.x
zypper in open-xchange-guard-s3-storage
Step 4 - Install open-xchange-rest
OX Guard requires access to existing middleware hosts. The endpoint is configured within the guard-core.properties which you can find in /opt/open-xchange/etc. The endpoint configured via com.openexchange.guard.restApiHostname need to have the package open-xchange-rest installed to be able to retrieve and process OX Guard REST requests.
Normally the post installation scripts from the open-xchange-guard package should take over the formerly configured endpoint so that you do not have to do anything.
Step 5 - Ensure that the server keys are readable by the sever
The oxguardpass file should be owned by the open-xchange user and have the permission 400, meaning read access only by the owner of the file.
ls -l /opt/open-xchange/etc/oxguardpass
-r-------- 1 open-xchange root 64 Nov 17 19:14 /opt/open-xchange/etc/oxguardpass
Step 6 - Ensure that the Apache balancer directive uses the correct port
Since OX Guard is part of the OSGi stack and Grizzly is used to register and serve the OX Guard servlets, the oxguard and oxguardpgp balancer directives in the Apache configuration have to be adjusted in order to set the correct port and possibly removed.
If the Proxy balancer for the backend (oxcluster) is the same as the Guard servers, then you can remove the balancer entries for Guard and just redirect the ProxyPass directive to the oxcluster like so
ProxyPass /appsuite/api/oxguard balancer://oxcluster/oxguard ProxyPass /pks balancer://oxcluster/pgp
Otherwise, if the Guard servers are different from your backend, find the balancers containing
BalancerMember http://guardserver:8080/oxguard timeout=1800 smax=0 ttl=60 retry=60 loadfactor=100 route=OX1
BalancerMember http://guardserver:8080/pgp timeout=1800 smax=0 ttl=60 retry=60 loadfactor=100 route=OX1
The second Balancer (oxguardpgp) for PGP should just be removed. The primary Guard balancer should look like below, removing the sub-directory "/oxuard"
<Proxy balancer://oxguard> BalancerMember http://guardserver:8009 timeout=1800 smax=0 ttl=60 retry=60 loadfactor=100 route=OX1
The find the lines with the ProxyPass for </appsuite/api/oxguard> and change the balancer to //oxguard/oxguard Add the PGP balancer if you are exposing the PGP Public key server.
<Proxy /appsuite/api/oxguard> ProxyPass balancer://oxguard/oxguard </Proxy> <Proxy /pks> ProxyPass balancer://oxguard/pgp </Proxy>
Please note : You cannot have more than one BalancerMember with the same URL. So, if changing to the same URL as the OX Backend, you should just remove the Guard entries otherwise there will be some errors reported by Apache, and only the first entry will be used.
Step 7 - Adapt configuration
This chapter is only valid for OX Guard-only node where no additional middleware services haven't been desired and installed before! As Guard is now using existing middleware mechanisms to access the databases you have to configure the following properties.
configdb.properties- copy an already configured file from a middleware node to
/opt/open-xchange/etcon the OX Guard node. system.properties- copy an already configured file from a middleware node to
/opt/open-xchange/etcon the OX Guard node. server.properties- set
com.openexchange.connector.networkListenerHostto your desired hosts (e. g.*) logback.xml- consolidate your old OX Guard
logback.xmlwith the new one from within/opt/open-xchange/etc jolokia.properties- copy an already configured file from a middleware node to
/opt/open-xchange/etcon the OX Guard node. mpasswd- copy your existing
oxadminmastercredential storage to/opt/open-xchange/etcon the OX Guard node. guard-api.properties- Adapt the property
com.openexchange.guard.endpoint. This can be as simple as replacing the old port 8080 with 8009 but may differ depending on your setup - Adapt the property
com.openexchange.guard.externalReaderUrl. Adjust to the new location of the Guard reader under apache. Defaults to domain/guard/reader/reader.html
After copying those files, please make sure that they are owned by the group open-xchange, otherwise they cannot be read by the middleware service. In addition to the above mentioned properties you may would like to change the default configuration if the underlying middleware instance is only used for OX Guard:
- Disable Hazelcast cluster discovery
hazelcast.properties: com.openexchange.hazelcast.network.join=empty- remove all permissions set via configuration
permissions.properties: set entries tofalse
Step 8 - Restart the groupware
Last, and certainly not least, you need to restart the groupware process:
Debian 7.x/RedHat Enterprise Linux 6.x/CentOS 6.x/SUSE Linux Enterprise Server 11.x
service open-xchange restart
Debian 8.x/RedHat Enterprise Linux 7.x/CentOS 7.x/SUSE Linux Enterprise Server 12.x
systemctl restart open-xchange
Step 9 - Execute update tasks
Before OX Guard should receive the first requests you have to ensure that all database schemas are up to date. This can be done by executing /opt/open-xchange/sbin/runupdate.
Congratulations! You should now have a running OX Guard 2.4.0!
Common Problems
When trying to decode a message I get a 404
This happens probably because the AppSuite ProxyPass Location directives are overwriting the OX Guard ones. To solve this, you will have to move the OX Guard ProxyPass Location directive right before the AppSuite ones.
Change the ordering of the locations from this:
<Location /ajax>
ProxyPass balancer://oxcluster/ajax
</Location>
<Location /appsuite/api>
ProxyPass balancer://oxcluster/ajax
</Location>
<Location /appsuite/api/oxguard>
ProxyPass balancer://oxguard
</Location>
to this:
<Location /appsuite/api/oxguard>
ProxyPass balancer://oxguard
</Location>
<Location /ajax>
ProxyPass balancer://oxcluster/ajax
</Location>
<Location /appsuite/api>
ProxyPass balancer://oxcluster/ajax
</Location>
After migration, OX Guard fails to accept any request
This may happen if configuration files that were copied during migration do not have the correct permission set. Please have a look at the OX Guard middleware log file to find any ERROR messages. The getmissingservices command-line tool may also help to identify services that have not been started and why they failed to start.
After migration, Guest reader doesn't work and report too many redirects
The location of the Guest reader has been changed, and is now being handled by apache rather than the Jetty server previously used by Guard.
To maintain compatibility with the Guest emails previously sent, Guard tries to redirect the old URL to the new...which is configured as the externalReaderPath in the guard-core.properties file. Be sure to update this setting to the new location as mentioned above.