AppSuite:File Storages per User

This information is valid from 7.8.0

File Storages per user

This document tries to outline and explain the new concept of having the possibility to specify file storages on a per user basis that is introduced with Open‐Xchange Server v7.8.0.

The previous situation

Before the introduction of the file storages per user feature, it was only possible to specify the file storage association per context. That file storage was used for all modules/functions that are supposed to store/retrieve files:

• Drive
• Thumbnails
• PIM attachments
• Signature attachments

File storages are registered and maintained in file store table held in configDb database. File storage obtains a unique identifier, a base URI and a size limitation.

For each context a file storage identifier is specified alongside with a context quota that defines how much space that context is allowed to occupy in assigned file storage. Thus the sum of all quotas from those contexts assigned to file storage must not exceed the file storage’s size.

Motivation

The motivation is to differentiate between file storages for common context‐associated files and Drive file. Those Drive files may further reside in the common context‐associated storage, but it also possible to specify user‐associated file storages where Drive files are maintained. The association is determined based on a folder's creator/owner: The quota gets accounted to that user that created a folder (provided the user has a user‐associated file storage).

That kind of separation allows different provisioning scenarios that an administrator possibly wants to achieve. With regard to Drive files it is now possible to define per context that:

• All Drive files reside in the file storage associated with the context, which is either automatically determined or explicitly set (through -F,--destination-store-id option) when creating a context. All Drive files (from users of that context) and all other file resources (Snippets, PIM attachments, ...) share the same quota usage and limit settings. The file storage associated with the context is called context file storage.
• Certain users are supposed to store their Drive files in their own file storage with individual quota settings different from the context file storage. All files created in Drive folders owned by such a user are then accounted to that user-associated file storage having their own quota usage and limitation. The user-associated file storage may be set on user creation as it is for context creation. The file storage associated with a user is called user file storage. Introducing a user file storage is suitable when an administrator wants to have user-sensitive quota settings for Drive files and/or wants to assign a different type of storage for Drive files uploaded by users; e.g. let seldom used/accessed file resources reside in context file storage, but let frequently, highly accessed Drive files reside in a better storage sink
• A certain user has his own (bigger) file storage for Drive files and other users of that context are supposed to use that file storage to share the quota usage and limitation specified for that file storage. A file storage owned by a user that is also used by other users is called master file storage.

Having that possibilities of separation for quota usage and limitation it is possible to apply a certain use case scenario to contexts; e.g.

• Slow mass storage vs. fast Drive storage setup
• OX Drive Stand‐Alone setup

When assigning dedicated file storage for Drive to a certain user, the user’s file occupation is no more accounted to the overall context in which that user resides, but gets accounted to a dedicated user quota. Such a user quota can be used for different sales strategies that include tracking the user’s file occupation.

To achieve the file storage association on a per user basis, the existing user table has been enhanced by file storage and quota information as it is for the context table. Moreover the filestore_usage table has also been enhanced by the capability to account storage occupation to a certain user

In addition the user table has also been extended by filestore_owner column. That column is used to let a user’s file storage be combined with another user’s storage. This serves the scenario in which a grouped OX Drive Stand‐Alone setup is operated. Multiple users’ file occupation gets accounted to a master user account.

Furthermore Open‐Xchange allows changing the running setups at any time to adjust the file storage usages/associations according to possibly changing needs/requirements.

Note

Please note that with introducing the concept of having user-associated file storages, the output for "listfilestore" command-line tool changed. Since also a user can be accounted to a file storage's capacity, the output changed to be:

/opt/open-xchange/sbin/listfilestore -A oxadminmaster -P secret
 id path                      size reserved used max-entities cur-entities
  2 file:///var/opt/filestore 1000      300    0         5000            2

Hence, "maxctx" is replaced with "max-entities" and "curctx" is replaced with "cur-entities"

Seamless integration and flexibility

The new more flexible approach fits seamlessly into existing context/file storage setups. Thus there is no need to change anything, unless demanded by administrator

The fall‐back setup

Everything stays as is. There is a storage‐per‐context association and every file is held in that storage. There are no user‐specific file storage attributes set in user table.

Slow mass storage vs. fast file storage setup

There are two storages available: a file‐based and an S3 storage (assuming that the S3 one would be faster). While the context has the file‐based storage set, users in that context do have individual name spaces in the S3 storage.

That way common files (PIM/Signature attachments, thumbnails) are stored to context‐ associated file‐based storage while Drive documents are stored in the individual user name spaces in the S3 storage. Every user is allowed to occupy up to 10GB of space in the S3 storage (this can be varied as well).

OX Drive Stand‐Alone – Single User

Again there two file storages. Again the file‐based one is assumed to serve the slow mass storage behavior while S3 is assumed to be the fast storage.

Context designated to the Single User scenario for OX Drive Stand‐Alone operation, do hold only one user that has its individual name space associated in the S3 storage.

Again common files (PIM/Signature attachments, thumbnails) are stored in the context‐ associated file‐based storage while Drive documents are stored in the individual user name spaces in the S3 storage of each context. Every Drive document accounts the user‐sensitive quota of the associated user.

OX Drive Stand‐Alone – Business Account

There two different file storages as in the previous setups. Again the file‐based one is assumed to serve the slow mass storage behavior while S3 is assumed to be the fast storage.

For this setup users do not hold their own individual name space in the S3 storage, but all share the same name space of the master user in the S3 storage. Moreover there are no dedicated user quotas, but all Drive documents do account to the master user’s quota

Extensions to provisioning command‐line tools

createuser

The existing createuser CLI is enhanced by the following options

‐q/‐‐quota The file storage quota in MB for associated user
‐f/‐‐filestore The identifier for the file storage.
‐o/‐‐owner The owner for the file storage. If no owner is given and f/filestore option is set, the user itself is taken as owner

changeuser

The existing changeuser CLI is enhanced by the following option

‐q/‐‐quota The file storage quota in MB for associated user

Note: Option ALLOW_CHANGING_QUOTA_IF_NO_FILESTORE_SET in file 'AdminUser.properties' controls if a user file storage is automatically enabled if the ‐q/‐‐quota option is given on a 'changeuser' invocation as an alternative to 'movecontextfilestore2user'. For users without a user file storage the 'changeuser' CLT needs to be called with a quota value different to unlimited (-1) and unlimited quota will only be enabled in case the user already has a user file storage.

movecontextfilestore2user

Move a user’s files from a context to an individual storage name space.
Usage: movecontextfilestore2user

‐h,‐‐help	Prints a help text
‐‐environment	Show info about commandline environment
‐‐nonl	Remove all newlines (\n) from output
‐i,‐‐userid <userid>	| Id of the user
‐u,‐‐username <username>	| Username of the user
‐A,‐‐adminuser <adminuser>	? Admin username
‐P,‐‐adminpass <adminpass>	? Admin password
‐c,‐‐contextid <contextid>	| The id of the context
‐N,‐‐contextname <contextname>	| context name
‐f,‐‐filestore <filestore>	* The identifier for the file storage.
‐q,‐‐quota <quota>	* The file storage quota in MB for associated user.

Entries marked with an asterisk (*) are mandatory. Entries marked with an question mark (?) are mandatory depending on your configuration. Entries marked with a pipe (|) are mandatory for one another which means that at least one of them must be set.

moveuserfilestore2context

The counterpart for the previous CLI. Move a user's files from his individual storage name space to the context storage name space.
Usage: moveuserfilestore2context

‐h,‐‐help	Prints a help text
‐‐environment	Show info about commandline environment
‐‐nonl	Remove all newlines (\n) from output
‐i,‐‐userid <userid>	| Id of the user
‐u,‐‐username <username>	| Username of the user
‐A,‐‐adminuser <adminuser>	? Admin username
‐P,‐‐adminpass <adminpass>	? Admin password
‐c,‐‐contextid <contextid>	| The id of the context
‐N,‐‐contextname <contextname>	| context name

Entries marked with an asterisk (*) are mandatory. Entries marked with an question mark (?) are mandatory depending on your configuration. Entries marked with a pipe (|) are mandatory for one another that means that at least one of them must be set.

moveuserfilestore2master

Move a user's files from his individual storage name space to the storage name space of specified master.
Usage: moveuserfilestore2master

‐h,‐‐help	Prints a help text
‐‐environment	Show info about commandline environment
‐‐nonl	Remove all newlines (\n) from output
‐i,‐‐userid <userid>	| Id of the user
‐u,‐‐username <username>	| Username of the user
‐A,‐‐adminuser <adminuser>	? Admin username
‐P,‐‐adminpass <adminpass>	? Admin password
‐c,‐‐contextid <contextid>	| The id of the context
‐N,‐‐contextname <contextname>	| context name
‐m,‐‐master <master>	Master user id. If not set, the context administrator is assumed to be the master user.

Entries marked with an asterisk (*) are mandatory. Entries marked with an question mark (?) are mandatory depending on your configuration. Entries marked with a pipe (|) are mandatory for one another, which means that at least one of them, must be set.

movemasterfilestore2user

Move a user's files from a master account name space to an individual storage name space.
Usage: movemasterfilestore2user

‐h,‐‐help	Prints a help text
‐‐environment	Show info about commandline environment
‐‐nonl	Remove all newlines (\n) from output
‐i,‐‐userid <userid>	| Id of the user
‐u,‐‐username <username>	| Username of the user
‐A,‐‐adminuser <adminuser>	? Admin username
‐P,‐‐adminpass <adminpass>	? Admin password
‐c,‐‐contextid <contextid>	| The id of the context
‐N,‐‐contextname <contextname>	| context name
‐m,‐‐master <master>	Master user id. If not set, the context administrator is assumed to be the master user.
‐f,‐‐filestore <filestore>	* The identifier for the file storage.
‐q,‐‐quota <quota>	* The file storage quota in MB for associated user.

Entries marked with an asterisk (*) are mandatory. Entries marked with an question mark (?) are mandatory depending on your configuration. Entries marked with a pipe (|) are mandatory for one another, which means that at least one of them, must be set.

Creating a user file storage

Having the extensions to command-line tools a user-associated file storage can be enabled:

• After using 'movecontextfilestore2user' command-line tool
• Using 'changeuser' command-line tool with quota and filestore options or only quota options and "ALLOW_CHANGING_QUOTA_IF_NO_FILESTORE_SET" set to "true" in 'AdminUser.properties' file
• By provisioning via 'createuser' command-line tool with quota and filestore options

Useful command‐line tools

The following command‐line tools might be useful to get an overview

• 'listfilestore' to list available file storages that can be assigned to contexts/users
• 'listcontext' to get the quota limit specified per context or -1 if none
• 'listuser' to get the quota limit specified per user or -1 if none
• 'listuserfilestores' to get a listing of users that use their own user file storage

FAQs

When is a user filestore enabled or when does a user have an own filestore?

A user filestore has an own filestore if a filestore is assigned to this user. This can either be done via the create and update clt's or via the movecontextfilestore2user clt.

When is which quota affected?

In case the user has no own filestore the quota of the context filestore is affected. In case the user has an own filestore or in case the user is a slave user of another users filestore the quota of this user filestore is affected.

How can a user check his quota usage?

The 'Quota' portal widget shows quota and usage.

How to change the filestore quota shown in the portal widget?

To change the quota one just has to use the changeuser or changecontext CLT with the --quota (-q) argument. E.g.:

changecontext -A adminmaster -P masterpassword -c -q 1024

How does quota work for public files?

The quota of the folder creator/owner is in charge for files other users put inside. This affects the quota for users having an own filestore

What happens with ownership of public files and folders when the creator/owner gets deleted?

This depends which argument is used during the delete operation. In the default case, the context admin is assigned as the owner. But it is also possible that the data is assigned to a different user or that the data is completed removed.

What happens in over quota scenarios by deleting users?

If the data is assigned to a different user the quota limit is ignored. But the user (inheriting deleted user's files) is supposed to clean existing files as any attempt to create a Drive file will fail due to over-quota

How to deal with quota issues for the context admin caused by public files?

If the quota is too limited for the context admin the data should be completely removed or assigned to a different user. E.g.:

deleteuser -A ctxadmin -P password -c -i --no-reassign