Difference between revisions of "AppSuite:Running a cluster"

Line 1: Line 1:
<div class="title">Running a cluster</div>
+
<div class="title">OX Drive API</div>
  
 
__TOC__
 
__TOC__
  
= Concepts =
+
= Introduction =
  
For inter-OX-communication over the network, multiple Open-Xchange servers can form a cluster. This brings different advantages regarding distribution and caching of volatile data, load balancing, scalability, fail-safety and robustness. Additionally, it provides the infrastructure for upcoming features of the Open-Xchange server.
+
The module <code>drive</code> is used to synchronize files and folders between server and client, using a server-centric approach to allow an easy implementation on the client-side.
The clustering capabilities of the Open-Xchange server are mainly built up on [http://hazelcast.com Hazelcast], an open source clustering and highly scalable data distribution platform for Java. The following article provides an overview about the current featureset and configuration options.
 
  
= Requirements on HTTP routing =
+
The synchronization is based on checksums for files and folders, differences between the server- and client-side are determined using a three-way comparison of server, client and previously acknowledged file- and directory-versions. The synchronization logic is performed by the server, who instructs the client with a set of actions that should be executed in order to come to a synchronized state.
  
An OX cluster is always part of a larger picture. Usually there is front level loadbalancer as central HTTPS entry point to the platform. This loadbalancer optionally performs HTTPS termination and forwards HTTP(S) requests to webservers (the usual and only supported choice as of now is Apache). These webservers are performing HTTPS termination (if this is not happening on the loadbalancer) and serve static content, and (which is what is relevant for our discussion here) they forward dynamic requests to the OX backends.
+
Therefore, the client takes a snapshot of it's local files and directories, calculates their checksums, and sends them as a list to the server, along with a list of previously acknowledged checksums. The server takes a similar snapshot of the files and directories on the underlying file storages and evaluates which further actions are necessary for synchronization. After executing the server-side actions, the client receives a list of actions that should be executed on the client-side. These steps are repeated until the server-state matches the client-state.  
  
A central requirement for the interaction of these components (loadbalancer, webservers, OX nodes) is that we have session stability based on the JSESSIONID cookie / jsessionid path component suffix. This means that our application sets a cookie named JSESSIONID which has a value like <large decimal number>.<route identifier>, e.g. "5661584529655240315.OX1". The route identifier here ("OX1" in this example) is taken by the OX node from a configuration setting from a config file and is specific to one OX node. HTTP routing must happen such that HTTP requests with a cookie with such a suffix always end up the corresponding OX node. There are furthermore specific cirumstances when passing this information via cookie is not possible. Then the JSESSIONID is transferred in a path component as "jsessionid=..." in the HTTP request. The routing mechanism needs to take that into account also.
+
Key concept is that the synchronization works stateless, i.e. it can be interrupted and restarted at any time, following the eventual consistency model.  
  
There are mainly two options to implement this. If the Apache processes are running co-located on the same machines running the OX groupware processes, it is often desired to have the front level loadbalancer perform HTTP routing to the correct machines. If dedicated Apache nodes are employed, is is usually sufficient to have the front-level loadbalancer do HTTP routing to the Apache nodes in a round-robin fashion and perform routing to the correct OX nodes in the Apache nodes.
+
Entry point for the synchronization is the [[#Synchronize_folders|<code>syncfolders</code>]] request, where the directories are compared, and further actions are determined by the server, amongst others actions to synchronize the files in a specific directory using the [[#Synchronize_files_in_a_folder|<code>syncfiles</code>]] request. After executing the actions, the client should send another <code>syncfolders</code> request to the server and execute the returned actions (if present), or finish the synchronization if there are no more actions to execute. In pseudo-code, the synchronization routine could be implemented as follows:
  
We provide sample configuration files to configure Apache (with mod_proxy_http) to perform HTTP routing correctly in our guides on OXpedia, e.g. [[AppSuite:Main_Page_AppSuite#quickinstall]]. Central elements are the directives "ProxySet stickysession=JSESSIONID|jsessionid scolonpathdelim=On" in conjunction with the "route=OX1" parameters to the BalancerMember lines in the Proxy definition. This is valid for Apache 2.2 as of Sep-2014.
+
WHILE TRUE
 +
{
 +
  response = SYNCFOLDERS()
 +
  IF 0 == response.actions.length
 +
    BREAK
 +
  ELSE
 +
    EXECUTE(response.actions)
 +
}
  
How to configure a front level loadbalancer to perform HTTP equivalent HTTP routing is dependent on the specific loadbalancer implementation. If Apache is used as front level loadbalancer, the same configuration as discussed in the previous section can be employed. As of time of writing this text (Sep 2014), the alternative choices are thin. F5 BigIP is reported to be able to implement "jsessionid based persistence using iRules". nginx has the functionality in their commercial "nginx plus" product. (Both of these options have not been tested by OX.) Other loadbalancers with this functionality are not known to us.
+
Basically, it's up to the client how often such a synchronization cycle is initiated. For example, he could start a new synchronization cycle after a fixed interval, if he recognizes that the client directories have changed, or if he is informed that something has changed on the server by an event. It's also up to the client to interrupt the synchronization cycle at any time during execution of the actions and continue later on, however, it's recommended to start a new synchronization cycle each time to avoid possibly outdated actions.
  
If the front level loadbalancer is not capable of performing correct HTTP routing, is is required to configure correct HTTP routing on Apache level, even if Apache runs co-located on the OX nodes and thus cross-routing happens.
+
= API =
  
There are several reasons why we require session stability in exactly this way. We require session stabilty for horizontal scale-out; while we support transparent resuming / migration of user sessions in the OX cluster without need for users to re-authenticate, sessions wandering around randomly will consume a fixed amount resources corresponding to a running session on each OX node in the cluster, while a session sticky to one OX node will consume this fixed amount of resources only on one OX node. Furthermore there are mechanisms in OX like TokenLogin which work only of all requests beloning to one sequence get routed to the same OX node even if they stem from different machines with different IPs. Only the JSESSIONID (which in this case is transferred as jsessionid path component, as cookies do not work during a 302 redirect, which is part of this sequence) carries the required information where the request must be routed to.
+
As part of the [[HTTP_API|HTTP API]], the basic conventions for exchanging messages that described there are also valid for this case, especially the [[HTTP_API#Low_level_protocol|low level protocol]] and [[HTTP_API#Error_handling|error handling]]. Each request against the Drive API assumes a valid server session that is uniquely identified by the session id and the corresponding cookies and are sent with each request. A new session can be created via the [[HTTP_API#Module_.22login.22|login module]].
  
Usual "routing based on cookie hash" is not sufficient here since it disregards the information which machine originally issued the cookie. It only ensures that the session will be sticky to any target, which statistically will not be the same machine that issued the cookie. OX will then set a new JSESSIONID cookie, assuming the session had been migrated. The loadbalancer will then route the session to a different target, as the hash of the cookie will differ. This procedure then happens iteratively until by chance the routing based on cookie hash will route the session to the correct target. By then, a lot of resources will have been wasted, by creating full (short-term) sessions on all OX nodes. Furthermore, processes like TokenLogin will not work this way.
+
The root folder plays another important role for the message exchange. The root folder has a unique identifier. It is the parent server folder for the synchronization. All path details for directories and files are relative to this folder. This folder's id is sent with each request. To select the root folder during initial client configuration, the client may get a list of synchronizable folders with the [[#Get_synchronizable_Folders|<code>subfolders</code>]] action.
  
= Configuration =
+
Subsequently all transferred objects and all possible actions are listed.
  
All settings regarding cluster setup are located in the configuration file ''hazelcast.properties''. The former used additional files ''cluster.properties'', ''mdns.properties'' and ''static-cluster-discovery.properties'' are no longer needed. The following gives an overview about the most important settings - please refer to the inline documentation of the configuration file for more advanced options.
+
== File Version ==
  
Note: The configuration guide targets v7.4.0 of the OX server (and above). For older versions, please consult the history of this page.
+
A file in a directory is uniquely identified by its filename and the checksum of its content.  
  
== General ==
+
{| id="FileVersion" cellspacing="0" border="1"
 +
|+ align="bottom" | File Version
 +
! Name !! Type !! Value
 +
|-
 +
| name || String || The name of the file, including its extension, e.g. <code>test.doc</code>.
 +
|-
 +
| checksum || String || The MD5 hash of the file, expressed as a lowercase hexadecimal number string, 32 characters long, e.g. <code>f8cacac95379527cd4fa15f0cb782a09</code>.
 +
|}
  
To restrict access to the cluster and to separate the cluster from others in the local network, a name and password needs to be defined. Only backend nodes having the same values for those properties are able to join and form a cluster.
+
== Directory Version ==
  
  # Configures the name of the cluster. Only nodes using the same group name  
+
A directory is uniquely identified by its full path, relative to the root folder, and the checksum of its content.
  # will join each other and form the cluster. Required if  
+
 
  # "com.openexchange.hazelcast.network.join" is not "empty" (see below).
+
{| id="DirectoryVersion" cellspacing="0" border="1"
  com.openexchange.hazelcast.group.name=
+
|+ align="bottom" | Directory Version
 +
! Name !! Type !! Value
 +
|-
 +
| path || String || The path of the directory, including the directory's name, relative to the root folder, e.g. <code>/sub/test/letters</code>.
 +
|-
 +
| checksum || String || The MD5 hash of the directory, expressed as a lowercase hexadecimal number string, 32 characters long, e.g. <code>f8cacac95379527cd4fa15f0cb782a09</code>.
 +
|}
 +
 
 +
Note: the checksum of a directory is calculated based on its contents in the following algorithm:
 +
 
 +
* Build a list containing each file in the directory (not including subfolders or files in subfolders)
 +
* Ensure a lexicographically order in the following way:
 +
** Normalize the filename using the <code>NFC</code> normalization form (canonical decomposition, followed by canonical composition) - see http://www.unicode.org/reports/tr15/tr15-23.html for details
 +
** Encode the filename to an array of UTF-8 unsigned bytes (array of codepoints)
 +
** Compare the filename (encoded as byte array "fn1") to another one "fn2" using the following comparator algorithm:
 +
 
 +
min_length = MIN(LENGTH(fn1), LENGTH(fn2))
 +
FOR i = 0; i < min_length; i++
 +
{
 +
  result = fn1[i] - fn2[i]
 +
  IF 0 != result RETURN result
 +
}
 +
RETURN LENGTH(fn1) - LENGTH(fn2)
 +
 
 +
* Calculate the aggregated MD5 checksum for the directory based on each file in the ordered list:
 +
** Append the file's NFC-normalized (see above) name, encoded as UTF-8 bytes
 +
** Append the file's MD5 checksum string, encoded as UTF-8 bytes
 +
 
 +
== Actions ==
 +
 
 +
All actions are encoded in the following format. Depending on the action type, not all properties may be present.
 +
 
 +
{| id="Actions" cellspacing="0" border="1"
 +
|+ align="bottom" | Actions
 +
! Name !! Type !! Value
 +
|-
 +
| action || String || The type of action to execute, currently one of <code>acknowledge</code>, <code>edit</code>, <code>download</code>, <code>upload</code>, <code>remove</code>, <code>sync</code>, <code>error</code>.
 +
|-
 +
| version || Object || The (original) file- or directory-version referenced by the action.
 +
|-
 +
| newVersion || Object || The (new) file- or directory-version referenced by the action.
 +
|-
 +
| path || String || The path to the synchronized folder, relative to the root folder.
 +
|-
 +
| offset || Number || The requested start offset in bytes for file uploads.
 +
|-
 +
| totalLength || Number || The total length in bytes for file downloads.
 +
|-
 +
| contentType || String || The file's content type for downloads.
 +
|-
 +
| created || Timestamp || The file's creation time (always UTC, not translated into user time).
 +
|-
 +
| modified || Timestamp || The file's last modification time (always UTC, not translated into user time).
 +
|-
 +
| error || Object || The error object in case of synchronization errors.
 +
|-
 +
| quarantine || Boolean || The flag to indicate whether versions need to be excluded from synchronization.
 +
|-
 +
| reset || Boolean || The flag to indicate whether locally stored checksums should be invalidated.
 +
|-
 +
| stop || Boolean || The flag to signal that the client should stop the current synchronizsation cycle.
 +
|-
 +
| acknowledge || Boolean || The flag to signal if the client should not update it's stored checksums when performing an <code>EDIT</code> action.
 +
|-
 +
| thumbnailLink || String || A direct link to a small thumbnail image of the file if available (deprecated, available until API version 2).
 +
|-
 +
| previewLink || String || A direct link to a medium-sized preview image of the file if available (deprecated, available until API version 2).
 +
|-
 +
| directLink || String || A direct link to the detail view of the file in the web interface (deprecated, available until API version 2).
 +
|-
 +
| directLinkFragments || String || The fragments part of the direct link (deprecated, available until API version 2).
 +
|}
 +
 
 +
The following list gives an overview about the used action types:
 +
 
 +
=== <code>acknowledge</code> ===
 +
Acknowledges the successful synchronization of a file- or directory version, i.e., the client should treat the version as synchronized by updating the corresponding entry in its metadata store and including this updated information in all following <code>originalVersions</code> arrays of the <code>syncfiles</code> / <code>syncfolders</code> actions. Depending on the <code>version</code> and <code>newVersion</code> parameters of the action, the following acknowledge operations should be executed (exemplarily for directory versions, file versions are acknowledged in the same way):
 +
 
 +
* Example 1: Acknowledge a first time synchronized directory <br /> The server sends an <code>acknowledge</code> action where the newly synchronized directory version is encoded in the  <code>newVersion</code> parameter. The client should store the version in his local checksum store and send this version in the <code>originalVersions</code> array in upcoming <code>syncfolders</code> requests.
 +
{
 +
  "action" : "acknowledge",
 +
  "newVersion" : {
 +
      "path" : "/",
 +
      "checksum" : "d41d8cd98f00b204e9800998ecf8427e"
 +
  }
 +
}
 +
 
 +
* Example 2: Acknowledge a synchronized directory after updates <br /> The server sends an <code>acknowledge</code> action where the previous directory version is encoded in the <code>version</code>, and the newly synchronized directory in the <code>newVersion</code> parameter. The client should replace any previously stored entries of the directory version in his local checksum store with the updated version, and send this version in the <code>originalVersions</code> array in upcoming <code>syncfolders</code> requests.
 +
{
 +
  "action" : "acknowledge",
 +
  "newVersion" : {
 +
    "path" : "/",
 +
    "checksum" : "7bb1f1a550e9b9ab4be8a12246f9d5fb"
 +
  },
 +
  "version" : {
 +
    "path" : "/",
 +
    "checksum" : "d41d8cd98f00b204e9800998ecf8427e"
 +
  }
 +
}
 +
 
 +
* Example 3: Acknowledge the deletion of a previously synchronized directory <br /> The server sends an <code>acknowledge</code> where the <code>newVersion</code> parameter is set to <code>null</code> to acknowledge the deletion of the previously synchronized directory version as found in the <code>version</code> parameter. The client should remove any stored entries for this directory from his local checksum store, and no longer send this version in the <code>originalVersions</code> array in upcoming <code>syncfolders</code> requests. <br /> Note that an acknowledged deletion of a directory implicitly acknowledges the deletion of all contained files and subfolders, too, so the client should also remove those <code>originalVersion</code>s from his local checksum store.
 +
{
 +
  "action" : "acknowledge",
 +
  "version" : {
 +
    "path" : "/test",
 +
    "checksum" : "3525d6f28eb8cb30eb61ab7932367c35"
 +
  }
 +
}
 +
 
 +
=== <code>edit</code> ===
 +
Instructs the client to edit a file- or directory version. This is used for move/rename operations. The <code>version</code> parameter is set to the version as sent in the <code>clientVersions</code> array of the preceding <code>syncfiles</code>/</code>syncfolders</code> action. The <code>newVersion</code> contains the new name/path the client should use. Unless the optional boolean parameter <code>acknowledge</code> is set to <code>false</code> an <code>edit</code> action implies that the client updates its known versions store accordingly, i.e. removes the previous entry for <code>version</code> and adds a new entry for <code>newVersion</code>.
 +
When editing a directory version, the client should implicitly take care to create any not exisiting subdirectories in the <code>path</code> of the <code>newVersion</code> parameter.
 +
A concurrent client-side modification of the file/directory version can be detected by the client by comparing the current checksum against the one in the passed <code>newVersion</code> parameter.
 +
 
 +
* Example 1: Rename a file <br /> The server sends an <code>edit</code> action where the source file is encoded in the <code>version</code>, and the target file in the <code>newVersion</code> parameter. The client should rename the file identified by the <code>version</code> parameter to the name found in the <code>newVersion</code> parameter. Doing so, the stored checksum entry for the file in <code>version</code> should be updated, too, to reflect the changes.
 +
{
 +
  "path" : "/",
 +
  "action" : "edit",
 +
  "newVersion" : {
 +
    "name" : "test_1.txt",
 +
    "checksum" : "03395a94b57eef069d248d90a9410650"
 +
  },
 +
  "version" : {
 +
    "name" : "test.txt",
 +
    "checksum" : "03395a94b57eef069d248d90a9410650"
 +
  }
 +
}
 +
 
 +
* Example 2: Move a directory <br /> The server sends an <code>edit</code> action where the source directory is encoded in the <code>version</code>, and the target directory in the <code>newVersion</code> parameter. The client should move the directory identified by the <code>version</code> parameter to the path found in the <code>newVersion</code> parameter. Doing so, the stored checksum entry for the directory in <code>version</code> should be updated, too, to reflect the changes.
 +
{
 +
  "action" : "edit",
 +
  "newVersion" : {
 +
    "path" : "/test2",
 +
    "checksum" : "3addd6de801f4a8650c5e089769bdb62"
 +
  },
 +
  "version" : {
 +
    "path" : "/test1/test2",
 +
    "checksum" : "3addd6de801f4a8650c5e089769bdb62"
 +
  }
 +
  }
 +
 
 +
* Example 3: Rename a conflicting file <br /> The server sends an <code>edit</code> action where the original client file is encoded in the <code>version</code>, and the target filename in the <code>newVersion</code> parameter. The client should rename the file identified by the <code>version</code> parameter to the new filename found in the <code>newVersion</code> parameter. If the <code>acknowledge</code> parameter is set to <code>true</code> or is not set, the stored checksum entry for the file in <code>version</code> should be updated, too, to reflect the changes, otherwise, as in this example, no changes should be done to the stored checksums.
 +
{
 +
  "action" : "edit",
 +
  "version" : {
 +
    "checksum" : "fade32203220752f1fa0e168889cf289",
 +
    "name" : "test.txt"
 +
  },
 +
  "newVersion" : {
 +
    "checksum" : "fade32203220752f1fa0e168889cf289",
 +
    "name" : "test (TestDrive).txt"
 +
  },
 +
  "acknowledge" : false,
 +
  "path" : "/"
 +
}
 +
 
 +
=== <code>download</code> ===
 +
Contains information about a file version the client should download. For updates of existing files, the previous client version is supplied in the <code>version</code> parameter. For new files, the <code>version</code> parameter is omitted. The <code>newVersion</code> holds the target file version, i.e. filename and checksum, and should be used for the following <code>download</code> request. The <code>totalLength</code> parameter is set to the file size in bytes, allowing the client to recognize when a download is finished. Given the supplied checksum, the client may decide on its own if the target file needs to be downloaded from the server, or can be created by copying a file with the same checksum to the target location, e.g. from a trash folder. The file's content type can be retrieved from the <code>contentType</code> parameter, similar to the file's creation and modification times that are availble in the <code>created</code> and <code>modified</code> parameters.
 +
 
 +
* Example 1: Download a new file <br /> The server sends a <code>download</code> action where the file version to download is encoded in the <code>newVersion</code> paramter. The client should download and save the file as indicated by the <code>name</code> property of the <code>newVersion</code> in the directory identified by the supplied <code>path</code>. After downloading, the <code>newVersion</code> should be added to the client's known file versions database.
 +
{
 +
  "totalLength" : 536453,
 +
  "path" : "/",
 +
  "action" : "download",
 +
  "newVersion" : {
 +
    "name" : "test.pdf",
 +
    "checksum" : "3e0d7541b37d332c42a9c3adbe34aca2"
 +
  },
 +
  "contentType" : "application/pdf",
 +
  "created" : 1375276738232,
 +
  "modified" : 1375343720985
 +
}
 +
 
 +
* Example 2: Download an updated file <br /> The server sends a <code>download</code> action where the previous file version is encoded in the <code>version</code>, and the file version to download in the <code>newVersion</code> parameter. The client should download and save the file as indicated by the <code>name</code> property of the <code>newVersion</code> in the directory identified by the supplied <code>path</code>, replacing the previous file. After downloading, the <code>newVersion</code> should be added to the client's known file versions database, replacing an existing entry for the previous <code>version</code>.
 +
  {
 +
  "totalLength" : 1599431,
 +
  "path" : "/",
 +
  "action" : "download",
 +
  "newVersion" : {
 +
    "name" : "test.pdf",
 +
    "checksum" : "bb198790904f5a1785d7402b0d8c390e"
 +
  },
 +
  "contentType" : "application/pdf",
 +
  "version" : {
 +
    "name" : "test.pdf",
 +
    "checksum" : "3e0d7541b37d332c42a9c3adbe34aca2"
 +
  },
 +
  "created" : 1375276738232,
 +
  "modified" : 1375343720985
 +
}
 +
 
 +
=== <code>upload</code> ===
 +
Instructs the client to upload a file to the server. For updates of existing files, the previous server version is supplied in the <code>version</code> parameter, and should be used for the following <code>upload</code> request. For new files, the <code>version</code> parameter is omitted. The <code>newVersion</code> holds the target file version, i.e. filename and checksum, and should be used for the following <code>upload</code> request. When resuming a previously partly completed upload, the <code>offset</code> parameter contains the offset in bytes from which the file version should be uploaded by the client. If possible, the client should set the <code>contentType</code> parameter for the uploaded file, otherwise, the content type falls back to <code>application/octet-stream</code>.
 +
 
 +
=== <code>remove</code> ===
 +
Instructs the client to delete a file or directory version. The <code>version</code> parameter contains the version to delete. A deletion also implies a removal of the corresponding entry in the client's known versions store.
 +
A concurrent client-side modification of the file/directory version can be detected by comparing the current checksum against the one in the passed <code>version</code> parameter.
 +
 
 +
* Example 1: Remove a file <br /> The server sends a <code>remove</code> action where the file to be removed is encoded as <code>version</code> parameter. The <code>newVersion</code> parameter is not set in the action. The client should delete the file identified by the <code>version</code> parameter. A stored checksum entry for the file in <code>version</code> should be removed, too, to reflect the changes. The <code>newVersion</code> parameter is not set in the action.
 +
{
 +
  "path" : "/test2",
 +
  "action" : "remove",
 +
  "version" : {
 +
    "name" : "test.txt",
 +
    "checksum" : "03395a94b57eef069d248d90a9410650"
 +
  }
 +
}
 +
 
 +
* Example 2: Remove a directory <br /> The server sends a <code>remove</code> action where the directory to be removed is encoded as <code>version</code> parameter. The <code>newVersion</code> parameter is not set in the action. The client should delete the directory identified by the <code>version</code> parameter. A stored checksum entry for the directory in <code>version</code> should be removed, too, to reflect the changes.
 +
{
 +
  "action" : "remove",
 +
  "version" : {
 +
    "path" : "/test1",
 +
    "checksum" : "d41d8cd98f00b204e9800998ecf8427e"
 +
  }
 +
}
 +
 
 +
=== <code>sync</code> ===
 +
The client should trigger a synchronization of the files in the directory supplied in the <code>version</code> parameter using the <code>syncfiles</code> request. A <code>sync</code> action implies the client-side creation of the referenced directory if it not yet exists, in case of a new directory on the server. \\
 +
If the <code>version</code> parameter is not specified, a synchronization of all folders using the <code>syncfolders</code> request should be initiated by the client. \\
 +
If the <code>reset</code> flag in the <code>SYNC</code> action is set to <code>true</code>, the client should reset his local state before synchronizing the files in the directory. This may happen when the server detects a synchronization cycle, or believes something else is going wrong. Reset means that the client should invalidate any stored original checksums for the directory itself and any contained files, so that they get re-calculated upon the next synchronization. If the <code>reset</code> flag is set in a <code>SYNC</code> action without a apecific directory version, the client should invalidate any stored checksums, so that all file- and directory-versions get re-calculated during the following synchronizations.
 +
 
 +
* Example 1: Synchronize folder <br /> The server sends a <code>sync</code> action with a <code>version</code>. The client should trigger a <code>syncfiles</code> request for the specified folder.
 +
{
 +
  "action": "sync",
 +
  "version": {
 +
    "path": "<folder>",
 +
    "checksum": "<md5>"
 +
  }
 +
}
 +
 
 +
* Example 2: Synchronize all folders <br /> The server sends a <code>sync</code> action without <code>version</code> (or version is //null//). The client should trigger a <code>syncfolder</code> request, i.e. the client should synchronize all folders.
 +
{
 +
  "action": "sync",
 +
  "version": null
 +
}
 +
 
 +
=== <code>error</code> ===
 +
With the <code>error</code> action, file- or directory versions causing a synchronization problem can be identified. The root cause of the error is encoded in the <code>error</code> parameter as described at the [[HTTP_API#Error_handling|HTTP API]].
 +
 
 +
Basically, there are two scenarios where either the errorneous version affects the synchronization state or not. For example, a file that was deleted at the client without sufficient permissions on the server can just be downloaded again by the client, and afterwards, client and server are in-sync again. On the other hand, e.g. when creating a new file at the client and this file can't be uploaded to the server due to missing permissions, the client is out of sync as long as the file is present. Therefore, the boolean parameter <code>quarantine</code> instructs the client whether the file or directory version must be excluded from the synchronization or not. If it is set to <code>true</code>, the client should exclude the version from the <code>clientVersions</code> array, and indicate the issue to the enduser. However, if the synchronization itself is not affected and the <code>quarantine</code> flag is set to <code>false</code>, the client may still indicate the issue once to the user in the background, e.g. as a balloontip notification.
 +
 
 +
The client may reset it's quarantined versions on it's own, e.g. if the user decides to "try again", or automatically after a configurable interval.
 +
 
 +
The server may also decide that further synchronization should be suspended, e.g. in case of repeated synchronization problems. Such a situation is indicated with the parameter <code>stop</code> set to <code>true</code>. In this case, the client should at least cancel the current synchronization cycle. If appropriate, the client should also be put into a 'paused' mode, and the user should be informed accordingly.
 +
 
 +
There may also be situations where a error or warning is sent to the client, independently of a file- or directory version, e.g. when the client version is outdated and a newer version is available for download.
 +
 
 +
The most common examples for errors are insufficient permissions or exceeded quota restrictions, see examples below.
 +
 
 +
* Example 1: Create a file in a read-only folder <br /> The server sends an <code>error</code> action where the errorneous file is encoded in the <code>newVersion</code> parameter and the <code>quarantine</code> flag is set to <code>true</code>. The client should exclude the version from the <code>clientVersions</code> array in upcoming <code>syncFiles</code> requests so that it doesn't affect the synchronization algorithm. The error message and further details are encoded in the <code>error</code> object of the action.
 +
  {
 +
  "error" : {
 +
    "category" : 3,
 +
    "error_params" : ["/test"],
 +
    "error" : "You are not allowed to create files at \"/test\"",
 +
    "error_id" : "1358320776-69",
 +
    "categories" : "PERMISSION_DENIED",
 +
    "code" : "DRV-0012"
 +
  },
 +
  "path" : "/test",
 +
  "quarantine" : true,
 +
  "action" : "error",
 +
  "newVersion" : {
 +
    "name" : "test.txt",
 +
    "checksum" : "3f978a5a54cef77fa3a4d3fe9a7047d2"
 +
  }
 +
}
 +
 
 +
* Example 2: Delete a file without sufficient permissions <br /> Besides a new <code>download</code> action to restore the locally deleted file again, the server sends an <code>error</code> action where the errorneous file is encoded in the <code>version</code> parameter and the <code>quarantine</code> flag is set to <code>false</code>. Further synchronizations are not affected, but the client may still inform the user about the rejected operation. The error message and further details are encoded in the <code>error</code> object of the action.
 +
{
 +
  "error" : {
 +
    "category" : 3,
 +
    "error_params" : ["test.png", "/test"],
 +
    "error" : "You are not allowed to delete the file \"test.png\" at \"/test\"",
 +
    "error_id" : "1358320776-74",
 +
    "categories" : "PERMISSION_DENIED",
 +
    "code" : "DRV-0011"
 +
  },
 +
  "path" : "/test",
 +
  "quarantine" : false,
 +
  "action" : "error",
 +
  "newVersion" : {
 +
    "name" : "test.png",
 +
    "checksum" : "438f06398ce968afdbb7f4db425aff09"
 +
  }
 +
}
 +
 
 +
* Example 3: Upload a file that exceeds the quota <br /> The server sends an <code>error</code> action where the errorneous file is encoded in the <code>newVersion</code> parameter and the <code>quarantine</code> flag is set to <code>true</code>. The client should exclude the version from the <code>clientVersions</code> array in upcoming <code>syncFiles</code> requests so that it doesn't affect the synchronization algorithm. The error message and further details are encoded in the <code>error</code> object of the action.
 +
{
 +
  "error" : {
 +
    "category" : 3,
 +
    "error_params" : [],
 +
    "error" : "The allowed Quota is reached",
 +
    "error_id" : "-485491844-918",
 +
    "categories" : "PERMISSION_DENIED",
 +
    "code" : "DRV-0016"
 +
  },
 +
  "path" : "/",
 +
  "quarantine" : true,
 +
  "action" : "error",
 +
  "newVersion" : {
 +
    "name" : "test.txt",
 +
    "checksum" : "0ca6033e2a9c2bea1586a2984bf111e6"
 +
  }
 +
}
 +
 
 +
* Example 4: Synchronize with a client where the version is no longer supported. <br /> The server sends an <code>error</code> action with code <code>DRV-0028</code> and an appropriate error message. The <code>stop</code> flag is set to <code>true</code> to interrupt the synchronization cycle.
 +
{
 +
  "stop" : true,
 +
  "error" : {
 +
    "category" : 13,
 +
    "error_params" : [],
 +
    "error" : "The client application you're using is outdated and no longer supported - please upgrade to a newer version.",
 +
    "error_id" : "103394512-13",
 +
    "categories" : "WARNING",
 +
    "code" : "DRV-0028",
 +
    "error_desc" : "Client outdated - current: \"0.9.2\", required: \"0.9.10\""
 +
  },
 +
  "quarantine" : false,
 +
  "action" : "error"
 +
}
 +
 
 +
* Example 5: Synchronize with a client where a new version of the client application is available. <br /> The server sends an <code>error</code> action with code <code>DRV-0029</code> and an appropriate error message. The <code>stop</code> flag is set to <code>false</code> to indicate that the synchronization can continue.
 +
{
 +
  "stop" : false,
 +
  "error" : {
 +
    "category" : 13,
 +
    "error_params" : [],
 +
    "error" : "A newer version of your client application is available for download.",
 +
    "error_id" : "103394512-29",
 +
    "categories" : "WARNING",
 +
    "code" : "DRV-0029",
 +
    "error_desc" : "Client update available - current: \"0.9.10\", available: \"0.9.12\""
 +
  },
 +
  "quarantine" : false,
 +
  "action" : "error"
 +
}
 +
 +
== Synchronize folders ==
 +
 
 +
This request performs the synchronization of all folders, resulting in different actions that should be executed on the client afterwards. This operation typically serves as an entry point for a synchronization cycle.
 +
 
 +
PUT <code>/ajax/drive?action=syncfolders</code>
 +
 
 +
Parameters:
 +
* <code>session</code> - A session ID previously obtained from the login module.
 +
* <code>root</code> - The ID of the referenced root folder on the server.
 +
* <code>version</code> - The current client version (matching the pattern <code>^[0-9]+(\\.[0-9]+)*$</code>). If not set, the initial version <code>0</code> is assumed.
 +
* <code>apiVersion</code> - The API version that the client is using. If not set, the initial version <code>0</code> is assumed.
 +
* <code>diagnostics</code> (optional) - If set to <code>true</code>, an additional diagnostics trace is supplied in the response.
 +
* <code>pushToken</code> (optional) - The client's push registration token to associate it to generated events.
 +
 
 +
Request Body: <br />
 +
A JSON object containing two JSON arrays named <code>clientVersions</code> and <code>originalVersions</code>. The client versions array lists all current directories below the root directory as a flat list, encoded as [[#Directory_Version|Directory Versions]]. The original versions array contains all previously known directories, i.e. all previously synchronized and acknowledged directories, also encoded as [[#Directory_Version|Directory Versions]]. \\
 +
Optionally, available since API version 2, the JSON object may also contain two arrays named <code>fileExclusions</code> and <code>directoryExclusions</code> to define client-side exclusion filters, with each element encoded as [[#File_pattern|File patterns]] and [[#Directory_pattern|Directory patterns]] accordingly. See [[#Client_side_filtering]] for details.
 +
 
 +
Response: <br />
 +
A JSON array containing all actions the client should execute for synchronization. Each array element is an action as described in [[#Actions | Actions]]. <br /> If the <code>diagnostics</code> flag was set (either to <code>true</code> or <code>false</code>), this array is wrapped into an additional JSON object in the <code>actions</code> parameter, and the diagnostics trace is provided at <code>diagnostics</code>.
 +
 
 +
Example:
 +
==> PUT http://192.168.32.191/ajax/drive?action=syncfolders&root=56&session=5d0c1e8eb0964a3095438b450ff6810f
 +
  > Content:
 +
    {
 +
      "clientVersions" : [{
 +
          "path" : "/",
 +
          "checksum" : "7b744b13df4b41006495e1a15327368a"
 +
        }, {
 +
          "path" : "/test1",
 +
          "checksum" : "3ecc97334d7f6bf2b795988092b8137e"
 +
        }, {
 +
          "path" : "/test2",
 +
          "checksum" : "56534fc2ddcb3b7310d3ef889bc5ae18"
 +
        }, {
 +
          "path" : "/test2/test3",
 +
          "checksum" : "c193fae995d9f9431986dcdc3621cd98"
 +
        }
 +
      ],
 +
      "originalVersions" : [{
 +
          "path" : "/",
 +
          "checksum" : "7b744b13df4b41006495e1a15327368a"
 +
        }, {
 +
          "path" : "/test2/test3",
 +
          "checksum" : "c193fae995d9f9431986dcdc3621cd98"
 +
        }, {
 +
          "path" : "/test2",
 +
          "checksum" : "35d1b51fdefbee5bf81d7ae8167719b8"
 +
        }, {
 +
          "path" : "/test1",
 +
          "checksum" : "3ecc97334d7f6bf2b795988092b8137e"
 +
        }
 +
      ]
 +
    }
 +
 
 +
<== HTTP 200 OK (8.0004 ms elapsed, 102 bytes received)
 +
<  Content:
 +
    {
 +
      "data" : [{
 +
          "action" : "sync",
 +
          "version" : {
 +
            "path" : "/test2",
 +
            "checksum" : "56534fc2ddcb3b7310d3ef889bc5ae18"
 +
          }
 +
        }
 +
      ]
 +
    }
 +
 
 +
Example 2:
 +
==> PUT http://192.168.32.191/ajax/drive?action=syncfolders&root=56&session=5d0c1e8eb0964a3095438b450ff6810f
 +
  > Content:
 +
    {
 +
      "clientVersions" : [{
 +
          "path" : "/",
 +
          "checksum" : "7b744b13df4b41006495e1a15327368a"
 +
        }, {
 +
          "path" : "/test1",
 +
          "checksum" : "3ecc97334d7f6bf2b795988092b8137e"
 +
        }, {
 +
          "path" : "/test2",
 +
          "checksum" : "56534fc2ddcb3b7310d3ef889bc5ae18"
 +
        }, {
 +
          "path" : "/test2/test3",
 +
          "checksum" : "c193fae995d9f9431986dcdc3621cd98"
 +
        }
 +
      ],
 +
      "originalVersions" : [{
 +
          "path" : "/",
 +
          "checksum" : "7b744b13df4b41006495e1a15327368a"
 +
        }, {
 +
          "path" : "/test2/test3",
 +
          "checksum" : "c193fae995d9f9431986dcdc3621cd98"
 +
        }, {
 +
          "path" : "/test2",
 +
          "checksum" : "35d1b51fdefbee5bf81d7ae8167719b8"
 +
        }, {
 +
          "path" : "/test1",
 +
          "checksum" : "3ecc97334d7f6bf2b795988092b8137e"
 +
        }
 +
      ]
 +
      "fileExclusions" : [{
 +
          "path" : "/",
 +
          "name" : "excluded.txt",
 +
          "type" : "exact"
 +
        }
 +
      ], "directoryExclusions" : [{
 +
          "path" : "/temp",
 +
          "type" : "exact"
 +
        }, {
 +
          "path" : "/temp/*",
 +
          "type" : "glob"
 +
        }
 +
      ]
 +
    }
 +
 
 +
<== HTTP 200 OK (8.0004 ms elapsed, 102 bytes received)
 +
  <  Content:
 +
    {
 +
      "data" : [{
 +
          "action" : "sync",
 +
          "version" : {
 +
            "path" : "/test2",
 +
            "checksum" : "56534fc2ddcb3b7310d3ef889bc5ae18"
 +
          }
 +
        }
 +
      ]
 +
    }
 +
 
 +
 
 +
== Synchronize files in a folder ==
 +
 
 +
This request performs the synchronization of a single folder, resulting in different actions that should be executed on the client afterwards. This action is typically executed as result of a <code>syncfolders</code> action.
 +
 
 +
PUT <code>/ajax/drive?action=syncfiles</code>
 +
 
 +
Parameters:
 +
* <code>session</code> - A session ID previously obtained from the login module.
 +
* <code>root</code> - The ID of the referenced root folder on the server.
 +
* <code>path</code> - The path to the synchronized folder, relative to the root folder.
 +
* <code>device</code> (optional) - A friendly name identifying the client device from a user's point of view, e.g. "My Tablet PC".
 +
* <code>apiVersion</code> - The API version that the client is using. If not set, the initial version <code>0</code> is assumed.
 +
* <code>diagnostics</code> (optional) - If set to <code>true</code>, an additional diagnostics trace is supplied in the response.
 +
* <code>columns</code> (optional) -  A comma-separated list of columns representing additional metadata that is relevant for the client. Each column is specified by a numeric column identifier. Column identifiers for file metadata are defined in [[#File Metadata]]. If available, the requested metadata of files is included in the corresponsing <code>DOWNLOAD</code> and <code>ACKNOWLEDGE</code> actions (deprecated, available until API version 2).
 +
* <code>pushToken</code> (optional) - The client's push registration token to associate it to generated events.
 +
 
 +
Request Body: <br />
 +
A JSON object containing two JSON arrays named <code>clientVersions</code> and <code>originalVersions</code>. The client versions array lists all current files in the client directory, encoded as [[#File Version | File Versions]]. The original versions array contains all previously known files, i.e. all previously synchronized and acknowledged files, also encoded as [[#File Version | File Versions]]. \\
 +
Optionally, available since API version 2, the JSON object may also contain an array named <code>fileExclusions</code> to define client-side exclusion filters, with each element encoded as [[#File pattern | File patterns]]. See [[#Client side filtering]] for details.
 +
 
 +
Response: <br />
 +
A JSON array containing all actions the client should execute for synchronization. Each array element is an action as described in [[#Actions | Actions]]. <br /> If the <code>diagnostics</code> flag was set (either to <code>true</code> or <code>false</code>), this array is wrapped into an additional JSON object in the <code>actions</code> parameter, and the diagnostics trace is provided at <code>diagnostics</code>.
 +
 
 +
Example:
 +
==> PUT http://192.168.32.191/ajax/drive?action=syncfiles&root=56&path=/test2&device=Laptop&session=5d0c1e8eb0964a3095438b450ff6810f
 +
  > Content:
 +
    {
 +
      "clientVersions" : [{
 +
          "name" : "Jellyfish.jpg",
 +
          "checksum" : "5a44c7ba5bbe4ec867233d67e4806848"
 +
        }, {
 +
          "name" : "Penguins.jpg",
 +
          "checksum" : "9d377b10ce778c4938b3c7e2c63a229a"
 +
        }
 +
      ],
 +
      "originalVersions" : [{
 +
          "name" : "Jellyfish.jpg",
 +
          "checksum" : "5a44c7ba5bbe4ec867233d67e4806848"
 +
        }
 +
      ]
 +
    }
 
   
 
   
  # The password used when joining the cluster. Defaults to "wtV6$VQk8#+3ds!a".  
+
  <== HTTP 200 OK (6.0004 ms elapsed, 140 bytes received)
# Please change this value, and ensure it's equal on all nodes in the cluster.
+
<  Content:
com.openexchange.hazelcast.group.password=wtV6$VQk8#+3ds!a
+
    {
 +
      "data" : [{
 +
          "path" : "/test2",
 +
          "action" : "upload",
 +
          "newVersion" : {
 +
            "name" : "Penguins.jpg",
 +
            "checksum" : "9d377b10ce778c4938b3c7e2c63a229a"
 +
          },
 +
          "offset" : 0
 +
        }
 +
      ]
 +
    }
  
== Network ==
+
Example 2:
 +
==> PUT http://192.168.32.191/ajax/drive?action=syncfiles&root=56&path=/test2&device=Laptop&session=5d0c1e8eb0964a3095438b450ff6810f
 +
  > Content:
 +
    {
 +
      "clientVersions" : [{
 +
          "name" : "Jellyfish.jpg",
 +
          "checksum" : "5a44c7ba5bbe4ec867233d67e4806848"
 +
        }, {
 +
          "name" : "Penguins.jpg",
 +
          "checksum" : "9d377b10ce778c4938b3c7e2c63a229a"
 +
        }
 +
      ],
 +
      "originalVersions" : [{
 +
          "name" : "Jellyfish.jpg",
 +
          "checksum" : "5a44c7ba5bbe4ec867233d67e4806848"
 +
        }
 +
      ]
 +
      "fileExclusions" : [{
 +
          "path" : "*",
 +
          "name" : "*.tmp",
 +
          "type" : "glob"
 +
        }
 +
      ]
 +
    }
 +
 +
<== HTTP 200 OK (6.0004 ms elapsed, 140 bytes received)
 +
<  Content:
 +
    {
 +
      "data" : [{
 +
          "path" : "/test2",
 +
          "action" : "upload",
 +
          "newVersion" : {
 +
            "name" : "Penguins.jpg",
 +
            "checksum" : "9d377b10ce778c4938b3c7e2c63a229a"
 +
          },
 +
          "offset" : 0
 +
        }
 +
      ]
 +
    }
  
It's required to define the network interface that is used for cluster communication via ''com.openexchange.hazelcast.network.interfaces''. By default, the interface is restricted to the local loopback address only. To allow the same configuration amongst all nodes in the cluster, it's recommended to define the value using a wildcard matching the IP addresses of all nodes participating in the cluster, e.g. ''192.168.0.*''
+
== Download a file ==
  
# Comma-separated list of interface addresses hazelcast should use. Wildcards
+
Downloads a file from the server.
# (*) and ranges (-) can be used. Leave blank to listen on all interfaces
 
# Especially in server environments with multiple network interfaces, it's
 
# recommended to specify the IP-address of the network interface to bind to
 
# explicitly. Defaults to "127.0.0.1" (local loopback only), needs to be
 
# adjusted when building a cluster of multiple backend nodes.
 
com.openexchange.hazelcast.network.interfaces=127.0.0.1
 
  
To form a cluster of multiple OX server nodes, different discovery mechanisms can be used. The discovery mechanism is specified via the property ''com.openexchange.hazelcast.network.join'':
+
GET <code>/ajax/drive?action=download</code>
  
# Specifies which mechanism is used to discover other backend nodes in the
+
or
# cluster. Possible values are "empty" (no discovery for single-node setups),
 
# "static" (fixed set of cluster member nodes) or "multicast" (automatic
 
# discovery of other nodes via multicast). Defaults to "empty". Depending on
 
# the specified value, further configuration might be needed, see "Networking"
 
# section below.
 
com.openexchange.hazelcast.network.join=empty
 
  
Generally, it's advised to use the same network join mechanism for all nodes in the cluster, and, in most cases, it's strongly recommended to use a ''static'' network join configuration. This will allow the nodes to join the cluster directly upon startup. With a ''multicast'' based setup, nodes will merge to an existing cluster possibly at some later time, thus not being able to access the distributed data until they've joined.
+
PUT <code>/ajax/drive?action=download</code>
  
Depending on the network join setting, further configuration may be necessary, as decribed in the following paragraphs.
+
Parameters:
 +
* <code>session</code> - A session ID previously obtained from the login module.
 +
* <code>root</code> - The ID of the referenced root folder on the server.
 +
* <code>path</code> - The path to the synchronized folder, relative to the root folder.
 +
* <code>name</code> - The name of the file version to download.
 +
* <code>checksum</code> - The checksum of the file version to download.
 +
* <code>apiVersion</code> - The API version that the client is using. If not set, the initial version <code>0</code> is assumed.
 +
* <code>offset</code> (optional) - The start offset in bytes for the download. If not defined, an offset of <code>0</code> is assumed.
 +
* <code>length</code> (optional) - The number of bytes to include in the download stream. If not defined, the file is read until the end.
  
=== empty ===
+
Request Body: <br />
 +
Optionally, available since API version 3, if client-side file- and/or directory exclusion filters are active, a PUT request can be used. The request body then holds a JSON object containing two arrays named <code>fileExclusions</code> and <code>directoryExclusions</code> to define client-side exclusion filters, with each element encoded as [[File_pattern|File patterns]] and [[Directory_pattern|Directory patterns]] accordingly. See [[Client_side_filtering|Client side filtering]] for details.
  
When using the default value ''empty'', no other nodes are discovered in the cluster. This value is suitable for single-node installations. Note that other nodes that are configured to use other network join mechanisms may be still able to still to connect to this node, e.g. using a ''static'' network join, having the IP address of this host in the list of potential cluster members (see below).
+
Response: <br />
 +
The binary content of the requested file version. Note that in case of errors, an exception is not encoded in the default JSON error format here. Instead, an appropriate HTTP error with a status code != 200 is returned. For example, in case of the requested file being deleted or modified in the meantime, a response with HTTP status code 404 (not found) is sent.
  
=== static ===
+
Example:
 +
==> GET http://192.168.32.191/ajax/drive?action=download&root=56&path=/test2&name=Jellyfish.jpg&checksum=5a44c7ba5bbe4ec867233d67e4806848&offset=0&length=-1&session=5d0c1e8eb0964a3095438b450ff6810f
 +
 +
<== HTTP 200 OK (20.0011 ms elapsed, 775702 bytes received)
  
The most common setting for ''com.openexchange.hazelcast.network.join'' is ''static''. A static cluster discovery uses a fixed list of IP addresses of the nodes in the cluster. During startup and after a specific interval, the underlying Hazelcast library probes for not yet joined nodes from this list and adds them to the cluster automatically. The address list is configured via ''com.openexchange.hazelcast.network.join.static.nodes'':
+
== Upload a file ==
  
# Configures a comma-separated list of IP addresses / hostnames of possible
+
Uploads a file to the server.
# nodes in the cluster, e.g. "10.20.30.12, 10.20.30.13:5701, 192.178.168.110".
 
# Only used if "com.openexchange.hazelcast.network.join" is set to "static".
 
# It doesn't hurt if the address of the local host appears in the list, so
 
# that it's still possible to use the same list throughout all nodes in the  
 
# cluster.
 
com.openexchange.hazelcast.network.join.static.nodes=
 
  
For a fixed set of backend nodes, it's recommended to simply include the IP addresses of all nodes in the list, and use the same configuration for each node. However, it's only required to add the address of at least one other node in the cluster to allow the node to join the cluster. Also, when adding a new node to the cluster and this list is extended accordingly, existing nodes don't need to be shut down to recognize the new node, as long as the new node's address list contains at least one of the already running nodes.
+
PUT <code>/ajax/drive?action=upload</code>
  
=== multicast ===
+
Parameters:
 +
* <code>session</code> - A session ID previously obtained from the login module.
 +
* <code>root</code> - The ID of the referenced root folder on the server.
 +
* <code>path</code> - The path to the synchronized folder, relative to the root folder.
 +
* <code>newName</code> - The target name of the file version to upload.
 +
* <code>newChecksum</code> - The target checksum of the file version to upload.
 +
* <code>name</code> (optional) - The previous name of the file version being uploaded. Only set when uploading an updated version of an existing file to the server.
 +
* <code>checksum</code> - The previous checksum of the file version to upload. Only set when uploading an updated version of an existing file to the server.
 +
* <code>apiVersion</code> - The API version that the client is using. If not set, the initial version <code>0</code> is assumed.
 +
* <code>contentType</code> (optional) - The content type of the file. If not defined, <code>application/octet-stream</code> is assumed.
 +
* <code>offset</code> (optional) - The start offset in bytes for the upload when resuming a previous partial upload. If not defined, an offset of <code>0</code> is assumed.
 +
* <code>totalLength</code> (optional) - The total expected length of the file (required to support resume of uploads). If not defined, the upload is assumed completed after the operation.
 +
* <code>created</code> (optional) - The creation time of the file as timestamp.
 +
* <code>modified</code> (optional) - The last modification time of the file as timestamp. Defaults to the current server time if no value or a value larger than the current time is supplied.
 +
* <code>binary</code> - Expected to be set to <code>true</code> to indicate the binary content.
 +
* <code>device</code> (optional) - A friendly name identifying the client device from a user's point of view, e.g. "My Tablet PC".
 +
* <code>diagnostics</code> (optional) - If set to <code>true</code>, an additional diagnostics trace is supplied in the response.
 +
* <code>pushToken</code> (optional) - The client's push registration token to associate it to generated events.
  
For highly dynamic setups where nodes are added and removed from the cluster quite often and/or the host's IP addresses are not fixed, it's also possible to configure the network join via multicast. During startup and after a specific interval, the backend nodes initiate the multicast join process automatically, and discovered nodes form or join the cluster afterwards. The multicast group and port can be configured as follows:
+
Request body: <br />
 +
The binary content of the uploaded file version.  
  
# Configures the multicast address used to discover other nodes in the cluster
+
Response: <br />
# dynamically. Only used if "com.openexchange.hazelcast.network.join" is set
+
A JSON array containing all actions the client should execute for synchronization. Each array element is an action as described in [[#Actions | Actions]]. <br /> If the <code>diagnostics</code> flag was set (either to <code>true</code> or <code>false</code>), this array is wrapped into an additional JSON object in the <code>actions</code> parameter, and the diagnostics trace is provided at <code>diagnostics</code>.
# to "multicast". If the nodes reside in different subnets, please ensure that
+
 
# multicast is enabled between the subnets. Defaults to "224.2.2.3".  
+
Example:
  com.openexchange.hazelcast.network.join.multicast.group=224.2.2.3
+
  ==> PUT http://192.168.32.191/ajax/drive?action=upload&root=56&path=/test2&newName=Penguins.jpg&newChecksum=9d377b10ce778c4938b3c7e2c63a229a&contentType=image/jpeg&offset=0&totalLength=777835&binary=true&device=Laptop&created=1375343426999&modified=1375343427001&session=5d0c1e8eb0964a3095438b450ff6810f
 +
  > Content:
 +
    [application/octet-stream;, 777835 bytes]
 
   
 
   
  # Configures the multicast port used to discover other nodes in the cluster
+
  <== HTTP 200 OK (108.0062 ms elapsed, 118 bytes received)
  # dynamically. Only used if "com.openexchange.hazelcast.network.join" is set
+
  <  Content:
# to "multicast". Defaults to "54327".
+
    {
com.openexchange.hazelcast.network.join.multicast.port=54327
+
      "data" : [{
 +
          "action" : "acknowledge",
 +
          "newVersion" : {
 +
            "name" : "Penguins.jpg",
 +
            "checksum" : "9d377b10ce778c4938b3c7e2c63a229a"
 +
          }
 +
        }
 +
      ]
 +
    }
 +
 
 +
== Listen for changes (long polling) ==
 +
 
 +
Listens for server-side changes. The request blocks until new actions for the client are available, or the specified waiting time elapses. May return immediately if previously received but not yet processed actions are available for this client.
 +
 
 +
GET <code>/ajax/drive?action=listen</code>
  
== Example ==
+
Parameters:
 +
* <code>session</code> - A session ID previously obtained from the login module.
 +
* <code>root</code> - The ID of the referenced root folder on the server.
 +
* <code>timeout</code> (optional) - The maximum timeout in milliseconds to wait.
 +
* <code>pushToken</code> (optional) - The client's push registration token to associate it to generated events.
  
The following example shows how a simple cluster named ''MyCluster'' consisting of 4 backend nodes can be configured using ''static'' cluster discovery. The node's IP addresses are 10.0.0.15, 10.0.0.16, 10.0.0.17 and 10.0.0.18. Note that the same ''hazelcast.properties'' is used by all nodes.
+
Response: <br />
 +
A JSON array containing all actions the client should execute for synchronization. Each array element is an action as described in [[#Actions | Actions]]. If there no changes were detected, an empty array is returned. Typically, the client will continue with the next <code>listen</code> request after the response was processed.
  
  com.openexchange.hazelcast.group.name=MyCluster
+
Example:
  com.openexchange.hazelcast.group.password=secret
+
  ==> GET http://192.168.32.191/ajax/drive?action=listen&root=65841&session=51378e29f82042b4afe4af1c034c6d68
  com.openexchange.hazelcast.network.join=static
+
   
com.openexchange.hazelcast.network.join.static.nodes=10.0.0.15,10.0.0.16,10.0.0.17,10.0.0.18
+
  <== HTTP 200 OK (63409.6268 ms elapsed, 28 bytes received)
com.openexchange.hazelcast.network.interfaces=10.0.0.*
+
<  Content:
 +
    {
 +
      "data" : [{
 +
          "action" : "sync",
 +
        }
 +
      ]
 +
    }
  
 +
== Get quota ==
  
== Advanced Configuration ==
+
Gets the quota limits and current usage for the storage the supplied root folder belongs to. Depending on the filestore configuration, this may include both restrictions on the number of allowed files and the total size of all contained files in bytes. If there's no limit, -1 is returned.
  
=== Custom Partitioning (preliminary) ===
+
GET <code>/ajax/drive?action=quota</code>
  
While originally being desgined to separate the nodes holding distributed data into different risk groups for increased fail safety, a custom partioning strategy may also be used to distinguish between nodes holding distributed data from those who should not.  
+
Parameters:
 +
* <code>session</code> - A session ID previously obtained from the login module.
 +
* <code>root</code> - The ID of the referenced root folder on the server.
  
This approach of custom partitioning may be used in a OX cluster, where usually different backend nodes serve different purposes. A common scenario is that there are nodes handling requests from the web interfaces, and others being responsible for USM/EAS traffic. Due to their nature of processing large chunks of synchronization data in memory, the USM/EAS nodes may encounter small delays when the Java garbage collector kicks in and suspends the Java Virtual Machine. Since those delays may also have an influence on hazelcast-based communication in the cluster, the idea is to instruct hazelcast to not store distributed data on that nodes. This is where a custom partitioning scheme comes into play.
+
Response: <br />
 +
A JSON object containing the quota restrictions inside a JSON array with the property name <code>quota</code>. The JSON array contains zero, one or two <code>quota</code> objects as described below, depending on the filestore configuration. If one or more quota <code>type</code>s are missing in the array, the client can expect that there are no limitations for that type. Besides the array, the JSON object also contains a hyperlink behind the <code>manageLink</code> parameter, pointing to an URL where the user could manage his quota restrictions.
  
To setup a custom paritioning scheme in the cluster, an additional ''hazelcast.xml'' configuration file is used, which should be placed into the ''hazelcast'' subdirectory of the OX configuration folder, usually at ''/opt/openexchange/etc/hazelcast''. Please note that it's vital that each node in the cluster is configured equally here, so the same ''hazelcast.xml'' file should be copied to each server. The configuration read from there is used as basis for all further settings that are taken from the ordinary ''hazelcast.properties'' config file.  
+
{| id="Quota" cellspacing="0" border="1"
 +
|+ align="bottom" | Quota
 +
! Name !! Type !! Value
 +
|-
 +
| limit || Number || The allowed limit (either number of files or sum of filesizes in bytes).
 +
|-
 +
| use || Number || The current usage (again either number of files or sum of filesizes in bytes).
 +
|-
 +
| type || String || The kind of quota restriction, currently either <code>storage</code> (size of contained files in bytes) or <code>file</code> (number of files).
 +
|}
  
To setup a custom paritioning scheme, the partition groups must be defined in the ''hazelcast.xml'' file. See the following file for an example configuration, where the three nodes ''10.10.10.60'', ''10.10.10.61'' and ''10.10.10.62'' are defined to form an own paritioning group each. Doing so, all distributed data will be stored at one of those nodes physically, while the corresponding backup data (if configured) at one of the other two nodes. All other nodes in the cluster will not be used to store distributed data, but will still be "full" hazelcast members, which is necessary for other cluster-wide operations the OX backends use.  
+
Example:
 +
==> GET http://192.168.32.191/ajax/drive?action=quota&root=56&session=35cb8c2d1423480692f0d5053d14ba52
 +
 
 +
<== HTTP 200 OK (9.6854 ms elapsed, 113 bytes received)
 +
<  Content:
 +
    {
 +
      "data" : {
 +
        "quota" : [{
 +
            "limit" : 107374182400,
 +
            "use" : 1109974882,
 +
            "type" : "storage"
 +
          }, {
 +
            "limit" : 800000000000,
 +
            "use" : 1577,
 +
            "type" : "file"
 +
          }
 +
        ],
 +
        "manageLink" : "https://www.example.com/manageQuota"
 +
      }
 +
    }
  
Please note that the configured backup count in the map configurations should be smaller than the number of nodes here, otherwise, there may be problems if one of those data nodes is shut down temporarily for maintenance. So, the minimum number of nodes to define in the partition group sections is implicitly bound to the sum of a map's ''backupCount'' and ''asyncBackupCount'' properties, plus ''1'' for the original data partition.
+
== Get Settings ==
  
 +
Gets various settings applicable for the drive clients.
  
<?xml version="1.0" encoding="UTF-8"?>
+
GET <code>/ajax/drive?action=settings</code>
<!--
+
 
  ~ Copyright (c) 2008-2013, Hazelcast, Inc. All Rights Reserved.
+
Parameters:
  ~
+
* <code>session</code> - A session ID previously obtained from the login module.
  ~ Licensed under the Apache License, Version 2.0 (the "License");
+
* <code>root</code> - The ID of the referenced root folder on the server.
  ~ you may not use this file except in compliance with the License.
+
* <code>language</code> (optional) - The locale to use for language-sensitive settings (in the format <code><2-letter-language>_<2-letter-region></code>, e.g. <code>de_CH</code> or <code>en_GB</code>). Defaults to the user's configured locale on the server.
   ~ You may obtain a copy of the License at
+
 
  ~
+
Response:<br />
  ~ http://www.apache.org/licenses/LICENSE-2.0
+
A JSON object holding the settings as described below. This also includes a JSON array with the property name <code>quota</code> that contains zero, one or two quota objects as described below, depending on the filestore configuration. If one or more quota types are missing in the array, the client can expect that there are no limitations for that type.  
  ~
+
 
  ~ Unless required by applicable law or agreed to in writing, software
+
{| id="Quota" cellspacing="0" border="1"
  ~ distributed under the License is distributed on an "AS IS" BASIS,
+
|+ align="bottom" | Quota
  ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+
! Name !! Type !! Value
  ~ See the License for the specific language governing permissions and
+
|-
  ~ limitations under the License.
+
| limit || Number || The allowed limit (either number of files or sum of filesizes in bytes).
  -->
+
|-
 +
| use || Number || The current usage (again either number of files or sum of filesizes in bytes).
 +
|-
 +
| type || String || The kind of quota restriction, currently either <code>storage</code> (size of contained files in bytes) or <code>file</code> (number of files).
 +
|}
 +
 
 +
{| id="Settings" cellspacing="0" border="1"
 +
|+ align="bottom" | Settings
 +
! Name !! Type !! Value
 +
|-
 +
| helpLink || String || A hyperlink to the online help.
 +
|-
 +
| quotaManageLink || String || A hyperlink to an URL where the user could manage his quota restrictions.
 +
|-
 +
| quota || Array || A JSON array containing the quota restrictions as described above.
 +
|-
 +
| serverVersion || String || The server version string.
 +
|-
 +
| supportedApiVersion || String || The API version supported by the server.
 +
|-
 +
| minApiVersion || String || The API version required to synchronize with the server.
 +
|}
 +
 
 +
Example:
 +
==> GET http://192.168.32.191/ajax/drive?action=settings&root=56&session=35cb8c2d1423480692f0d5053d14ba52
 +
    
 +
<== HTTP 200 OK (11.3530 ms elapsed, 318 bytes received)
 +
<  Content:
 +
    {
 +
      "data" : {
 +
        "quota" : [{
 +
            "limit" : 107374182400,
 +
            "use" : 8828427,
 +
            "type" : "storage"
 +
          }, {
 +
            "limit" : 800000000000,
 +
            "use" : 1559,
 +
            "type" : "file"
 +
          }
 +
        ],
 +
        "helpLink" : "http://192.168.32.191/ajax/help/en_US/index.html",
 +
        "quotaManageLink" : "https://192.168.32.191/manageQuota",
 +
        "serverVersion" : "7.4.2-Rev1",
 +
        "supportedApiVersion" : "2",
 +
        "minApiVersion" : "1"
 +
      }
 +
    }
 +
 
 +
== Subscribe to Push-Events ==
 +
 
 +
Registers a client device to receive push notifications from the server. The subscription is performed based on the configured root folder ID of the client application that identifies itself with it's device token. Supported services currently include the Apple Push Notification Service (APN) and Google Cloud Messaging (GCM). Trying to perform an identical subscription (same <code>root</code>, <code>service</code> and <code>token</code>) from the same user account again is treated as a no-op.
 +
 
 +
GET <code>/ajax/drive?action=subscribe</code>
 +
 
 +
Parameters:
 +
* <code>session</code> - A session ID previously obtained from the login module.
 +
* <code>root</code> - The ID of the referenced root folder on the server.
 +
* <code>service</code> - The name of the underlying push service to use, currently one of <code>gcm</code>, <code>apn</code> or <code>apn.macos</code>.
 +
* <code>token</code> - The device's registration token as assigned by the service.
 +
 
 +
Response:<br />
 +
An empty JSON result.
 +
 
 +
Example:
 +
==> GET http://192.168.32.191/ajax/drive?action=subscribe&root=65841&session=51378e29f82042b4afe4af1c034c6d68&service=apn&token=28919862989a1b5ba59c11d5f7cb7ba2b9678be9dd18b033184d04f682013677
 
   
 
   
  <hazelcast xsi:schemaLocation="http://www.hazelcast.com/schema/config hazelcast-config-3.1.xsd"
+
  <== HTTP 200 OK (13.6268 ms elapsed, 11 bytes received)
            xmlns="http://www.hazelcast.com/schema/config"
+
<  Content:  
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
+
     {
     <partition-group enabled="true" group-type="CUSTOM">
+
      "data" : {
        <member-group>
+
      }
            <interface>10.10.10.60</interface>
+
     }
        </member-group>
 
        <member-group>
 
            <interface>10.10.10.61</interface>
 
        </member-group>
 
        <member-group>
 
            <interface>10.10.10.62</interface>
 
        </member-group>
 
     </partition-group>
 
</hazelcast>
 
  
More general information regarding custom partioning is available at http://hazelcast.org/docs/latest/manual/html/partitiongroupconfig.html .
 
  
It's also recommended to use a "static" cluster discovery for the network join, and list same the nodes that are also configured in the parition groups here, so that join requests are handled by those nodes, too (and not the other nodes that are potentially prone to garbage collection delays.
+
== Unsubscribe from Push-Events ==
  
After configuring a custom paritioning scheme, the data distribution may be verified, e.g. by inspecting the MBeans of the distributed maps via JMX.
+
Unregisters a previously registered client device to stop receiving push notifications from the server. The same parameters that were used to perform the subscription need to be passed again, which includes the root folder ID, the device token and the service name.
  
= Features =
+
GET <code>/ajax/drive?action=unsubscribe</code>
  
The following list gives an overview about different features that were implemented using the new cluster capabilities.
+
Parameters:
 +
* <code>session</code> - A session ID previously obtained from the login module.
 +
* <code>root</code> - The ID of the referenced root folder on the server.
 +
* <code>service</code> - The name of the underlying push service to use, currently one of <code>gcm</code>, <code>apn</code> or <code>apn.macos</code>.
 +
* <code>token</code> - The device's registration token as assigned by the service.
  
== Distributed Session Storage ==
+
Response:<br />
 +
An empty JSON result.
  
Previously, when an Open-Xchange server was shutdown for maintenance, all user sessions that were bound to that machine were lost, i.e. the users needed to login again. With the distributed session storage, all sessions are backed by a distributed map in the cluster, so that they are no longer bound to a specific node in the cluster. When a node is shut down, the session data is still available in the cluster and can be accessed from the remaining nodes. The load-balancing techniques of the webserver then seamlessly routes the user session to another node, with no ''session expired'' errors. The distributed session storage comes with the package ''open-xchange-sessionstorage-hazelcast''. It's recommended to install this optional package in all clustered environments with multiple groupware server nodes.
+
Example:
 +
==> GET http://192.168.32.191/ajax/drive?action=unsubscribe&root=65841&session=51378e29f82042b4afe4af1c034c6d68&service=apn&token=28919862989a1b5ba59c11d5f7cb7ba2b9678be9dd18b033184d04f682013677
 +
 +
<== HTTP 200 OK (26.0015 ms elapsed, 11 bytes received)
 +
<  Content:
 +
    {
 +
      "data" : {
 +
      }
 +
    }
  
'''Note:''' While there's some kind of built-in session distribution among the nodes in the cluster, this should not be seen as a replacement for session-stickiness between the loadbalancer and groupware nodes, i.e. one should still configure the webserver to use sticky sessions for performance reasons.
+
== Update the subscription token ==
  
Depending on the cluster infrastructure, different backup-count configuration options might be set for the distributed session storage in the map configuration file ''sessions.properties'' in the ''hazelcast'' subdirectory:
+
Updates a device's registration token in case a new one was assigned by the service.
  
  com.openexchange.hazelcast.configuration.map.backupCount=1
+
GET <code>/ajax/drive?action=updateToken</code>
  
The ''backupcount'' property configures the number of nodes with synchronized backups. Synchronized backups block operations until backups are successfully copied and acknowledgements are received. If 1 is set as the backup-count for example, then all entries of the map will be copied to another JVM for fail-safety. 0 means no backup. Any integer between 0 and 6. Default is 1, setting bigger than 6 has no effect.
+
Parameters:
 +
* <code>session</code> - A session ID previously obtained from the login module.
 +
* <code>service</code> - The name of the underlying push service to use, currently one of <code>gcm</code>, <code>apn</code> or <code>apn.macos</code>.
 +
* <code>token</code> - The previous registration token as assigned by the service.
 +
* <code>newToken</code> - The new registration token as assigned by the service.
  
  com.openexchange.hazelcast.configuration.map.asyncBackupCount=0
+
Response:<br />
 +
An empty JSON result.
  
The ''asyncbackup'' property configures the number of nodes with async backups. Async backups do not block operations and do not require acknowledgements. 0 means no backup. Any integer between 0 and 6. Default is 0, setting bigger than 6 has no effect.
+
Example:
 +
==> GET http://192.168.32.191/ajax/drive?action=updateToken&service=apn&session=51378e29f82042b4afe4af1c034c6d68&token=28919862989a1b5ba59c11d5f7cb7ba2b9678be9dd18b033184d04f682013677&newToken=38919862989a1b5ba59c11d5f7cb7ba2b9678be9dd18b033184d04f682013677
 +
 +
<== HTTP 200 OK (15.6653 ms elapsed, 11 bytes received)
 +
<  Content:
 +
    {
 +
      "data" : {
 +
      }
 +
    }
  
Since session data is backed up by default continously by multiple nodes in the cluster, the steps described in [[ Session_Migration ]] to trigger session mirgration to other nodes explicitly is obsolete and no longer needed with the distributed session storage.
+
== Get file metadata ==
  
Normally, sessions in the distributed storages are not evicted automatically, but are only removed when they're also removed from the session handler, either due to a logout operation or when exceeding the long-term session lifetime as configured by ''com.openexchange.sessiond.sessionLongLifeTime'' in ''sessiond.properties''. Under certain circumstances, i.e. the session is no longer accessed by the client and the OX node hosting the session in it's long-life container being shutdown, the remove operation from the distributed storage might not be triggered. Therefore, additionaly a maximum idle time of map-entries can be configured for the distributed sessions map via
+
Deprecated, available until API version 2. <br />
 +
Additional metadata of synchronized files is made available via the <code>fileMetadata</code> request.
  
com.openexchange.hazelcast.configuration.map.maxIdleSeconds=640000
+
PUT <code>/ajax/drive?action=fileMetata</code>
  
To avoid unnecessary eviction, the value should be higher than the configured ''com.openexchange.sessiond.sessionLongLifeTime'' in ''sessiond.properties''.
+
Parameters:
 +
* <code>session</code> - A session ID previously obtained from the login module.
 +
* <code>root</code> - The ID of the referenced root folder on the server.
 +
* <code>path</code> - The path to the synchronized folder, relative to the root folder.
 +
* <code>columns</code> -  A comma-separated list of columns to return. Each column is specified by a numeric column identifier. Column identifiers for file metadata are defined in [[#File Metadata]].
  
== Distributed Indexing Jobs ==
+
Request Body:<br />
 +
A JSON array containing the file versions to get the metadata for. Each object in the array should be sent as [[#File Version | File Versions]], and needs to be present in the referenced path.
  
Groupware data is indexed in the background to yield faster search results. See the article on the [[Indexing Bundle]] for more.
+
Response:<br />
 +
A JSON array containing the file metadata in the order of the requested file versions. Each array element describes one file metadata and is itself an array. The elements of each array contain the information specified by the corresponding identifiers in the columns parameter.
  
== Remote Cache Invalidation ==
+
{| id="FileMetadataDeprecated" cellspacing="0" border="1"
 +
|+ align="bottom" | File Metadata (deprecated)
 +
! ID !! Name !! Type !! Value
 +
|-
 +
| name || String || The name of the file version.
 +
|-
 +
| 4 || created || Timestamp || The file's last modification time (always UTC, not translated into user time).
 +
|-
 +
| 5 || modified || Timestamp || The file's last modification time (always UTC, not translated into user time).
 +
|-
 +
| 702 || name || String || The name of the file, including it's extension, e.g. <code>test.doc</code>.
 +
|-
 +
| 703 || contentType || String || The file's content type, e.g. "image/png".
 +
|-
 +
| 708 || checksum || String || The MD5 hash of the file, expressed as a lowercase hexadecimal number string, 32 characters long, e.g. <code>f8cacac95379527cd4fa15f0cb782a09</code>.
 +
|-
 +
| 750 || previewLink || String || A direct link to a medium-sized preview image of the file if available.
 +
|-
 +
| 751 || directLinkFragments || String | The fragments part of the direct link that can be used in combination with the [[http://oxpedia.org/index.php?title=HTTP_API#Token_Login_.28since_7.0.1.29 |token login]] method to jump directly to the detail view of the file in the web interface, bypassing the need to login manually.
 +
|-
 +
| 752 || directLink || String || A direct link to the detail view of the file in the web interface.
 +
|-
 +
| 753 || thumbnailLink || String || A direct link to a small thumbnail image of the file if available.
 +
|}
  
For faster access, groupware data is held in different caches by the server. Formerly, the caches utilized the TCP Lateral Auxiliary Cache plug in (LTCP) for the underlying JCS caches to broadcast updates and removals to caches on other OX nodes in the cluster. This could potentially lead to problems when remote invalidation was not working reliably due to network discovery problems. As an alternative, remote cache invalidation can also be performed using reliable publish/subscribe events built up on Hazelcast topics. This can be configured in the ''cache.properties'' configuration file, where the 'eventInvalidation' property can either be set to 'false' for the legacy behavior or 'true' for the new mechanism:
+
Example:
 +
==> PUT http://192.168.32.191/ajax/drive?action=fileMetadata&root=97974&path=%2f&columns=702%2c708%2c752%2c750%2c753&session=43aca91a80de42559ff0c2493dd973d0
 +
  > Content:
 +
    [
 +
      {
 +
        "name" : "image.jpg",
 +
        "checksum" : "2b04df3ecc1d94afddff082d139c6f15"
 +
      }, {
 +
        "name" : "song.mp3",
 +
        "checksum" : "5a9a91184e611dae3fed162b8787ce5f"
 +
      }, {
 +
        "name" : "test1.txt",
 +
        "checksum" : "7e36f409a042f06ecb88606a97a88c8f"
 +
      }, {
 +
        "name" : "test3.txt",
 +
        "checksum" : "703bc9aabff33faf07cf121dcda12ec8"
 +
      }
 +
    ] 
 +
 +
<== HTTP 200 OK (6.0004 ms elapsed, 140 bytes received)
 +
<  Content:
 +
    [
 +
      ["image.jpg", "2b04df3ecc1d94afddff082d139c6f15", "https://192.168.32.191/ox6/index.html#m=infostore&f=97974&i=179629", "https://192.168.32.191/ajax/files?action=document&folder=97974&id=179629&version=1&delivery=download&scaleType=contain&width=128&height=90", "m=infostore&f=97974&i=179629"],
 +
      ["song.mp3", "5a9a91184e611dae3fed162b8787ce5f", "https://192.168.32.191/ox6/index.html#m=infostore&f=97974&i=179630", "https://192.168.32.191/ajax/image/file/mp3Cover?folder=97974&id=179630&version=1&delivery=download&scaleType=contain&width=128&height=90", "m=infostore&f=97974&i=179630"],
 +
      ["test1.txt", "7e36f409a042f06ecb88606a97a88c8f", "https://192.168.32.191/ox6/index.html#m=infostore&f=97974&i=179626", null, "m=infostore&f=97974&i=179626"],
 +
      ["test3.txt", "703bc9aabff33faf07cf121dcda12ec8", "https://192.168.32.191/ox6/index.html#m=infostore&f=97974&i=179624", null, "m=infostore&f=97974&i=179624"]
 +
    ]
 +
 
 +
== Get a direct link for a folder/a file into appsuite ==
 +
 
 +
Generate a direct link into appsuite UI for a synchronized file/a synchronized folder and a token for token-based login.
 +
 
 +
POST <code>/ajax/drive?action=jump</code>
 +
 
 +
Parameters:
 +
* <code>session</code> - A session ID previously obtained from the login module.
 +
* <code>root</code> - The ID of the referenced root folder on the server.
 +
* <code>path</code> - The path to the synchronized folder, relative to the root folder.
 +
* <code>name</code> - The name of the file in the synchronized folder given in <code>path</code>-parameter. Optional
 +
* <code>method</code> - [[#Methods | Methods]]
 +
* <code>authId</code> - Identifier for tracing every single login request passed between different systems in a cluster. The value should be some token that is unique for every login request. This parameter must be given as URL parameter and not inside the body of the POST request.
 +
* <code>clientToken</code> - Client side identifier for accessing the session later. The value should be some token that is unique for every login request.
 +
 
 +
Methods:<br />
 +
* <code>edit</code>: Open the file in appsuite editor or in text/spreadsheet (if available).
 +
* <code>permissions</code>: Open the file's/folder's change-permission dialog.
 +
* <code>version_history</code>: Open the file's version history summary.
 +
* <code>preview</code>: Open the file's/folder's preview.
 +
 
 +
Response:<br />
 +
A JSON array containing the direct link to the file/folder including a server token for token based login.
 +
 
 +
Example:
 +
==> POST http://localhost/ajax/drive?action=jump&session=48a289898ad949faaa46c04e7fb422f5&root=9547&path=/path/to/file&name=file_to_edit.txt&method=edit&authId=41763584-8460-11e4-b116-123b93f75dba
 +
  > Content: clientToken=47d74b1c-81df-11e4-b116-123b93f75cba
 +
 +
<== HTTP 200 OK
 +
<  Content:
 +
    {
 +
      "data": {
 +
        "redirectUrl": "http://localhost/appsuite#app=io.ox/editor&folder=273264&id=273264/307438&serverToken=7b90972628e34e89bb9a3946d1372c68"
 +
      }
 +
    }
  
com.openexchange.caching.jcs.eventInvalidation=true
+
== Use direct link and token with token-based login ==
  
All nodes participating in the cluster should be configured equally.
+
Login to appsuite UI with token-based login via the link created with [[#Get a direct link for a folder/a file into appsuite | Get a direct link for a folder/a file into appsuite]].
  
Internally, if ''com.openexchange.caching.jcs.eventInvalidation'' is set to ''true'', LTCP is disabled in JCS caches. Instead, an internal mechanism based on distributed Hazelcast event topics is used to invalidate data throughout all nodes in the cluster after local update- and remove-operations. Put-operations aren't propagated (and haven't been with LTCP either), since all data put into caches can be locally loaded/evaluated at each node from the persistent storage layer.
+
GET <code>[direct link]</code>
  
Using Hazelcast-based cache invalidation also makes further configuration of the JCS auxiliaries obsolete in the ''cache.ccf'' configuration file. In that case, all ''jcs.auxiliary.LTCP.*'' configuration settings are virtually ignored. However, it's still required to mark caches that require cluster-wide invalidation via ''jcs.region.<cache_name>=LTCP'', just as before. So basically, when using the new default setting ''com.openexchange.caching.jcs.eventInvalidation=true'', it's recommended to just use the stock ''cache.ccf'' file, since no further LTCP configuration is required.
+
Parameters:
 +
* <code>clientToken</code> – Client side identifier for accessing the session. The value must be the same as in [[#Get a direct link for a folder/a file into appsuite | Get a direct link for a folder/a file into appsuite]].
  
 +
Example:
 +
==> GET http://localhost/appsuite#app=io.ox/editor&folder=273264&id=273264/307438&serverToken=7b90972628e34e89bb9a3946d1372c68&clientToken=47d74b1c-81df-11e4-b116-123b93f75cba
 
   
 
   
= Adminstration / Troubleshooting =
+
<== HTTP 200 OK
 +
 
 +
== Get synchronizable Folders ==
 +
 
 +
Available since API version 4. <br />
 +
 
 +
Allows getting a list of folders that are available on the server for synchronization. This request should be used to build up a folder tree and let the user select the root synchronization folder(s).
 +
 
 +
GET <code>/ajax/drive?action=subfolders</code>
 +
 
 +
Parameters:
 +
* <code>session</code> - A session ID previously obtained from the login module.
 +
* <code>parent</code> - The ID of the parent folder to get the subfolders for as read from a previously fetched directory metadata object. Optional; if not set, the root available root folders are returned.
 +
 
 +
Response:<br />
 +
A JSON array holding metadata information for all subfolders as defined in [[#Directory Metadata | Directory Metadata]], with the <code>files</code> array being left out.
 +
 
 +
Example:
 +
==> GET http://192.168.32.191/ajax/drive?action=subfolders&session=35cb8c2d1423480692f0d5053d14ba52
 +
 
 +
<== HTTP 200 OK (241.0252 ms elapsed, 966 bytes received)
 +
<  Content:
 +
    {
 +
      "data": [{
 +
        "id": "com.openexchange.file.storage.googledrive://1/",
 +
        "name": "Google Drive",
 +
        "path": "/Google Drive",
 +
        "has_subfolders": true,
 +
        "own_rights": 403710016,
 +
        "permissions": [{
 +
          "bits": 403710016,
 +
          "group": false,
 +
          "entity": 182,
 +
          "display_name": "Mander, Jens",
 +
          "email_address": "jens.mander@example.com",
 +
          "guest": false
 +
        }],
 +
        "jump": ["permissions"]
 +
      },
 +
      {
 +
        "id": "10",
 +
        "name": "Freigegebene Dateien",
 +
        "path": "/Freigegebene Dateien",
 +
        "created": 1224493261628,
 +
        "modified": 1417164170136,
 +
        "has_subfolders": true,
 +
        "own_rights": 1,
 +
        "permissions": [{
 +
          "bits": 1,
 +
          "group": true,
 +
          "entity": 0,
 +
          "display_name": "All users",
 +
          "guest": false
 +
        },
 +
        {
 +
          "bits": 1,
 +
          "group": true,
 +
          "entity": 2147483647,
 +
          "display_name": "Guests",
 +
          "guest": false
 +
        }],
 +
        "jump": ["permissions"],
 +
        "shared": true
 +
      },
 +
      {
 +
        "id": "15",
 +
        "name": "Öffentliche Dateien",
 +
        "path": "/Öffentliche Dateien",
 +
        "created": 1224493261628,
 +
        "modified": 1418383637250,
 +
        "has_subfolders": true,
 +
        "own_rights": 403709956,
 +
        "permissions": [{
 +
          "bits": 403709956,
 +
          "group": true,
 +
          "entity": 0,
 +
          "display_name": "All users",
 +
          "guest": false
 +
        },
 +
        {
 +
          "bits": 1,
 +
          "group": true,
 +
          "entity": 2147483647,
 +
          "display_name": "Guests",
 +
          "guest": false
 +
        }],
 +
        "jump": ["permissions"],
 +
        "shared": true
 +
      },
 +
      {
 +
        "id": "com.openexchange.file.storage.dropbox://1/",
 +
        "name": "Dropbox",
 +
        "path": "/Dropbox",
 +
        "has_subfolders": true,
 +
        "own_rights": 403710016,
 +
        "permissions": [{
 +
          "bits": 403710016,
 +
          "group": false,
 +
          "entity": 182,
 +
          "display_name": "Mander, Jens",
 +
          "email_address": "jens.mander@example.com",
 +
          "guest": false
 +
        }],
 +
        "jump": ["permissions"]
 +
      },
 +
      {
 +
        "id": "9542",
 +
        "name": "Meine Dateien",
 +
        "path": "/Meine Dateien",
 +
        "created": 1320230546147,
 +
        "modified": 1426764458823,
 +
        "default_folder": true,
 +
        "has_subfolders": true,
 +
        "own_rights": 403710016,
 +
        "permissions": [{
 +
          "bits": 403710016,
 +
          "group": false,
 +
          "entity": 182,
 +
          "display_name": "Mander, Jens",
 +
          "email_address": "jens.mander@example.com",
 +
          "guest": false
 +
        }],
 +
        "jump": ["permissions"]
 +
      }]
 +
    }
 +
 
 +
= File- and Directory Name Restrictions =
 +
 
 +
Regarding the case sensitivity of file and directory names, OX Drive works in a case-insensitive, but case-preserving way. That means that there cannot be two files with an equal name ignoring case in the same directory, but it's still possible to synchronize the names in a case-sensitive manner, as well as it's possible to change only the case of file- and directory names.
 +
 
 +
The same applies to equally named files and directories on the same level in the folder hierarchy, i.e. it's not possible to create a new file in a directory where an equally (ignoring case) named subdirectory already exists and vice versa.
 +
 
 +
There is a similar restriction regarding file and directory names in the same directory having different unicode normalization forms, yet the same textual representation. OX Drive requires uniqueness regarding this textual representaion of potentially different encoded unicode strings. So, in case the client tries to synchronize two textually equal files or directories, he is instructed to put one of them into quarantine. Internally the server performs an equals-check of the "NFC" normalization forms of the strings, i.e. an unicode string is normalized using full canonical decomposition, followed by the replacement of sequences with their primary composites, if possible. Details regarding unicode normalization can be found at http://www.unicode.org/reports/tr15/tr15-23.html .
 +
 
 +
== Invalid and ignored Filenames ==
 +
 
 +
There are some filenames that are invalid or ignored and therefore not synchronized. This means that files with these names should not be taken into account when sending the directory contents to the server, or when calculating the directory checksum (see below). The following list describes when a filename is considered invalid:
 +
* If it contains one or of the following reserved characters:
 +
** <code><</code> (less than),
 +
** <code>></code> (greater than)
 +
** <code>:</code> (colon)
 +
** <code>"</code> (double quote)
 +
** <code>/</code> (forward slash)
 +
** <code>\</code> (backslash)
 +
** <code>|</code> (vertical bar or pipe)
 +
** <code>?</code> (question mark)
 +
** <code>*</code> (asterisk)
 +
** Characters whose integer representations are in the range from 0 through 31
 +
* The last character is a <code>.</code> (dot) or <code>' '</code> (space)
 +
* It's case-invariant name without an optional extension matches one of the reserved names <code>CON</code>, <code>PRN</code>, <code>AUX</code>, <code>NUL</code>, <code>COM1</code>, <code>COM2</code>, <code>COM3</code>, <code>COM4</code>, <code>COM5</code>, <code>COM6</code>, <code>COM7</code>, <code>COM8</code>, <code>COM9</code>, <code>LPT1</code>, <code>LPT2</code>, <code>LPT3</code>, <code>LPT4</code>, <code>LPT5</code>, <code>LPT6</code>, <code>LPT7</code>, <code>LPT8</code>, or <code>LPT9</code>
 +
* It consists solely of whitespace characters
 +
 
 +
The following list gives an overview about the ignored filenames:
 +
* <code>desktop.ini</code>
 +
* <code>Thumbs.db</code>
 +
* <code>.DS_Store</code>
 +
* <code>icon\r</code>
 +
* Any filename ending with <code>.drivepart</code>
 +
* Any filename starting with <code>.msngr_hstr_data_</code> and ending with <code>.log</code>
 +
 
 +
Nevertheless, if the client still insists to send a file version with an invalid or ignored filename, the file creation on the server is refused with a corresponding <code>error</code> action (see below).
  
== Hazelcast Configuration ==
+
== Invalid and ignored Directory Names ==
  
The underlying Hazelcast library can be configured using the file ''hazelcast.properties''.
+
There are also similar restrictions regarding invalid directory names. Any try to include them in the list of directory versions will be responded with a corresponding error action for the directory version. The following list describes when a path is considered invalid:
 +
* If it contains one or of the following reserved characters:
 +
** <code><</code> (less than),
 +
** <code>></code> (greater than)
 +
** <code>:</code> (colon)
 +
** <code>"</code> (double quote)
 +
** <code>\</code> (backslash)
 +
** <code>|</code> (vertical bar or pipe)
 +
** <code>?</code> (question mark)
 +
** <code>*</code> (asterisk)
 +
** Characters whose integer representations are in the range from 0 through 31
 +
* The last character of any subpath (i.e. the last part of the whole path or the part preceding the spearator character <code>/</code>) is a <code>.</code> (dot) or <code>' '</code> (space)
 +
* It consists solely of whitespace characters
 +
* It not equals the root path <code>/</code>, but ends with a <code>/</code> (forward slash) character
 +
* It contains two or more consecutive <code>/</code> (forward slash) characters
  
'''Important''':<br>
+
The following list gives an overview about the ignored directory names:
By default property ''com.openexchange.hazelcast.network.interfaces'' is set to ''127.0.0.1''; meaning Hazelcast listens only to loop-back device. To build a cluster among remote nodes the appropriate network interface needs to be configured there. Leaving that property empty lets Hazelcast listen to all available network interfaces.
+
* <code>/.drive</code>
 +
* Any directory whose path ends with <code>/.msngr_hstr_data</code>
  
The Hazelcast JMX MBean can be enabled or disabled with the property ''com.openexchange.hazelcast.jmx''. The properties ''com.openexchange.hazelcast.mergeFirstRunDelay'' and ''com.openexchange.hazelcast.mergeRunDelay'' control the run intervals of the so-called ''Split Brain Handler'' of Hazelcast that initiates the cluster join process when a new node is started. More details can be found at http://www.hazelcast.com/docs/2.5/manual/single_html/#NetworkPartitioning.
+
== Length Restrictions ==
  
The port ranges used by Hazelcast for incoming and outgoing connections can be controlled via the configuration parameters ''com.openexchange.hazelcast.networkConfig.port'', ''com.openexchange.hazelcast.networkConfig.portAutoIncrement'' and ''com.openexchange.hazelcast.networkConfig.outboundPortDefinitions''.
+
The maximum allowed length for path segments, i.e. the parts between forawrd slashes (</code>/</code>) in directory and filenames, is restricted to 255 characters. Synchronizing a file or directory version that contains path segments longer than this limit leads to those versions being put into quarantine.
  
== Commandline Tool ==
+
= Client side filtering =
  
To print out statistics about the cluster and the distributed data, the ''showruntimestats'' commandline tool can be executed witht the ''clusterstats'' ('c') argument. This provides an overview about the runtime cluster configuration of the node, other members in the cluster and distributed data structures.
+
Client-side filtering is available since API version 2. <br />
  
== JMX ==
+
OX Drive clients may define a user- and/or application-defined list of file- and directory name exclusions. Those exclusion filters are then taken into account during synchronization, i.e. files and directories matching a defined exclusion pattern are ignored when comparing the list of server-, client- and original versions. Also, the file exclusion lists are considered for the calculation of aggergated directory checksums.
  
In the Open-Xchange server Java process, the MBean ''com.hazelcast'' can be used to monitor and manage different aspects of the underlying Hazelcast cluster. The ''com.hazelcast'' MBean provides detailed information about the cluster configuration and distributed data structures.
+
The exclusion filters may be set, changed or unset at any time during synchronization, there are no additional requests needed to set them up. Instead, the list of excluded files and directories is simply sent along with each <code>syncFolders</code>, <code>syncFiles</code> and <code>download</code> request. The following tables show the JSON representation of file- and directory patterns that are used to build up the exlcusion lists:
  
== Hazelcast Errors ==
+
== Directory pattern ==
  
When experiencing hazelcast related errors in the logfiles, most likely different versions of the packages are installed, leading to different message formats that can't be understood by nodes using another version. Examples for such errors are exceptions in hazelcast components regarding (de)serialization or other message processing.
+
A directory pattern is defined by a pattern string and further attributes.
This may happen when performing a consecutive update of all nodes in the cluster, where temporarily nodes with a heterogeneous setup try to communicate with each other. If the errors don't disappear after all nodes in the cluster have been update to the same package versions, it might be necessary to shutdown the cluster completely, so that all distributed data is cleared.
 
  
== Cluster Discovery Errors ==
+
{| id="DirectoryPattern" cellspacing="0" border="1"
 +
|+ align="bottom" | Directory Pattern
 +
! Name !! Type !! Value
 +
|-
 +
| type || String || The pattern type, currently one of <code>exact</code> or <code>glob</code>.
 +
|-
 +
| path || String || The path pattern, in a format depending on the pattern type.
 +
|-
 +
| caseSensitive || Boolean | Optional flag to enable case-sensitive matching, defaults to <code>false</code>
 +
|}
  
* If the started OX nodes don't form a cluster, please double-check your configuration in ''hazelcast.properties''
+
== File pattern ==
* It's important to have the same cluster name defined in ''hazelcast.properties'' throughout all nodes in the cluster
 
* Especially when using multicast cluster discovery, it might take some time until the cluster is formed
 
* When using ''static'' cluster discovery, at least one other node in the cluster has to be configured in ''com.openexchange.hazelcast.network.join.static.nodes'' to allow joining, however, it's recommended to list all nodes in the cluster here
 
  
== Disable Cluster Features ==
+
A file pattern is defined by pattern strings for the filename and path, as well as further attributes.
  
The Hazelcast based clustering features can be disabled with the following property changes:
+
{| id="FilePattern" cellspacing="0" border="1"
* Disable cluster discovery by setting ''com.openexchange.hazelcast.network.join'' to ''empty'' in ''hazelcast.properties''
+
|+ align="bottom" | File Pattern
* Disable Hazelcast by setting ''com.openexchange.hazelcast.enabled'' to false in ''hazelcast.properties''
+
! Name !! Type !! Value
* Disable message based cache event invalidation by setting ''com.openexchange.caching.jcs.eventInvalidation'' to ''false'' in ''cache.properties''
+
|-
 +
| type || String || The pattern type, currently one of <code>exact</code> or <code>glob</code>.
 +
|-
 +
| path || String || The path pattern, in a format depending on the pattern type.  
 +
|-
 +
| name || String || The filename pattern, in a format depending on the pattern type.
 +
|-
 +
| caseSensitive || Boolean | Optional flag to enable case-sensitive matching, defaults to <code>false</code>
 +
|}
  
== Update from 6.22.1 to version 6.22.2 and above ==
+
== Pattern types ==
  
As hazelcast will be used by default for the distribution of sessions starting 6.22.2 you have to adjust hazelcast according to our old cache configuration. First of all it's important that you install the open-xchange-sessionstorage-hazelcast package. This package will add the binding between hazelcast and the internal session management. Next you have to set a cluster name to the cluster.properties file (see [[#Cluster Discovery Errors]]). Furthermore you will have to add one of the two discovery modes mentioned in [[#Cluster Discovery]].
+
A pattern currently may be defined in two formats: <code>exact</code> or <code>glob</code>.
  
 +
* <code>exact</code> <br /> An exact pattern, matching the file- or directory version literally. For example, to exclude the file <code>Backup.pst</code> in the subfolder <code>Mail</code> below the root synchronization folder, an <code>exact</code> file pattern would look like: <code>{"path":"/Mail","name":"Backup.pst","type":"exact"}</code>, or, an <code>exact</code> directory pattern for the directory <code>/Archive</code> would be represented as <code>{"path":"/Archive","type":"exact"}</code>.
 +
* <code>glob</code> <br /> A simple pattern allowing to use the common wildcards <code>*</code> and <code>?</code> to match file- and directory versions. For example, to exclude all files ending with <code>.tmp</code> across all directories, the <code>glob</code> file pattern could be defined as <code>{"path":"*","name":"*.tmp","type":"glob"}</code>, or, to exclude the directory <code>/Project/.git</code> and all its subdirectories recursively, this would be expressed using a combination of the following two directory patterns: <code>[{"path":"/Project/.git","type":"exact"},{"path":"/Project/.git*","type":"glob"}]</code>.
  
= Updating a Cluster =
+
== Further considerations ==
  
Running a cluster means built-in failover on the one hand, but might require some attention when it comes to the point of upgrading the services on all nodes in the cluster. This chapter gives an overview about general concepts and hints for silent updates of the cluster.
+
* It's possible to exclude a (parent) directory with an appropriate pattern, while still subfolders below that directory being synchronized. This usually results in the excluded directory being created ob both client- and server side, but no file contents within the excluded directory being exchanged. If subfolders should be excluded, too, a wildcard should be used in the pattern to match any subdirectories.
 +
* If the client tries to synchronize a file- or directory version that is ignored, i.e. a version that would match any of the provided exclusion filters, the server behaves similarly to the handling of invalid and ignored file- and directory names (see above), i.e. the client would be instructed to put those versions into quarantine.
 +
* For the calculation of directory checksums, it's important that the server and client perform exactly the same matching for ignored filenames: A <code>*</code> character matches zero or more characters, a <code>?</code> character matches exactly one character. All other characters are matched literally. Advanced glob flavors like braces to define subpattern alternatives or square brackets for character sets are not used.
 +
* Client-side filtering is available with API version 2. The API version that is supported by the server is included in the response of the [[#Get Settings | Settings]] request.
 +
* Whenever there are active exclusion filters, the <code>syncFolders</code> request should contain all of both directory and file exclusion filter lists. For the <code>syncFiles</code> request, it's sufficient to include the list of file exclusions.
  
== Limitations ==
 
  
While in most cases a seamless, rolling upgrade of all nodes in the cluster is possible, there may be situations where nodes running a newer version of the Open-Xchange Server are not able to communicate with older nodes in the cluster, i.e. can't access distributed data or consume incompatible event notifications - especially, when the underlying Hazelcast library is part of the update, which does not support this scenario at the moment. In such cases, the release notes will contain corresponding information, so please have a look there before applying an update.
+
= Metadata Synchronization =
  
Additionally, there may always be some kind of race conditions during an update, i.e. client requests that can't be completed successfully or internal events not being deliverd to all nodes in the cluster. That's why the following information should only serve as a best-practices guide to minimize the impact of upgrades to the user experience.
+
The synchronization of metadata is available since API version 3. <br />
  
== Upgrading a single Node ==
+
== Introduction ==
  
Upgrading all nodes in the cluster should usually be done sequentially, i.o.w. one node after the other. This means that during the upgrade of one node, the node is temporarily disconnected from the other nodes in the cluster, and will join the cluster again after the update is completed. From the backend perspective, this is as easy as stopping the open-xchange service. other nodes in the cluster will recognize the disconnected node and start to repartition the shared cluster data automatically. But wait a minute - doing so would potentially lead to the webserver not registering the node being stopped immediately, resulting in temporary errors for currently logged in users until they are routed to another machine in the cluster. That's why it's good practice to tell the webserver's load balancer that the node should no longer fulfill incoming requests. The Apache Balancer Manager is an excellent tool for this ([http://httpd.apache.org/docs/2.2/mod/mod_status.html module ''mod_status'']). Look at the screen shot. Every node can be put into a disabled mode. Further requests will the redirected to other nodes in the cluster:
+
Previously, only the "raw" folders and files were synchronized between server and clients. While this is sufficient for basic synchronization, there are cases where the clients could benefit from additional data - "metadata" - that is already available on the server. For example, clients could display directories that have been shared or published to other people in a different way. Or, clients could consider folder permissions directly in case the user is performing a local change that would be rejected by the server in the next synchronization cycle anyway.
  
 +
To supply the clients with those additional information without any influence on the existing synchronization protocol (!), <code>.drive-meta</code> files are introduced for each synchronized directory. Regarding synchronization, such files are treated like any other ordinary file. Especially, those files are taken into account when it comes to directory checksum calculation. Doing so, metadata updates result in a changed <code>.drive-meta</code> file, which in turn causes the parent directory checksum to change, hence synchronization is triggered.
  
[[Image:balancer_manager.jpg]]
+
However, some special handling applies for those files:
  
 +
* Clients are not allowed to change metadata, so modifications of metadata files or the deletion of them is rejected. Recovery is done via the protocol here, i.e. the client is instructed to re-download the file.
 +
* <code>.drive-meta</code> files are actually not stored physically on the file storage backend, but created on the fly based on the actual metadata of the directory.
 +
* Client applications may either store such files on the client file system, or evaluate and store the contained metadata information in a local database for later retrieval. If the file is not saved physically on the client (which is actually recommended), the client is responsible to consider the metadata file in a virtual way and include it's checksum for the directory checksum calculation - similar to the server's internal handling.
  
Afterwards, the open-xchange service on the disabled node can be stopped by executing:
+
== Metadata format ==
  
  $ /etc/init.d/open-xchange stop
+
The metadata in <code>.drive-meta</code> files is serialized in JSON format to allow easy processing at the clients. The following shows an example of the contents:
 +
  {
 +
  "path": "/",
 +
  "created": 1418024049629,
 +
  "modified": 1418024189166,
 +
  "own_rights": 403710016,
 +
  "permissions": [{
 +
    "bits": 403710016,
 +
    "group": false,
 +
    "entity": 182,
 +
    "display_name": "Mander, Jens",
 +
    "email_address": "jens.mander@example.com",
 +
    "guest": false
 +
  }],
 +
  "jump": ["permissions"],
 +
  "files": [{
 +
    "name": "Koala.jpg",
 +
    "created": 1418024190565,
 +
    "modified": 1418026995663,
 +
    "created_by": {
 +
      "group": false,
 +
      "entity": 182,
 +
      "display_name": "Mander, Jens",
 +
      "email_address": "jens.mander@example.com",
 +
      "guest": false
 +
    },
 +
    "modified_by": {
 +
      "group": false,
 +
      "entity": 182,
 +
      "display_name": "Mander, Jens",
 +
      "email_address": "jens.mander@example.com",
 +
      "guest": false
 +
    },
 +
    "preview": "http://192.168.32.191/ajax/files?action=document&folder=268931&id=268931/297620&version=1&delivery=download&scaleType=contain&width=800&height=800&rotate=true",
 +
    "thumbnail": "http://192.168.32.191/ajax/files?action=document&folder=268931&id=268931/297620&version=1&delivery=download&scaleType=contain&width=100&height=100&rotate=true",
 +
    "object_permissions": [{
 +
      "bits": 1,
 +
      "group": false,
 +
      "entity": 10,
 +
      "display_name": "Jan Ot/to Finsel",
 +
      "email_address": "jan.finsel@premium",
 +
      "guest": false
 +
    },
 +
    {
 +
      "bits": 1,
 +
      "group": false,
 +
      "entity": 8338,
 +
      "email_address": "horst@example.com",
 +
      "guest": true
 +
    }],
 +
    "shared": true,
 +
    "number_of_versions": 1,
 +
    "version": "1",
 +
    "jump": ["preview",
 +
    "permissions",
 +
    "version_history"]
 +
  },
 +
  {
 +
    "name": "test.txt",
 +
    "created": 1418024198520,
 +
    "modified": 1418027394897,
 +
    "created_by": {
 +
      "group": false,
 +
      "entity": 182,
 +
      "display_name": "Mander, Jens",
 +
      "email_address": "jens.mander@example.com",
 +
      "guest": false
 +
    },
 +
    "modified_by": {
 +
      "group": false,
 +
      "entity": 182,
 +
      "display_name": "Mander, Jens",
 +
      "email_address": "jens.mander@example.com",
 +
      "guest": false
 +
    },
 +
    "preview": "http://192.168.32.191/ajax/files?action=document&format=preview_image&folder=268931&id=268931/297621&version=6&delivery=download&scaleType=contain&width=800&height=800",
 +
    "thumbnail": "http://192.168.32.191/ajax/files?action=document&format=preview_image&folder=268931&id=268931/297621&version=6&delivery=download&scaleType=contain&width=100&height=100",
 +
    "locked": true,
 +
    "number_of_versions": 4,
 +
    "version": "6",
 +
    "version_comment": "Uploaded with OX Drive (TestDrive)",
 +
    "versions": [{
 +
      "name": "test.txt",
 +
      "file_size": 23,
 +
      "created": 1418024198520,
 +
      "modified": 1418024202878,
 +
      "created_by": {
 +
        "group": false,
 +
        "entity": 182,
 +
        "display_name": "Mander, Jens",
 +
        "email_address": "jens.mander@example.com",
 +
        "guest": false
 +
      },
 +
      "modified_by": {
 +
        "group": false,
 +
        "entity": 182,
 +
        "display_name": "Mander, Jens",
 +
        "email_address": "jens.mander@example.com",
 +
        "guest": false
 +
      },
 +
      "version": "1",
 +
      "version_comment": "Uploaded with OX Drive (TestDrive)"
 +
    },
 +
    {
 +
      "name": "test.txt",
 +
      "file_size": 54,
 +
      "created": 1418024234782,
 +
      "modified": 1418024231522,
 +
      "created_by": {
 +
        "group": false,
 +
        "entity": 182,
 +
        "display_name": "Mander, Jens",
 +
        "email_address": "jens.mander@example.com",
 +
        "guest": false
 +
      },
 +
      "modified_by": {
 +
        "group": false,
 +
        "entity": 182,
 +
        "display_name": "Mander, Jens",
 +
        "email_address": "jens.mander@example.com",
 +
        "guest": false
 +
      },
 +
      "version": "2",
 +
      "version_comment": "Uploaded with OX Drive (TestDrive)"
 +
    },
 +
    {
 +
      "name": "test.txt",
 +
      "file_size": 120,
 +
      "created": 1418027349026,
 +
      "modified": 1418027355957,
 +
      "created_by": {
 +
        "group": false,
 +
        "entity": 182,
 +
        "display_name": "Mander, Jens",
 +
        "email_address": "jens.mander@example.com",
 +
        "guest": false
 +
      },
 +
      "modified_by": {
 +
        "group": false,
 +
        "entity": 182,
 +
        "display_name": "Mander, Jens",
 +
        "email_address": "jens.mander@example.com",
 +
        "guest": false
 +
      },
 +
      "version": "5"
 +
    },
 +
    {
 +
      "name": "test.txt",
 +
      "file_size": 127,
 +
      "created": 1418027370051,
 +
      "modified": 1418027366945,
 +
      "created_by": {
 +
        "group": false,
 +
        "entity": 182,
 +
        "display_name": "Mander, Jens",
 +
        "email_address": "jens.mander@example.com",
 +
        "guest": false
 +
      },
 +
      "modified_by": {
 +
        "group": false,
 +
        "entity": 182,
 +
        "display_name": "Mander, Jens",
 +
        "email_address": "jens.mander@example.com",
 +
        "guest": false
 +
      },
 +
      "version": "6",
 +
      "version_comment": "Uploaded with OX Drive (TestDrive)"
 +
    }],
 +
    "jump": ["preview",
 +
    "edit",
 +
    "permissions",
 +
    "version_history"]
 +
  },
 +
  {
 +
    "name": "Kalimba.mp3",
 +
    "created": 1418026529047,
 +
    "modified": 1247549551659,
 +
    "created_by": {
 +
      "group": false,
 +
      "entity": 182,
 +
      "display_name": "Mander, Jens",
 +
      "email_address": "jens.mander@example.com",
 +
      "guest": false
 +
    },
 +
    "modified_by": {
 +
      "group": false,
 +
      "entity": 182,
 +
      "display_name": "Mander, Jens",
 +
      "email_address": "jens.mander@example.com",
 +
      "guest": false
 +
    },
 +
    "preview": "http://192.168.32.191/ajax/image/file/mp3Cover?folder=268931&id=268931/297623&version=1&delivery=download&scaleType=contain&width=800&height=800",
 +
    "thumbnail": "http://192.168.32.191/ajax/image/file/mp3Cover?folder=268931&id=268931/297623&version=1&delivery=download&scaleType=contain&width=100&height=100",
 +
    "number_of_versions": 1,
 +
    "version": "1",
 +
    "version_comment": "Uploaded with OX Drive (TestDrive)",
 +
    "jump": ["preview",
 +
    "permissions",
 +
    "version_history"]
 +
  }]
 +
}
  
or
+
The following objects describe the JSON structure of the metadata for a directory:
  
$ service open-xchange stop
+
{| id="DirectoryMetadata" cellspacing="0" border="1"
 +
|+ align="bottom" | Directory Metadata
 +
! Name !! Type !! Value
 +
|-
 +
| id || String || The server-side unique identifier of the directory.
 +
|-
 +
| name || String || The display name of the directory.
 +
|-
 +
| path || String || The path of the directory the metadata belongs to.
 +
|-
 +
| created || Timestamp || The folder's last modification time (always UTC, not translated into user time).
 +
|-
 +
| modified || Timestamp || The folder's last modification time (always UTC, not translated into user time).
 +
|-
 +
| own_rights || Number|| Folder permissions which apply to the current user, as described in [[http://oxpedia.org/index.php?title=HTTP_API#PermissionFlags | permission flags]].
 +
|-
 +
| permissions || Array || All folder permissions, each element is an object as described in [[#Folder Permission | Folder Permission]].
 +
|-
 +
| default_folder || Boolean || <code>true</code> if the folder is a default folder, <code>false</code> or not set, otherwise.
 +
|-
 +
| has_subfolders || Boolean || <code>true</code> if the folder (potentially) has subfolders, <code>false</code> or not set, otherwise.
 +
|-
 +
| shared || Boolean || <code>true</code> if the folder is shared, <code>false</code> or not set, otherwise.
 +
|-
 +
| type || Number || The special folder type, or not set, if not available.
 +
|-
 +
| jump || Array || An array containing the names of possible <code>jump</code> methods to use for the folder.
 +
|-
 +
| files || Array || Metadata for the contained files, each element is an object as described in [[#File Metadata | File Metadata]].
 +
|}
  
Now, the node is effectively in maintenance mode and any updates can take place. One could now verify the changed cluster infrastructure by accessing the Hazelcast MBeans either via JMX or the ''showruntimestats -c'' commandline tool (see above for details). There, the shut down node should no longer appear in the 'Member' section (com.hazelcast:type=Member).
 
  
When all upgrades are processed, the node open-xchange service can be started again by executing:
+
{| id="FileMetadata" cellspacing="0" border="1"
 +
|+ align="bottom" | File Metadata
 +
! Name !! Type !! Value
 +
|-
 +
| name || String || The name of the file the metadata belongs to.
 +
|-
 +
| created || Timestamp || The file's last modification time (always UTC, not translated into user time).
 +
|-
 +
| modified || Timestamp || The file's last modification time (always UTC, not translated into user time).
 +
|-
 +
| created_by || Object || Information about the file's creator as described in  [[#Entity Information | Entity Information]].
 +
|-
 +
| modified_by || Object || Information about the file's last editor as described in  [[#Entity Information | Entity information]].
 +
|-
 +
| preview || String || A URL to a preview image for the file.
 +
|-
 +
| thumbnail || String || A URL to a thumbnail image for the file.
 +
|-
 +
| object_permissions || Array | All file permissions, each element is an object as described in [[#Object Permission | Object Permission]].
 +
|-
 +
| shared || Boolean || <code>true</code> if the file is shared, <code>false</code> or not set, otherwise.
 +
|-
 +
| locked || Boolean || <code>true</code> if the file is locked, <code>false</code> or not set, otherwise.
 +
|-
 +
| jump || Array || An array containing the names of possible <code>jump</code> methods to use for the file.
 +
|-
 +
| number_of_versions || Number | The number of all versions of the file.
 +
|-
 +
| version || String || The current version identifier (usually, but not necessarily a numerical value) of the file.
 +
|-
 +
| version_comment || String | An additional comment for the file version.
 +
|-
 +
| versions || Array || Metadata for all versions of the file, each element is an object as described in [[#File Version | File Version]].
 +
|}
  
$ /etc/init.d/open-xchange start
+
{| id="EntityInformation" cellspacing="0" border="1"
 +
|+ align="bottom" | Entity Information
 +
! Name !! Type !! Value
 +
|-
 +
| entity || Number || The unique identifier of the entity.
 +
|-
 +
| group || Boolean || <code>true</code> if the entity is a group, <code>false</code> or not set, if it is a single user or guest.
 +
|-
 +
| display_name || String || A display name for the entity if available.
 +
|-
 +
| email_address || String || An e-mail address for the entity if available.
 +
|-
 +
| group || Boolean || <code>true</code> if the entity is an external guest, <code>false</code> or not set, otherwise.
 +
|}
  
or  
+
{| id="FolderPermission" cellspacing="0" border="1"
 +
|+ align="bottom" | Folder Permission
 +
! Name !! Type !! Value
 +
|-
 +
| entity || Number || The unique identifier of the entity.
 +
|-
 +
| group || Boolean || <code>true</code> if the entity is a group, <code>false</code> or not set, if it is a single user or guest.
 +
|-
 +
| display_name || String || A display name for the entity if available.
 +
|-
 +
| email_address || String || An e-mail address for the entity if available.
 +
|-
 +
| group || Boolean || <code>true</code> if the entity is an external guest, <code>false</code> or not set, otherwise.
 +
|-
 +
| bits || Number|| Permission level, as described in [[http://oxpedia.org/index.php?title=HTTP_API#PermissionFlags | permission flags]].
 +
|}
  
$ service open-xchange start
+
{| id="ObjectPermission" cellspacing="0" border="1"
 +
|+ align="bottom" | Object Permission
 +
! Name !! Type !! Value
 +
|-
 +
| entity || Number || The unique identifier of the entity.
 +
|-
 +
| group || Boolean || <code>true</code> if the entity is a group, <code>false</code> or not set, if it is a single user or guest.
 +
|-
 +
| display_name || String || A display name for the entity if available.
 +
|-
 +
| email_address || String || An e-mail address for the entity if available.
 +
|-
 +
| group || Boolean || <code>true</code> if the entity is an external guest, <code>false</code> or not set, otherwise.
 +
|-
 +
| bits || Number|| Object permission level, as described in [[http://oxpedia.org/index.php?title=HTTP_API#ObjectPermissionFlags | permission flags]].
 +
|}
  
As stated above, depending on the chosen cluster discovery mechanism, it might take some time until the node joins the cluster again. When using static cluster discovery, it will join the existing cluster usually directly during serivce startup, i.o.w. before other depending OSGi services are started. Otherwise, there might also be situations where the node cannot join the cluster directly, for example when there were no mDNS advertisments for other nodes in the cluster received yet. Then, it can take some additional time until the node finally joins the cluster. During startup of the node, you can observe the JMX console or the output of ''showruntimestats -c'' (com.hazelcast:type=Member) of another node in the cluster to verify when the node has joined.  
+
{| id="FileVersion" cellspacing="0" border="1"
 +
|+ align="bottom" | File Version
 +
! Name !! Type !! Value
 +
|-
 +
| name || String || The name of the file version.  
 +
|-
 +
| file_size || Number || The file size of the version in bytes.  
 +
|-
 +
| created || Timestamp || The file version's last modification time (always UTC, not translated into user time).  
 +
|-
 +
| modified || Timestamp || The file version's last modification time (always UTC, not translated into user time).
 +
|-
 +
| created_by || Object || Information about the file version's creator as described in  [[#Entity Information | Entity Information]].  
 +
|-
 +
| modified_by || Object || Information about the file version's last editor as described in  [[#Entity Information | Entity information]].
 +
|-
 +
| version || String || The version identifier (usually, but not necessarily a numerical value) of the file version.
 +
|-
 +
| version_comment || String || An additional comment for the file version.  
 +
|}
  
After the node has joined, distributed data is re-partioned automatically, and the node is ready to server incoming requests again - so now the node can finally be enabled again in the load balancer configuration of the webserver. Afterwards, the next node in the cluster can be upgraded using the same procedure, until all nodes were processed.
+
== Client-side implementation ==
  
== Other Considerations ==
+
In order to make use of the metadata, clients should roughly implement the following:
 +
* Include the <code>apiVersion</code> parameter in each request, and set it to at least <code>3</code> in order to include <code>.drive-meta</code> during synchronization
 +
* Evaluate <code>.drive-meta</code> files and store the information, as well as the file's checksums in a local database
 +
* Include this file in the calculation of the parent directory checksum, just like an ordinary file in that directory
 +
* Do something useful with the metadata information.
  
* It's always recommended to only upgrade one node after the other, always ensuring that the cluster has formed correctly between each shutdown/startup of a node.
+
== Additional notes ==
* Do not stop a node while running the runUpdate script or the associated update task.
 
* During the time of such a rolling upgrade of all nodes, we have effectively heterogeneous software versions in the cluster, which potentially might lead to temporary inconsistencies. Therefore, all nodes in the cluster should be updated in one cycle (but still one after the other).
 
* Following the above guideline, it's also possible to add or remove nodes dynamically to the cluster, not only when disconnecting a node temporary for updates.
 
* In case of trouble, i.e. a node refuses to join the cluster again after restart, consult the logfiles first for any hints about what is causing the problem - both on the disconnected node, and also on other nodes in the network
 
* If there are general incompatibilities between two revisions of the Open-Xchange Server that prevent an operation in a cluster (release notes), it's recommended to choose another name for the cluster in ''cluster.properties'' for the nodes with the new version. This will temporary lead to two separate clusters during the rolling upgrade, and finally the old cluster being shut down completely after the last node was updated to the new version. While distributed data can't be migrated from one server version to another in this scenario due to incompatibilities, the uptime of the system itself is not affected, since the nodes in the new cluster are able to serve new incoming requests directly.
 
  
 +
* The metadata synchronization via <code>.drive-meta</code> files embedded into the synchronization protocol obsoletes the previously used methods to receive metadata information ([[#Get file metadata]] and <code>columns</code> parameter in [[#Synchronize files in a folder]].
 +
* Depending on the underlying file storage backend, the included metadata may vary, so each information should be treatened as optional.
  
[[Category: AppSuite]] [[Category: Administration]] [[Category: Cluster]]
+
== Possible use cases ==
 +
* For files where the <code>locked</code> property is <code>true</code>, display some kind of "lock" icon (-overlay) in the file list / explorer view
 +
* For files or folders where the <code>shared</code> property is <code>true</code>, display some kind of "cloud" icon (-overlay) in the file list / explorer view
 +
* For files or folders where the user is not allowed to perform an action with, don't offer such actions (e.g. if a file cannot be deleted or renamed by the user due to insufficient permissions, disable the corresponding options)
 +
* Use the URLs in <code>preview</code> and <code>thumbnail</code> to get a preview image for the files
 +
* Display the server creation / last modification timestamps of files and folders
 +
* Embed a version history for files with multiple versions
 +
* Show to which users a file or folder is currently shared
 +
* Offer appropriate "jump" actions to the groupware web interface for more advanced options (e.g. to directly edit an .xlsx file in the spreadsheet application of the web interface, or to manage a folder's permission

Revision as of 07:36, 26 March 2015

OX Drive API

Introduction

The module drive is used to synchronize files and folders between server and client, using a server-centric approach to allow an easy implementation on the client-side.

The synchronization is based on checksums for files and folders, differences between the server- and client-side are determined using a three-way comparison of server, client and previously acknowledged file- and directory-versions. The synchronization logic is performed by the server, who instructs the client with a set of actions that should be executed in order to come to a synchronized state.

Therefore, the client takes a snapshot of it's local files and directories, calculates their checksums, and sends them as a list to the server, along with a list of previously acknowledged checksums. The server takes a similar snapshot of the files and directories on the underlying file storages and evaluates which further actions are necessary for synchronization. After executing the server-side actions, the client receives a list of actions that should be executed on the client-side. These steps are repeated until the server-state matches the client-state.

Key concept is that the synchronization works stateless, i.e. it can be interrupted and restarted at any time, following the eventual consistency model.

Entry point for the synchronization is the syncfolders request, where the directories are compared, and further actions are determined by the server, amongst others actions to synchronize the files in a specific directory using the syncfiles request. After executing the actions, the client should send another syncfolders request to the server and execute the returned actions (if present), or finish the synchronization if there are no more actions to execute. In pseudo-code, the synchronization routine could be implemented as follows:

WHILE TRUE
{
  response = SYNCFOLDERS()
  IF 0 == response.actions.length
    BREAK
  ELSE
    EXECUTE(response.actions)
}

Basically, it's up to the client how often such a synchronization cycle is initiated. For example, he could start a new synchronization cycle after a fixed interval, if he recognizes that the client directories have changed, or if he is informed that something has changed on the server by an event. It's also up to the client to interrupt the synchronization cycle at any time during execution of the actions and continue later on, however, it's recommended to start a new synchronization cycle each time to avoid possibly outdated actions.

API

As part of the HTTP API, the basic conventions for exchanging messages that described there are also valid for this case, especially the low level protocol and error handling. Each request against the Drive API assumes a valid server session that is uniquely identified by the session id and the corresponding cookies and are sent with each request. A new session can be created via the login module.

The root folder plays another important role for the message exchange. The root folder has a unique identifier. It is the parent server folder for the synchronization. All path details for directories and files are relative to this folder. This folder's id is sent with each request. To select the root folder during initial client configuration, the client may get a list of synchronizable folders with the subfolders action.

Subsequently all transferred objects and all possible actions are listed.

File Version

A file in a directory is uniquely identified by its filename and the checksum of its content.

File Version
Name Type Value
name String The name of the file, including its extension, e.g. test.doc.
checksum String The MD5 hash of the file, expressed as a lowercase hexadecimal number string, 32 characters long, e.g. f8cacac95379527cd4fa15f0cb782a09.

Directory Version

A directory is uniquely identified by its full path, relative to the root folder, and the checksum of its content.

Directory Version
Name Type Value
path String The path of the directory, including the directory's name, relative to the root folder, e.g. /sub/test/letters.
checksum String The MD5 hash of the directory, expressed as a lowercase hexadecimal number string, 32 characters long, e.g. f8cacac95379527cd4fa15f0cb782a09.

Note: the checksum of a directory is calculated based on its contents in the following algorithm:

  • Build a list containing each file in the directory (not including subfolders or files in subfolders)
  • Ensure a lexicographically order in the following way:
    • Normalize the filename using the NFC normalization form (canonical decomposition, followed by canonical composition) - see http://www.unicode.org/reports/tr15/tr15-23.html for details
    • Encode the filename to an array of UTF-8 unsigned bytes (array of codepoints)
    • Compare the filename (encoded as byte array "fn1") to another one "fn2" using the following comparator algorithm:
min_length = MIN(LENGTH(fn1), LENGTH(fn2))
FOR i = 0; i < min_length; i++ 
{
  result = fn1[i] - fn2[i]
  IF 0 != result RETURN result
}
RETURN LENGTH(fn1) - LENGTH(fn2)
  • Calculate the aggregated MD5 checksum for the directory based on each file in the ordered list:
    • Append the file's NFC-normalized (see above) name, encoded as UTF-8 bytes
    • Append the file's MD5 checksum string, encoded as UTF-8 bytes

Actions

All actions are encoded in the following format. Depending on the action type, not all properties may be present.

Actions
Name Type Value
action String The type of action to execute, currently one of acknowledge, edit, download, upload, remove, sync, error.
version Object The (original) file- or directory-version referenced by the action.
newVersion Object The (new) file- or directory-version referenced by the action.
path String The path to the synchronized folder, relative to the root folder.
offset Number The requested start offset in bytes for file uploads.
totalLength Number The total length in bytes for file downloads.
contentType String The file's content type for downloads.
created Timestamp The file's creation time (always UTC, not translated into user time).
modified Timestamp The file's last modification time (always UTC, not translated into user time).
error Object The error object in case of synchronization errors.
quarantine Boolean The flag to indicate whether versions need to be excluded from synchronization.
reset Boolean The flag to indicate whether locally stored checksums should be invalidated.
stop Boolean The flag to signal that the client should stop the current synchronizsation cycle.
acknowledge Boolean The flag to signal if the client should not update it's stored checksums when performing an EDIT action.
thumbnailLink String A direct link to a small thumbnail image of the file if available (deprecated, available until API version 2).
previewLink String A direct link to a medium-sized preview image of the file if available (deprecated, available until API version 2).
directLink String A direct link to the detail view of the file in the web interface (deprecated, available until API version 2).
directLinkFragments String The fragments part of the direct link (deprecated, available until API version 2).

The following list gives an overview about the used action types:

acknowledge

Acknowledges the successful synchronization of a file- or directory version, i.e., the client should treat the version as synchronized by updating the corresponding entry in its metadata store and including this updated information in all following originalVersions arrays of the syncfiles / syncfolders actions. Depending on the version and newVersion parameters of the action, the following acknowledge operations should be executed (exemplarily for directory versions, file versions are acknowledged in the same way):

  • Example 1: Acknowledge a first time synchronized directory
    The server sends an acknowledge action where the newly synchronized directory version is encoded in the newVersion parameter. The client should store the version in his local checksum store and send this version in the originalVersions array in upcoming syncfolders requests.
{
  "action" : "acknowledge",
  "newVersion" : {
     "path" : "/",
     "checksum" : "d41d8cd98f00b204e9800998ecf8427e"
  }
}
  • Example 2: Acknowledge a synchronized directory after updates
    The server sends an acknowledge action where the previous directory version is encoded in the version, and the newly synchronized directory in the newVersion parameter. The client should replace any previously stored entries of the directory version in his local checksum store with the updated version, and send this version in the originalVersions array in upcoming syncfolders requests.
{
  "action" : "acknowledge",
  "newVersion" : {
    "path" : "/",
    "checksum" : "7bb1f1a550e9b9ab4be8a12246f9d5fb"
  },
  "version" : {
    "path" : "/",
    "checksum" : "d41d8cd98f00b204e9800998ecf8427e"
  }
}
  • Example 3: Acknowledge the deletion of a previously synchronized directory
    The server sends an acknowledge where the newVersion parameter is set to null to acknowledge the deletion of the previously synchronized directory version as found in the version parameter. The client should remove any stored entries for this directory from his local checksum store, and no longer send this version in the originalVersions array in upcoming syncfolders requests.
    Note that an acknowledged deletion of a directory implicitly acknowledges the deletion of all contained files and subfolders, too, so the client should also remove those originalVersions from his local checksum store.
{
  "action" : "acknowledge",
  "version" : {
    "path" : "/test",
    "checksum" : "3525d6f28eb8cb30eb61ab7932367c35"
  }
}

edit

Instructs the client to edit a file- or directory version. This is used for move/rename operations. The version parameter is set to the version as sent in the clientVersions array of the preceding syncfiles/syncfolders action. The newVersion contains the new name/path the client should use. Unless the optional boolean parameter acknowledge is set to false an edit action implies that the client updates its known versions store accordingly, i.e. removes the previous entry for version and adds a new entry for newVersion. When editing a directory version, the client should implicitly take care to create any not exisiting subdirectories in the path of the newVersion parameter. A concurrent client-side modification of the file/directory version can be detected by the client by comparing the current checksum against the one in the passed newVersion parameter.

  • Example 1: Rename a file
    The server sends an edit action where the source file is encoded in the version, and the target file in the newVersion parameter. The client should rename the file identified by the version parameter to the name found in the newVersion parameter. Doing so, the stored checksum entry for the file in version should be updated, too, to reflect the changes.
{
  "path" : "/",
  "action" : "edit",
  "newVersion" : {
    "name" : "test_1.txt",
    "checksum" : "03395a94b57eef069d248d90a9410650"
  },
  "version" : {
    "name" : "test.txt",
    "checksum" : "03395a94b57eef069d248d90a9410650"
  }
}
  • Example 2: Move a directory
    The server sends an edit action where the source directory is encoded in the version, and the target directory in the newVersion parameter. The client should move the directory identified by the version parameter to the path found in the newVersion parameter. Doing so, the stored checksum entry for the directory in version should be updated, too, to reflect the changes.
{
  "action" : "edit",
  "newVersion" : {
    "path" : "/test2",
    "checksum" : "3addd6de801f4a8650c5e089769bdb62"
  },
  "version" : {
    "path" : "/test1/test2",
    "checksum" : "3addd6de801f4a8650c5e089769bdb62"
  }
}
  • Example 3: Rename a conflicting file
    The server sends an edit action where the original client file is encoded in the version, and the target filename in the newVersion parameter. The client should rename the file identified by the version parameter to the new filename found in the newVersion parameter. If the acknowledge parameter is set to true or is not set, the stored checksum entry for the file in version should be updated, too, to reflect the changes, otherwise, as in this example, no changes should be done to the stored checksums.
{
  "action" : "edit",
  "version" : {
    "checksum" : "fade32203220752f1fa0e168889cf289",
    "name" : "test.txt"
  },
  "newVersion" : {
    "checksum" : "fade32203220752f1fa0e168889cf289",
    "name" : "test (TestDrive).txt"
  },
  "acknowledge" : false,
  "path" : "/"
}

download

Contains information about a file version the client should download. For updates of existing files, the previous client version is supplied in the version parameter. For new files, the version parameter is omitted. The newVersion holds the target file version, i.e. filename and checksum, and should be used for the following download request. The totalLength parameter is set to the file size in bytes, allowing the client to recognize when a download is finished. Given the supplied checksum, the client may decide on its own if the target file needs to be downloaded from the server, or can be created by copying a file with the same checksum to the target location, e.g. from a trash folder. The file's content type can be retrieved from the contentType parameter, similar to the file's creation and modification times that are availble in the created and modified parameters.

  • Example 1: Download a new file
    The server sends a download action where the file version to download is encoded in the newVersion paramter. The client should download and save the file as indicated by the name property of the newVersion in the directory identified by the supplied path. After downloading, the newVersion should be added to the client's known file versions database.
{
  "totalLength" : 536453,
  "path" : "/",
  "action" : "download",
  "newVersion" : {
    "name" : "test.pdf",
    "checksum" : "3e0d7541b37d332c42a9c3adbe34aca2"
  },
  "contentType" : "application/pdf",
  "created" : 1375276738232,
  "modified" : 1375343720985
}
  • Example 2: Download an updated file
    The server sends a download action where the previous file version is encoded in the version, and the file version to download in the newVersion parameter. The client should download and save the file as indicated by the name property of the newVersion in the directory identified by the supplied path, replacing the previous file. After downloading, the newVersion should be added to the client's known file versions database, replacing an existing entry for the previous version.
{
  "totalLength" : 1599431,
  "path" : "/",
  "action" : "download",
  "newVersion" : {
    "name" : "test.pdf",
    "checksum" : "bb198790904f5a1785d7402b0d8c390e"
  },
  "contentType" : "application/pdf",
  "version" : {
    "name" : "test.pdf",
    "checksum" : "3e0d7541b37d332c42a9c3adbe34aca2"
  },
  "created" : 1375276738232,
  "modified" : 1375343720985
}

upload

Instructs the client to upload a file to the server. For updates of existing files, the previous server version is supplied in the version parameter, and should be used for the following upload request. For new files, the version parameter is omitted. The newVersion holds the target file version, i.e. filename and checksum, and should be used for the following upload request. When resuming a previously partly completed upload, the offset parameter contains the offset in bytes from which the file version should be uploaded by the client. If possible, the client should set the contentType parameter for the uploaded file, otherwise, the content type falls back to application/octet-stream.

remove

Instructs the client to delete a file or directory version. The version parameter contains the version to delete. A deletion also implies a removal of the corresponding entry in the client's known versions store. A concurrent client-side modification of the file/directory version can be detected by comparing the current checksum against the one in the passed version parameter.

  • Example 1: Remove a file
    The server sends a remove action where the file to be removed is encoded as version parameter. The newVersion parameter is not set in the action. The client should delete the file identified by the version parameter. A stored checksum entry for the file in version should be removed, too, to reflect the changes. The newVersion parameter is not set in the action.
{
  "path" : "/test2",
  "action" : "remove",
  "version" : {
    "name" : "test.txt",
    "checksum" : "03395a94b57eef069d248d90a9410650"
  }
}
  • Example 2: Remove a directory
    The server sends a remove action where the directory to be removed is encoded as version parameter. The newVersion parameter is not set in the action. The client should delete the directory identified by the version parameter. A stored checksum entry for the directory in version should be removed, too, to reflect the changes.
{
  "action" : "remove",
  "version" : {
    "path" : "/test1",
    "checksum" : "d41d8cd98f00b204e9800998ecf8427e"
  }
}

sync

The client should trigger a synchronization of the files in the directory supplied in the version parameter using the syncfiles request. A sync action implies the client-side creation of the referenced directory if it not yet exists, in case of a new directory on the server. \\ If the version parameter is not specified, a synchronization of all folders using the syncfolders request should be initiated by the client. \\ If the reset flag in the SYNC action is set to true, the client should reset his local state before synchronizing the files in the directory. This may happen when the server detects a synchronization cycle, or believes something else is going wrong. Reset means that the client should invalidate any stored original checksums for the directory itself and any contained files, so that they get re-calculated upon the next synchronization. If the reset flag is set in a SYNC action without a apecific directory version, the client should invalidate any stored checksums, so that all file- and directory-versions get re-calculated during the following synchronizations.

  • Example 1: Synchronize folder
    The server sends a sync action with a version. The client should trigger a syncfiles request for the specified folder.
{
  "action": "sync",
  "version": {
    "path": "<folder>",
    "checksum": "<md5>"
  }
}
  • Example 2: Synchronize all folders
    The server sends a sync action without version (or version is //null//). The client should trigger a syncfolder request, i.e. the client should synchronize all folders.
{
  "action": "sync",
  "version": null
}

error

With the error action, file- or directory versions causing a synchronization problem can be identified. The root cause of the error is encoded in the error parameter as described at the HTTP API.

Basically, there are two scenarios where either the errorneous version affects the synchronization state or not. For example, a file that was deleted at the client without sufficient permissions on the server can just be downloaded again by the client, and afterwards, client and server are in-sync again. On the other hand, e.g. when creating a new file at the client and this file can't be uploaded to the server due to missing permissions, the client is out of sync as long as the file is present. Therefore, the boolean parameter quarantine instructs the client whether the file or directory version must be excluded from the synchronization or not. If it is set to true, the client should exclude the version from the clientVersions array, and indicate the issue to the enduser. However, if the synchronization itself is not affected and the quarantine flag is set to false, the client may still indicate the issue once to the user in the background, e.g. as a balloontip notification.

The client may reset it's quarantined versions on it's own, e.g. if the user decides to "try again", or automatically after a configurable interval.

The server may also decide that further synchronization should be suspended, e.g. in case of repeated synchronization problems. Such a situation is indicated with the parameter stop set to true. In this case, the client should at least cancel the current synchronization cycle. If appropriate, the client should also be put into a 'paused' mode, and the user should be informed accordingly.

There may also be situations where a error or warning is sent to the client, independently of a file- or directory version, e.g. when the client version is outdated and a newer version is available for download.

The most common examples for errors are insufficient permissions or exceeded quota restrictions, see examples below.

  • Example 1: Create a file in a read-only folder
    The server sends an error action where the errorneous file is encoded in the newVersion parameter and the quarantine flag is set to true. The client should exclude the version from the clientVersions array in upcoming syncFiles requests so that it doesn't affect the synchronization algorithm. The error message and further details are encoded in the error object of the action.
{
  "error" : {
    "category" : 3,
    "error_params" : ["/test"],
    "error" : "You are not allowed to create files at \"/test\"",
    "error_id" : "1358320776-69",
    "categories" : "PERMISSION_DENIED",
    "code" : "DRV-0012"
  },
  "path" : "/test",
  "quarantine" : true,
  "action" : "error",
  "newVersion" : {
    "name" : "test.txt",
    "checksum" : "3f978a5a54cef77fa3a4d3fe9a7047d2"
  }
}
  • Example 2: Delete a file without sufficient permissions
    Besides a new download action to restore the locally deleted file again, the server sends an error action where the errorneous file is encoded in the version parameter and the quarantine flag is set to false. Further synchronizations are not affected, but the client may still inform the user about the rejected operation. The error message and further details are encoded in the error object of the action.
{
  "error" : {
    "category" : 3,
    "error_params" : ["test.png", "/test"],
    "error" : "You are not allowed to delete the file \"test.png\" at \"/test\"",
    "error_id" : "1358320776-74",
    "categories" : "PERMISSION_DENIED",
    "code" : "DRV-0011"
  },
  "path" : "/test",
  "quarantine" : false,
  "action" : "error",
  "newVersion" : {
    "name" : "test.png",
    "checksum" : "438f06398ce968afdbb7f4db425aff09"
  }
}
  • Example 3: Upload a file that exceeds the quota
    The server sends an error action where the errorneous file is encoded in the newVersion parameter and the quarantine flag is set to true. The client should exclude the version from the clientVersions array in upcoming syncFiles requests so that it doesn't affect the synchronization algorithm. The error message and further details are encoded in the error object of the action.
{
  "error" : {
    "category" : 3,
    "error_params" : [],
    "error" : "The allowed Quota is reached",
    "error_id" : "-485491844-918",
    "categories" : "PERMISSION_DENIED",
    "code" : "DRV-0016"
  },
  "path" : "/",
  "quarantine" : true,
  "action" : "error",
  "newVersion" : {
    "name" : "test.txt",
    "checksum" : "0ca6033e2a9c2bea1586a2984bf111e6"
  }
}
  • Example 4: Synchronize with a client where the version is no longer supported.
    The server sends an error action with code DRV-0028 and an appropriate error message. The stop flag is set to true to interrupt the synchronization cycle.
{
  "stop" : true,
  "error" : {
    "category" : 13,
    "error_params" : [],
    "error" : "The client application you're using is outdated and no longer supported - please upgrade to a newer version.",
    "error_id" : "103394512-13",
    "categories" : "WARNING",
    "code" : "DRV-0028",
    "error_desc" : "Client outdated - current: \"0.9.2\", required: \"0.9.10\""
  },
  "quarantine" : false,
  "action" : "error"
}
  • Example 5: Synchronize with a client where a new version of the client application is available.
    The server sends an error action with code DRV-0029 and an appropriate error message. The stop flag is set to false to indicate that the synchronization can continue.
{
  "stop" : false,
  "error" : {
    "category" : 13,
    "error_params" : [],
    "error" : "A newer version of your client application is available for download.",
    "error_id" : "103394512-29",
    "categories" : "WARNING",
    "code" : "DRV-0029",
    "error_desc" : "Client update available - current: \"0.9.10\", available: \"0.9.12\""
  },
  "quarantine" : false,
  "action" : "error"
}

Synchronize folders

This request performs the synchronization of all folders, resulting in different actions that should be executed on the client afterwards. This operation typically serves as an entry point for a synchronization cycle.

PUT /ajax/drive?action=syncfolders

Parameters:

  • session - A session ID previously obtained from the login module.
  • root - The ID of the referenced root folder on the server.
  • version - The current client version (matching the pattern ^[0-9]+(\\.[0-9]+)*$). If not set, the initial version 0 is assumed.
  • apiVersion - The API version that the client is using. If not set, the initial version 0 is assumed.
  • diagnostics (optional) - If set to true, an additional diagnostics trace is supplied in the response.
  • pushToken (optional) - The client's push registration token to associate it to generated events.

Request Body:
A JSON object containing two JSON arrays named clientVersions and originalVersions. The client versions array lists all current directories below the root directory as a flat list, encoded as Directory Versions. The original versions array contains all previously known directories, i.e. all previously synchronized and acknowledged directories, also encoded as Directory Versions. \\ Optionally, available since API version 2, the JSON object may also contain two arrays named fileExclusions and directoryExclusions to define client-side exclusion filters, with each element encoded as File patterns and Directory patterns accordingly. See #Client_side_filtering for details.

Response:
A JSON array containing all actions the client should execute for synchronization. Each array element is an action as described in Actions.
If the diagnostics flag was set (either to true or false), this array is wrapped into an additional JSON object in the actions parameter, and the diagnostics trace is provided at diagnostics.

Example:

==> PUT http://192.168.32.191/ajax/drive?action=syncfolders&root=56&session=5d0c1e8eb0964a3095438b450ff6810f
  > Content:
    {
      "clientVersions" : [{
          "path" : "/",
          "checksum" : "7b744b13df4b41006495e1a15327368a"
        }, {
          "path" : "/test1",
          "checksum" : "3ecc97334d7f6bf2b795988092b8137e"
        }, {
          "path" : "/test2",
          "checksum" : "56534fc2ddcb3b7310d3ef889bc5ae18"
        }, {
          "path" : "/test2/test3",
          "checksum" : "c193fae995d9f9431986dcdc3621cd98"
        }
      ],
      "originalVersions" : [{
          "path" : "/",
          "checksum" : "7b744b13df4b41006495e1a15327368a"
        }, {
          "path" : "/test2/test3",
          "checksum" : "c193fae995d9f9431986dcdc3621cd98"
        }, {
          "path" : "/test2",
          "checksum" : "35d1b51fdefbee5bf81d7ae8167719b8"
        }, {
          "path" : "/test1",
          "checksum" : "3ecc97334d7f6bf2b795988092b8137e"
        }
      ]
    }
  
<== HTTP 200 OK (8.0004 ms elapsed, 102 bytes received)
<   Content: 
    {
      "data" : [{
          "action" : "sync",
          "version" : {
            "path" : "/test2",
            "checksum" : "56534fc2ddcb3b7310d3ef889bc5ae18"
          }
        }
      ]
    }

Example 2:

==> PUT http://192.168.32.191/ajax/drive?action=syncfolders&root=56&session=5d0c1e8eb0964a3095438b450ff6810f
  > Content:
    {
      "clientVersions" : [{
          "path" : "/",
          "checksum" : "7b744b13df4b41006495e1a15327368a"
        }, {
          "path" : "/test1",
          "checksum" : "3ecc97334d7f6bf2b795988092b8137e"
        }, {
          "path" : "/test2",
          "checksum" : "56534fc2ddcb3b7310d3ef889bc5ae18"
        }, {
          "path" : "/test2/test3",
          "checksum" : "c193fae995d9f9431986dcdc3621cd98"
        }
      ],
      "originalVersions" : [{
          "path" : "/",
          "checksum" : "7b744b13df4b41006495e1a15327368a"
        }, {
          "path" : "/test2/test3",
          "checksum" : "c193fae995d9f9431986dcdc3621cd98"
        }, {
          "path" : "/test2",
          "checksum" : "35d1b51fdefbee5bf81d7ae8167719b8"
        }, {
          "path" : "/test1",
          "checksum" : "3ecc97334d7f6bf2b795988092b8137e"
        }
      ]
      "fileExclusions" : [{
          "path" : "/",
          "name" : "excluded.txt",
          "type" : "exact"
        }
      ], "directoryExclusions" : [{
          "path" : "/temp",
          "type" : "exact"
        }, {
          "path" : "/temp/*",
          "type" : "glob"
        }
      ]
    }
  
<== HTTP 200 OK (8.0004 ms elapsed, 102 bytes received)
<   Content: 
    {
      "data" : [{
          "action" : "sync",
          "version" : {
            "path" : "/test2",
            "checksum" : "56534fc2ddcb3b7310d3ef889bc5ae18"
          }
        }
      ]
    }


Synchronize files in a folder

This request performs the synchronization of a single folder, resulting in different actions that should be executed on the client afterwards. This action is typically executed as result of a syncfolders action.

PUT /ajax/drive?action=syncfiles

Parameters:

  • session - A session ID previously obtained from the login module.
  • root - The ID of the referenced root folder on the server.
  • path - The path to the synchronized folder, relative to the root folder.
  • device (optional) - A friendly name identifying the client device from a user's point of view, e.g. "My Tablet PC".
  • apiVersion - The API version that the client is using. If not set, the initial version 0 is assumed.
  • diagnostics (optional) - If set to true, an additional diagnostics trace is supplied in the response.
  • columns (optional) - A comma-separated list of columns representing additional metadata that is relevant for the client. Each column is specified by a numeric column identifier. Column identifiers for file metadata are defined in #File Metadata. If available, the requested metadata of files is included in the corresponsing DOWNLOAD and ACKNOWLEDGE actions (deprecated, available until API version 2).
  • pushToken (optional) - The client's push registration token to associate it to generated events.

Request Body:
A JSON object containing two JSON arrays named clientVersions and originalVersions. The client versions array lists all current files in the client directory, encoded as File Versions. The original versions array contains all previously known files, i.e. all previously synchronized and acknowledged files, also encoded as File Versions. \\ Optionally, available since API version 2, the JSON object may also contain an array named fileExclusions to define client-side exclusion filters, with each element encoded as File patterns. See #Client side filtering for details.

Response:
A JSON array containing all actions the client should execute for synchronization. Each array element is an action as described in Actions.
If the diagnostics flag was set (either to true or false), this array is wrapped into an additional JSON object in the actions parameter, and the diagnostics trace is provided at diagnostics.

Example:

==> PUT http://192.168.32.191/ajax/drive?action=syncfiles&root=56&path=/test2&device=Laptop&session=5d0c1e8eb0964a3095438b450ff6810f
  > Content: 
    {
      "clientVersions" : [{
          "name" : "Jellyfish.jpg",
          "checksum" : "5a44c7ba5bbe4ec867233d67e4806848"
        }, {
          "name" : "Penguins.jpg",
          "checksum" : "9d377b10ce778c4938b3c7e2c63a229a"
        }
      ],
      "originalVersions" : [{
          "name" : "Jellyfish.jpg",
          "checksum" : "5a44c7ba5bbe4ec867233d67e4806848"
        }
      ]
    }

<== HTTP 200 OK (6.0004 ms elapsed, 140 bytes received)
<   Content:
    {
      "data" : [{
          "path" : "/test2",
          "action" : "upload",
          "newVersion" : {
            "name" : "Penguins.jpg",
            "checksum" : "9d377b10ce778c4938b3c7e2c63a229a"
          },
          "offset" : 0
        }
      ]
    }

Example 2:

==> PUT http://192.168.32.191/ajax/drive?action=syncfiles&root=56&path=/test2&device=Laptop&session=5d0c1e8eb0964a3095438b450ff6810f
  > Content: 
    {
      "clientVersions" : [{
          "name" : "Jellyfish.jpg",
          "checksum" : "5a44c7ba5bbe4ec867233d67e4806848"
        }, {
          "name" : "Penguins.jpg",
          "checksum" : "9d377b10ce778c4938b3c7e2c63a229a"
        }
      ],
      "originalVersions" : [{
          "name" : "Jellyfish.jpg",
          "checksum" : "5a44c7ba5bbe4ec867233d67e4806848"
        }
      ]
      "fileExclusions" : [{
          "path" : "*",
          "name" : "*.tmp",
          "type" : "glob"
        }
      ]
    }

<== HTTP 200 OK (6.0004 ms elapsed, 140 bytes received)
<   Content:
    {
      "data" : [{
          "path" : "/test2",
          "action" : "upload",
          "newVersion" : {
            "name" : "Penguins.jpg",
            "checksum" : "9d377b10ce778c4938b3c7e2c63a229a"
          },
          "offset" : 0
        }
      ]
    }

Download a file

Downloads a file from the server.

GET /ajax/drive?action=download

or

PUT /ajax/drive?action=download

Parameters:

  • session - A session ID previously obtained from the login module.
  • root - The ID of the referenced root folder on the server.
  • path - The path to the synchronized folder, relative to the root folder.
  • name - The name of the file version to download.
  • checksum - The checksum of the file version to download.
  • apiVersion - The API version that the client is using. If not set, the initial version 0 is assumed.
  • offset (optional) - The start offset in bytes for the download. If not defined, an offset of 0 is assumed.
  • length (optional) - The number of bytes to include in the download stream. If not defined, the file is read until the end.

Request Body:
Optionally, available since API version 3, if client-side file- and/or directory exclusion filters are active, a PUT request can be used. The request body then holds a JSON object containing two arrays named fileExclusions and directoryExclusions to define client-side exclusion filters, with each element encoded as File patterns and Directory patterns accordingly. See Client side filtering for details.

Response:
The binary content of the requested file version. Note that in case of errors, an exception is not encoded in the default JSON error format here. Instead, an appropriate HTTP error with a status code != 200 is returned. For example, in case of the requested file being deleted or modified in the meantime, a response with HTTP status code 404 (not found) is sent.

Example:

==> GET http://192.168.32.191/ajax/drive?action=download&root=56&path=/test2&name=Jellyfish.jpg&checksum=5a44c7ba5bbe4ec867233d67e4806848&offset=0&length=-1&session=5d0c1e8eb0964a3095438b450ff6810f

<== HTTP 200 OK (20.0011 ms elapsed, 775702 bytes received)

Upload a file

Uploads a file to the server.

PUT /ajax/drive?action=upload

Parameters:

  • session - A session ID previously obtained from the login module.
  • root - The ID of the referenced root folder on the server.
  • path - The path to the synchronized folder, relative to the root folder.
  • newName - The target name of the file version to upload.
  • newChecksum - The target checksum of the file version to upload.
  • name (optional) - The previous name of the file version being uploaded. Only set when uploading an updated version of an existing file to the server.
  • checksum - The previous checksum of the file version to upload. Only set when uploading an updated version of an existing file to the server.
  • apiVersion - The API version that the client is using. If not set, the initial version 0 is assumed.
  • contentType (optional) - The content type of the file. If not defined, application/octet-stream is assumed.
  • offset (optional) - The start offset in bytes for the upload when resuming a previous partial upload. If not defined, an offset of 0 is assumed.
  • totalLength (optional) - The total expected length of the file (required to support resume of uploads). If not defined, the upload is assumed completed after the operation.
  • created (optional) - The creation time of the file as timestamp.
  • modified (optional) - The last modification time of the file as timestamp. Defaults to the current server time if no value or a value larger than the current time is supplied.
  • binary - Expected to be set to true to indicate the binary content.
  • device (optional) - A friendly name identifying the client device from a user's point of view, e.g. "My Tablet PC".
  • diagnostics (optional) - If set to true, an additional diagnostics trace is supplied in the response.
  • pushToken (optional) - The client's push registration token to associate it to generated events.

Request body:
The binary content of the uploaded file version.

Response:
A JSON array containing all actions the client should execute for synchronization. Each array element is an action as described in Actions.
If the diagnostics flag was set (either to true or false), this array is wrapped into an additional JSON object in the actions parameter, and the diagnostics trace is provided at diagnostics.

Example:

==> PUT http://192.168.32.191/ajax/drive?action=upload&root=56&path=/test2&newName=Penguins.jpg&newChecksum=9d377b10ce778c4938b3c7e2c63a229a&contentType=image/jpeg&offset=0&totalLength=777835&binary=true&device=Laptop&created=1375343426999&modified=1375343427001&session=5d0c1e8eb0964a3095438b450ff6810f
  > Content: 
    [application/octet-stream;, 777835 bytes]

<== HTTP 200 OK (108.0062 ms elapsed, 118 bytes received)
<   Content: 
    {
      "data" : [{
          "action" : "acknowledge",
          "newVersion" : {
            "name" : "Penguins.jpg",
            "checksum" : "9d377b10ce778c4938b3c7e2c63a229a"
          }
        }
      ]
    }

Listen for changes (long polling)

Listens for server-side changes. The request blocks until new actions for the client are available, or the specified waiting time elapses. May return immediately if previously received but not yet processed actions are available for this client.

GET /ajax/drive?action=listen

Parameters:

  • session - A session ID previously obtained from the login module.
  • root - The ID of the referenced root folder on the server.
  • timeout (optional) - The maximum timeout in milliseconds to wait.
  • pushToken (optional) - The client's push registration token to associate it to generated events.

Response:
A JSON array containing all actions the client should execute for synchronization. Each array element is an action as described in Actions. If there no changes were detected, an empty array is returned. Typically, the client will continue with the next listen request after the response was processed.

Example:

==> GET http://192.168.32.191/ajax/drive?action=listen&root=65841&session=51378e29f82042b4afe4af1c034c6d68

<== HTTP 200 OK (63409.6268 ms elapsed, 28 bytes received)
<   Content: 
    {
      "data" : [{
          "action" : "sync",
        }
      ]
    }

Get quota

Gets the quota limits and current usage for the storage the supplied root folder belongs to. Depending on the filestore configuration, this may include both restrictions on the number of allowed files and the total size of all contained files in bytes. If there's no limit, -1 is returned.

GET /ajax/drive?action=quota

Parameters:

  • session - A session ID previously obtained from the login module.
  • root - The ID of the referenced root folder on the server.

Response:
A JSON object containing the quota restrictions inside a JSON array with the property name quota. The JSON array contains zero, one or two quota objects as described below, depending on the filestore configuration. If one or more quota types are missing in the array, the client can expect that there are no limitations for that type. Besides the array, the JSON object also contains a hyperlink behind the manageLink parameter, pointing to an URL where the user could manage his quota restrictions.

Quota
Name Type Value
limit Number The allowed limit (either number of files or sum of filesizes in bytes).
use Number The current usage (again either number of files or sum of filesizes in bytes).
type String The kind of quota restriction, currently either storage (size of contained files in bytes) or file (number of files).

Example:

==> GET http://192.168.32.191/ajax/drive?action=quota&root=56&session=35cb8c2d1423480692f0d5053d14ba52
  
<== HTTP 200 OK (9.6854 ms elapsed, 113 bytes received)
<   Content: 
    {
      "data" : {
        "quota" : [{
            "limit" : 107374182400,
            "use" : 1109974882,
            "type" : "storage"
          }, {
            "limit" : 800000000000,
            "use" : 1577,
            "type" : "file"
          }
        ],
        "manageLink" : "https://www.example.com/manageQuota"
      }
    }

Get Settings

Gets various settings applicable for the drive clients.

GET /ajax/drive?action=settings

Parameters:

  • session - A session ID previously obtained from the login module.
  • root - The ID of the referenced root folder on the server.
  • language (optional) - The locale to use for language-sensitive settings (in the format <2-letter-language>_<2-letter-region>, e.g. de_CH or en_GB). Defaults to the user's configured locale on the server.

Response:
A JSON object holding the settings as described below. This also includes a JSON array with the property name quota that contains zero, one or two quota objects as described below, depending on the filestore configuration. If one or more quota types are missing in the array, the client can expect that there are no limitations for that type.

Quota
Name Type Value
limit Number The allowed limit (either number of files or sum of filesizes in bytes).
use Number The current usage (again either number of files or sum of filesizes in bytes).
type String The kind of quota restriction, currently either storage (size of contained files in bytes) or file (number of files).
Settings
Name Type Value
helpLink String A hyperlink to the online help.
quotaManageLink String A hyperlink to an URL where the user could manage his quota restrictions.
quota Array A JSON array containing the quota restrictions as described above.
serverVersion String The server version string.
supportedApiVersion String The API version supported by the server.
minApiVersion String The API version required to synchronize with the server.

Example:

==> GET http://192.168.32.191/ajax/drive?action=settings&root=56&session=35cb8c2d1423480692f0d5053d14ba52
  
<== HTTP 200 OK (11.3530 ms elapsed, 318 bytes received)
<   Content: 
    {
      "data" : {
        "quota" : [{
            "limit" : 107374182400,
            "use" : 8828427,
            "type" : "storage"
          }, {
            "limit" : 800000000000,
            "use" : 1559,
            "type" : "file"
          }
        ],
        "helpLink" : "http://192.168.32.191/ajax/help/en_US/index.html",
        "quotaManageLink" : "https://192.168.32.191/manageQuota",
        "serverVersion" : "7.4.2-Rev1",
        "supportedApiVersion" : "2",
        "minApiVersion" : "1"
      }
    }

Subscribe to Push-Events

Registers a client device to receive push notifications from the server. The subscription is performed based on the configured root folder ID of the client application that identifies itself with it's device token. Supported services currently include the Apple Push Notification Service (APN) and Google Cloud Messaging (GCM). Trying to perform an identical subscription (same root, service and token) from the same user account again is treated as a no-op.

GET /ajax/drive?action=subscribe

Parameters:

  • session - A session ID previously obtained from the login module.
  • root - The ID of the referenced root folder on the server.
  • service - The name of the underlying push service to use, currently one of gcm, apn or apn.macos.
  • token - The device's registration token as assigned by the service.

Response:
An empty JSON result.

Example:

==> GET http://192.168.32.191/ajax/drive?action=subscribe&root=65841&session=51378e29f82042b4afe4af1c034c6d68&service=apn&token=28919862989a1b5ba59c11d5f7cb7ba2b9678be9dd18b033184d04f682013677

<== HTTP 200 OK (13.6268 ms elapsed, 11 bytes received)
<   Content: 
    {
      "data" : {
      }
    }


Unsubscribe from Push-Events

Unregisters a previously registered client device to stop receiving push notifications from the server. The same parameters that were used to perform the subscription need to be passed again, which includes the root folder ID, the device token and the service name.

GET /ajax/drive?action=unsubscribe

Parameters:

  • session - A session ID previously obtained from the login module.
  • root - The ID of the referenced root folder on the server.
  • service - The name of the underlying push service to use, currently one of gcm, apn or apn.macos.
  • token - The device's registration token as assigned by the service.

Response:
An empty JSON result.

Example:

==> GET http://192.168.32.191/ajax/drive?action=unsubscribe&root=65841&session=51378e29f82042b4afe4af1c034c6d68&service=apn&token=28919862989a1b5ba59c11d5f7cb7ba2b9678be9dd18b033184d04f682013677

<== HTTP 200 OK (26.0015 ms elapsed, 11 bytes received)
<   Content: 
    {
      "data" : {
      }
    }

Update the subscription token

Updates a device's registration token in case a new one was assigned by the service.

GET /ajax/drive?action=updateToken

Parameters:

  • session - A session ID previously obtained from the login module.
  • service - The name of the underlying push service to use, currently one of gcm, apn or apn.macos.
  • token - The previous registration token as assigned by the service.
  • newToken - The new registration token as assigned by the service.

Response:
An empty JSON result.

Example:

==> GET http://192.168.32.191/ajax/drive?action=updateToken&service=apn&session=51378e29f82042b4afe4af1c034c6d68&token=28919862989a1b5ba59c11d5f7cb7ba2b9678be9dd18b033184d04f682013677&newToken=38919862989a1b5ba59c11d5f7cb7ba2b9678be9dd18b033184d04f682013677

<== HTTP 200 OK (15.6653 ms elapsed, 11 bytes received)
<   Content: 
    {
      "data" : {
      }
    }

Get file metadata

Deprecated, available until API version 2.
Additional metadata of synchronized files is made available via the fileMetadata request.

PUT /ajax/drive?action=fileMetata

Parameters:

  • session - A session ID previously obtained from the login module.
  • root - The ID of the referenced root folder on the server.
  • path - The path to the synchronized folder, relative to the root folder.
  • columns - A comma-separated list of columns to return. Each column is specified by a numeric column identifier. Column identifiers for file metadata are defined in #File Metadata.

Request Body:
A JSON array containing the file versions to get the metadata for. Each object in the array should be sent as File Versions, and needs to be present in the referenced path.

Response:
A JSON array containing the file metadata in the order of the requested file versions. Each array element describes one file metadata and is itself an array. The elements of each array contain the information specified by the corresponding identifiers in the columns parameter.

File Metadata (deprecated)
ID Name Type Value
name String The name of the file version.
4 created Timestamp The file's last modification time (always UTC, not translated into user time).
5 modified Timestamp The file's last modification time (always UTC, not translated into user time).
702 name String The name of the file, including it's extension, e.g. test.doc.
703 contentType String The file's content type, e.g. "image/png".
708 checksum String The MD5 hash of the file, expressed as a lowercase hexadecimal number string, 32 characters long, e.g. f8cacac95379527cd4fa15f0cb782a09.
750 previewLink String A direct link to a medium-sized preview image of the file if available.
751 directLinkFragments The fragments part of the direct link that can be used in combination with the [|token login] method to jump directly to the detail view of the file in the web interface, bypassing the need to login manually.
752 directLink String A direct link to the detail view of the file in the web interface.
753 thumbnailLink String A direct link to a small thumbnail image of the file if available.

Example:

==> PUT http://192.168.32.191/ajax/drive?action=fileMetadata&root=97974&path=%2f&columns=702%2c708%2c752%2c750%2c753&session=43aca91a80de42559ff0c2493dd973d0
  > Content:
    [
      {
        "name" : "image.jpg",
        "checksum" : "2b04df3ecc1d94afddff082d139c6f15"
      }, {
        "name" : "song.mp3",
        "checksum" : "5a9a91184e611dae3fed162b8787ce5f"
      }, {
        "name" : "test1.txt",
        "checksum" : "7e36f409a042f06ecb88606a97a88c8f"
      }, {
        "name" : "test3.txt",
        "checksum" : "703bc9aabff33faf07cf121dcda12ec8"
      }
    ]  

<== HTTP 200 OK (6.0004 ms elapsed, 140 bytes received)
<   Content:
    [
      ["image.jpg", "2b04df3ecc1d94afddff082d139c6f15", "https://192.168.32.191/ox6/index.html#m=infostore&f=97974&i=179629", "https://192.168.32.191/ajax/files?action=document&folder=97974&id=179629&version=1&delivery=download&scaleType=contain&width=128&height=90", "m=infostore&f=97974&i=179629"], 
      ["song.mp3", "5a9a91184e611dae3fed162b8787ce5f", "https://192.168.32.191/ox6/index.html#m=infostore&f=97974&i=179630", "https://192.168.32.191/ajax/image/file/mp3Cover?folder=97974&id=179630&version=1&delivery=download&scaleType=contain&width=128&height=90", "m=infostore&f=97974&i=179630"], 
      ["test1.txt", "7e36f409a042f06ecb88606a97a88c8f", "https://192.168.32.191/ox6/index.html#m=infostore&f=97974&i=179626", null, "m=infostore&f=97974&i=179626"], 
      ["test3.txt", "703bc9aabff33faf07cf121dcda12ec8", "https://192.168.32.191/ox6/index.html#m=infostore&f=97974&i=179624", null, "m=infostore&f=97974&i=179624"]
    ]

Get a direct link for a folder/a file into appsuite

Generate a direct link into appsuite UI for a synchronized file/a synchronized folder and a token for token-based login.

POST /ajax/drive?action=jump

Parameters:

  • session - A session ID previously obtained from the login module.
  • root - The ID of the referenced root folder on the server.
  • path - The path to the synchronized folder, relative to the root folder.
  • name - The name of the file in the synchronized folder given in path-parameter. Optional
  • method - Methods
  • authId - Identifier for tracing every single login request passed between different systems in a cluster. The value should be some token that is unique for every login request. This parameter must be given as URL parameter and not inside the body of the POST request.
  • clientToken - Client side identifier for accessing the session later. The value should be some token that is unique for every login request.

Methods:

  • edit: Open the file in appsuite editor or in text/spreadsheet (if available).
  • permissions: Open the file's/folder's change-permission dialog.
  • version_history: Open the file's version history summary.
  • preview: Open the file's/folder's preview.

Response:
A JSON array containing the direct link to the file/folder including a server token for token based login.

Example:

==> POST http://localhost/ajax/drive?action=jump&session=48a289898ad949faaa46c04e7fb422f5&root=9547&path=/path/to/file&name=file_to_edit.txt&method=edit&authId=41763584-8460-11e4-b116-123b93f75dba
  > Content: clientToken=47d74b1c-81df-11e4-b116-123b93f75cba

<== HTTP 200 OK
<   Content: 
    {
      "data": {
        "redirectUrl": "http://localhost/appsuite#app=io.ox/editor&folder=273264&id=273264/307438&serverToken=7b90972628e34e89bb9a3946d1372c68"
      }
    }

Use direct link and token with token-based login

Login to appsuite UI with token-based login via the link created with Get a direct link for a folder/a file into appsuite.

GET [direct link]

Parameters:

Example:

==> GET http://localhost/appsuite#app=io.ox/editor&folder=273264&id=273264/307438&serverToken=7b90972628e34e89bb9a3946d1372c68&clientToken=47d74b1c-81df-11e4-b116-123b93f75cba

<== HTTP 200 OK

Get synchronizable Folders

Available since API version 4.

Allows getting a list of folders that are available on the server for synchronization. This request should be used to build up a folder tree and let the user select the root synchronization folder(s).

GET /ajax/drive?action=subfolders

Parameters:

  • session - A session ID previously obtained from the login module.
  • parent - The ID of the parent folder to get the subfolders for as read from a previously fetched directory metadata object. Optional; if not set, the root available root folders are returned.

Response:
A JSON array holding metadata information for all subfolders as defined in Directory Metadata, with the files array being left out.

Example:

==> GET http://192.168.32.191/ajax/drive?action=subfolders&session=35cb8c2d1423480692f0d5053d14ba52
  
<== HTTP 200 OK (241.0252 ms elapsed, 966 bytes received)
<   Content: 
    {
      "data": [{
        "id": "com.openexchange.file.storage.googledrive://1/",
        "name": "Google Drive",
        "path": "/Google Drive",
        "has_subfolders": true,
        "own_rights": 403710016,
        "permissions": [{
          "bits": 403710016,
          "group": false,
          "entity": 182,
          "display_name": "Mander, Jens",
          "email_address": "jens.mander@example.com",
          "guest": false
        }],
        "jump": ["permissions"]
      },
      {
        "id": "10",
        "name": "Freigegebene Dateien",
        "path": "/Freigegebene Dateien",
        "created": 1224493261628,
        "modified": 1417164170136,
        "has_subfolders": true,
        "own_rights": 1,
        "permissions": [{
          "bits": 1,
          "group": true,
          "entity": 0,
          "display_name": "All users",
          "guest": false
        },
        {
          "bits": 1,
          "group": true,
          "entity": 2147483647,
          "display_name": "Guests",
          "guest": false
        }],
        "jump": ["permissions"],
        "shared": true
      },
      {
        "id": "15",
        "name": "Öffentliche Dateien",
        "path": "/Öffentliche Dateien",
        "created": 1224493261628,
        "modified": 1418383637250,
        "has_subfolders": true,
        "own_rights": 403709956,
        "permissions": [{
          "bits": 403709956,
          "group": true,
          "entity": 0,
          "display_name": "All users",
          "guest": false
        },
        {
          "bits": 1,
          "group": true,
          "entity": 2147483647,
          "display_name": "Guests",
          "guest": false
        }],
        "jump": ["permissions"],
        "shared": true
      },
      {
        "id": "com.openexchange.file.storage.dropbox://1/",
        "name": "Dropbox",
        "path": "/Dropbox",
        "has_subfolders": true,
        "own_rights": 403710016,
        "permissions": [{
          "bits": 403710016,
          "group": false,
          "entity": 182,
          "display_name": "Mander, Jens",
          "email_address": "jens.mander@example.com",
          "guest": false
        }],
        "jump": ["permissions"]
      },
      {
        "id": "9542",
        "name": "Meine Dateien",
        "path": "/Meine Dateien",
        "created": 1320230546147,
        "modified": 1426764458823,
        "default_folder": true,
        "has_subfolders": true,
        "own_rights": 403710016,
        "permissions": [{
          "bits": 403710016,
          "group": false,
          "entity": 182,
          "display_name": "Mander, Jens",
          "email_address": "jens.mander@example.com",
          "guest": false
        }],
        "jump": ["permissions"]
      }]
    }

File- and Directory Name Restrictions

Regarding the case sensitivity of file and directory names, OX Drive works in a case-insensitive, but case-preserving way. That means that there cannot be two files with an equal name ignoring case in the same directory, but it's still possible to synchronize the names in a case-sensitive manner, as well as it's possible to change only the case of file- and directory names.

The same applies to equally named files and directories on the same level in the folder hierarchy, i.e. it's not possible to create a new file in a directory where an equally (ignoring case) named subdirectory already exists and vice versa.

There is a similar restriction regarding file and directory names in the same directory having different unicode normalization forms, yet the same textual representation. OX Drive requires uniqueness regarding this textual representaion of potentially different encoded unicode strings. So, in case the client tries to synchronize two textually equal files or directories, he is instructed to put one of them into quarantine. Internally the server performs an equals-check of the "NFC" normalization forms of the strings, i.e. an unicode string is normalized using full canonical decomposition, followed by the replacement of sequences with their primary composites, if possible. Details regarding unicode normalization can be found at http://www.unicode.org/reports/tr15/tr15-23.html .

Invalid and ignored Filenames

There are some filenames that are invalid or ignored and therefore not synchronized. This means that files with these names should not be taken into account when sending the directory contents to the server, or when calculating the directory checksum (see below). The following list describes when a filename is considered invalid:

  • If it contains one or of the following reserved characters:
    • < (less than),
    • > (greater than)
    • : (colon)
    • " (double quote)
    • / (forward slash)
    • \ (backslash)
    • | (vertical bar or pipe)
    • ? (question mark)
    • * (asterisk)
    • Characters whose integer representations are in the range from 0 through 31
  • The last character is a . (dot) or ' ' (space)
  • It's case-invariant name without an optional extension matches one of the reserved names CON, PRN, AUX, NUL, COM1, COM2, COM3, COM4, COM5, COM6, COM7, COM8, COM9, LPT1, LPT2, LPT3, LPT4, LPT5, LPT6, LPT7, LPT8, or LPT9
  • It consists solely of whitespace characters

The following list gives an overview about the ignored filenames:

  • desktop.ini
  • Thumbs.db
  • .DS_Store
  • icon\r
  • Any filename ending with .drivepart
  • Any filename starting with .msngr_hstr_data_ and ending with .log

Nevertheless, if the client still insists to send a file version with an invalid or ignored filename, the file creation on the server is refused with a corresponding error action (see below).

Invalid and ignored Directory Names

There are also similar restrictions regarding invalid directory names. Any try to include them in the list of directory versions will be responded with a corresponding error action for the directory version. The following list describes when a path is considered invalid:

  • If it contains one or of the following reserved characters:
    • < (less than),
    • > (greater than)
    • : (colon)
    • " (double quote)
    • \ (backslash)
    • | (vertical bar or pipe)
    • ? (question mark)
    • * (asterisk)
    • Characters whose integer representations are in the range from 0 through 31
  • The last character of any subpath (i.e. the last part of the whole path or the part preceding the spearator character /) is a . (dot) or ' ' (space)
  • It consists solely of whitespace characters
  • It not equals the root path /, but ends with a / (forward slash) character
  • It contains two or more consecutive / (forward slash) characters

The following list gives an overview about the ignored directory names:

  • /.drive
  • Any directory whose path ends with /.msngr_hstr_data

Length Restrictions

The maximum allowed length for path segments, i.e. the parts between forawrd slashes (/) in directory and filenames, is restricted to 255 characters. Synchronizing a file or directory version that contains path segments longer than this limit leads to those versions being put into quarantine.

Client side filtering

Client-side filtering is available since API version 2.

OX Drive clients may define a user- and/or application-defined list of file- and directory name exclusions. Those exclusion filters are then taken into account during synchronization, i.e. files and directories matching a defined exclusion pattern are ignored when comparing the list of server-, client- and original versions. Also, the file exclusion lists are considered for the calculation of aggergated directory checksums.

The exclusion filters may be set, changed or unset at any time during synchronization, there are no additional requests needed to set them up. Instead, the list of excluded files and directories is simply sent along with each syncFolders, syncFiles and download request. The following tables show the JSON representation of file- and directory patterns that are used to build up the exlcusion lists:

Directory pattern

A directory pattern is defined by a pattern string and further attributes.

Directory Pattern
Name Type Value
type String The pattern type, currently one of exact or glob.
path String The path pattern, in a format depending on the pattern type.
caseSensitive Optional flag to enable case-sensitive matching, defaults to false

File pattern

A file pattern is defined by pattern strings for the filename and path, as well as further attributes.

File Pattern
Name Type Value
type String The pattern type, currently one of exact or glob.
path String The path pattern, in a format depending on the pattern type.
name String The filename pattern, in a format depending on the pattern type.
caseSensitive Optional flag to enable case-sensitive matching, defaults to false

Pattern types

A pattern currently may be defined in two formats: exact or glob.

  • exact
    An exact pattern, matching the file- or directory version literally. For example, to exclude the file Backup.pst in the subfolder Mail below the root synchronization folder, an exact file pattern would look like: {"path":"/Mail","name":"Backup.pst","type":"exact"}, or, an exact directory pattern for the directory /Archive would be represented as {"path":"/Archive","type":"exact"}.
  • glob
    A simple pattern allowing to use the common wildcards * and ? to match file- and directory versions. For example, to exclude all files ending with .tmp across all directories, the glob file pattern could be defined as {"path":"*","name":"*.tmp","type":"glob"}, or, to exclude the directory /Project/.git and all its subdirectories recursively, this would be expressed using a combination of the following two directory patterns: [{"path":"/Project/.git","type":"exact"},{"path":"/Project/.git*","type":"glob"}].

Further considerations

  • It's possible to exclude a (parent) directory with an appropriate pattern, while still subfolders below that directory being synchronized. This usually results in the excluded directory being created ob both client- and server side, but no file contents within the excluded directory being exchanged. If subfolders should be excluded, too, a wildcard should be used in the pattern to match any subdirectories.
  • If the client tries to synchronize a file- or directory version that is ignored, i.e. a version that would match any of the provided exclusion filters, the server behaves similarly to the handling of invalid and ignored file- and directory names (see above), i.e. the client would be instructed to put those versions into quarantine.
  • For the calculation of directory checksums, it's important that the server and client perform exactly the same matching for ignored filenames: A * character matches zero or more characters, a ? character matches exactly one character. All other characters are matched literally. Advanced glob flavors like braces to define subpattern alternatives or square brackets for character sets are not used.
  • Client-side filtering is available with API version 2. The API version that is supported by the server is included in the response of the Settings request.
  • Whenever there are active exclusion filters, the syncFolders request should contain all of both directory and file exclusion filter lists. For the syncFiles request, it's sufficient to include the list of file exclusions.


Metadata Synchronization

The synchronization of metadata is available since API version 3.

Introduction

Previously, only the "raw" folders and files were synchronized between server and clients. While this is sufficient for basic synchronization, there are cases where the clients could benefit from additional data - "metadata" - that is already available on the server. For example, clients could display directories that have been shared or published to other people in a different way. Or, clients could consider folder permissions directly in case the user is performing a local change that would be rejected by the server in the next synchronization cycle anyway.

To supply the clients with those additional information without any influence on the existing synchronization protocol (!), .drive-meta files are introduced for each synchronized directory. Regarding synchronization, such files are treated like any other ordinary file. Especially, those files are taken into account when it comes to directory checksum calculation. Doing so, metadata updates result in a changed .drive-meta file, which in turn causes the parent directory checksum to change, hence synchronization is triggered.

However, some special handling applies for those files:

  • Clients are not allowed to change metadata, so modifications of metadata files or the deletion of them is rejected. Recovery is done via the protocol here, i.e. the client is instructed to re-download the file.
  • .drive-meta files are actually not stored physically on the file storage backend, but created on the fly based on the actual metadata of the directory.
  • Client applications may either store such files on the client file system, or evaluate and store the contained metadata information in a local database for later retrieval. If the file is not saved physically on the client (which is actually recommended), the client is responsible to consider the metadata file in a virtual way and include it's checksum for the directory checksum calculation - similar to the server's internal handling.

Metadata format

The metadata in .drive-meta files is serialized in JSON format to allow easy processing at the clients. The following shows an example of the contents:

{
  "path": "/",
  "created": 1418024049629,
  "modified": 1418024189166,
  "own_rights": 403710016,
  "permissions": [{
    "bits": 403710016,
    "group": false,
    "entity": 182,
    "display_name": "Mander, Jens",
    "email_address": "jens.mander@example.com",
    "guest": false
  }],
  "jump": ["permissions"],
  "files": [{
    "name": "Koala.jpg",
    "created": 1418024190565,
    "modified": 1418026995663,
    "created_by": {
      "group": false,
      "entity": 182,
      "display_name": "Mander, Jens",
      "email_address": "jens.mander@example.com",
      "guest": false
    },
    "modified_by": {
      "group": false,
      "entity": 182,
      "display_name": "Mander, Jens",
      "email_address": "jens.mander@example.com",
      "guest": false
    },
    "preview": "http://192.168.32.191/ajax/files?action=document&folder=268931&id=268931/297620&version=1&delivery=download&scaleType=contain&width=800&height=800&rotate=true",
    "thumbnail": "http://192.168.32.191/ajax/files?action=document&folder=268931&id=268931/297620&version=1&delivery=download&scaleType=contain&width=100&height=100&rotate=true",
    "object_permissions": [{
      "bits": 1,
      "group": false,
      "entity": 10,
      "display_name": "Jan Ot/to Finsel",
      "email_address": "jan.finsel@premium",
      "guest": false
    },
    {
      "bits": 1,
      "group": false,
      "entity": 8338,
      "email_address": "horst@example.com",
      "guest": true
    }],
    "shared": true,
    "number_of_versions": 1,
    "version": "1",
    "jump": ["preview",
    "permissions",
    "version_history"]
  },
  {
    "name": "test.txt",
    "created": 1418024198520,
    "modified": 1418027394897,
    "created_by": {
      "group": false,
      "entity": 182,
      "display_name": "Mander, Jens",
      "email_address": "jens.mander@example.com",
      "guest": false
    },
    "modified_by": {
      "group": false,
      "entity": 182,
      "display_name": "Mander, Jens",
      "email_address": "jens.mander@example.com",
      "guest": false
    },
    "preview": "http://192.168.32.191/ajax/files?action=document&format=preview_image&folder=268931&id=268931/297621&version=6&delivery=download&scaleType=contain&width=800&height=800",
    "thumbnail": "http://192.168.32.191/ajax/files?action=document&format=preview_image&folder=268931&id=268931/297621&version=6&delivery=download&scaleType=contain&width=100&height=100",
    "locked": true,
    "number_of_versions": 4,
    "version": "6",
    "version_comment": "Uploaded with OX Drive (TestDrive)",
    "versions": [{
      "name": "test.txt",
      "file_size": 23,
      "created": 1418024198520,
      "modified": 1418024202878,
      "created_by": {
        "group": false,
        "entity": 182,
        "display_name": "Mander, Jens",
        "email_address": "jens.mander@example.com",
        "guest": false
      },
      "modified_by": {
        "group": false,
        "entity": 182,
        "display_name": "Mander, Jens",
        "email_address": "jens.mander@example.com",
        "guest": false
      },
      "version": "1",
      "version_comment": "Uploaded with OX Drive (TestDrive)"
    },
    {
      "name": "test.txt",
      "file_size": 54,
      "created": 1418024234782,
      "modified": 1418024231522,
      "created_by": {
        "group": false,
        "entity": 182,
        "display_name": "Mander, Jens",
        "email_address": "jens.mander@example.com",
        "guest": false
      },
      "modified_by": {
        "group": false,
        "entity": 182,
        "display_name": "Mander, Jens",
        "email_address": "jens.mander@example.com",
        "guest": false
      },
      "version": "2",
      "version_comment": "Uploaded with OX Drive (TestDrive)"
    },
    {
      "name": "test.txt",
      "file_size": 120,
      "created": 1418027349026,
      "modified": 1418027355957,
      "created_by": {
        "group": false,
        "entity": 182,
        "display_name": "Mander, Jens",
        "email_address": "jens.mander@example.com",
        "guest": false
      },
      "modified_by": {
        "group": false,
        "entity": 182,
        "display_name": "Mander, Jens",
        "email_address": "jens.mander@example.com",
        "guest": false
      },
      "version": "5"
    },
    {
      "name": "test.txt",
      "file_size": 127,
      "created": 1418027370051,
      "modified": 1418027366945,
      "created_by": {
        "group": false,
        "entity": 182,
        "display_name": "Mander, Jens",
        "email_address": "jens.mander@example.com",
        "guest": false
      },
      "modified_by": {
        "group": false,
        "entity": 182,
        "display_name": "Mander, Jens",
        "email_address": "jens.mander@example.com",
        "guest": false
      },
      "version": "6",
      "version_comment": "Uploaded with OX Drive (TestDrive)"
    }],
    "jump": ["preview",
    "edit",
    "permissions",
    "version_history"]
  },
  {
    "name": "Kalimba.mp3",
    "created": 1418026529047,
    "modified": 1247549551659,
    "created_by": {
      "group": false,
      "entity": 182,
      "display_name": "Mander, Jens",
      "email_address": "jens.mander@example.com",
      "guest": false
    },
    "modified_by": {
      "group": false,
      "entity": 182,
      "display_name": "Mander, Jens",
      "email_address": "jens.mander@example.com",
      "guest": false
    },
    "preview": "http://192.168.32.191/ajax/image/file/mp3Cover?folder=268931&id=268931/297623&version=1&delivery=download&scaleType=contain&width=800&height=800",
    "thumbnail": "http://192.168.32.191/ajax/image/file/mp3Cover?folder=268931&id=268931/297623&version=1&delivery=download&scaleType=contain&width=100&height=100",
    "number_of_versions": 1,
    "version": "1",
    "version_comment": "Uploaded with OX Drive (TestDrive)",
    "jump": ["preview",
    "permissions",
    "version_history"]
  }]
}

The following objects describe the JSON structure of the metadata for a directory:

Directory Metadata
Name Type Value
id String The server-side unique identifier of the directory.
name String The display name of the directory.
path String The path of the directory the metadata belongs to.
created Timestamp The folder's last modification time (always UTC, not translated into user time).
modified Timestamp The folder's last modification time (always UTC, not translated into user time).
own_rights Number Folder permissions which apply to the current user, as described in [| permission flags].
permissions Array All folder permissions, each element is an object as described in Folder Permission.
default_folder Boolean true if the folder is a default folder, false or not set, otherwise.
has_subfolders Boolean true if the folder (potentially) has subfolders, false or not set, otherwise.
shared Boolean true if the folder is shared, false or not set, otherwise.
type Number The special folder type, or not set, if not available.
jump Array An array containing the names of possible jump methods to use for the folder.
files Array Metadata for the contained files, each element is an object as described in File Metadata.


File Metadata
Name Type Value
name String The name of the file the metadata belongs to.
created Timestamp The file's last modification time (always UTC, not translated into user time).
modified Timestamp The file's last modification time (always UTC, not translated into user time).
created_by Object Information about the file's creator as described in Entity Information.
modified_by Object Information about the file's last editor as described in Entity information.
preview String A URL to a preview image for the file.
thumbnail String A URL to a thumbnail image for the file.
object_permissions All file permissions, each element is an object as described in Object Permission.
shared Boolean true if the file is shared, false or not set, otherwise.
locked Boolean true if the file is locked, false or not set, otherwise.
jump Array An array containing the names of possible jump methods to use for the file.
number_of_versions The number of all versions of the file.
version String The current version identifier (usually, but not necessarily a numerical value) of the file.
version_comment An additional comment for the file version.
versions Array Metadata for all versions of the file, each element is an object as described in File Version.
Entity Information
Name Type Value
entity Number The unique identifier of the entity.
group Boolean true if the entity is a group, false or not set, if it is a single user or guest.
display_name String A display name for the entity if available.
email_address String An e-mail address for the entity if available.
group Boolean true if the entity is an external guest, false or not set, otherwise.
Folder Permission
Name Type Value
entity Number The unique identifier of the entity.
group Boolean true if the entity is a group, false or not set, if it is a single user or guest.
display_name String A display name for the entity if available.
email_address String An e-mail address for the entity if available.
group Boolean true if the entity is an external guest, false or not set, otherwise.
bits Number Permission level, as described in [| permission flags].
Object Permission
Name Type Value
entity Number The unique identifier of the entity.
group Boolean true if the entity is a group, false or not set, if it is a single user or guest.
display_name String A display name for the entity if available.
email_address String An e-mail address for the entity if available.
group Boolean true if the entity is an external guest, false or not set, otherwise.
bits Number Object permission level, as described in [| permission flags].
File Version
Name Type Value
name String The name of the file version.
file_size Number The file size of the version in bytes.
created Timestamp The file version's last modification time (always UTC, not translated into user time).
modified Timestamp The file version's last modification time (always UTC, not translated into user time).
created_by Object Information about the file version's creator as described in Entity Information.
modified_by Object Information about the file version's last editor as described in Entity information.
version String The version identifier (usually, but not necessarily a numerical value) of the file version.
version_comment String An additional comment for the file version.

Client-side implementation

In order to make use of the metadata, clients should roughly implement the following:

  • Include the apiVersion parameter in each request, and set it to at least 3 in order to include .drive-meta during synchronization
  • Evaluate .drive-meta files and store the information, as well as the file's checksums in a local database
  • Include this file in the calculation of the parent directory checksum, just like an ordinary file in that directory
  • Do something useful with the metadata information.

Additional notes

  • The metadata synchronization via .drive-meta files embedded into the synchronization protocol obsoletes the previously used methods to receive metadata information (#Get file metadata and columns parameter in #Synchronize files in a folder.
  • Depending on the underlying file storage backend, the included metadata may vary, so each information should be treatened as optional.

Possible use cases

  • For files where the locked property is true, display some kind of "lock" icon (-overlay) in the file list / explorer view
  • For files or folders where the shared property is true, display some kind of "cloud" icon (-overlay) in the file list / explorer view
  • For files or folders where the user is not allowed to perform an action with, don't offer such actions (e.g. if a file cannot be deleted or renamed by the user due to insufficient permissions, disable the corresponding options)
  • Use the URLs in preview and thumbnail to get a preview image for the files
  • Display the server creation / last modification timestamps of files and folders
  • Embed a version history for files with multiple versions
  • Show to which users a file or folder is currently shared
  • Offer appropriate "jump" actions to the groupware web interface for more advanced options (e.g. to directly edit an .xlsx file in the spreadsheet application of the web interface, or to manage a folder's permission