AppSuite:UI build system: Difference between revisions

From Open-Xchange
(Undo revision 22287 by Khgras (talk))
 
(21 intermediate revisions by 5 users not shown)
Line 1: Line 1:
{{Stability-experimental}}
{{Stability-deprecated}}


<div class="title">Build System</div>
<div class="title">Build System</div>
Line 17: Line 17:
== Installing ==
== Installing ==


The build system comes in two variants: as part of the OX App Suite source, and as a Software Development Kit (SDK). The SDK contains only the build system and can be installed as a package. Its only external dependency is Node.js, which should be installed automatically by your package manager. While the core of OX App Suite is supposed to be built using the included version of the build system, either the source or the SDK can be used to build external (i.e. independently installable) apps.
Installing the right environment for running the <code>appserver</code> and the UI build system is described in [[AppSuite:GettingStarted#Installing | the getting started article]].
 
The actual installation depends on the chosen variant:
 
=== SDK ===
 
First, if not already done, add the Open-Xchange repository to the list of Apt sources.
 
  # echo deb http://software.open-xchange.com/products/appsuite/\
  stable/appsuiteui/DebianSqueeze/ / >> /etc/apt/sources.list.d/ox.list
  # aptitude update
 
Then, the actual installation is a single command.
 
  # aptitude install open-xchange-appsuite-dev
 
Finally, the main executable of the build system should be put in the executable path for easier calling. This is typically done by either extending the <code>$PATH</code> environment variable
 
  $ export PATH=$PATH:/opt/open-xchange-appsuite-dev/bin
 
or symlinking the binary to a directory already in the path.
 
  # ln -s /opt/open-xchange-appsuite-dev/bin/build-appsuite /usr/local/bin
 
=== Source ===
 
Using the source variant can avoid the requirement for root permissions.
 
  $ git clone https://git.open-xchange.com/git/wd/frontend/web
 
Just like with the SDK variant, the executable should be put in the executable path either by extending the <code>$PATH</code> variable
 
  $ export PATH="$PATH:$(pwd)/web/ui/bin"
 
or by symlinking the executable.
 
  # ln -s "$(pwd)/web/ui/bin/build-appsuite" /usr/local/bin


== Running ==
== Running ==
Line 63: Line 27:
Since the build system is based on Jake, it also accepts all other Jake options. In addition, the environment variable <code>$nodeopts</code> can be used to pass command line parameters to Node.js. One of the most useful parameters is <code>--debug-brk</code>, which can be used to debug the build system.
Since the build system is based on Jake, it also accepts all other Jake options. In addition, the environment variable <code>$nodeopts</code> can be used to pass command line parameters to Node.js. One of the most useful parameters is <code>--debug-brk</code>, which can be used to debug the build system.


When developing external apps, the build system must be run from the top directory of the app's source. As a safety precaution, execution is aborted if the subdirectory <code>apps</code>, which usually contains JavaScript source code, is not found.
When developing external apps, the build system must be run from the top directory of the app's source. As a safety precaution, execution is aborted if the subdirectory <code>apps</code>, which usually contains JavaScript source code, is not found. This Article is written assuming, you're working in your workspace directory, containing the subfolder <code>apps</code>.


== Workflow ==
== Workflow ==
Line 69: Line 33:
The build system is used not only to create source archives for packaging. It can also directly install and update the built UI in a directory during development, help with the setup of the source of a new external app and more. While all of these tasks are described in the reference section, daily work involves just a few of them.
The build system is used not only to create source archives for packaging. It can also directly install and update the built UI in a directory during development, help with the setup of the source of a new external app and more. While all of these tasks are described in the reference section, daily work involves just a few of them.


=== Create Namespace ===
To have an easy example about how to [[AppSuite:GettingStarted#Writing | write]], [[AppSuite:GettingStarted#Building | build]] & [[AppSuite:GettingStarted#Running | host]] your javascript code and how to [[AppSuite:GettingStarted#Initialization | initialize]] & [[AppSuite:GettingStarted#Building_Packages | build]] your packages, please have a look at the [[AppSuite:GettingStarted | GettingStarted article]].
 
In order to have a proper space for your app/plugin create a workspace prospectivly containing all your code.
This folder should contain the subfolder <tt>apps</tt>.
The following article is written assuming, you're working in your workspace directory.
 
=== Create packaging metadata ===
 
The first task which should be executed when creating a new external app is <code>[[#init-packaging|init-packaging]]</code>. It creates packaging metadata in the current directory. It requires the parameter <code>package</code> to specify the package name in the generated files. All subsequent tasks will automatically extract the package name from the files generated by <code>[[#init-packaging|init-packaging]]</code>.
 
Before execution, the <code>apps</code> subdirectory must be created. It indicates that the current directory is actually a valid source directory.
 
  $ mkdir apps
  $ build-appsuite init-packaging package=example-app
  Build path: build
  Build version: 0.0.1-1.20130424.123835
 
  Version [0.0.1]:
 
  Maintainer (Name <e-mail>): Maintainer <maintainer@example.com>
 
  Copyright line [2013 Open-Xchange, Inc]:
 
  License name [CC-BY-NC-SA-3.0]:
 
  Short description: Hello World app to demonstrate usage of the build system
 
The task presents a number of interactive prompts to get the necessary information about the generated packages. The entered values should follow the Debian Maintainer's Guide. Some or even all prompts can be skipped by explicitly specifying the information as a build variable. The list of variable names is available in the reference of <code>[[#init-packaging|init-packaging]]</code>.
 
After the task is finished, the generated files can be customized manually to account for any additional packaging requirements.
 
=== Building apps ===
 
The main workhorse task is <code>[[#app|app]]</code>. It is used to process all source files to an optimized deployable app. During development, it is usually called automatically by an IDE or a directory monitoring tool. Since the builds are incremental, the task can be called after every source modification.
 
For quickest roundtrip times, the directory with the generated files of the app should be made available through the OX App Suite back end by overlaying an existing OX App Suite installation.
Therefor you should run [[AppSuite:Appserver | Appserver]] using an existing OX AppSuite server and  (with local parts of the AppSuite) automaticly get the recent stable source packages from the AppSuite servers. For using your namespace as a source for hosting your app, simply add the build-path of your app as an parameter while running appserver.
 
Deploying your app through running appserver would look like this for example:
  $ appserver -s https://appsuite-dev.open-xchange.com/appsuite/ /path/to/your/workspace/build
 
WARNING: Do not use build variables <code>[[#builddir|builddir]]</code> or <code>[[#manifestDir|manifestDir]]</code> when developing an external app to generate the files directly in their final destination! The <code>[[#clean|clean]]</code> task will delete these directories and all their contents! In general, don't point <code>[[#builddir|builddir]]</code> or any other <code>*Dir</code> variables at existing directories.
 
As an example, let's create a small app and build it. It requires only two files: <code>apps/com.example/main.js</code> for the source code of the app
 
<pre class="language-javascript">
define('com.example/main', function () {
    'use strict';
    var app = ox.ui.createApp({ name: 'com.example' });
    app.setLauncher(function () {
        var win = ox.ui.createWindow({
            name: 'com.example',
            title: 'Hello World App'
        });
        app.setWindow(win);
        win.nodes.main.append($('<h1>').text('Hello, World!'));
        win.show();
    });
    return { getApp: app.getInstance };
});
</pre>
 
and <code>apps/com.example/manifest.json</code> for the manifest:
 
<pre class="language-javascript">{ title: 'Hello World App' }</pre>
 
Building the app is as easy as calling
 
  $ build-appsuite app


==== JSHINT ====
==== JSHINT ====

Latest revision as of 09:00, 10 August 2016

API status: Deprecated

Build System

Abstract: This document describes the build system of OX App Suite. It is intended for app developers who use the build system to create apps as well as for OX App Suite developers who not only use the build system but also may wish to extend it.

The build system is used to create source archives from source files. These source archives can be used to compile installable packages for various Linux distributions. The build system can generate archives for the core UI as well as for independently installed apps.

The OX App Suite build system uses Jake, a port of Rake from Ruby to Node.js. Both Rake and Jake are dependency-based build systems like Make, which allows quick incremental builds. But unlike Make, Jake doesn't have its own syntax. Instead, it provides an API in JavaScript. The API is used not only to specify dependencies between files using a full programming language, but also to implement the generation of files in the same language. This allows easy implementation of complex build systems, of which the OX App Suite build system is an example. Using the same language for the developed project and for its build system also allows any core developer to quickly extend the build system without having to switch to another language.

Using the Build System

While easily extensible, most of the time the build system will be used as-is. This chapter describes how to set up and use the build system to develop apps and the OX App Suite core.

All command examples are given for the Debian operating system. The instructions should work similarly on other POSIX systems. The first character of each command indicates whether it should be executed as root (#) or as a normal user ($).

Installing

Installing the right environment for running the appserver and the UI build system is described in the getting started article.

Running

The build system is executed by invoking the command build-appsuite. Similar to most build systems, the build system can perform multiple tasks, which are specified as parameters on the command line. Each task can require any number of parameters. These parameters can be specified either on the command line, using the syntax name=value, or as environment variables.

If present, the file local.conf is sourced by a shell script before the build process starts. This file can export environment variables which are specific to the local system, without checking them into a version control system. Typically, it defines values for builddir and debug.

Since the build system is based on Jake, it also accepts all other Jake options. In addition, the environment variable $nodeopts can be used to pass command line parameters to Node.js. One of the most useful parameters is --debug-brk, which can be used to debug the build system.

When developing external apps, the build system must be run from the top directory of the app's source. As a safety precaution, execution is aborted if the subdirectory apps, which usually contains JavaScript source code, is not found. This Article is written assuming, you're working in your workspace directory, containing the subfolder apps.

Workflow

The build system is used not only to create source archives for packaging. It can also directly install and update the built UI in a directory during development, help with the setup of the source of a new external app and more. While all of these tasks are described in the reference section, daily work involves just a few of them.

To have an easy example about how to write, build & host your javascript code and how to initialize & build your packages, please have a look at the GettingStarted article.

JSHINT

One step of the build process is running jshint to statically analyse the sources for certain errors. JSHint is configured with (what we think) sane defaults, but if you want to configure your own rules, edit .jshintrc in your project`s root folder, according to your needs.

default

The default task is used instead of the app task when building the core OX App Suite. Since it is the default Jake task, it is not necessary to specify it on the command line when it's the only task.

The top directory of OX App Suite source code includes the script build.sh, which should be used instead of calling a potentially unrelated version of build-appsuite. The script changes the current directory to its own, so that it can be called from any directory. 

  $ web/ui/build.sh

clean

The build system uses dependencies and file timestamps to decide which files to rebuild. This assumes that any change to a file increases its timestamp to a value which is greater than the timestamp of any existing file. When this assumption is violated (e.g. after switching to a different source control branch with older files) it may become necessary to rebuild everything to restore the assumption about timestamps. The simplest way to achieve this is the clean task, which simply deletes all generated files.

WARNING: This can be potentially dangerous, since the clean task simply deletes the directories specified by the variables builddir, destDir, l10nDir, manifestDir, and helpDir.

dist

When the app is ready to be shipped, or rather all the time on a continuous build system, the app needs to be packaged in a format suitable for installation on a production system. Since there already exist tools to create packages from suitably arranged source code archives, the OX App Suite build system merely prepares such source archives.

The dist task creates an archive with the source (the one ending in .orig.tar.bz2) and a few additional files necessary for Debian packaging. RPM packages can be generated using the same source archive and the .spec file created by init-packaging. The version of the package is extracted from the newest entry in the file debian/changelog.

Debian packages can also be generated manually either from the temporary directory left behind by dist, or even directly from the source tree. The second option pollutes the source tree with generated files, so it is not recommended, although the .gitignore file created by init-packaging can handle these generated files. 

  $ build-appsuite dist
  $ ls tmp/packaging/
  example-app-0.0.1                   example-app_0.0.1-1.dsc
  example-app_0.0.1-1.debian.tar.bz2  example-app_0.0.1.orig.tar.bz2
  $ cd tmp/packaging/example-app-0.0.1/
  $ dpkg-buildpackage -b

Reference

Variables

BASEDIR

The top directory of the build system.

Used by: all tasks.

Default: installation directory of the build system

Required to build external apps, since in this case, the build system is not installed in the current directory. This variable is automatically set as an environment variable by the build system executable based on $OX_APPSUITE_DEV.

branch

The Subversion branch of the CLDR to checkout.

Used by: update-i18n.

builddir

The target directory for generated files.

Used by: app, clean, default, dist, docs, jakedeps.

Default: build

copyright

The copyright line to be included in packaging metadata.

Used by: init-packaging.

Example: 2012 Open-Xchange, Inc

coreDir

Location of OX App Suite UI files.

Used by: app, default.

Default: same as builddir

Some tasks depend on installed files from the OX App Suite. When building external apps, coreDir specified the directory which contains these files.

Currently, this parameter is used to compile .less files for every installed theme. This can be disabled by setting skipLess.

debug

Enables a debug build.

Used by: app, default.

Default: false

To simplify debugging of OX App Suite, compression of source code can be disabled by specifying on, yes, true or 1.

description

Used by: init-packaging.

The description of the app to be included in packaging metadata.

destDir

Output directory for source archives created by the dist task.

Used by: clean, dist.

Default: tmp/packaging

disableStrictMode

Removes all "use strict" directives from processed JavaScript code.

Used by: app, default.

Default: false

Some debugging tools which use code instrumentation have problems when the debugged code uses strict mode. This setting enables code processing even whe using debug mode, so line numbers will not match the original source code.

forceDeb

Whether an error during the generation of Debian source packages is an error.

Used by: dist.

Default: false

This is useful when Debian packages are generated automatically, and a failure of dpkg-source is also a failure of the entire build. By default it merely produces a warning.

from

The root of the printed dependency tree between Jake tasks.

Used by: jakedeps.

helpDir

The location of online help files.

Used by: clean, default, dist.

Default: same as builddir

If the value contains the string @lang@, it will be replaced by the lowercase language code (e.g. en-us) to allow per-language directories.

l10nDir

The location of compiled l10n files.

Used by: app, clean, default, dist.

Default: same as builddir

If the value contains the string @lang@, it will be replaced by the lowercase language code (e.g. en-us) to allow per-language directories.

license

File name of the full text of the distribution license.

Used by: init-packaging.

Default: based on licenseName.

licenseName

Name of the distribution license to be included in packaging metadata.

Used by: init-packaging.

Default: CC-BY-NC-SA-3.0

manifestDir

The location of the combined manifest file.

Used by: app, clean, default, dist.

Default: same as builddir

maintainer

Name and email address of the package maintainer.

Used by: init-packaging.

Format: Name <email>

package

The name of the package for the built app.

Used by: all tasks.

Default: the package name in the first line of debian/changelog

Since the name of the manifest file contains the package name and it is required to determine build dependencies, the package name must be always known. This means either debian/changelog must exist and contain at least one entry, or the parameter must be explicitly specified.

reverse

Reverses the direction of printed dependencies.

Used by: deps.

When specified, the deps task prints modules which depend on the specified modules, instead of modules on which the specified module depends.

revision

Revision number of the package for the app.

Used by: app, default, dist.

Default: 1

The revision number must increase with each rebuild of the same version to enable the creation of unique version strings. These are required in package names and to control content caching in clients.

root

Specifies for which module to print the dependencies.

Used by: deps.

Default: print all roots

If specified, only the dependencies of the specified module are printed. Otherwise, the dependencies of all modules are printed.

skipDeb

Whether to skip the generation of Debian source packages.

Used by: dist.

Default: false

This is useful when Debian packages are not required and/or dpkg-source is not available, e.g. on RPM based systems.

Even when using this flag, at least the file debian/changelog is still required, because it is used to store the package name and version.

skipLess

Whether to skip the preprocessing of LessCSS files.

Used by: app, default.

Default: false

This flag skips the generation of CSS files in the apps/themes/*/less directories of all themes. It is used by the packaging system, where the LessCSS files are precompiled after installation on the target system instead of while building the package.

tag

The Subversion tag of the CLDR to checkout.

Used by: update-i18n.

to

The leaf task in the printed dependency path between Jake tasks.

Used by: jakedeps.

When specified, only a single path from the root to the leaf task is printed (in reverse order).

version

Version number of the app.

Used by: app, default, dist, init-packaging.

Default: 0.0.1

The version should consist of a major, minor and patch version separated by dots.

Tasks

An up-to-date list of tasks can be printed using the -T command line option.

app

Builds an external app.

Variables: BASEDIR, builddir, coreDir, debug, disableStrictMode, l10nDir, manifestDir, package, revision, version.

This is the main task used to build external apps.

It works without explicitly specifying any variables, but during development, builddir is usually pointed to the directory of a locally installed OX App Suite UI to avoid additional copying steps. For debugging, debug is also often used. Note that clean must be called when changing any of the variables.

clean

Removes all generated files.

Variables: BASEDIR, builddir, destDir, l10nDir, manifestDir, package.

This task should be executed before a normal build using app or default after changing any build variables and after a switch between Git branches. Normal incremental builds can miss changed files if a branch switch replaces files by older versions.

default

Builds OX App Suite.

Variables: BASEDIR, builddir, debug, disableStrictMode, l10nDir, manifestDir, package, revision, version.

This is the main task to build OX App Suite. Since it is the default Jake task, it does not need to be specified explicitly on the command line when it is the only task.

It works without explicitly specifying any variables, but during development, builddir is usually pointed to a directory which is accessible to a local web server. For debugging, debug is also often used. Note that clean must be called when changing any of the variables.

deps

Prints module dependencies.

Variables: BASEDIR, package, reverse, root.

This task visualizes dependencies of RequireJS modules. It prints a tree of dependencies to stdout.

If root is specified, then only the dependencies of that module are printed. Otherwise, the dependencies of all modules, on which no other module depends are printed in sequence.

If reverse is specified, then this task prints dependants instead of dependencies, i.e. modules which depend on the specified module instead of modules on which the specified module depends.

dist

Creates source packages.

Variables: BASEDIR, builddir, destDir, forceDeb, l10nDir, manifestDir, package, revision, skipDeb, version.

This task cleans the source tree by calling clean and packs the source into an archive which can be used to create Debian and RPM packages. The necessary Debian metadata is created alongside the source archive. All files necessary for Debian packaging are placed in the directory specified by destDir. The generated .orig.tar.bz2 archive can also be used together with the .spec file to generate RPM packages.

Unless the variable skipDeb is set to true, the program dpkg-source is required by this task. It is used to generate Debian-specific packaging metadata.

init-packaging

Initializes packaging information for a new app.

Variables: BASEDIR, copyright, description, license, licenseName, maintainer, package, version.

This task is the first task called when starting with the development of a new external app. The directory from which it is called must already contain at least the apps subdirectory. This is also the only task where the package variable must be specified explicitly. Afterwards, the package name is looked up automatically in the file debian/changelog, which is created by this task.

The variables version, maintainer, copyright, licenseName, license, and description are required to fill out all necessary packaging metadata. Any of these variables which are not specified explicitly will cause an interactive prompt. This avoids the need to remember the list of variables before one can start developing an app.

jakedeps

Shows the dependency chain between two Jake tasks.

Variables: BASEDIR, from, package, to.

This task visualized dependencies between Jake tasks. The variable from specifies the root of a dependency tree, with all dependencies of from as inner and leaf nodes. If to is not specified, then that entire tree is printed.

If to is also specified, then only the first found dependency path from from to to is .printed with to at the top and from at the bottom.

merge

Updates all .po files with the generated ox.pot.

Variables: BASEDIR, builddir, debug, package, revision, version.

This task updates the list of extracted i18n strings in ox.pot and calls the GNU Gettext tool msgmerge for every language in i18n/*.po.

i18n/en_US.po is not updated by this task because the original strings are already in the en_US locale. It would only end up with every translation mapping every string to itself. The only entries in that file should be for cases when the en_US text is not a suitable fallback for missing translations. The original string in such a case would be something internationally appropriate (e. g. a date written as "2013-01-01" instead of "01/01/2013") and i18n/en_US.po would contain a translation.

update-i18n

Updates CLDR data in the source tree.

Variables: BASEDIR, branch, package, tag.

This task downloads data from the Unicode CLDR and updates date translations for all languages in i18n/*.po.

The exact version of CLDR data is specified as Suubversion tag or branch by the variables tag and branch, respectively. If neither is specified, then the Subversion trunk is checked out. If both are specified, tag is used.

verify-doc

Generates a documentation skeleton for extension points.

Variables: BASEDIR, package.

This task is still under development. Currently, it creates a list of all extension points, which have a constant name in the source code. In the future it is supposed to update an existing list instead of overwriting it, and to handle non-constant extension point names.

The list of extension points is stored as an HTML snippet in doc/extensionpoints.html.

Retrieved from "https://wiki.open-xchange.com/wiki/index.php?title=AppSuite:UI_build_system&oldid=22288"