Difference between revisions of "CrawlerArchitecture"

(Architecture of the crawler bundle)
(Architecture of the crawler bundle)
Line 2: Line 2:
  
 
Each 3rd-Party-Service (e.g. Facebook) is represented by a separate
 
Each 3rd-Party-Service (e.g. Facebook) is represented by a separate
OSGI-Service-instance (type ”GenericSubscribeService”). These services
+
OSGI-Service-instance (type "GenericSubscribeService"). These services
each contain a description (type “CrawlerDescription”) of how to access
+
each contain a description (type "CrawlerDescription") of how to access
“their” 3rd-Party-Service. Every time the OSGI-Bundle
+
"their" 3rd-Party-Service. Every time the OSGI-Bundle
“com.openexchange.subscribe.crawler” is started these descriptions get updated via
+
"com.openexchange.subscribe.crawler" is started these descriptions get updated via
 
reading YML-files (one for each 3rd-Party-Service). So to update a
 
reading YML-files (one for each 3rd-Party-Service). So to update a
“Crawler” a new YML-file has to be placed in the appropriate directory
+
"Crawler" a new YML-file has to be placed in the appropriate directory
 
and the OSGI-Bundle has to be restarted.
 
and the OSGI-Bundle has to be restarted.
  
 
When a user hits the refresh-button of a subscription the respective
 
When a user hits the refresh-button of a subscription the respective
OSGI-Service creates a new workflow-instance (type “Workflow”) from its
+
OSGI-Service creates a new workflow-instance (type "Workflow") from its
 
description, inserts the user´s credentials (decrypted using his login
 
description, inserts the user´s credentials (decrypted using his login
 
password) and executes the workflow. The workflow then executes each each
 
password) and executes the workflow. The workflow then executes each each
Step (type “Step”) in order. The output of each step is given to its
+
Step (type "Step") in order. The output of each step is given to its
successor as input. The output of a workflow´s final step is the wanted
+
successor as input. The output of a workflow's final step is the wanted
 
content (e.g. a list of Contacts) and is returned, via the OSGI-Service, to
 
content (e.g. a list of Contacts) and is returned, via the OSGI-Service, to
 
the OX-Server.
 
the OX-Server.

Revision as of 10:08, 4 March 2010

Architecture of the crawler bundle

Each 3rd-Party-Service (e.g. Facebook) is represented by a separate OSGI-Service-instance (type "GenericSubscribeService"). These services each contain a description (type "CrawlerDescription") of how to access "their" 3rd-Party-Service. Every time the OSGI-Bundle "com.openexchange.subscribe.crawler" is started these descriptions get updated via reading YML-files (one for each 3rd-Party-Service). So to update a "Crawler" a new YML-file has to be placed in the appropriate directory and the OSGI-Bundle has to be restarted.

When a user hits the refresh-button of a subscription the respective OSGI-Service creates a new workflow-instance (type "Workflow") from its description, inserts the user´s credentials (decrypted using his login password) and executes the workflow. The workflow then executes each each Step (type "Step") in order. The output of each step is given to its successor as input. The output of a workflow's final step is the wanted content (e.g. a list of Contacts) and is returned, via the OSGI-Service, to the OX-Server.

Crawler Architecture.jpg