Configuring clustered mode settings in general.properties
Clustered WorldServer application machines share a common database. The settings in the general.properties file on each system tell WorldServer how to handle clustered activities.
Indicating how many engines to create
workflow_daemon and background_daemon properties of the general.properties file. Note that the WorldServer 11 interface and the legacy interface use different sets of background engines, as follows:
- For the legacy interface, configure the number of background daemons in the general.properties file under %WS_CONFIG%.
- For the WorldServer 11 interface, configure the number of background daemons in the general.properties file under %WS_CONFIG%/ws-api.
workflow_daemon and background_daemon properties determine the level of concurrency for either engine:
- A value of
0means that the engine is disabled on this machine.For example, the following settings indicate that thebackground_daemonis disabled on this machine and that there is one workflow engine on this machine:workflow_daemon=1 background_daemon=0 - A value of
1(the default) means that there is a single engine of that type running on this machine. Each engine can handle one task or one job at a time. - A value of
2means that there are two engines of that type running on this machine and that two tasks or jobs can run in parallel.
To achieve the best performance, assign approximately the same number of engines as the number of CPUs. In dual-CPU application servers, set the number of engines to 2 or 3.
To create one or more dedicated WorldServer application machines, first install WorldServer on each machine using the standard installation instructions, but do not create a new WorldServer database for each installation. Make sure that each server points to the same WorldServer database. Then, set the workflow_daemon and background_daemon properties in general.properties to 0 on each of these machines. This will disable the workflow engine and background engine on these machines and ensure that the machines can be dedicated to the WorldServer application.
Whether you have multiple engines on a single machine or multiple engines on separate machines, the scaling works the same way. Each engine looks for a task or job that it can execute. If it finds one, it claims it, executes it, and then moves on to the next one. At any given time, each engine is capable of handling only a single task or job, and a single task or job cannot be shared across multiple engines. If there was one very large task or job, you would see a single engine processing it and all other engines would be idle. On the other hand, for a large number of tasks or jobs and multiple engines, you would see each of the engines claiming tasks to process one at a time.
Providing a clone name
WorldServer provides several dialog boxes that display information about the tasks and background jobs that are executing or that are waiting to execute. On these dialog boxes, an Assigned To column identifies the specific workflow or background engine on which a task or job is running, in the following format:
host-name:[clone-name:]engine-number
- host-name is the name of the server.
- clone-name indicates the instance of WorldServer. If you are running in a cluster, that is,
cluster_environment=true. Otherwise, clone-name is not displayed. The instances can run on a single server or multiple servers. If you do not provide a clone name, WorldServer displays a unique hash-code to identify the clone. - engine-number is a number assigned to each running workflow engine in each WorldServer instance.
- Set
cluster_environmenttotrue - Configure the
clone_nameproperty. For example, for a two-machine cluster, you might specify a name as follows:
Then, on the other machine, you might specify the name as follows:clone_name=juliet
The Assigned To column will now display the names you have specified instead of the internally-generated codes.clone_name=romeo
Specifying a location for files related to background processing
For each job, WorldServer stores files related to background processing, specifically the log file and any temporary data files related to the job. Seven days after completing or canceling the job, the WorldServer garbage collection feature deletes the files.
background_file_storage property in general.properties. For example,
background_file_storage = c:\WorldServer\background-jobsresets the storage area to the directory: c:\WorldServer\background-jobs.
background.process.log.dateTimeFormat in WEB-INF/classes/config/strings.properties. The default configuration is as follows:
# This determines how dates are formatted in background process log files.
background.process.log.dateTimeFormat=yyyy-MM-dd hh:mm:ss,SSS
Sharing files in a clustered environment
If you are working in a clustered environment, the background_file_storage property must point to one directory that is shared across the cluster by all WorldServer instances. If WorldServer is running on multiple servers, make sure that this directory is shared by, and accessible to, all the WorldServer servers in the cluster.
background_file_storage=//hosting_machine/background_share Drive letters on different nodes in the cluster do not have to match, but they must all point to the same fileshare location.
A local file path may be used only for the machine hosting the background file storage, if the machine is a member node in the cluster. Also, ensure that necessary read/write permissions are granted so that all background processes on each node in the cluster can read and write both from and to the background file storage location.
Working with other daemons
The following settings in general.properties control the startup of the Recurrence and Notification daemons:
recurrence_engine=on
notification_engine=on
These daemons are required by workflows for time scheduled processing and automatic notifications. You can turn off the Recurrence and Notification engines by setting the properties to off. Note that turning off the Recurrence engine also disables the garbage collection process.
Unlike the WorldServer application or the workflow engine, the Recurrence and Notification engines do not process requests in parallel; there is only one active Recurrence Engine or Notification Engine at any given time. If you have multiple installations with these engines enabled, the engines simply share the work among the different instances in a serial manner. Hence, there is no benefit to having a large cluster of machines in which these are turned on. However, you can enable these engines on multiple machines to get automatic failover: if one of the machines needs to be brought down, the other machine will take over the tasks. Thus, it is a good idea to have at least two machines with these engines enabled.
Avoiding collisions between multiple engines
You can avoid collisions between multiple engines in the entire cluster by configuring <add key="TotalNumberOfDaemons" value="0" />. This is an optional configuration, but, if configured, it will be used as the number of runnable task steps each engine should load each time for execution (the default value is 10). It will also be used to determine the maximum allowed collisions between engines and other engines in a WorldServer engine cluster. Currently, you must configure this property only if there are more than 10 engines in your cluster. If you configure this property, you must use the same value for each member of the cluster to manage collisions caused by long running task steps. If the execution time for a task-step exceeds 5 minutes, it will always get loaded by all available engines for running, and will stop the engines from considering other runnable steps if other runnable steps are not also loaded by the engine. This parameter, if configured correctly, will ensure that any runnable steps that are not currently being executed by any other engines will always be loaded and executed simultaneously in spite of collisions on long running task steps.