Jobs

This Technical Concept is relevant to all services

Characteristic

Short description

A job is an alternative to synchronous requests. It is the representation of an asynchronous transaction.

Use

Jobs are used for potentially long running requests.

Detailed Consideration

As an alternative to synchronous requests there is a protocol for potentially long running requests. Instead of single HTTP requests that stay idle until a response is being sent, the initial request will start a background job running on the server. Clients have to query the job status to see when it has finished, then ask for the result.

A job is the representation of an asynchronous transaction. It has a status which may include further detailed progress information. When finished, a job also has a result which may be an error message. Jobs are kept persistent on the server for a certain period: A shorter period for fetched jobs and a longer one for unfetched jobs (see the job properties in the PTV xServer Configuration File).

Benefits

Using the asynchronous protocol meets several requirements that occur in some use cases:

Interruptions of the HTTP connection will no longer lose the result in the process, and transactions are much more reliable.
Some middleware proxies will forcibly terminate connections that are idle for too long. This problem vanishes when using the asynchronous protocol.
There are convenient operations that can report status and detailed progress information.
There are convenient operations that can stop or delete jobs. Results of stopped jobs can still be fetched.

Operation Naming Scheme

Every potentially long running operation of the form

runLong(RequestType): ResponseType

also provides a pair of operations:

startRunLong(RequestType): Job
fetchResponseType(JobRequest): ResponseType

All start operations begin with the prefix start, and all operations retrieving the result begin with fetch. In addition, the protocol requires the use of status operations provided by the xruntime service:

Job Status and Progress

The Job object models meta information about the server job. It contains the following attributes:

id: string - The globally unique id of the job, used to retrieve the job meta data and the results.
status: JobStatus - The current status of the job.
elapsedTime: int - The time since the job has been queued.
progress: JobProgress - The progress information for the job. This is an optional attribute and may be missing. If a progress is available, its concrete type depends on the operation. For instance, bulk operations always return a BulkJobProgress. See the PTV xServer API documentation for details. If the request is finished, the progress values represent the state when the request was finished.

All job-related operations except fetch will return a Job object. The status of the job will be QUEUING when returned by the start operation, STOPPING when returned by the stopJob operation, and DELETED when returned by the deleteJob operation. The watchJob operation may return any of the possible status values:"

job status	description	observable	has progress	fetchable	stoppable	deletable
`QUEUING`	the job has been scheduled for execution	when server is under load and the job has to wait	no	no	no	yes
`RUNNING`	the job is being executed	if the result is not yet available	yes	no	yes	yes
`STOPPING`	preparing for a clean exit and return the intermidate result	until the current processing step has finished	yes	no	yes (ignored)	yes
`SUCCEEDED`	the job was successful	after processing has terminated	yes	yes	yes (ignored)	yes (erases)
`FAILED`	the job has failed	after processing has terminated	yes	yes (returns with the exception)	yes (ignored)	yes (erases)
`DELETED`	the job has been deleted	while the job has not yet been terminated and cleaned up	no	no	no	yes (ignored)
`UNKNOWN`	no current job with this id	if id is wrong, or job has already been deleted or fetched	no	no	no	yes (ignored)

Workflow

A typical transaction is executed like the following pseudocode illustrates:

            jobId = startRunLong(request).id
            do  
                jobStatus = watchJob(jobId).status
            while jobStatus not in [ FAILED, SUCCEEDED, DELETED, UNKNOWN ]
        
            if jobStatus in [ FAILED, SUCCEEDED ] then
                response = fetchResponseType(jobId)
            end

The example illustrates the most simple form of the transaction. watchJob will return immediately if the status represents a finished job, i.e. it is one of FAILED, SUCCEEDED, DELETED, UNKNOWN. If the job is still running it will return only after the first progress update (by default one second minimum) so that a sleep command is not necessary in the above loop. It will return after 50 seconds even though no progress update is available.

In order to obtain more or less frequent progress updates (if supported by the service), specify the progressUpdatePeriod with the minimum period after which a progress update shall be returned at the earliest. This value is limited to 250ms at minimum and 50000ms at maximum. In order to wait for a shorter period for a progress update and to return earlier than the maximum of 50 seconds even though no progress update is available, specify the maximumPollingPeriod which must be greater or equal to the progressUpdatePeriod and must not exceed 50000ms.

Usually you might want to do more:

handle error conditions,
automatic retries in case of temporary connectivity issues,
display progress for users (in interactive scenarios),
allow users to stop/delete jobs (in interactive scenarios).

Administrator's Guide	Configuring the job database
Administrator's Guide	Setting up Database structures