Skip to main content

Async API

Overview​

Elide APIs are designed for synchronous request and response behavior. The time allowed to service a synchronous response can be limited by proxy servers and gateways. Analytic queries can often take longer than these limits and result in a server timeout. Elide's asynchronous API decouples the submission of a request and the delivery of the response into separate client calls. Some of the features available are:

  • Queries are run in a background thread that posts the results into a persistence store.
  • Results can be retrieved as an embedded response or URL for downloading later.
  • Supported downloading formats - CSV and JSON.
  • Queries can be configured to execute synchronously before switching to asynchronous mode if not finished by a client provided threshold.
  • Queries that are interrupted due to an application crash/reboot are automatically flagged as TIMEDOUT.
  • Persisted queries and results are deleted after a configurable threshold.

Design​

API​

The Async API supports two different query abstractions built using standard Elide models (AsyncQuery and TableExport):

  • Any read-only API request in JSON-API or GraphQL can be submitted by creating an AsyncQuery object. After creation, the client can poll the AsyncQuery object asynchronously for status updates. Once complete, the AsyncQuery object stores the query result as an embedded attribute.

  • For large response bodies, Elide supports a streaming abstraction called TableExport. It works similarly to the AsyncQuery model but with three important differences:

    1. TableExport can only be leveraged when:

      • The query is a simple read.
      • The model being queried has no relationships.
      • Only one model is being queried in the request.

    2. Upon successful completion, the TableExport model includes a separate URL attribute that references the query results for streaming downloads.

    3. The results of the GraphQL or JSON-API query are converted into a simple, flat JSON or CSV format.

Example API requests and responses can be found here.

Threading​

Each Elide instance runs a scheduler that is responsible for executing these requests in background threads. New async requests are initially marked in the QUEUED state. The requests are picked for execution as the threads become available. Upon completion, the background thread posts the query status and results to a persistent store. The size of the thread pool can be configured as mentioned here.

Async After​

The Async requests can be configured to execute synchronously before switching to asynchronous mode. The requests not finished synchronously by the client provided threshold are handed off to a separate thread for posting the results once complete. The default value for async-after is 10 seconds. Setting async-after to 0 will execute the request in asynchronous mode upon submission.

Background Cleaner​

Each Elide instance will also run a scheduler for maintenance and cleanup.

  • It cleans up requests and results stored in the persistent store.

  • It scans and flags the queries that failed due to an application crash/reboot automatically as TIMEDOUT.

  • It is responsible for the graceful cancellation of async requests.

    • The user can update the status of the async requests to CANCELLED. The cleaner polls the AsyncQuery and TableExport models to find any new requests that were CANCELLED and tries to terminate the transaction associated with that execution and change the status to CANCEL_COMPLETE.
    • Long-running transactions that exceed the run time threshold are terminated. Any background thread executing the original request is interrupted.

  • The retention, polling interval, and max run time thresholds can be configured during application startup.

  • This scheduler can be disabled by setting cleanup.enabled to false as mentioned here.

TableExport Results Download​

Elide has built-in support for streaming the results of a TableExport request through the export endpoint. Upon successful completion, the TableExport model includes a separate URL attribute where results can be downloaded from.

Enabling the end-point, timeouts, path, download attachment extensions, etc. can be configured during application startup as mentioned here.

Supported Query Types​

Below are the supported values for query type in asynchronous calls:

  • GRAPHQL_V1_0
  • JSONAPI_V1_0

Supported Result Types​

Elide can transform the results into a pre-selected format while persisting them via the ResultStorageEngine. Below are the supported formats for Table Export results:

  • JSON
  • CSV

Query Status​

Below are the different states of an asynchronous request:

StatusDescription
QUEUEDRequest is submitted and waiting to be picked up for execution.
PROCESSINGRequest has been picked up for execution.
COMPLETERequest has completed.
CANCELLEDThe client has requested to cancel a running request.
TIMEDOUTRequest did not finish within the configured maximum run time.
FAILURERequest not completed due to one or more failures encountered by the scheduler.
CANCEL_COMPLETERequest has been canceled by the background cleaner.

Malformed or invalid queries provided in the Async request will finish with COMPLETE status and the actual error message will be available in the result property of AsyncQuery and TableExport models.

Security​

The Async API models (AsyncQuery and TableExport) have a simple permission model: Only the principal who submitted a query and principals which belong to an administrative role are allowed to retrieve its status or results. Principals can be assigned roles when constructing the Elide user object.

Enable the Async API​

By default the async API is disabled. The elide models (AsyncQuery and TableExport) needed to support the Async API are JPA models that are mapped to a specific database schema. This schema must be created in our target database. Feel free to modify the query/result column sizes if needed.

NameDescriptionDefault
elide.async.enabledEnable the Async API feature.false
elide.async.cleanup.enabledEnable cleaning up of Async API requests history, update the status of interrupted/timedout requests, and cancel requests.false

Configure in application.yaml.

If we rely on Spring to autodiscover the entities which are placed in the same package/sub-package as the application class with @SpringBootApplication annotation, we will have to add the @EntityScan annotation to that application class for those entities to be discovered after async is enabled.

elide:
  async:
    enabled: true
    cleanup:
      enabled: true

Additional Configuration​

These additional configuration settings control timeouts, cleanup, export end-point, resultStorageEngine and the sizes of thread pools.

NameDescriptionDefault Value
elide.async.enabledWhether or not the async feature is enabled.false
elide.async.thread-pool-sizeDefault thread pool size.5
elide.async.max-async-afterDefault maximum permissible time to wait synchronously for the query to complete before switching to asynchronous mode.10s
elide.async.cleanup.enabledWhether or not the cleanup is enabled.false
elide.async.cleanup.query-max-run-timeMaximum query run time.3600s
elide.async.cleanup.query-retention-durationRetention period of async query and results before being cleaned up.7d
elide.async.cleanup.query-cancellation-intervalPolling interval to identify async queries that should be canceled.300s
elide.async.export.enabledWhether or not the controller is enabled.false
elide.async.export.pathThe URL path prefix for the controller./export
elide.async.export.append-file-extensionEnable Adding Extension to table export attachments.false
elide.async.export.storage-destinationStorage engine destination./tmp
elide.async.export.format.csv.write-headerGenerates the header in a CSV formatted export.true

These additional configuration settings are only applicable for Elide’s Standalone module. When using Spring, please configure the TaskExecutor used by Spring MVC for executing and managing the asynchronous requests.

NameDescriptionDefault Value
exportAsyncResponseTimeoutDefault timeout for TableExport's result download end-point.30s
exportAsyncResponseExecutorExecutor for executing TableExport's result download request asynchronously.A java.util.concurrent.ExecutorService instance

Configure in application.yaml.

elide:
  async:
    thread-pool-size: 10
    max-async-after: 30s
    cleanup:
      enabled: true
        query-max-run-time: 120s
        query-retention-duration: 10d
        query-cancellation-check-interval: 600s
    export:
      enabled: true
      path: /export
      storage-destination: /tmp
      format:
        csv:
          write-header: true

Running​

After configuring and starting our service, the following commands illustrate how to make asynchronous requests. Don't forget to change localhost:8080 accordingly. The example below makes use of the models and sample data that the liquibase migrations added through our example is available here.

Submitting query​

curl -X POST http://localhost:8080/api/v1/asyncQuery/ \
    -H"Content-Type: application/vnd.api+json" -H"Accept: application/vnd.api+json" \
    -d'{
            "data": {
                       "type": "asyncQuery",
                       "id": "ba31ca4e-ed8f-4be0-a0f3-12088fa9263d",
                       "attributes": {
                                       "query": "/group?sort=commonName&fields%5Bgroup%5D=commonName,description",
                                       "queryType": "JSONAPI_V1_0",
                                       "status": "QUEUED"
                                     }
                    }
        }'

Here are the respective responses:

{
    "data":{
        "type":"asyncQuery",
        "id":"ba31ca4e-ed8f-4be0-a0f3-12088fa9263d",
        "attributes":{
            "asyncAfterSeconds":10,
            "principalName":null,
            "createdOn":"2020-04-08T23:29Z",
            "query":"/group?sort=commonName&fields%5Bgroup%5D=commonName,description",
            "queryType":"JSONAPI_V1_0",
            "status":"COMPLETE",
            "updatedOn":"2020-04-08T23:29Z",
            "result":{
                "recordCount":2,
                "httpStatus":200,
                "completedOn":"2020-04-08T23:29Z",
                "contentLength":282,
                "responseBody":"{\"data\":[{\"type\":\"group\",\"id\":\"com.paiondata.elide\",\"attributes\":{\"commonName\":\"Elide\",\"description\":\"The magical library powering this project\"}},{\"type\":\"group\",\"id\":\"com.example.repository\",\"attributes\":{\"commonName\":\"Example Repository\",\"description\":\"The code for this project\"}}]}"
            }
        }
    }
}

Retrieving status and result​

Long-running queries in the QUEUED or PROCESSING state may not return with the result property populated in the responses above. The client can poll the AsyncQuery and TableExport objects asynchronously for status updates.

curl -X GET http://localhost:8080/api/v1/asyncQuery/ba31ca4e-ed8f-4be0-a0f3-12088fa9263d \
    -H"Content-Type: application/vnd.api+json" -H"Accept: application/vnd.api+json"

Here are the respective responses:

{
    "data":{
        "type":"asyncQuery",
        "id":"ba31ca4e-ed8f-4be0-a0f3-12088fa9263d",
        "attributes":{
            "asyncAfterSeconds":10,
            "principalName":null,
            "createdOn":"2020-04-08T21:25Z",
            "query":"/group?sort=commonName&fields%5Bgroup%5D=commonName,description",
            "queryType":"JSONAPI_V1_0",
            "status":"COMPLETE",
            "updatedOn":"2020-04-08T21:25Z",
            "result":{
                "recordCount":2,
                "httpStatus":200,
                "completedOn":"2020-04-08T23:29Z",
                "contentLength":282,
                "responseBody":"{\"data\":[{\"type\":\"group\",\"id\":\"com.paiondata.elide\",\"attributes\":{\"commonName\":\"Elide\",\"description\":\"The magical library powering this project\"}},{\"type\":\"group\",\"id\":\"com.example.repository\",\"attributes\":{\"commonName\":\"Example Repository\",\"description\":\"The code for this project\"}}]}"
            }
        }
    }
}

Downloading the TableExport results​

The TableExport request will return a URL to download the results as shown in the example response below.

{
    "result":{
        "completedOn":"2020-04-08T21:25Z",
        "url":"http://localhost:8080/export/bb31ca4e-ed8f-4be0-a0f3-12088fb9263g",
        "message":null,
        "httpStatus":200,
        "recordCount":2
    }
}

Overriding the AsyncApiDao​

The Async API interacts with the persistence layer through an abstraction - the AsyncApiDao, for status updates, query cleanup, etc. This can be customized by providing our own implementation. Elide provides a default implementation of AsyncApiDao.

Create a @Configuration class that defines our custom implementation as a @Bean.

@Configuration
public class ElideConfiguration {
    /**
     * Configure the AsyncApiDao used by async requests.
     * @return an AsyncApiDao object.
     */
    @Bean
    public AsyncApiDao asyncApiDao() {
        return new CustomAsyncApiDao();
    }
}

Overriding the ResultStorageEngine​

Table exports leverage a reactive abstraction (ResultStorageEngine) for streaming results to and from a persistence backend. This can be customized by providing our own implementation. Elide provides default implementation of ResultStorageEngine.

Create a @Configuration class that defines our custom implementation as a @Bean.

@Configuration
public class ElideConfiguration {
    /**
     * Configure the ResultStorageEngine used by TableExport requests.
     * @return a ResultStorageEngine object.
     */
    @Bean
    public ResultStorageEngine resultStorageEngine() {
        return new CustomResultStorageEngine();
    }
}

Internals​

Error loading elide-async-design.png