HTTP interface for imports

The Import HTTP API allows you to load JSON data in bulk into ArangoDB

Import JSON data as documents

imports documents from JSON

POST /_api/import

Query Parameters

  • collection (string, required): The name of the target collection. The collection needs to exist already.

  • type (string, optional): Determines how the body of the request is interpreted.

    • documents: JSON Lines (JSONL) format. Each line is expected to be one JSON object.

      Example:

      {"_key":"john","name":"John Smith","age":35}
{"_key":"katie","name":"Katie Foster","age":28}

    • array (or list): JSON format. The request body is expected to be a JSON array of objects. This format requires ArangoDB to parse the complete array and keep it in memory for the duration of the import. This is more resource-intensive than the line-wise JSONL processing.

      Any whitespace outside of strings is ignored, which means the JSON data can be a single line or be formatted as multiple lines.

      Example:

      [
  {"_key":"john","name":"John Smith","age":35},
  {"_key":"katie","name":"Katie Foster","age":28}
]

    • auto: automatically determines the type (either documents or array).

    • Omit the type parameter entirely to import JSON arrays of tabular data, similar to CSV.

      The first line is an array of strings that defines the attribute keys. The subsequent lines are arrays with the attribute values. The keys and values are matched by the order of the array elements.

      Example:

      ["_key","name","age"]
["john","John Smith",35]
["katie","Katie Foster",28]

  • fromPrefix (string, optional): An optional prefix for the values in _from attributes. If specified, the value is automatically prepended to each _from input value. This allows specifying just the keys for _from.

  • toPrefix (string, optional): An optional prefix for the values in _to attributes. If specified, the value is automatically prepended to each _to input value. This allows specifying just the keys for _to.

  • overwrite (boolean, optional): If this parameter has a value of true or yes, then all data in the collection will be removed prior to the import. Note that any existing index definitions will be preserved.

  • waitForSync (boolean, optional): Wait until documents have been synced to disk before returning.

  • onDuplicate (string, optional): Controls what action is carried out in case of a unique key constraint violation. Possible values are:

    • error: this will not import the current document because of the unique key constraint violation. This is the default setting.
    • update: this will update an existing document in the database with the data specified in the request. Attributes of the existing document that are not present in the request will be preserved.
    • replace: this will replace an existing document in the database with the data specified in the request.
    • ignore: this will not update an existing document and simply ignore the error caused by a unique key constraint violation.

    Note that update, replace and ignore will only work when the import document in the request contains the _key attribute. update and replace may also fail because of secondary unique key constraint violations.

  • complete (boolean, optional): If set to true, the whole import fails if any error occurs. Otherwise, the import continues even if some documents are invalid and cannot be imported, skipping the problematic documents.

  • details (boolean, optional): If set to true, the result includes a details attribute with information about documents that could not be imported.

Request Body

(string, required)

The body must either be a JSON-encoded array of objects or a string with multiple JSON objects separated by newlines.

Load JSON data and store it as documents into the specified collection.

The request body can have different JSON formats:

  • One JSON object per line (JSONL)
  • A JSON array of objects
  • One JSON array per line (CSV-like)

If you import documents into edge collections, all documents require a _from and a _to attribute.

Responses

HTTP 201: is returned if all documents could be imported successfully.

The response is a JSON object with the following attributes:

  • created (integer): The number of imported documents.

  • errors (integer): The number of documents that were not imported due to errors.

  • empty (integer): The number of empty lines found in the input. Only greater than zero for the types documents and auto.

  • updated (integer): The number of updated/replaced documents. Only greater than zero if onDuplicate is set to either update or replace.

  • ignored (integer): The number of failed but ignored insert operations. Only greater than zero if onDuplicate is set to ignore.

  • details (array of strings): An array with the error messages caused by documents that could not be imported. Only present if details is set to true.

HTTP 400: is returned if type contains an invalid value, no collection is specified, the documents are incorrectly encoded, or the request is malformed.

HTTP 404: is returned if collection or the _from or _to attributes of an imported edge refer to an unknown collection.

HTTP 409: is returned if the import would trigger a unique key violation and complete is set to true.

HTTP 500: is returned if the server cannot auto-generate a document key (out of keys error) for a document with no user-defined key.

Examples

Importing documents with heterogenous attributes from an array of JSON objects:

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=list <<EOF
[ 
  { 
    "_key" : "abc", 
    "value1" : 25, 
    "value2" : "test", 
    "allowed" : true 
  }, 
  { 
    "_key" : "foo", 
    "name" : "baz" 
  }, 
  { 
    "name" : { 
      "detailed" : "detailed name", 
      "short" : "short name" 
    } 
  } 
]
EOF

HTTP/1.1 201 Created
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 72
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Importing documents using JSON objects separated by new lines (JSONL):

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=documents <<EOF
{ "_key": "abc", "value1": 25, "value2": "test","allowed": true }
{ "_key": "foo", "name": "baz" }

{ "name": { "detailed": "detailed name", "short": "short name" } }

EOF

HTTP/1.1 201 Created
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 72
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Using the auto type detection:

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=auto <<EOF
[ 
  { 
    "_key" : "abc", 
    "value1" : 25, 
    "value2" : "test", 
    "allowed" : true 
  }, 
  { 
    "_key" : "foo", 
    "name" : "baz" 
  }, 
  { 
    "name" : { 
      "detailed" : "detailed name", 
      "short" : "short name" 
    } 
  } 
]
EOF

HTTP/1.1 201 Created
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 72
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Importing JSONL into an edge collection, with _from, _to and name attributes:

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=links&type=documents <<EOF
{ "_from": "products/123", "_to": "products/234" }
{"_from": "products/332", "_to": "products/abc",   "name": "other name" }
EOF

HTTP/1.1 201 Created
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 72
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Importing an array of JSON objects into an edge collection, omitting _from or _to:

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=links&type=list&details=true <<EOF
[ 
  { 
    "name" : "some name" 
  } 
]
EOF

HTTP/1.1 201 Created
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 182
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Violating a unique constraint, but allowing partial imports:

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=documents&details=true <<EOF
{ "_key": "abc", "value1": 25, "value2": "test" }
{ "_key": "abc", "value1": "bar", "value2": "baz" }
EOF

HTTP/1.1 201 Created
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 244
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Violating a unique constraint, not allowing partial imports:

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=documents&complete=true <<EOF
{ "_key": "abc", "value1": 25, "value2": "test" }
{ "_key": "abc", "value1": "bar", "value2": "baz" }
EOF

HTTP/1.1 409 Conflict
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 85
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Using a non-existing collection:

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=documents <<EOF
{ "name": "test" }
EOF

HTTP/1.1 404 Not Found
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 97
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Using a malformed body with an array of JSON objects being expected:

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=list <<EOF
{ }
EOF

HTTP/1.1 400 Bad Request
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 95
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Importing two documents using the JSON arrays format. The documents have a _key, value1, and value2 attribute each. One line in the import data is empty and skipped:

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products <<EOF
[ "_key", "value1", "value2" ]
[ "abc", 25, "test" ]

[ "foo", "bar", "baz" ]
EOF

HTTP/1.1 201 Created
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 72
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Importing JSON arrays into an edge collection, with _from, _to, and name attributes:

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=links <<EOF
[ "_from", "_to", "name" ]
[ "products/123","products/234", "some name" ]
[ "products/332", "products/abc", "other name" ]
EOF

HTTP/1.1 201 Created
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 72
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Importing JSON arrays into an edge collection, omitting _from or _to:

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=links&details=true <<EOF
[ "name" ]
[ "some name" ]
[ "other name" ]
EOF

HTTP/1.1 201 Created
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 281
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Using a malformed body with JSON arrays being expected:

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products <<EOF
{ "_key": "foo", "value1": "bar" }
EOF

HTTP/1.1 400 Bad Request
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 92
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body
❮  Analyzers

© ArangoDB - Graph and Beyond