Indexing in Bulk
Once you've established a solid mapping, you'll want to index multiple documents at once using the Bulk API. A typical payload to the _bulk endpoint would be sent as newline-delimited JSON (ndjson) but since this format is quite verbose and often hard to get right, it's helpful to use the client libraries' helpers instead. Nonetheless, we'll cover the ndjson format too in case you don't plan on using any client library.
Working with GeoJSON data? If you're ingesting geographic features exported from mapping applications, our Google Maps GeoJSON export guide shows how to convert user-drawn shapes into properly formatted GeoJSON that's ready for ElasticSearch ingestion.
There's no "optimal" payload chunk size due to a plethora of factors but a good amount to start with is 1,000 documents, or 5MB per request.
How do I index a JSON file consisting of an array of objects in python?
Using no python client lib, only requests
import requests
import json
index_name = 'my_index'
# we can either append the index name to the url like here
# or add it as one of the the `_index` attributes down the line
endpoint = 'http://localhost:9200/_bulk/' + index_name
data = []
with open('file.json', 'r') as json_in:
docs = json.loads(json_in.read())
for doc in docs:
# if '_id' is blank or left out, it'll be auto-generated
data.append({'index': {'_id': doc['_id']}})
data.append(doc)
# rudimentary conversion into ndjson
payload = '\n'.join([json.dumps(line) for line in data]) + '\n'
r = requests.put(endpoint,
# `data` instead of `json`
data=payload,
headers={
# it's a requirement
'Content-Type': 'application/x-ndjson'
})
print(r.json()) Using the elasticsearch-py library