Path Matching in Objects
{
"attrs": {
"useless_key": "jibber_jabber",
"batch": {
"created_at": 1605443039,
"id": "batch_1213"
},
"session": {
"created_at": 1605443039,
"tags": ["affiliate_1", "checkout"]
}
}
} with potentially lots of currently unknown attrs.* sub-objects.
I want to map all attrs.*.created_at fields as timestamps and the rest (attrs.*.*) as keywords.
At the same time, I want to set index: false on everything else inside attrs to not make it searchable.
Let's use dynamic_templates with path_match and path_unmatch.
PUT myindex
{
"mappings": {
"dynamic_templates": [
{
"catch_created_at": {
"path_match": "attrs.*.created_at",
"mapping": {
"type": "date",
"format": "epoch_second"
}
}
},
{
"catch_keywords": {
"path_match": "attrs.*.*",
"path_unmatch": "attrs.*.created_at",
"mapping": {
"type": "keyword"
}
}
},
{
"disable_all_else": {
"path_match": "attrs.*",
"path_unmatch": "attrs.*.*",
"mapping": {
"index": false
}
}
}
]
}
}POST myindex/_doc
{
"attrs": {
"useless_key": 123,
"another_useless_key": "jibber_jabber",
"batch": {
"created_at": 1605443039,
"payload": "YmF0Y2g="
},
"session": {
"created_at": 1605443039,
"payload": "c2Vzc2lvbl9zdW5kYXk="
}
}
}{
"myindex" : {
"mappings" : {
"dynamic_templates" : [
...
],
"properties" : {
"attrs" : {
"properties" : {
// note that index: false
"useless_key" : { "type" : "long", "index" : false },
"another_useless_key" : { "type" : "text", "index" : false },
"batch" : {
"properties" : {
"created_at" : { "type" : "date", "format" : "epoch_second" },
"payload" : { "type" : "keyword" }
}
},
"session" : {
"properties" : {
"created_at" : { "type" : "date", "format" : "epoch_second" },
"payload" : { "type" : "keyword" }
}
}
}
}
}
}
}
} While the above solution works perfectly fine, dynamic_templates tend to monkey-patch the underlying issue of not being able to clearly define an index's mapping. On the other hand, there are situations where hard-coding the full mapping is not only onerous, but also impractical, and dynamic_templates do come in handy!
While wildcard attribute paths are supported in dynamic_templates, they are not in other parts of the Elasticsearch API such as aggregations. In practical terms this means that aggregating attrs.*.payload is not possible unless you resort to 6. (Painless) Scripting .
The official documentation is available here