Meta field for the document

Problem: Currently document contains no information on how to interpret it's contents. E.g. whether text in a certain field is tokenized as sentences.

Idea:

  • Add special field called "_meta" inside the document.
  • Something like a mapping definition in Elasticsearch.
  • Should be minimal and not puke out too much useless information.

Open questions:

  • how is the meta for nested fields stored?
  • what actually needs to be done and where?

Tokenization & spans should always default to text!!!

Edited by Raul Sirel