ElasticSearch Ingest Pipeline：创建和更新时间戳字段

要在我的索引上创建时间戳字段，根据this https://stackoverflow.com/a/66958236/758836回答，我创建了一个摄取管道 https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html运行特定索引：

PUT _ingest/pipeline/auto_now_add
{
  "description": "Assigns the current date if not yet present and if the index name is whitelisted",
  "processors": [
    {
      "script": {
        "source": """
          // skip if not whitelisted
          if (![ "my_index_1",
                 "my_index_2"
              ].contains(ctx['_index'])) { return; }
          
          // always update updated_at
          ctx['updated_at'] = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date());
          
        """
      }
    }
  ]
}

然后我将所有索引设置应用为默认管道

PUT _all/_settings
{
  "index": {
    "default_pipeline": "auto_now_add"
  }
}

之后，我开始将我的对象索引到这些索引中。当我查询索引项时，我将使用以下命令获取该项updated_at索引时更新的字段，例如：

{
  _index: 'my_index_1',
  _type: '_doc',
  _id: 'r1285044056',
  _version: 11,
  _seq_no: 373,
  _primary_term: 2,
  found: true,
  _source: {
    updated_at: '2021-07-07 04:35:39',
    ...
  }
}

我现在想要一个created_at字段，仅在第一次更新，所以我尝试以这种方式更新上面的脚本：

PUT _ingest/pipeline/auto_now_add
{
  "description": "Assigns the current date if not yet present and if the index name is whitelisted",
  "processors": [
    {
      "script": {
        "source": """
          // skip if not whitelisted
          if (![ "my_index_1",
                 "my_index_2",
                 "..."
              ].contains(ctx['_index'])) { return; }
          
           // always update updated_at
          ctx['updated_at'] = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date());
          // don't overwrite if present
          if (ctx != null && ctx['created_at'] != null) { return; }
          
          ctx['created_at'] = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date());
        """
      }
    }
  ]
}

但这个解决方案似乎不起作用：条件

if (ctx != null && ctx['created_at'] != null) { return; }

总是会失败，从而导致更新created_at在索引上的每个对象更新时，以相同的方式updated_at场，使其无用。 那么，如何防止这种情况发生，并确保该字段created_at由摄取管道创建后是否存在？

正如@Val 在中所描述的这个答案 https://stackoverflow.com/a/51435984/8160318:

...摄取管道处理器将仅运行在上下文中 https://www.elastic.co/guide/en/elasticsearch/painless/7.13/painless-ingest-processor-context.html#painless-ingest-processor-context您发送的文档，而不是存储的文档（如果有）。

因此，您将无权访问底层_source nor doc因为摄取管道是为ingest阶段，而不是update phase.

您当然可以保留您的auto_now_add自动添加管道updated_at，你可以用它来扩展它created_at（如果尚未存在于摄取有效负载中）通过检查ctx.containsKey- 自从ctx本质上是一个javaMap:

PUT _ingest/pipeline/auto_now_add
{
  "description": "Assigns the current date if not yet present and if the index name is whitelisted",
  "processors": [
    {
      "script": {
        "source": """
          // skip if not whitelisted
          if (![ "my_index_1",
                 "my_index_2",
                 "..."
              ].contains(ctx['_index'])) { return; }
          
          def now = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date());
          
          // guaranteee updated_at
          ctx['updated_at'] = now;
          
          // add created_at only if nonexistent in the payload
          if (!ctx.containsKey('created_at')) {
            ctx['created_at'] = now;
          }  
        """
      }
    }
  ]
}

但是，这仅在您第一次摄取文档时有效！

Running:

POST my_index_1/_doc/some_id
{ 
  "some": "param"
}

将产生：

{
  "some" : "param",
  "updated_at" : "2021-07-08 10:35:13",
  "created_at" : "2021-07-08 10:35:13"
}

现在，为了自动递增updated_at每次更新文档时，你还需要一个脚本— 这次存储在_scripts, not _ingest/pipeline:

PUT _scripts/incement_update_at__plus_new_params
{
  "script": {
    "lang": "painless", 
    "source": """
      // add whatever is in the params
      ctx._source.putAll(params);
      
      // increment updated_at no matter what was in the params
      ctx._source['updated_at'] = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date());
    """
  }
}

然后，当你运行你的_update打电话，通过提到上述内容来做到这一点script:

POST my_index_1/_doc/some_id/_update
{
  "script": {
    "id": "incement_update_at__plus_new_params",
    "params": {
      "your": "new params"
    }
  }
}

这会增加updated_at不碰created_at并添加任何其他参数：

{
   "some":"param",
   "updated_at":"2021-07-08 10:49:44",    <--
   "created_at":"2021-07-08 10:39:55",
   "your":"new params"                    <--
}

无耻插件：我讨论管道和脚本 https://elasticsearchbook.com/learn-elasticsearch/post-indexing-updates-488d69498920464999ef35e36e3e8095非常详细地在我的Elasticsearch 手册 https://elasticsearchbook.com/.