Logstash 新手,真的很喜欢它。
尝试解析包含时间戳的 CSV 文件。想要解析时间戳并将其用作@timestamp字段。
我的 CSV 输入示例
input {
stdin {}
}
filter {
# filter the input by csv (i.e. comma-separated-value)
csv {
columns => [
"Job ID",
"Server Name",
"Status Code",
"Job Type",
"Client Name",
"Start Time",
"End Time"
]
}
# parse the start time to create a real date
date {
# Examples of times in this log file
# "May 29, 2015 10:00:01 PM"
# "May 9, 2015 4:47:23 AM"
match => [ "End Time",
"MMM dd, YYYY HH:mm:ss aa",
"MMM d, YYYY HH:mm:ss aa" ]
}
}
# send the output to stdout, using the rubydebug codec
# rubydedug uses the Ruby Awesome Print library
output {
stdout { codec => rubydebug }
}
我的输入样本
108628,anmuswcnbu01,1,Backup,anmuswcrfax01.na.jnj.com,"May 29, 2015 10:00:01 PM","May 30, 2015 6:21:29 AM"
108629,anmuswcnbu01,1,Backup,anmuswcapps01.na.jnj.com,"May 29, 2015 10:00:01 PM","May 9, 2015 10:51:39 pm"
108630,anmuswcnbu01,1,Backup,anmuswcapps03.na.jnj.com,"May 29, 2015 10:00:01 PM","May 29, 2015 9:31:19 PM"
我的输出示例
Logstash startup completed
{
"message" => [
[0] "108628,anmuswcnbu01,1,Backup,anmuswcrfax01.na.jnj.com,\"May 29, 2015 10:00:01 PM\",\"May 30, 2015 6:21:29 AM\"\r"
],
"@version" => "1",
"@timestamp" => "2015-05-30T06:21:29.000Z",
"host" => "ip-172-31-34-14",
"Job ID" => "108628",
"Server Name" => "anmuswcnbu01",
"Status Code" => "1",
"Job Type" => "Backup",
"Client Name" => "anmuswcrfax01.na.jnj.com",
"Start Time" => "May 29, 2015 10:00:01 PM",
"End Time" => "May 30, 2015 6:21:29 AM"
}
{
"message" => [
[0] "108629,anmuswcnbu01,1,Backup,anmuswcapps01.na.jnj.com,\"May 29, 2015 10:00:01 PM\",\"May 9, 2015 10:51:39 pm\"\r"
],
"@version" => "1",
"@timestamp" => "2015-05-09T10:51:39.000Z",
"host" => "ip-172-31-34-14",
"Job ID" => "108629",
"Server Name" => "anmuswcnbu01",
"Status Code" => "1",
"Job Type" => "Backup",
"Client Name" => "anmuswcapps01.na.jnj.com",
"Start Time" => "May 29, 2015 10:00:01 PM",
"End Time" => "May 9, 2015 10:51:39 pm"
}
{
"message" => [
[0] "108630,anmuswcnbu01,1,Backup,anmuswcapps03.na.jnj.com,\"May 29, 2015 10:00:01 PM\",\"May 29, 2015 9:31:19 PM\"\r"
],
"@version" => "1",
"@timestamp" => "2015-05-29T09:31:19.000Z",
"host" => "ip-172-31-34-14",
"Job ID" => "108630",
"Server Name" => "anmuswcnbu01",
"Status Code" => "1",
"Job Type" => "Backup",
"Client Name" => "anmuswcapps03.na.jnj.com",
"Start Time" => "May 29, 2015 10:00:01 PM",
"End Time" => "May 29, 2015 9:31:19 PM"
}
Logstash shutdown completed
例如,在最后(第三)行数据中,而不是:
"@timestamp" => "2015-05-29T09:31:19.000Z",
我觉得我应该得到
"@timestamp" => "2015-05-29T21:31:19.000Z",
据我所知,日期过滤器似乎忽略了我的“半天”语法
match => [ "End Time",
"MMM dd, YYYY HH:mm:ss aa",
"MMM d, YYYY HH:mm:ss aa" ]
Logstash 新手,所以想知道我是否做错了什么?
-Chad