1
Ich analysiere dieses Log in Logstash. Dieses Protokoll enthält verschachtelte JSON.Parse nested JSON in Logstash
[[email protected] conf.d]# cat dummy.txt
{"dwo":"0","rs":{"$date":"2016-02-11T00:52:50.348Z"},"first":[{"timestamp":"2016-02-11T00:52:51Z","a":"abc","b":"243253456","c":"fdsfg444323fef"},{"timestamp":"2016-02-11T00:52:51Z","a":"xyz","b":"543q653q654","c":"fewdfsdf","d":"Thu Feb 11 2016 11:52:47 GMT 1100 (AUS Eastern Daylight Time)","e":"4235345"}],"bwv":"40","cp_tz":{"$date":"2016-02-11T11:52:50.000Z"}}
Hier ist meine logstash Konfiguration
[[email protected] conf.d]# cat hadoop.conf
input {
stdin {}
}
filter {
json {
source => message
}
}
output {
stdout { codec => rubydebug }
}
und das ist, wie ich logstash leite
[[email protected] conf.d]# /opt/logstash/bin/logstash -f /etc/logstash/conf.d/hadoop.conf
Logstash startup completed
{"dwo":"0","rs":{"$date":"2016-02-11T00:52:50.348Z"},"first":[{"timestamp":"2016-02-11T00:52:51Z","a":"abc","b":"243253456","c":"fdsfg444323fef"},{"timestamp":"2016-02-11T00:52:51Z","a":"xyz","b":"543q653q654","c":"fewdfsdf","d":"Thu Feb 11 2016 11:52:47 GMT 1100 (AUS Eastern Daylight Time)","e":"4235345"}],"bwv":"40","cp_tz":{"$date":"2016-02-11T11:52:50.000Z"}}
{
"message" => "{\"dwo\":\"0\",\"rs\":{\"$date\":\"2016-02-11T00:52:50.348Z\"},\"first\":[{\"timestamp\":\"2016-02-11T00:52:51Z\",\"a\":\"abc\",\"b\":\"243253456\",\"c\":\"fdsfg444323fef\"},{\"timestamp\":\"2016-02-11T00:52:51Z\",\"a\":\"xyz\",\"b\":\"543q653q654\",\"c\":\"fewdfsdf\",\"d\":\"Thu Feb 11 2016 11:52:47 GMT 1100 (AUS Eastern Daylight Time)\",\"e\":\"4235345\"}],\"bwv\":\"40\",\"cp_tz\":{\"$date\":\"2016-02-11T11:52:50.000Z\"}}",
"@version" => "1",
"@timestamp" => "2016-04-04T03:04:17.722Z",
"host" => "localhost",
"dwo" => "0",
"rs" => {
"$date" => "2016-02-11T00:52:50.348Z"
},
"first" => [
[0] {
"timestamp" => "2016-02-11T00:52:51Z",
"a" => "abc",
"b" => "243253456",
"c" => "fdsfg444323fef"
},
[1] {
"timestamp" => "2016-02-11T00:52:51Z",
"a" => "xyz",
"b" => "543q653q654",
"c" => "fewdfsdf",
"d" => "Thu Feb 11 2016 11:52:47 GMT 1100 (AUS Eastern Daylight Time)",
"e" => "4235345"
}
],
"bwv" => "40",
"cp_tz" => {
"$date" => "2016-02-11T11:52:50.000Z"
}
}
Wenn Sie auf dem Bild unten sehen, werden Sie feststellen, dass die Komponente innerhalb der "first" json ist nicht indiziert und daher nicht suchbar. Das ist meine Frage. Ich bin nicht in der Lage, verschachtelte JSON zu parsen.
!kibana]https://cdn.discourse.org/elastic/uploads/default/optimized/2X/e/e71d3a6734a6b49e3722b0b140d9ae77711ab73a_1_690x381.png