Fluentular: a Fluentd regular expression editor

Regular Expression Test String Custom Time Format (See also ruby document; strptime)

Example (Apache)

Regular expression:

^(?<host>[^ ]*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^ ]*) +\S*)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?$

Time Format:

%d/%b/%Y:%H:%M:%S %z

Configuration

Copy and paste to fluent.conf or td-agent.conf

<source>
  type tail
  path /var/log/foo/bar.log
  pos_file /var/log/td-agent/foo-bar.log.pos
  tag foo.bar
  format /^(?<remote_addr>[^ ]*) - (?<remote_user>[^ ]*) \[(?<time>[^\]]*)\]\s+(?<request_type>[^ ]*) (?<request_url>[^ ]*) (?<request_http_protocol>[^ ]*) "(?<status>[^"]*)" (?<body_bytes_sent>[^ ]*) "(?<http_referer>[^"]*)" "(?<http_host>[^"]*)" (?<host>[^ ]*) "(?<http_user_agent>[^"]*)" "(?<http_x_forwarded_for>[^"]*)" upstream_response_time (?<upstream_response_time>[^ ]*) upstream_addr (?<upstream_addr>[^ ]*) msec (?<msec request_time>[^ ]*) request_time (?<request_time>[^ ]*)/
</source>

Data Inspector

Attributes

Key	Value

Records

Key	Value