W3C plugin

[jsirianni]

I can help test this. It depends on a release that does not exists (yet) Stanza v1.1.9. You will need to comment min_stanza_version for now. You can test with a dev branch of stanza, I can help set you up.

The W3C plugin wraps the CSV parser, making use of dynamic field names

• File input detects W3C headers and attaches them as labls
• CSV operator will use the Fields label to determine the CSV field names dynamically, for every entry.

This allows the user to read arbitrary W3C files without specifying the field names.

[jsirianni]

You can test this plugin by building this branch observIQ/stanza#404

[ericwholt]

Using the following config:

    - type: w3c
      file_log_path: 
      - /Users/ericholt/bindplane/bplogagent-dev/msiis-logs/iis.log
      header_delimiter: " "
      delimiter: " "
      start_at: beginning

IIS Logs Output:

{
  "timestamp": "2021-08-23T21:48:43.197971-04:00",
  "severity": 0,
  "labels": {
    "Date": "2020-11-03 16:13:15",
    "Software": "Microsoft Internet Information Services 10.0",
    "Version": "1.0",
    "file_name": "iis.log",
    "log_type": "w3c",
    "plugin_id": "w3c"
  },
  "record": {
    "c-ip": "10.0.0.2",
    "cs(Referer)": "-",
    "cs(User-Agent)": "Go-http-client/1.1",
    "cs-method": "GET",
    "cs-uri-query": "-",
    "cs-uri-stem": "/status",
    "cs-username": "-",
    "date": "2020-11-03",
    "s-ip": "10.0.0.1",
    "s-port": "80",
    "sc-status": "404",
    "sc-substatus": "0",
    "sc-win32-status": "2",
    "time": "16:13:15",
    "time-taken": "1"
  }
}