Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix json_v2 parser to handle nested objects in arrays properly #9479

Merged
merged 1 commit into from
Jul 8, 2021

Conversation

sspaink
Copy link
Contributor

@sspaink sspaink commented Jul 7, 2021

  • Wrote appropriate unit tests.

@sjwang90 found an issue when parsing the following JSON that would result in empty metrics:

{
    "features": [
        {
            "properties": {
                "mag": 5.17
            }
        }
    ]
}

Config used:

[[inputs.file]]
    files = ["./testdata/test/input.json"]
    data_format = "json_v2"
    [[inputs.file.json_v2]]
            [[inputs.file.json_v2.object]]
                path = "features"

This pull request resolves this problem by updating the part where the metrics were being discarded.

@telegraf-tiger telegraf-tiger bot added the fix pr to fix corresponding bug label Jul 7, 2021
@sspaink sspaink requested review from akrantz01 and sjwang90 July 7, 2021 20:50
@sspaink sspaink marked this pull request as ready for review July 7, 2021 20:50
@sspaink sspaink changed the title Resolve bug with array of objects Fix json_v2 parser to handle nested objects in arrays properly Jul 7, 2021
@sjwang90
Copy link
Contributor

sjwang90 commented Jul 7, 2021

Nice!!! This worked for me

Config

[[inputs.http]]
  urls = [
    "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/significant_week.geojson"
  ]
  name_override = "earthquake"
  tagexclude = ["url", "host"]
  data_format = "json_v2"
  [[inputs.http.json_v2]]
        [[inputs.http.json_v2.object]]
            path = "features"
            timestamp_key = "properties_time"
            timestamp_format = "unix_ms"
            tags = ["properties_place"]
            # excluded_keys = ["properties_detail", "properties_url", "properties_status", "properties_ids", "properties_sources", "properties_types"]
            included_keys = ["properties_mag"]

Output:

> earthquake,properties_place=12\ km\ NNW\ of\ Kukuihaele\,\ Hawaii properties_mag=5.17 1625700002000000000
> earthquake,properties_place=4km\ NNW\ of\ Fontana\,\ CA properties_mag=3.41 1625700002000000000

@sspaink sspaink merged commit 1b20680 into influxdata:master Jul 8, 2021
reimda pushed a commit that referenced this pull request Jul 28, 2021
@Hipska Hipska added the area/json json and json_v2 parser/serialiser related label Feb 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/json json and json_v2 parser/serialiser related fix pr to fix corresponding bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants