fluent-bit-plugin: Auto add Kubernetes labels to Loki labels #1204
Conversation
|
|
There was a problem hiding this comment.
I like this change, but I think this should be a root configuration not an extension of the line format that is used to format the actual line of a record not its label.
If you can refactor to use a new property a boolean for instance.
auto_kubernetes_labels boolean to automatically set kubernetes metadata on each records, may be ignore labelMap, LabelKeys, RemoveKeys.
Why wasn't the labelMap not working for you ?
|
@cyriltovena Thanks your advice. I had already refactored the code. The |
cmd/fluent-bit/README.md
Outdated
| @@ -16,6 +16,7 @@ This plugin is implemented with [Fluent Bit's Go plugin](https://github.com/flue | |||
| | Labels | labels for API requests. | {job="fluent-bit"} | | |||
| | LogLevel | LogLevel for plugin logger. | "info" | | |||
| | RemoveKeys | Specify removing keys. | none | | |||
| | AutoKubernetesLabels | If set to ture, it will auto parse kubernetes metadata into labels | false | | |||
There was a problem hiding this comment.
| | AutoKubernetesLabels | If set to ture, it will auto parse kubernetes metadata into labels | false | | |
| | AutoKubernetesLabels | If set to true, it will add all kubernetes labels to loki labels | false | |
cmd/fluent-bit/loki.go
Outdated
| kuberneteslbs := model.LabelSet{} | ||
| replacer := strings.NewReplacer("/", "_", ".", "_", "-", "_") | ||
| for k, v := range records["kubernetes"].(map[interface{}]interface{}) { | ||
| if k.(string) == "labels" { |
There was a problem hiding this comment.
could you use a switch instead
switch key := k.(string) {
case "labels":
....
case "docker_id", "pod_id","annotations":
continue
case default:
....
}
cmd/fluent-bit/README.md
Outdated
| @@ -27,6 +28,10 @@ Labels are used to [query logs](../../docs/logql.md) `{container_name="nginx", c | |||
|
|
|||
| You can use `Labels`, `RemoveKeys` , `LabelKeys` and `LabelMapPath` to how the output plugin will perform labels extraction. | |||
|
|
|||
| ### AutoKubernetesLabels | |||
|
|
|||
| If set to ture, it will parser kubernetes metadata into labels automatically and ignore the paramater `LabelKeys`, LabelMapPath. | |||
There was a problem hiding this comment.
| If set to ture, it will parser kubernetes metadata into labels automatically and ignore the paramater `LabelKeys`, LabelMapPath. | |
| If set to true, it will add Kubernetes metadata into Loki labels automatically and ignore parameters `LabelKeys`, LabelMapPath. |
|
I like this feature, good job @allanhung. |
|
@cyriltovena I had fixed the typo and use switch instead of if, I also run go fmt. Thanks for your suggestion. |
|
if you have time to rebase to master let me know. |
|
@cyriltovena Rebase has been done. |
What this PR does / why we need it:
It is too complex to change LabelMap every time when we deploy appliction in new labels. If you install 3rd add-on, you even don't know what the labels are. This PR provides a way to automatic parse the metadata when you set lineformat with "kubernetes".
Checklist