Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The ring configuration based on gossip memberlist #2499

Closed
iamparamvs opened this issue Aug 13, 2020 · 4 comments
Closed

The ring configuration based on gossip memberlist #2499

iamparamvs opened this issue Aug 13, 2020 · 4 comments

Comments

@iamparamvs
Copy link

Describe the bug
The configuration provided at the URL below is not working properly.
https://github.com/grafana/loki/blob/master/docs/sources/configuration/examples.md#almost-zero-dependencies-setup

The error is prompted under two sections:

  1. The first error is for the ring store specified under the distributor
  2. The second error is for the attributes "max_join_backoff, max_join_retries, min_join_backoff" under the memberlist_config

To Reproduce
Steps to reproduce the behavior:

  1. Create a configmap with the configuration for Loki
  2. Deploy Loki using manifest files
  3. Check the logs of the Loki pod

Expected behavior
The configuration is supposed to work without the errors. I change the distributor_config as follows and it worked fine:

    distributor:
      ring:
        kvstore:
          store: memberlist

Request you to update the same.

However, the configuration "max_join_backoff, max_join_retries, min_join_backoff" under memberlist_config still throws an error.

Environment:

  • Infrastructure: Openshift
  • Deployment tool: Manifest files

Screenshots, Promtail config, or terminal output

[user@host-xxx-xx-xx-xx loki-memberlist]$ oc logs loki-mani-0
failed parsing config: /etc/loki/loki.yaml: yaml: unmarshal errors:
  line 4: field store not found in type distributor.RingConfig
  line 20: field max_join_backoff not found in type memberlist.KVConfig
  line 21: field max_join_retries not found in type memberlist.KVConfig
  line 22: field min_join_backoff not found in type memberlist.KVConfig

@owen-d
Copy link
Member

owen-d commented Aug 14, 2020

Hey, thanks for this and sorry it didn't work. We plan to eventually add a process to automatically test these configs, but time is a scarce resource :)

I'll push up a fix for distributor kvstore yaml path, but the memberlist specific fields not existing is very confusing. I've just tested and they work (and it definitely looks correct in the source).

Can you paste your loki --version output as well as the entire config file you were using?

owen-d added a commit to owen-d/loki that referenced this issue Aug 14, 2020
@iamparamvs
Copy link
Author

Hey @owen-d

I think the first issue which was being faced (distributor kvstore yaml path) is being fixed by you.

However the second part of the issue(max_join_backoff, max_join_retries, min_join_backoff) is fixed after I updated my Loki version from 1.5.0 to 1.6.0. Also, I am attaching the configuration anyway, please verify that and let me know if it is a correct configuration for a MONOLITHIC deployment of Loki in a kubernetes environment.

loki.yaml

  auth_enabled: false
    chunk_store_config:
      max_look_back_period: 0s
    distributor:
      ring:
        kvstore:
          store: memberlist
    ingester:
      chunk_block_size: 262144
      chunk_idle_period: 3m
      chunk_retain_period: 1m
      lifecycler:
        ring:
          kvstore:
            store: memberlist
          replication_factor: 1
        final_sleep: 0s
      max_transfer_retries: 0
    memberlist:
      abort_if_cluster_join_fails: false
      bind_port: 7946
      join_members:
        - loki-mani-headless.loki-project.svc.cluster.local
      max_join_backoff: 1m
      max_join_retries: 10
      min_join_backoff: 1s
    limits_config:
      max_entries_limit_per_query: 10000
      enforce_metric_name: false
      reject_old_samples: true
      reject_old_samples_max_age: 168h
    schema_config:
      configs:
      - from: "2018-04-15"
        index:
          period: 168h
          prefix: index_
        object_store: filesystem
        schema: v11
        store: boltdb
    server:
      http_listen_port: 3100
    storage_config:
      boltdb:
        directory: /data/loki/index
      filesystem:
        directory: /data/loki/chunks
    table_manager:
      retention_deletes_enabled: false
      retention_period: 0s

@owen-d
Copy link
Member

owen-d commented Aug 20, 2020

Hey, the yaml you linked is incorrectly indented. Once I fixed that, I ran this locally.

@iamparamvs
Copy link
Author

Hey! This could have been a copy paste error, but other than the indentation the rest of the configuration is fine?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants