Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement RPC to dump in memory config #666

Merged
merged 5 commits into from
Sep 28, 2015

Conversation

cosmo0920
Copy link
Contributor

Related to #633.

Add <endpoint>/api/config.dump to RPC.

e.g)

example conf:

<source>
  type http
  bind 0.0.0.0
  port 9880
  body_size_limit 32MB
  keepalive_timeout 10
  # backlog 0
  add_http_headers false
  format default
</source>

<match test>
  type stdout
</match>
<system>
  rpc_endpoint 127.0.0.1:9000
</system>

Dump API

$ curl http://127.0.0.1:9000//api/config.dump
{"ok":true}

then, got:

2015-08-28 15:45:28 +0900 [info]: dump in-memory config
2015-08-28 15:45:28 +0900 [info]: <ROOT>
  <source>
    type http
    bind 0.0.0.0
    port 9880
    body_size_limit 32MB
    keepalive_timeout 10
    add_http_headers false
    format default
  </source>
  <match test>
    type stdout
  </match>
  <system>
    rpc_endpoint 127.0.0.1:9000
  </system>
</ROOT>

via HTTP

$ curl http://127.0.0.1:9000/api/config.getDump
{"conf":"<ROOT>\n  <source>\n    type http\n    bind 0.0.0.0\n    port 9880\n    body_size_limit 32MB\n    keepalive_timeout 10\n    add_http_headers false\n    format default\n  </source>\n  <match test>\n    type stdout\n  </match>\n  <system>\n    rpc_endpoint 127.0.0.1:9000\n  </system>\n</ROOT>\n","ok":true}

@sonots
Copy link
Member

sonots commented Aug 28, 2015

Do we really need two ways to dump?

@cosmo0920
Copy link
Contributor Author

I'm not sure for now.

I think that dump in-memory config in log is really needed.
But, I think that via HTTP version needs to pay attention to implement. (detailed as below)

At least, both ways makes easy to dump in-memory configurations.
Two ways to dump has different advantages and disadvantages.

config.dump

Advantage

  • Display in-memory config with pretty print

Disadvantage

  • Need to login or open terminal where fluentd running

config.getDump

Advantage

  • Be able to get dump without login or open terminal where fluentd running

Disadvantage

  • Too easy to see in-memory config when someone know Fluentd RPC endpoints (maybe violate security???)

@tagomoris
Copy link
Member

In my opinion, we can monitor Fluentd configuration from remote point continuously with config.getDump. It's helpful to find Fluentd node/process which are running with old/outdated configuration.

If all plugins uses :secret feature to mask sensitive values, there seems no security risk. But right now, we can't expect it.
I think that config.getDump should be disabled in default, and it's better to be enabled by command line option or <system> configuration. Fluentd v0.14 may be able to enable it in default.

@cosmo0920
Copy link
Contributor Author

I think that config.getDump should be disabled in default, and it's better to be enabled by command line option or configuration. Fluentd v0.14 may be able to enable it in default.

👍
I think so, too.
I want to toggle enable/disable it by <system> configuration.
Because using <system> configuration has a consistency for other dump config implementation.

@repeatedly
Copy link
Member

@sonots @tagomoris If you are Okay, please merge this PR.

@sonots
Copy link
Member

sonots commented Sep 14, 2015

👍

@repeatedly
Copy link
Member

Can we merge this?

repeatedly added a commit that referenced this pull request Sep 28, 2015
@repeatedly repeatedly merged commit 2223fa4 into fluent:master Sep 28, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants