Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

emit backtrace on UNRECOVERABLE_ERRORS #2149

Merged

Conversation

Etsukata
Copy link
Contributor

UNRECOVERABLE_ERRORS are thought to be rather critical, so we will
benefit from backtrace output.

Originally emitted in fluentd v1.3 or earlier, but suppressed in
c011256bec95eca

Signed-off-by: Eiichi Tsukata devel@etsukata.com

UNRECOVERABLE_ERRORS are thought to be rather critical, so we will
benefit from backtrace output.

Originally emitted in fluentd v1.3 or earlier, but suppressed in
fluent@c011256bec95eca

Signed-off-by: Eiichi Tsukata <devel@etsukata.com>
@Etsukata
Copy link
Contributor Author

Example output log

  • before
2018-10-09 11:52:13 +0900 [warn]: #0 got unrecoverable error in primary and no secondary error_class=NoMethodError error="undefined method `has_key?' for #<String:0x00007f0ee9f4bf60>" 
2018-10-09 11:52:13 +0900 [warn]: #0 bad chunk is moved to /tmp/fluent/backup/worker0/object_3f8775b4a618/577c2d31e98de1fc80472d4d2edd9c7c.log
  • after
2018-10-11 09:00:04 +0900 [warn]: #0 got unrecoverable error in primary and no secondary error_class=NoMethodError error="undefined method `has_key?' for #<String:0x00007fdd141c22a0>" 
  2018-10-11 09:00:04 +0900 [warn]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-elasticsearch-2.10.3/lib/fluent/plugin/elasticsearch_error_handler.rb:66:in `block in handle_error'
  2018-10-11 09:00:04 +0900 [warn]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/event.rb:323:in `each'
  2018-10-11 09:00:04 +0900 [warn]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/event.rb:323:in `block in each'
  2018-10-11 09:00:04 +0900 [warn]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/plugin/buffer/memory_chunk.rb:80:in `open'
  2018-10-11 09:00:04 +0900 [warn]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/plugin/buffer/memory_chunk.rb:80:in `open'
  2018-10-11 09:00:04 +0900 [warn]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/event.rb:322:in `each'
  2018-10-11 09:00:04 +0900 [warn]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-elasticsearch-2.10.3/lib/fluent/plugin/elasticsearch_error_handler.rb:27:in `handle_error'
  2018-10-11 09:00:04 +0900 [warn]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-elasticsearch-2.10.3/lib/fluent/plugin/out_elasticsearch.rb:516:in `send_bulk'
  2018-10-11 09:00:04 +0900 [warn]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-elasticsearch-2.10.3/lib/fluent/plugin/out_elasticsearch.rb:414:in `write'
  2018-10-11 09:00:04 +0900 [warn]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/plugin/output.rb:1099:in `try_flush'
  2018-10-11 09:00:04 +0900 [warn]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/plugin/output.rb:1379:in `flush_thread_run'
  2018-10-11 09:00:04 +0900 [warn]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/plugin/output.rb:440:in `block (2 levels) in start'
  2018-10-11 09:00:04 +0900 [warn]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.2.2/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
2018-10-11 09:00:04 +0900 [warn]: #0 bad chunk is moved to /tmp/fluent/backup/worker0/object_3fee8de6d690/577e8a7043da108ac692cde87efa8b6d.log

@repeatedly repeatedly merged commit 8ed8aeb into fluent:master Nov 9, 2018
@repeatedly
Copy link
Member

Merged. Thanks!

@Etsukata Etsukata deleted the emit-backtrace-on-unrecoverable-errors branch November 14, 2018 09:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants