-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tail plug-in fails to keep track of state of file when parser is used #499
Comments
Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>
Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>
@2x-bullet-time thanks for raising this issue. I did some local tests and the problem that I found in in_tail plugin, is that if a parser fails against a log line (bad regex), that content is missed, the right behavior is to keep that information in "raw text" mode. I placed a fix for that behavior in GIT master and 0.12 branches. |
Thank you for your work. I will give it a try! |
@edsiper
This is the sosreport associated with the above configuration file:
This is the sosreport associated with the above command:
in_tail plugin does not keep track of the state of the file when parser is used, either in command or config-file mode. This is the sosreport associated with a configuration file with parser enabled:
Thanks! |
@edsiper
However, it fails to detect it in these scenario(config file without parser, config file with parser, command with parser) according to this log:
Thanks! |
I cannot reproduce the problem you have stated. This is my test log file (apache.log):
1. Parser + DB:$ fluent-bit -R parsers.conf -i tail -p path=apache.log -p parser=apache -p db=test.db -o stdout -f 1
Fluent-Bit v0.13.0
Copyright (C) Treasure Data
[2018/02/06 11:27:11] [ info] [engine] started
[0] tail.0: [1517937781.000000000, {"host"=>"213.217.69.115", "user"=>"-", "method"=>"GET", "path"=>"/churchy", "code"=>"204", "size"=>"2216", "referer"=>"", "agent"=>""}]
[1] tail.0: [1517937781.000000000, {"host"=>"193.56.73.36", "user"=>"-", "method"=>"GET", "path"=>"/signable", "code"=>"500", "size"=>"2216", "referer"=>"", "agent"=>""}]
[2] tail.0: [1517937781.000000000, {"host"=>"18.240.253.3", "user"=>"-", "method"=>"GET", "path"=>"/beclamour", "code"=>"500", "size"=>"2216", "referer"=>"", "agent"=>""}]
[3] tail.0: [1517937781.000000000, {"host"=>"205.219.243.165", "user"=>"-", "method"=>"GET", "path"=>"/mowe", "code"=>"200", "size"=>"2216", "referer"=>"", "agent"=>""}]
[4] tail.0: [1517937781.000000000, {"host"=>"183.43.42.232", "user"=>"-", "method"=>"GET", "path"=>"/Deimos", "code"=>"200", "size"=>"2216", "referer"=>"", "agent"=>""}]
[5] tail.0: [1517937781.000000000, {"host"=>"30.30.47.28", "user"=>"-", "method"=>"GET", "path"=>"/basify", "code"=>"204", "size"=>"2216", "referer"=>"", "agent"=>""}]
[6] tail.0: [1517937781.000000000, {"host"=>"106.165.252.167", "user"=>"-", "method"=>"GET", "path"=>"/triphase", "code"=>"400", "size"=>"2216", "referer"=>"", "agent"=>""}]
[7] tail.0: [1517937781.000000000, {"host"=>"204.168.117.164", "user"=>"-", "method"=>"GET", "path"=>"/self-contentedly", "code"=>"200", "size"=>"2216", "referer"=>"", "agent"=>""}]
[8] tail.0: [1517937781.000000000, {"host"=>"251.244.238.241", "user"=>"-", "method"=>"GET", "path"=>"/overblessedness", "code"=>"400", "size"=>"2216", "referer"=>"", "agent"=>""}]
[9] tail.0: [1517937781.000000000, {"host"=>"119.96.224.125", "user"=>"-", "method"=>"GET", "path"=>"/senhora", "code"=>"404", "size"=>"2216", "referer"=>"", "agent"=>""}] 2. Only DB (no parser):
no data output, because database offset is being used. 3. Only Parser (no database, expected to read from the beginning)$ fluent-bit -R parsers.conf -i tail -p path=apache.log -p parser=apache -o stdout -f 1
Fluent-Bit v0.13.0
Copyright (C) Treasure Data
[2018/02/06 11:29:05] [ info] [engine] started
[0] tail.0: [1517937781.000000000, {"host"=>"213.217.69.115", "user"=>"-", "method"=>"GET", "path"=>"/churchy", "code"=>"204", "size"=>"2216", "referer"=>"", "agent"=>""}]
[1] tail.0: [1517937781.000000000, {"host"=>"193.56.73.36", "user"=>"-", "method"=>"GET", "path"=>"/signable", "code"=>"500", "size"=>"2216", "referer"=>"", "agent"=>""}]
[2] tail.0: [1517937781.000000000, {"host"=>"18.240.253.3", "user"=>"-", "method"=>"GET", "path"=>"/beclamour", "code"=>"500", "size"=>"2216", "referer"=>"", "agent"=>""}]
[3] tail.0: [1517937781.000000000, {"host"=>"205.219.243.165", "user"=>"-", "method"=>"GET", "path"=>"/mowe", "code"=>"200", "size"=>"2216", "referer"=>"", "agent"=>""}]
[4] tail.0: [1517937781.000000000, {"host"=>"183.43.42.232", "user"=>"-", "method"=>"GET", "path"=>"/Deimos", "code"=>"200", "size"=>"2216", "referer"=>"", "agent"=>""}]
[5] tail.0: [1517937781.000000000, {"host"=>"30.30.47.28", "user"=>"-", "method"=>"GET", "path"=>"/basify", "code"=>"204", "size"=>"2216", "referer"=>"", "agent"=>""}]
[6] tail.0: [1517937781.000000000, {"host"=>"106.165.252.167", "user"=>"-", "method"=>"GET", "path"=>"/triphase", "code"=>"400", "size"=>"2216", "referer"=>"", "agent"=>""}]
[7] tail.0: [1517937781.000000000, {"host"=>"204.168.117.164", "user"=>"-", "method"=>"GET", "path"=>"/self-contentedly", "code"=>"200", "size"=>"2216", "referer"=>"", "agent"=>""}]
[8] tail.0: [1517937781.000000000, {"host"=>"251.244.238.241", "user"=>"-", "method"=>"GET", "path"=>"/overblessedness", "code"=>"400", "size"=>"2216", "referer"=>"", "agent"=>""}]
[9] tail.0: [1517937781.000000000, {"host"=>"119.96.224.125", "user"=>"-", "method"=>"GET", "path"=>"/senhora", "code"=>"404", "size"=>"2216", "referer"=>"", "agent"=>""}] I don't see any issue. |
I gave it another try using your apache log and here is the result: I can now isolate parser from contributing to the problem. My regex was missing an end of line. Opps :/ in_tail plugin behaves properly when it is invoked via command line. However, it does not utilize offset when it is executed using a configuration file. Here is the configuration file (db + parser):
The same behavior is observed when parser is commented out. |
This might be a system/conf specific problem. I used the same set up on my ubuntu VM and it was working just fine in either mode. I will close this issue for now. Thank you. |
Hello :) |
@2x-bullet-time once compiled in your armv7l, do you still face the issue ? |
@edsiper |
@2x-bullet-time ok, so there are two concerns:
I will focus on #2. would you please specify the kernel version and linux distribution that device/machine uses ? |
@edsiper armv7l machine works properly if parameters are inserted as a part of the command as opposed to a config file. Is tail still an issue? The kernel+dist come with a custom board. The kernel is a custom variation of 4.1.15 and distro is Debian GNU/Linux 8.10 (jessie). I think I am out of luck here. |
@2x-bullet-time it's preferred to set all parameters either in the config file or in the command line only, avoid to mix them up (only use config file). |
@edsiper Yea. I tested it with all parameters in config file vs. in command. |
@2x-bullet-time to clarify: when you are using the configuration file, do you see an unexpected behavior ? |
@edsiper Yes. The unexpected behavior is observed when a configuration file is used. This behavior is not observed on the ubuntu vm with the exact same configuration set up. |
please paste "command line arguments" that works and the configuration file content.. |
command line arguments:
configuration file content:
|
@2x-bullet-time do you know if the problem can be reproduced in a Raspberry Pi or similar ? |
@edsiper Hello. I am able to reproduce this problem on a bbb. Here is the sosreport from the exact same setup:
|
bbb == beaglebone black ?, if so, which OS version (image URL?) |
Yea. It is a beaglebone black. /etc/dogtag gives:
I checked BeagleBoard.org and this is the only image corresponds to that date: |
Signed-off-by: Eduardo Silva <eduardo@treasure-data.com>
I've tried to replicate without success the issue you just described in a beagle bone black board using the debian image you have provided. The database file offsets are set properly (except when Fluent Bit don't have write access to /var/log/... but a message is triggered) would you please check if you can reproduce the problem with Fluent Bit 0.14.1 version ? |
Closing due to timeout. |
Signed-off-by: Sanya Kochhar <kochhars@microsoft.com>
I have noticed tail plugin does not obey the state of the file when a parser is defined. Plugin will always start at the head of the file unless parser is blanked out. This issue can also be replicated using command line:
fluent-bit --parser=parsers.conf -i tail -p path=/var/log/syslog -p db=/var/log/xyz.sqlite -p parser=syslog-custom -o stdout
This command will work as expected if parser parameter is taken out.
This is my sosreport:
I am hoping I have done something wrong on my end. Thanks!
The text was updated successfully, but these errors were encountered: