Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logstash won’t read 1 line input file (and might miss last line of data in any file) #5481

Closed
itaydvir opened this issue Jun 13, 2016 · 3 comments
Assignees
Labels

Comments

@itaydvir
Copy link

I spent a lot of time on this issue until I found the problem:
Like many people who start using logstash, I have created 1 demo input file with 1 line of data.
When I used stdin() for the same input line, everything worked great. but when tried to read same input line from file, logstash didn't read the data.
I think it's because the single line ends with "EOF" and there is no "\n". so only when added a new empty line at the end logstash was able to read the data.

Here is the discussion thread: https://discuss.elastic.co/t/logstash-wont-read-1-line-input-file-and-might-miss-last-line-of-data-in-any-file/51855

It's a small fix that can save a lot of hours around the globe.

  • Version: 2.3.1
  • Config File:
input {
     # stdin {}
    file {
      path => "/Users/me/Applications/logstash-2.3.1/examples/json_input.1.log"
      start_position => "beginning"
    }
}
filter {

    json{
        source => "message"
    }

    geoip {
        source => "ip"
        target => "geoip"
    }
}
output {
    stdout { 
        codec => rubydebug 
    }
    elasticsearch { 
        hosts => ["localhost:9200"]
        index => "logstash-app-errors"
    }
}
  • Steps to Reproduce: just put in json_input.1.log 1 line of input (without new line at the end)
@guyboertje
Copy link
Contributor

Changes to support this are part of the Event Milling redesign. See this discussion issue
Closing

@suyograo
Copy link
Contributor

@guyboertje Can we improve our docs to add this information to help troubleshooting until this is fixed in the product?

@suyograo suyograo added the docs label Jun 13, 2016
@suyograo
Copy link
Contributor

/cc @dedemorton

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants