Launcher.db growth

[wkleinhenz2]

Currently we run launcher on multiple windows 10 workstations and noticed that the size of the launcher.db grow very large, more than 100+ gigs. I saw in an old PR that the DB should start overwriting after 500,000 entries but it doesn't appear that this or another log rotation is occurring.
launcher version is 0.10.1 and the fleet version is currently 2.2.0 but for a time was 2.1.2 or 2.1.1

i can provide log files from one of the systems on request

[directionless]

That sounds pretty weird. Sure, can you send me the log file? (I'm seph on the osquery slack)

I probably need to familiarize myself with this part of the code base. I think I probably need to make some tools to help understand what'a in one of those files

[directionless]

relates to #70

[directionless]

Looking at these logs, I see some instances of #445

I also see a lot of:

{"caller":"extension.go:548","limit":2097152,"msg":"dropped log","severity":"info","size":7065204,"ts":"2019-08-01T12:17:05.8433136Z"}

I wonder if your configs are producing more logs than you can push over your fleet connection. (eg: is there an issue there:) I also wonder if #445 is a problem here, or mearly noise.

[directionless]

(I've been chatting a lot with @wkleinhenz2 on slack)

My theory is that launcher limits to 500k entries, so if the query logs are large, it's easy to hit 100gb. I think this is borne by the dropped log messages.

Talking with @wkleinhenz2 it sounds like there was an accidental cross join in their queries.

I'm closing this for now

[wkleinhenz2]

Would it be unreasonable to make a feature request to have the launcher database rotate based on size in addition to the number of entries. while this was user error on my part having something like this could cause issues in the future.