Plugin creates the file but does not append alerts

[zbalkan]

The application creates the alert file but it does not append the alerts.

-rw-r--r--   1 nobody nobody     0 Jan 24 14:35 crowdsec_alerts.json

When I check the logs with cat /var/log/crowdsec.log | grep notif command, I saw that the plugin process is exited.

time="24-01-2023 16:15:24" level=debug msg="starting plugin" args="[/usr/lib64/crowdsec/plugins/notification-file]" path=/usr/lib64/crowdsec/plugins/notification-file
time="24-01-2023 16:15:24" level=debug msg="plugin started" path=/usr/lib64/crowdsec/plugins/notification-file pid=172371
time="24-01-2023 16:15:24" level=debug msg="waiting for RPC address" path=/usr/lib64/crowdsec/plugins/notification-file
time="24-01-2023 16:15:24" level=debug msg="plugin process exited" path=/usr/lib64/crowdsec/plugins/notification-file pid=172336

but htop shows that process is running, yet does not use any CPU resources. There's something wrong with this setup.

[zbalkan]

Hi @LaurenceJJones, do you have any idea?

[xFlum3]

Hey @zbalkan ,
I installed notification-file plugin for Crowdsec and after a while the logs come like a 3 of them,
Today i checked with BF simulation and i can see that the file not appending the new logs but when i check Crowdsec Dashboard i can see the alert, please advice solving this issue to make Crowdsec sending logs into wazuh

[LaurenceJJones]

Just for context the above user also brought this to my attention and I been putting of creating an official plugin, so just to make it easier for everyone here a wip official plugin

crowdsecurity/crowdsec#2932

The TLDR; I made it compatible with this current plugin so it will be a "drop in" replacement, key difference is im not using logrus dependency since it was mostly a hack since originally I didnt want to write rotation by scratch

[zbalkan]

At last, his project will be finalized. It was already done by @LaurenceJJones mostly. I'll be glad to archive this repo as soon as the said PR is merged.

[xFlum3]

Hey @zbalkan ,
So crowdsec published an new version for notification-file plugin and its fixed the appending logs issue,
But now i got an issue with wazuh that filebeat doesnt indexing the alerts:

Do you maybe know how can i fix it ?

[LaurenceJJones]

Hey @zbalkan wonder if we can tap into your wazuh experience, the user above is getting an error on filebeat that shows the original JSON but is complaining about

object mapping for [data.program] tried to parse field [program] as object, but found a concrete value

dont suppose you ever encountered this? know if off topic to original issue but im clueless for wazuh problems 😆

more info: wazuh community support is suggestin we alter the log format to be syslog compatible but imo that not the problem the problem seems to be when the data is ingested on elastic side its not decoding properly

more info: they linked us to an upstream issue

[zbalkan]

Hi. Can you please let me know the configuration and overall setup you used so that I can reproduce the issue?

[xFlum3]

Hey @zbalkan ,
I am using wazuh version 4.7.3, this is my server properties:

Wazuh Agent config:

Rule File:

Wazuh About:

please let me know which configuration you need ?

[zbalkan]

Agent and rule are enough. I'll try to recreate and let you know.

[LaurenceJJones]

Just for some digging on the upstream issue people are complaining that the indexer on elastic might be complaining when a json has different values for different objects.

We did create this index map for elastic using http plugin https://docs.crowdsec.net/docs/next/notification_plugins/elastic#potential-mapping-issues

However, I dont know if the filebeat agent on the machine is not doing something correct 🤷🏻 but like I said im like a fish out of water here just throwing information hoping something sticks

[xFlum3]

Hey @zbalkan , thanks for helping waiting for your response.

[LaurenceJJones]

Agent and rule are enough. I'll try to recreate and let you know.

Wazuh community support managed to find the issue is within filebeat as program is a reserved top level key, changing the value to anything other than program resolved it. So I guess the difference was @xFlum3 used filebeat and you use the wazuh agent which doesnt create this conflict.

Do you know in the official plugin if I can choose a different value and we can use that within wazuh filter?

[zbalkan]

I believe it is just easier if we use a root with different name like "crowdsec" : {...}, that would solve all conflicts. It'd also make rule creating easier.

[LaurenceJJones]

I believe it is just easier if we use a root with different name like "crowdsec" : {...}, that would solve all conflicts. It'd also make rule creating easier.

Awesome! If this can be showcased in wazuh configuration example, then I can make sure to update the default format and include examples in the crowdsec docs

[zbalkan]

I currently don't have a setup in my sandbox. I can try to collect with a new lab environment tomorrow. It'd be easier as Crowdsec already has correlated the events. At that point, it's just forwarding the alert as is, as all of them are important. If we need to work with different levels of severity, we can just create multiple rules and it'd be fine. That was what I intended since the beginning.