Fix #2095, Automatic suppression of flooding events

[jhnphm]

Checklist (Please check before submitting)

• I reviewed the Contributing Guide.
• I signed and emailed the appropriate Contributor License Agreement to GSFC-SoftwareRelease@mail.nasa.gov and copied cfs-program@lists.nasa.gov.

Describe the contribution

• Fixes Automatic suppression of flooding events #2095

Testing performed

1. Built with SIMULATION=i686-native ENABLE_UNIT_TESTS=true
2. Ran unit tests, all passing except time due to local time configuration changes
3. Built for VxWorks
4. Ran cFS on vxWorks and observed events being squelched and the squelch event message being emitted

Expected behavior changes
A clear and concise description of how this contribution will change behavior and level of impact.

• API Change: Squelch error code gets returned by event send functions if a squelch happens
• Behavior Change: Events get squelched after 16 events within 1 second per app (default in cpu1_platform_cfg); tokens are returned at a rate of 8/sec (default in cpu1_platform_cfg) to allow a sustained event rate of 8/sec/app. Event is emitted upon squelching, and counters incremented. Counters added to event data stored to file and realtime telemetry.

System(s) tested on

• Hardware: SP0-s (Simics)
• OS: VxWorks 6.9
• Versions: cFE 7.0.0+dev127, OSAL v6.0.0-rc4+dev66, OSAL v6.0.0-rc4+dev66 w/ JSC changes

Additional context
N/A

Third party code
N/A

Contributor Info - All information REQUIRED for consideration of pull request
John N Pham (@jhnphm), Northrop Grumman, Space Systems

[jhnphm]

CLA incoming; will have to submit to my management to fill out

[skliper]

@jhnphm We just got a new consolidated form that includes all the apps, I'll email it.

[jhnphm]

Updated unit test to improve coverage, and fixed doc errors. Updated commit message to add more detail.

[jphickey]

I think overall there are some concerns that need to be resolved here. Simplify the logic, mainly.
I didn't look too closely at the unit tests but it seems to require modification to every test case. This is definitely not ideal, I'd try to avoid that if at all possible. I would think that it could be worked around by giving each app a huge amount of credit during the common setup function (e.g. INT32_MAX if you also use my suggestion of going with a signed credit value and a decrement)

[jhnphm]

I think overall there are some concerns that need to be resolved here. Simplify the logic, mainly. I didn't look too closely at the unit tests but it seems to require modification to every test case. This is definitely not ideal, I'd try to avoid that if at all possible. I would think that it could be worked around by giving each app a huge amount of credit during the common setup function (e.g. INT32_MAX if you also use my suggestion of going with a signed credit value and a decrement)

I simplified it a bit, but still requires resetting the token count to INT32_MAX after each successful call to CFE_EVS_Register, since that resets the token counter to CFE_PLATFORM_EVS_MAX_APP_EVENT_BURST, and I had to put a case in to ignore adding credits if the existing token count was > CFE_PLATFORM_EVS_MAX_APP_EVENT_BURST, which seems messy if it's just to satisfy tests. I guess I could add a flag to CFE_EVS_Global or one of it's substructs to properly disable squelching globally at runtime (and at that point might as well make it a command), but that would add more complexity.

[jhnphm]

@skliper @jphickey Added the changes requested/bugs pointed out during today's meeting. I'm using the global EVS mutex if that's okay. Also updated docs and took a first cut at requirements in the CSV.

Not sure if the diagram included is detailed enough; it's a notional state diagram, but there really isn't a state machine in the code. I have this activity diagram but it might be too "busy":

[skliper]

Closed in favor of #2117

[jusvit]

Closed in favor of #2117

Damn skliper you really did jhnphm dirty...

[skliper]

@justinvvitale - I should have been more verbose in my closure statement. Transfer was coordinated w/ all parties involved to straighten out contribution ownership based on the various organizations and contracts involved with this work. There's significant process involved behind the scenes in some external contribution situations, and I realize the optics don't look good w/o context. :)