MAYA-104552 improve diagnostics

[pierrebai-adsk]

Improve the message batching to be usable for errors too. Also, remove
hard-coded crash when we can. We should never explicitly choose crash Maya
under the user's feet just because of some-sub-system is unhappy. Errors are
fine for those cases.

Unfortunately, there are some messages that comes from OGS and which require a
Maya fix to avoid flooding the script editor, those cannot be fixed in the
plugin. (Well, it would need investigation why the VP2 delegate does what it
does and causes those OGS errors.)

• Always accumulate and coalesce messages.
• Use a thread to periodically write messages.
• Only write messages if there are some pending and at least a second has passed.
• For low-volume messages, this will print all messages.
• For high-volumes messages, this avoids a flood of messages.

The design goes like this:

• All messages are accumulated by the diagnostic delegates.
• Another delegate tells the diagnostic message flusher when
  any message arrives.
• The diagnostic message flusher has two purposes:
  • the first purpose is to detect bursts of messages and to withhold
    further messages from being written out when a burst is detected,
  • the second purpose is to write out (flush) the messages periodically.
• The condition for flushing are either:
  • that a forced flush is requested,
  • or that fewer than a maximum consecutive messages have been received,
  • or that one second has elapsed since the last time messages were flushed.
• Flushing can either be immediate or delayed.
  • Immediate flushing is done when a forced flush is requested or when
    outside of burst of messages.
  • Delayed flushing is done when a burst of messages is detected,
    to avoid writing too many messages in the log.
• Requesting a flushing of accumulated messages is done either directly
  or indirectly.
  • Direct flushing is done when the flushing request is triggered in the
    main thread.
  • Indirect flushing is done by queuing a task to be run on-idle in the
    main thread. If a task is already queued, nothing is done, to avoid
    queuing multiple redundant tasks to do the same thing.
• The actual flushing takes (extract and removes) all accumulated messages
  and prints them in the script console via MGlobal.
• This can only be done in the main thread due to the limitations of MGlobal.
• Printing of messages is done either fully or coalesced.
  • All messages are printed fully when not in a burst.
  • All messages are printed coalesced when in a burst. Coalesced messages
    only print a sample of the message followed by "and X similar".

[vlasovi]

Looks good

[dj-mcg]

Hey folks, we've stopped receiving TF warnings and errors in our unit test log files if we don't perform a diagnostic flush on teardown. I tried setting the PIXMAYA_DIAGNOSTICS_BATCH env var to false but that didn't do the trick. I'm wondering what the method is to completely disable batching of diagnostics because in these cases it seems like we want them as they are issued - thanks!

[pierrebai-adsk]

Hi, We've changed the code so that when too many warnings or errors and issued all at once, they get batched. You can change the maximum number of messages before batching in multiple ways: * In Python, call mayaUsd.lib.SetMaximumUnbatchedDiagnostics(1000 * 1000) * In C++ use the UsdMayaDiagnosticBatchContext(1000 * 1000) * Set the environment variable MAYAUSD_MAXIMUM_UNBATCHED_DIAGNOSTICS to some similar high value. It was changed in this pull request: #2799

[pierrebai-adsk]

Oh, wait, you are right, due to the way the message are buffered now, you might indeed need a flush. We probably should register an exit callback in the diagnostic delegate do it automatically or support not batching any messages, you are right.

[dj-mcg]

Ah ok - it would be great to ensure messages are flushed on exit so we wouldn't have to introduce any new env vars or function calls to our unit tests, but being able to explicitly disable batching also seems like a good idea... Thanks!

[pierrebai-adsk]

I've looked at the code. I could try to fix it, but given that I don't have repro step to make sure I've fixed the issue in your particular case, I'll describe what could be modified. You could tried it on your side and submit a PR.

1. In the DiagnosticFlusher constructor, check PIXMAYA_DIAGNOSTICS_BATCH and make the _maximumUnbatchedDiagnostics some huge number (i.e. INT_MAX).
2. Add a Maya-Exit callback to call UsdMayaDiagnosticDelegate::Flush(). Normally it gets called when the plugin is unloaded when Maya exits, but in release builds of Maya, Maya uses what it calls fast-exit, which skips a lot of cleanups and probably explains why the flush is not automatically done. Even in fast-exit, the exit callbacks do get called, so that would ensure the messages are flushed.

I think #2 would be enough to ensure flushing in your case, but #1 could be a nice extra.

[pierrebai-adsk]

I went ahead I wrote what I described in the PR: #2830