Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Wrap the
normalize_file
mongo operation inside a try/catch. This ac…
…complishes two things: (#2483) 1. It improves process log output, making any follow-on troubleshooting straightforward. We go from a generic OperationFailure log, which makes it look like the analysis doc was too big: ``` poetry[1072529]: 2025-01-31 11:24:41,229 [Task 1] [modules.reporting.mongodb] ERROR: Got MongoDB OperationFailure, code 10334 poetry[1072529]: 2025-01-31 11:24:41,229 [Task 1] [modules.reporting.mongodb] WARNING: Deleting behavior process tree children from results. ``` To this, which actually indicates that the hook failed and why: ``` poetry[1108630]: 2025-01-31 15:25:55,981 [Task 2] [dev_utils.mongo_hooks] ERROR: Mongo hook 'normalize_files' failed with code 10334: BSONObj size: 18852184 (0x11FA958) is invalid. Size must be between 0 and 16793600(16MB) First element: q: { _id: "...461a8" }, full error: {'ok': 0.0, 'errmsg': 'BSONObj size: 18852184 (0x11FA958) is invalid. Size must be between 0 and 16793600(16MB) First element: q: { _id: "...461a8" }', 'code': 10334, 'codeName': 'BSONObjectTooLarge'} ``` 2. If the `normalize_file` bulk write operation errors out, it no longer gets caught by the exception handler in `modules/reporting/mongodb.MongoDB.run` leading to the behaviour process tree children being removed from the analysis report (See logs above - not good!). What this does not fix, however, is the problem with large `strings` data that prevented it from being written to the `files` collection. It does provide a lot more visibility though, in addition to preventing the analysis document from being arbitrarily cut-down if an error does occur in the hook. On the topic of other hooks, there are probably other hooks that require the same treatment, but this is a good place to start.
- Loading branch information