Fix config validation failures caused by NVTX pipeline wrappers #11460
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
When loading pipelines from a config file, the arguments passed to individual pipeline components are validated by
pydantic
during init. For this, the validation model attempts to parse the function signature of the component's c'tor/entry point in order to check if all mandatory parameters are present in the config file.When using
models_and_pipes_with_nvtx_range
as anafter_pipeline_creation
callback, the methods of all pipeline components get replaced by an NVTX range wrapper before the above-mentioned validation takes place. This can be problematic for components that are implemented as Cython extension types - if the extension type is not compiled with Python bindings for its methods, they will have no signatures at runtime. This resulted inpydantic
validating the wrapper's parameters against those in the config, which raised errors.To avoid this, we no longer apply the wrapper to (Cython) methods that do not have signatures.
This PR also enables bindings for both the
Pipe
andTrainablePipe
Cython classes, thereby adding support for methods that aren't overridden by their subclasses.Types of change
Bug fix
Checklist