feat(pyrt)!: import_function materializer support
#699
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Yohe-Am
commented
Apr 29, 2024
|
|
||
| # Look for a spec under a certain module name | ||
| # https://peps.python.org/pep-0302/ | ||
| # https://peps.python.org/pep-0451/ |
There was a problem hiding this comment.
What I'm seeing in these PEPs and the docs imply the limitation/breaking change I've outlined in the PR desc is not necessary but I was not able to get it working despite hours.
Some context links:
| @@ -1,9 +1,10 @@ | |||
| from nested.dep import hello | |||
| from .nested.dep import hello | |||
There was a problem hiding this comment.
Here's the breaking change.
import_function materializer supportimport_function materializer support
zifeo
reviewed
Apr 29, 2024
731fa3e to
6019011
Compare
3 tasks
1ccf64c to
3733d53
Compare
Yohe-Am
added a commit
that referenced
this pull request
May 7, 2024
- Rewrites the PythonRuntime host using a `componentize-py` based component. - Leaf through this [memo](https://hackmd.io/@SC-qT-WXTROceKYdNA-Lpg/ryyAXiQlC/edit) for a mental model. Todo: - [x] `PythonRuntime.import_` support #699 - [x] ~~Add `pyrt.wasm` to release job~~ obviated by `build.rs` Items for other PRs: - Implemen `hostcall` #### Motivation and context MET-404. #### Migration notes _No end-user changes required_ ### Checklist - [x] The change come with new or modified tests - [x] Hard-to-understand functions have explanatory comments - [ ] End-user documentation is updated to reflect the change --------- Co-authored-by: afmika <afmichael73@gmail.com>
Yohe-Am
added a commit
that referenced
this pull request
May 22, 2024
wip: python mdk feat: represent union, either with Union feat: solve cycles, order python types by priority fix: self-reference fix: premature checks docs: add a comparison b/n metatype and other similar solutions/products. (#697) <!-- Pull requests are squash merged using: - their title as the commit message - their description as the commit body Having a good title and description is important for the users to get readable changelog. --> <!-- 1. Explain below WHAT the change is --> - Adds a comparison table between metatype and other similar services. - Add artifact upload protocol to `Architecture` section in docs. <!-- 2. Explain below WHY the change cannot be made simpler --> <!-- 3. Explain below WHY the was made or link an issue number --> [MET-443](https://linear.app/metatypedev/issue/MET-443/include-comparisons-with-other-products-similar-to-metatype) <!-- 4. Explain HOW users should update their code or remove that section --> _No Migration Needed_ <!-- 5. Readiness checklist - [ ] The change come with new or modified tests - [ ] Hard-to-understand functions have explanatory comments - [ ] End-user documentation is updated to reflect the change --> refactor(gate): wasi 0.2 pyrt (#687) - Rewrites the PythonRuntime host using a `componentize-py` based component. - Leaf through this [memo](https://hackmd.io/@SC-qT-WXTROceKYdNA-Lpg/ryyAXiQlC/edit) for a mental model. Todo: - [x] `PythonRuntime.import_` support #699 - [x] ~~Add `pyrt.wasm` to release job~~ obviated by `build.rs` Items for other PRs: - Implemen `hostcall` MET-404. _No end-user changes required_ - [x] The change come with new or modified tests - [x] Hard-to-understand functions have explanatory comments - [ ] End-user documentation is updated to reflect the change --------- Co-authored-by: afmika <afmichael73@gmail.com> cleanups fix: pre-commit fix: forward-refs fix: top level simple types test(metagen): python mdk cleanups and small fixes feat(metagen): codegen decorator and some cleanups feat(metagen): always merge defs if refered file is the same fix(metagen): prioritize relto typegraph path only if base.path is empty feat: Artifact removal (#668) - Add GC: remove artifacts when unreferenced by any deployed typegraph - Improve resource management: use `AsyncDisposable` and `AsyncDisposableStack` - Improve testability (for parallel testing): always read the tmpDir config from the `Typegate` object [MET-433](https://linear.app/metatypedev/issue/MET-433/file-removal) _N/A_ - [x] The change come with new or modified tests - [x] Hard-to-understand functions have explanatory comments - [x] End-user documentation is updated to reflect the change <!-- This is an auto-generated comment: release notes by coderabbit.ai --> - **New Features** - Enhanced search functionality with the addition of a new search bar. - Introduced new test configurations to improve script execution. - Updated artifact storage documentation to clarify management processes. - Added new extensions to support improved code commenting. - **Bug Fixes** - Removed outdated Deno import mapping settings to streamline development environment setup. - **Documentation** - Expanded documentation on artifact tracking and management, including reference counting and garbage collection mechanisms. - **Refactor** - Implemented interface changes in `QueryEngine` for better async disposal management. - Code restructuring in artifact management for enhanced performance and maintainability. - **Chores** - Adjusted settings and configurations in the development environment to align with current best practices. - **Tests** - Introduced new test cases for artifact upload and management functionalities. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Natoandro <anatoandro@hotmail.com> Co-authored-by: destifo <estifanosbireda@gmail.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> chore(release): prepare 0.4.0 (#710) Bumps version to release 0.4.0. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> - **New Features** - Updated the software across various components to version 0.4.0, enhancing functionality and potentially introducing new features or fixes. - **Documentation** - Updated version documentation in multiple configuration files to reflect new version 0.4.0. - **Bug Fixes** - Adjusted version constants and dependencies to ensure compatibility and stability with the new software version 0.4.0. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Natoandro <anatoandro@hotmail.com> chore(docs): final polish to comparison table. (#709) some changes to comparison table(docs) _No Migrations Needed_ <!-- 5. Readiness checklist - [ ] The change come with new or modified tests - [ ] Hard-to-understand functions have explanatory comments - [ ] End-user documentation is updated to reflect the change --> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> - **Documentation** - Introduced a new section on Artifact Tracking Protocol in the architecture documentation, explaining artifact classification and tracking modes in Metatype. - Updated comparisons documentation with additional platforms, criteria for choosing Metatype, and detailed feature comparison tables. - Renamed project directory for clarity and consistency in project setup documentation. - **Bug Fixes** - Removed outdated `TODO` comment in installation documentation. <!-- end of auto-generated comment: release notes by coderabbit.ai --> chore: bump to version 0.4.1-0 (#713) - Bumps version to 0.4.1-0. - Fixes broken release CI. - #719 - Adds 20 minutes to test-full timeout. <!-- 5. Readiness checklist - [ ] The change come with new or modified tests - [ ] Hard-to-understand functions have explanatory comments - [ ] End-user documentation is updated to reflect the change --> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> - **New Features** - Updated platform support for better compatibility with "x86_64-linux". - **Bug Fixes** - Minor version updates across multiple configurations to enhance stability. - **Chores** - Updated version numbers from "0.4.0" to "0.4.1-0" across various files and configurations. - **Refactor** - Adjusted build and test scripts for improved efficiency and compatibility. - **Documentation** - Enhanced internal documentation to reflect version and platform changes. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Yohe-Am <56622350+Yohe-Am@users.noreply.github.com> feat: polish documentation and project (#696) <!-- Pull requests are squash merged using: - their title as the commit message - their description as the commit body Having a good title and description is important for the users to get readable changelog. --> <!-- 1. Explain below WHAT the change is --> - update the headline, the overviews and many other documentation areas - upgrades the dependencies. <!-- 2. Explain below WHY the change cannot be made simpler --> <!-- 4. Explain HOW users should update their code or remove that section --> - [ ] The change come with new or modified tests - [ ] Hard-to-understand functions have explanatory comments - [x] End-user documentation is updated to reflect the change <!-- This is an auto-generated comment: release notes by coderabbit.ai --> - **Bug Fixes** - Updated Docker image version for the `typegate` service to ensure stability and compatibility. - **Documentation** - Revised `TAGLINE` for better clarity on supported languages: WASM, Typescript, and Python. - Updated version declarations for improved consistency and functionality across multiple files. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Teo Stocco <zifeo@users.noreply.github.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> feat(sdk): expose metagen to `typegraph/sdk` (#718) Expose metagen features to `typegraph/sdk` Depends on #707 and #696 None <!-- This is an auto-generated comment: release notes by coderabbit.ai --> - **New Features** - Introduced new `Metagen` class for code generation tasks in both TypeScript and Python SDKs. - Added functionality for defining policies and structures for deployment examples using Node.js. - **Improvements** - Enhanced `Metagen` class with methods for simulating and executing code generation tasks. - Simplified file reading and writing functions for better performance and maintainability. - **Bug Fixes** - Refined `compress_and_encode` function to streamline file handling processes. - **Tests** - Added comprehensive tests for `Metagen` functionality in both TypeScript and Python SDKs. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Teo Stocco <zifeo@users.noreply.github.com> Co-authored-by: Teo Stocco <teo@zifeo.com> Co-authored-by: Teo Stocco <zifeo@users.noreply.github.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Estifanos Bireda <77430541+destifo@users.noreply.github.com> Co-authored-by: Yohe-Am <56622350+Yohe-Am@users.noreply.github.com>
Yohe-Am
added a commit
that referenced
this pull request
May 22, 2024
wip: python mdk feat: represent union, either with Union feat: solve cycles, order python types by priority fix: self-reference fix: premature checks docs: add a comparison b/n metatype and other similar solutions/products. (#697) <!-- Pull requests are squash merged using: - their title as the commit message - their description as the commit body Having a good title and description is important for the users to get readable changelog. --> <!-- 1. Explain below WHAT the change is --> - Adds a comparison table between metatype and other similar services. - Add artifact upload protocol to `Architecture` section in docs. <!-- 2. Explain below WHY the change cannot be made simpler --> <!-- 3. Explain below WHY the was made or link an issue number --> [MET-443](https://linear.app/metatypedev/issue/MET-443/include-comparisons-with-other-products-similar-to-metatype) <!-- 4. Explain HOW users should update their code or remove that section --> _No Migration Needed_ <!-- 5. Readiness checklist - [ ] The change come with new or modified tests - [ ] Hard-to-understand functions have explanatory comments - [ ] End-user documentation is updated to reflect the change --> refactor(gate): wasi 0.2 pyrt (#687) - Rewrites the PythonRuntime host using a `componentize-py` based component. - Leaf through this [memo](https://hackmd.io/@SC-qT-WXTROceKYdNA-Lpg/ryyAXiQlC/edit) for a mental model. Todo: - [x] `PythonRuntime.import_` support #699 - [x] ~~Add `pyrt.wasm` to release job~~ obviated by `build.rs` Items for other PRs: - Implemen `hostcall` MET-404. _No end-user changes required_ - [x] The change come with new or modified tests - [x] Hard-to-understand functions have explanatory comments - [ ] End-user documentation is updated to reflect the change --------- Co-authored-by: afmika <afmichael73@gmail.com> cleanups fix: pre-commit fix: forward-refs fix: top level simple types test(metagen): python mdk cleanups and small fixes feat(metagen): codegen decorator and some cleanups feat(metagen): always merge defs if refered file is the same fix(metagen): prioritize relto typegraph path only if base.path is empty feat: Artifact removal (#668) - Add GC: remove artifacts when unreferenced by any deployed typegraph - Improve resource management: use `AsyncDisposable` and `AsyncDisposableStack` - Improve testability (for parallel testing): always read the tmpDir config from the `Typegate` object [MET-433](https://linear.app/metatypedev/issue/MET-433/file-removal) _N/A_ - [x] The change come with new or modified tests - [x] Hard-to-understand functions have explanatory comments - [x] End-user documentation is updated to reflect the change <!-- This is an auto-generated comment: release notes by coderabbit.ai --> - **New Features** - Enhanced search functionality with the addition of a new search bar. - Introduced new test configurations to improve script execution. - Updated artifact storage documentation to clarify management processes. - Added new extensions to support improved code commenting. - **Bug Fixes** - Removed outdated Deno import mapping settings to streamline development environment setup. - **Documentation** - Expanded documentation on artifact tracking and management, including reference counting and garbage collection mechanisms. - **Refactor** - Implemented interface changes in `QueryEngine` for better async disposal management. - Code restructuring in artifact management for enhanced performance and maintainability. - **Chores** - Adjusted settings and configurations in the development environment to align with current best practices. - **Tests** - Introduced new test cases for artifact upload and management functionalities. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Natoandro <anatoandro@hotmail.com> Co-authored-by: destifo <estifanosbireda@gmail.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> chore(release): prepare 0.4.0 (#710) Bumps version to release 0.4.0. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> - **New Features** - Updated the software across various components to version 0.4.0, enhancing functionality and potentially introducing new features or fixes. - **Documentation** - Updated version documentation in multiple configuration files to reflect new version 0.4.0. - **Bug Fixes** - Adjusted version constants and dependencies to ensure compatibility and stability with the new software version 0.4.0. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Natoandro <anatoandro@hotmail.com> chore(docs): final polish to comparison table. (#709) some changes to comparison table(docs) _No Migrations Needed_ <!-- 5. Readiness checklist - [ ] The change come with new or modified tests - [ ] Hard-to-understand functions have explanatory comments - [ ] End-user documentation is updated to reflect the change --> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> - **Documentation** - Introduced a new section on Artifact Tracking Protocol in the architecture documentation, explaining artifact classification and tracking modes in Metatype. - Updated comparisons documentation with additional platforms, criteria for choosing Metatype, and detailed feature comparison tables. - Renamed project directory for clarity and consistency in project setup documentation. - **Bug Fixes** - Removed outdated `TODO` comment in installation documentation. <!-- end of auto-generated comment: release notes by coderabbit.ai --> chore: bump to version 0.4.1-0 (#713) - Bumps version to 0.4.1-0. - Fixes broken release CI. - #719 - Adds 20 minutes to test-full timeout. <!-- 5. Readiness checklist - [ ] The change come with new or modified tests - [ ] Hard-to-understand functions have explanatory comments - [ ] End-user documentation is updated to reflect the change --> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> - **New Features** - Updated platform support for better compatibility with "x86_64-linux". - **Bug Fixes** - Minor version updates across multiple configurations to enhance stability. - **Chores** - Updated version numbers from "0.4.0" to "0.4.1-0" across various files and configurations. - **Refactor** - Adjusted build and test scripts for improved efficiency and compatibility. - **Documentation** - Enhanced internal documentation to reflect version and platform changes. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Yohe-Am <56622350+Yohe-Am@users.noreply.github.com> feat: polish documentation and project (#696) <!-- Pull requests are squash merged using: - their title as the commit message - their description as the commit body Having a good title and description is important for the users to get readable changelog. --> <!-- 1. Explain below WHAT the change is --> - update the headline, the overviews and many other documentation areas - upgrades the dependencies. <!-- 2. Explain below WHY the change cannot be made simpler --> <!-- 4. Explain HOW users should update their code or remove that section --> - [ ] The change come with new or modified tests - [ ] Hard-to-understand functions have explanatory comments - [x] End-user documentation is updated to reflect the change <!-- This is an auto-generated comment: release notes by coderabbit.ai --> - **Bug Fixes** - Updated Docker image version for the `typegate` service to ensure stability and compatibility. - **Documentation** - Revised `TAGLINE` for better clarity on supported languages: WASM, Typescript, and Python. - Updated version declarations for improved consistency and functionality across multiple files. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Teo Stocco <zifeo@users.noreply.github.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> feat(sdk): expose metagen to `typegraph/sdk` (#718) Expose metagen features to `typegraph/sdk` Depends on #707 and #696 None <!-- This is an auto-generated comment: release notes by coderabbit.ai --> - **New Features** - Introduced new `Metagen` class for code generation tasks in both TypeScript and Python SDKs. - Added functionality for defining policies and structures for deployment examples using Node.js. - **Improvements** - Enhanced `Metagen` class with methods for simulating and executing code generation tasks. - Simplified file reading and writing functions for better performance and maintainability. - **Bug Fixes** - Refined `compress_and_encode` function to streamline file handling processes. - **Tests** - Added comprehensive tests for `Metagen` functionality in both TypeScript and Python SDKs. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Teo Stocco <zifeo@users.noreply.github.com> Co-authored-by: Teo Stocco <teo@zifeo.com> Co-authored-by: Teo Stocco <zifeo@users.noreply.github.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Estifanos Bireda <77430541+destifo@users.noreply.github.com> Co-authored-by: Yohe-Am <56622350+Yohe-Am@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implements python runtime support for
import_functionmaterializers. This is an stacked PR on #673 which itself is appended on #687 for tackling MET-404.Migration notes
This PR introduced a breaking change in the importer logic for python modules brought in through
pyrt.import_and it's deps. After this PR, any path imports must be prefixed with.,..,...and so to point to relative path locations. This behavior is at odds with standard python which looks for relative modules and packages even if there are no dot prefixes.