Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build light weight PyRuntime without llvm or onnx-mlir #3044

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

chentong319
Copy link
Collaborator

@chentong319 chentong319 commented Jan 15, 2025

Motivation

Python driver is needed to run the compiled model. Currently, the driver is built with onnx-mlir and can only be run in the env where onnx-mlir is built, typically inside the onnx-mlir docker image. When the compilation can be done by calling the onnx-mlir docker image, we'd like to run the compiled .so with python driver in the local env, so that all the packages installed in the local env can be used, rather than installing them on top of docker.

In order to reach this goal, the PR tried to remove the unnecessary dependencies of pyruntime if the light-weight pyruntime is the target, how onnx-mlir is built and used remains as it was previously.
Details can be found in docs/build-pyruntime-lit.md.
I tried the build on a z16 machine: it takes less than 2 minutes.

Components in this PR

  1. CMakefile and source code changes to cut the dependencies of pyruntime. An option ONNX_MLIR_ENABLE_PYRUNTIME_LIT is used to control Cmake, and consequently a compile definition ENABLE_PYRUNTIME_LIT issued to control the source code.
  2. Wrap the built pyruntime driver into a python package
  3. This python package use python docker package to call the compiler. This is equivalent to docker/onnx-mlir.py in functionality but with different abstraction.

Test
Run successfully with utils/BuildPyRuntimeLit.sh.

Future works:

  1. Support of float16
  2. Not all llvm utilities are replaced. Only the essential ones have been implemented.
  3. Can third_party/onnx be removed?
  4. Try to integrate the precompiled lib for different os-arch into the package. Enable user to use pip install with package name remotely
  5. Try to integrate the utils/build-pyruntime-lit.sh into python package and invoke the build when pip install is executed.

Signed-off-by: Chen Tong <chentong@us.ibm.com>
Signed-off-by: Chen Tong <chentong@us.ibm.com>
Signed-off-by: Chen Tong <chentong@us.ibm.com>
Signed-off-by: Chen Tong <chentong@us.ibm.com>
@chentong319 chentong319 marked this pull request as draft January 15, 2025 21:00
Signed-off-by: Chen Tong <chentong@us.ibm.com>
Signed-off-by: Chen Tong <chentong@us.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant