[python] Add an iree.build package with API/tooling for program build…

…ing. (iree-org#18630)

This is the first step of providing a unified user-oriented build tool
for IREE export and compilation. This initial version supports the CLI
environment, network/fetch actions, and ONNX import+upgrade. A follow-on
step will build out the compiler invocation rules, specifically focusing
on getting option, device, and parameter handling and manifest/metadata
propagated properly.

The intent is that downstream users of the IREE/Turbine tools can have
one stop modules to efficiently export and compile arbitrarily
complicated pipelines without reinventing the wheel or needing to do the
advanced compiler juggling that comes with some of the more complicated
(and performant) workflows.

As a by-product this will standardize a directory layout and
manifest/metadata to accompany it for pipeline construction. The intent
is that this will simplify serving tooling, which currently has to get a
number of out of band parameters and files put together in the right way
and matching how they were compiled.

---------

Signed-off-by: Stella Laurenzo <stellaraccident@gmail.com>
Signed-off-by: Giacomo Serafini <179146510+giacs-epic@users.noreply.github.com>

.gitignore

@@ -22,6 +22,8 @@
 build/
 build-*/
 Testing/
+# Include iree.build package
+!compiler/bindings/python/iree/compiler/build/
 
 # Bazel artifacts
 **/bazel-*

compiler/bindings/python/CMakeLists.txt

@@ -234,6 +234,37 @@ add_mlir_python_modules(IREECompilerPythonModules
   )
 
 
+################################################################################
+# iree.build package
+# This is a pure Python part of the namespace, not rooted under iree.compiler
+# like the above. It is only using the same build support for compatibility
+# with the existing development flow.
+# If the build system for Python code is ever redone, this can just be
+# source namespace in the project definition.
+################################################################################
+
+# The iree.build package.
+declare_mlir_python_sources(IREECompilerBuildPythonPackage
+ROOT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/iree/build"
+SOURCES
+  __init__.py
+  __main__.py
+  executor.py
+  lang.py
+  main.py
+  net_actions.py
+  onnx_actions.py
+)
+
+add_mlir_python_modules(IREECompilerBuildPythonModules
+  ROOT_PREFIX "${_PYTHON_BUILD_PREFIX}/iree/build"
+  INSTALL_PREFIX "${_PYTHON_INSTALL_PREFIX}/iree/build"
+  DECLARED_SOURCES
+    IREECompilerBuildPythonPackage
+)
+
+add_dependencies(IREECompilerPythonModules IREECompilerBuildPythonModules)
+
 ################################################################################
 # Tools linked against the shared CAPI library
 ################################################################################

compiler/bindings/python/iree/build/__init__.py

@@ -0,0 +1,12 @@
+# Copyright 2024 The IREE Authors
+#
+# Licensed under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+import argparse
+
+from iree.build.lang import *
+from iree.build.main import *
+from iree.build.net_actions import *
+from iree.build.onnx_actions import *

compiler/bindings/python/iree/build/__main__.py

@@ -0,0 +1,11 @@
+# Copyright 2024 The IREE Authors
+#
+# Licensed under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+from .main import CliMain
+
+
+if __name__ == "__main__":
+    CliMain().run()