-
Notifications
You must be signed in to change notification settings - Fork 33
Conversation
Applied to the following packages: - api - llama-cpp-python
✅ Deploy Preview for leapfrogai-docs canceled.
|
I need to test this locally outside docker, but at this point the build and uds deploy should be working on mac. |
…b.com:defenseunicorns/leapfrogai into 491-chore-add-support-to-build-on-mac-silicon
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One small change request, but other than that this PR is looking good to me!
@alekst23 Can you update the I know vllm isn't MacOS compatible so it wasn't in the original scope of what you were doing here, but I think it's important enough to keep our builds consistent. |
-rm packages/llama-cpp-python/build/*.whl | ||
|
||
## The external link is needed to pull a pre-compiled cpu wheel for llama-cpp-python | ||
python -m pip wheel packages/llama-cpp-python -w packages/llama-cpp-python/build --find-links=${SDK_DEST} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The llama-cpp-python Dockerfile was taking a 'shortcut' to see if a model existed in the build/
directory to try to bypass the model-download script. If the build/
directory doesn't exist than our Dockerfile will run into an error when trying to build.
The Dockerfile is trying to do the copy here and the workflow is currently failing because the build/
directory doesn't exist.
I think we can get away with removing this shortcut, but I wouldn't be opposed to just adding another conditional check here either.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Everything, with the exception of @YrrepNoj comments, looks good to me!
Future reference: the copying over of src
into the build context's directory is an interesting move to get rid of the relative import within the pyproject.toml
. I think a clean-up of these, not just a .gitignore
, would be a nice touch to the Makefile
. (understanding that this isn't your original pattern, but an already present one)
Thanks for all the testing and improvements!
Closing in favor of #659 |
This PR adds support for building and deploy packages on Macs.
The issue arrises because the current build process is configured to build the packages on the local machine, and then copy the package for installation to a Docker image. Doing this on Mac causes the packages to be built for the local Mac architecture, and these file versions do not work in the Docker Linux container.
We address the issue in this PR by moving the build process inside Docker:
builder
layermake
command, which defaults toamd64
, and can be set toarm64
for building on Macmake
, to allow theregistry
container to change port if5000
is already taken on the local systemAdditional Notes:
lfaiui
toleapfrogai-ui
in theui
package