-
Notifications
You must be signed in to change notification settings - Fork 265
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci(java): build JNI release package #2516
Merged
Merged
Changes from all commits
Commits
Show all changes
36 commits
Select commit
Hold shift + click to select a range
f011a96
build
eddyxu 12303fd
use ubuntu 22.04
eddyxu 36181b0
show pwd
eddyxu 69b7655
copy lib
eddyxu 32b6f29
related path
eddyxu 5001ff9
Merge branch 'main' into lei/jni_multiplatform
eddyxu 6a6c0ff
Merge branch 'main' into lei/jni_multiplatform
eddyxu 2c11b61
Merge branch 'main' into lei/jni_multiplatform
eddyxu c9bc525
Merge branch 'main' into lei/jni_multiplatform
eddyxu cf6b2d4
fix: propagate storage_options to v2 writer (#2578)
jiachengdb 85bab4d
fix: be compatible with v1 index format (#2582)
BubbleCal a8603a1
feat: do flat search if too many rows are filtered out (#2583)
BubbleCal 7a841dc
feat: add label list index (#2581)
westonpace 0e65568
fix pom
eddyxu 2399d43
Merge branch 'main' into lei/jni_multiplatform
eddyxu b611b53
two dry runs
eddyxu 2a388c8
use token
eddyxu e7dda34
setup pgp
eddyxu 7fc46bc
release 0.0.4 for now
eddyxu 0cffc78
not dry run
eddyxu 6df0979
rm snapshot
eddyxu cf941a5
use snapshot
eddyxu 2345a1f
release in batch mode
eddyxu 3cef449
do not push change for now
eddyxu bbadc42
skip
eddyxu 4f0f2a9
set github config
eddyxu 699b2d7
Merge branch 'main' into lei/jni_multiplatform
eddyxu 5eb1771
pass
eddyxu 2416983
setup gdp tty
eddyxu 2e95685
Merge branch 'main' into lei/jni_multiplatform
eddyxu 718df33
expose tty
eddyxu 5df953b
Merge branch 'main' into lei/jni_multiplatform
eddyxu 41cda57
export
eddyxu 36ca1a3
setup java env
eddyxu 41e07c3
Merge branch 'main' into lei/jni_multiplatform
eddyxu 8357129
dont run prepare
eddyxu File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,110 @@ | ||
name: Build and publish Java packages | ||
on: | ||
release: | ||
types: [released] | ||
pull_request: | ||
paths: | ||
- .github/workflows/java-publish.yml | ||
|
||
jobs: | ||
macos-arm64: | ||
name: Build on MacOS Arm64 | ||
runs-on: macos-14 | ||
timeout-minutes: 30 | ||
defaults: | ||
run: | ||
working-directory: ./java | ||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
- uses: Swatinem/rust-cache@v2 | ||
- name: Install dependencies | ||
run: | | ||
brew install protobuf | ||
- name: Build release | ||
run: | | ||
cargo build --release | ||
- uses: actions/upload-artifact@v4 | ||
with: | ||
name: liblance_jni_darwin_aarch64.zip | ||
path: target/release/liblance_jni.dylib | ||
retention-days: 1 | ||
if-no-files-found: error | ||
linux-arm64: | ||
name: Build on Linux Arm64 | ||
runs-on: warp-ubuntu-2204-arm64-8x | ||
timeout-minutes: 30 | ||
defaults: | ||
run: | ||
working-directory: ./java | ||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
- uses: Swatinem/rust-cache@v2 | ||
- uses: actions-rust-lang/setup-rust-toolchain@v1 | ||
with: | ||
toolchain: "1.79.0" | ||
cache-workspaces: "src/rust" | ||
# Disable full debug symbol generation to speed up CI build and keep memory down | ||
# "1" means line tables only, which is useful for panic tracebacks. | ||
rustflags: "-C debuginfo=1" | ||
- name: Install dependencies | ||
run: | | ||
sudo apt -y -qq update | ||
sudo apt install -y protobuf-compiler libssl-dev pkg-config | ||
- name: Build release | ||
run: | | ||
cargo build --release | ||
cp ../target/release/liblance_jni.so liblance_jni.so | ||
- uses: actions/upload-artifact@v4 | ||
with: | ||
name: liblance_jni_linux_aarch64.zip | ||
path: target/release/liblance_jni.so | ||
retention-days: 1 | ||
if-no-files-found: error | ||
linux-x86: | ||
runs-on: warp-ubuntu-2204-x64-8x | ||
timeout-minutes: 30 | ||
needs: [macos-arm64, linux-arm64] | ||
defaults: | ||
run: | ||
working-directory: ./java | ||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
- uses: Swatinem/rust-cache@v2 | ||
- name: Set up Java 8 | ||
uses: actions/setup-java@v4 | ||
with: | ||
distribution: temurin | ||
java-version: 8 | ||
cache: "maven" | ||
server-id: ossrh | ||
server-username: SONATYPE_USER | ||
server-password: SONATYPE_TOKEN | ||
gpg-private-key: ${{ secrets.GPG_PRIVATE_KEY }} | ||
gpg-passphrase: ${{ secrets.GPG_PASSPHRASE }} | ||
- name: Install dependencies | ||
run: | | ||
sudo apt -y -qq update | ||
sudo apt install -y protobuf-compiler libssl-dev pkg-config | ||
- name: Download artifact | ||
uses: actions/download-artifact@v4 | ||
- name: Copy native libs | ||
run: | | ||
mkdir -p ./core/target/classes/nativelib/darwin-aarch64 ./core/target/classes/nativelib/linux-aarch64 | ||
cp ../liblance_jni_darwin_aarch64.zip/liblance_jni.dylib ./core/target/classes/nativelib/darwin-aarch64/liblance_jni.dylib | ||
cp ../liblance_jni_linux_aarch64.zip/liblance_jni.so ./core/target/classes/nativelib/linux-aarch64/liblance_jni.so | ||
- name: Set github | ||
run: | | ||
git config --global user.email "Lance Github Runner" | ||
git config --global user.name "dev+gha@lancedb.com" | ||
- name: Publish with Java 8 | ||
run: | | ||
echo "use-agent" >> ~/.gnupg/gpg.conf | ||
echo "pinentry-mode loopback" >> ~/.gnupg/gpg.conf | ||
export GPG_TTY=$(tty) | ||
mvn --batch-mode -DskipTests -DpushChanges=false -Dgpg.passphrase=${{ secrets.GPG_PASSPHRASE }} deploy -P deploy-to-ossrh | ||
env: | ||
SONATYPE_USER: ${{ secrets.SONATYPE_USER }} | ||
SONATYPE_TOKEN: ${{ secrets.SONATYPE_TOKEN }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i felt like it should not be 3 jobs. but instead we should have 3 run matrix and at the very end we need to copy all 3 share lib (linux-x86, linux-arm, macos-arm) into the same fat jar?
this will still release 3 different JAR IIUC?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, if we call
mvn deploy
in three different job, we have to release 3 different jars (with different names?).I am not sure that mvn will resolve dependency based on the system, like what pypi does.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Put them into separate jobs, so that i can pass
upload-artifcat/download-artifcat
between the jobs. If we just run 3 parallel metrics, we need the 4th one to download the artifact?