Add script to pull extism libraries and maven plugin to include them in jar

[Zwiterrion]

Hi, (#3)

I have used this method in other projects. It's really simple but effective. Before bundling the java-sdk, I run a script to fetch and include libextism in the resources folder. In the resources folder, we have every processor architecture.

When the java-sdk jar is included, the right libextism library is loaded.

[bhelx]

Wow this is awesome!

[bhelx]

The CI is not liking something about the bash script: https://github.com/extism/java-sdk/actions/runs/7853990948/job/21743265959?pr=20

[bhelx]

May want to add a shebang and set -e on the top of this script so it fails if something goes wrong.

[thomasdarimont]

I took a look at the update-extism.sh script and fixed a few shellcheck warnings:

#!/bin/bash
set -e

EXTISM_VERSION=$(curl https://api.github.com/repos/extism/extism/releases/latest | jq -r '.name')

echo "latest extism version is: ${EXTISM_VERSION}"

rm -rf src/main/resources/*
mkdir -p ./src/main/resources/natives/

create_librairies_folders() {
  archs=("darwin-aarch64" "darwin-x86-64" "linux-aarch64" "linux-x86-64" "win32-x86-64")
  for i in "${archs[@]}"; do
     mkdir "./src/main/resources/$i"
  done
}

fetch_and_unzip_library() {
  ARCH="$1"
  LIBRARY_FOLDER="$2"
  FILENAME="$3"

  curl -L -o "./src/main/resources/natives/${ARCH}-${EXTISM_VERSION}.tar.gz" "https://github.com/extism/extism/releases/download/${EXTISM_VERSION}/${ARCH}-${EXTISM_VERSION}.tar.gz"
  tar -xvf "./src/main/resources/natives/${ARCH}-${EXTISM_VERSION}.tar.gz" --directory ./src/main/resources/natives/
  mv "./src/main/resources/natives/${FILENAME}" "./src/main/resources/${LIBRARY_FOLDER}/${FILENAME}"
}

create_librairies_folders

fetch_and_unzip_library "libextism-aarch64-apple-darwin" "darwin-aarch64" "libextism.dylib"
fetch_and_unzip_library "libextism-x86_64-apple-darwin" "darwin-x86-64" "libextism.dylib"
fetch_and_unzip_library "libextism-aarch64-unknown-linux-gnu" "linux-aarch64" "libextism.so"
fetch_and_unzip_library "libextism-x86_64-unknown-linux-gnu" "linux-x86-64" "libextism.so"
fetch_and_unzip_library "libextism-x86_64-pc-windows-gnu" "win32-x86-64" "extism.dll"

rm -rf src/main/resources/natives 2> /dev/null

Besides that, I think it is a good idea to bundle the native lib with the jar, however this will effectively create a jar library that is 115MB+ large where ~80MB will be consumed by the library versions for other platforms.

Perhaps it would be better to build indiviual jar for the respective platform? This approach is used by other libraries too, e.g. the eclipse swt project: https://central.sonatype.com/namespace/org.eclipse.swt

[kevinduffey]

Is this going to get merged at some point? This seems quite helpful to have as presently we have to manually download the various shared libs and put them in to target/classes (which get wiped out on a mvn clean) or other path and configure it.

[bhelx]

I'll merge it when the tests are fixed. I haven't had a chance to look at it. If you'd like to contribute a fix that would be helpful.

[thomasdarimont]

Would it be possible to reduce the size of the binaries somehow?
Perhaps strip the binaries? https://www.baeldung.com/linux/strip-executables

Regarding the idea of creating os/arch specific jars that can be added later as an optimization to reduce the size of a jar file.

[thomasdarimont]

Small experiment: strip libextism.so yields 33% size reduction... not bad.

-rwxr-xr-x 1 tom tom  24M  5. Apr 17:13 libextism.so.orig*
-rwxr-xr-x 1 tom tom  16M  5. Apr 17:13 libextism.so.stripped*

[amysyk-viax-io]

/bin/sh is often a POSIX shell, which does not support arrays. Could you please try this #!/bin/bash as a first line