Support for macos_arm64 platform

[simonmaurer]

What do these changes do?

With the latest LCE release v0.6 the TensorFlow dependency has been upgraded to 2.5.0 including relevant changes to support compilation for the macos_arm64 platform using the recent Apple M1 ARM processor.
This also included upstream dependencies on XNNPACK and pthreadpool as needed by the LCE (lce_benchmark_model/lce_minimal) build process.

Starting with latest bazel versions for arm64 this small PR (given all upstream changes in TF) allows building LCE for Apple M1.

Feel free to check out the discussions here: XNNPACK/pthreadpool/TF pip package/TFLite

How Has This Been Tested?

bazel-3.7.2-arm64 build -c opt --config=macos_arm64 --macos_cpus=arm64 //larq_compute_engine/tflite/benchmark:lce_benchmark_model`

or with a more recent version of bazel (as of bazel@492829)

bazel-4.0.0-arm64 build -c opt --macos_cpus=arm64 //larq_compute_engine/tflite/benchmark:lce_benchmark_model`

Benchmark Results

The table below presents single-/multi-threaded performance of Larq Compute Engine v0.6 on
different versions of QuickNet (trained on ImageNet dataset, released on Larq Zoo)
on an Apple mac mini 2020 (M1):

Model	Top-1 Accuracy	mac mini M1, ms (1 thread)	mac mini M1, ms (1 thread w/ XNNPACK)
QuickNetSmall (.h5)	59.4 %	4.0	3.5
QuickNet (.h5)	63.3 %	5.8	5.4
QuickNetLarge (.h5)	66.9 %	9.9	9.5

Model	Top-1 Accuracy	mac mini M1, ms (4 threads)	mac mini M1, ms (4 threads w/ XNNPACK)
QuickNetSmall (.h5)	59.4 %	1.8	1.9
QuickNet (.h5)	63.3 %	2.5	2.6
QuickNetLarge (.h5)	66.9 %	3.9	4.1

Related issue number

#604

[AdamHillier]

That's awesome, thanks @simonmaurer! Since it's not currently possible for our CI to test macos_arm64 binaries, could you possibly run our kernel tests on your M1 machine just to double check they pass?

bazel test larq_compute_engine/tests:cc_tests --copt=-O2 --distinct_host_configuration=false --test_output=all

[simonmaurer]

@AdamHillier oh absolutely, my pleasure. the numbers are just crazy ;)

[simonmaurer]

..in the process of compiling:

test --macos_cpus=arm64 larq_compute_engine/tests:cc_tests --copt=-O2 --distinct_host_configuration=false --test_output=all

and here's the result:

[AdamHillier]

This is great!

[simonmaurer]

updated the PR with benchmark results

[lgeiger]

🚀

[lgeiger]

tensorflow/tensorflow#47639 (comment) and tensorflow/tensorflow#47639 (comment) mention that TFLite can make use of the Accelerate framework on macOS.
@simonmaurer do you think it makes sense to enables this for the benchmarks for LCE as well?

[simonmaurer]

tensorflow/tensorflow#47639 (comment) and tensorflow/tensorflow#47639 (comment) mention that TFLite can make use of the Accelerate framework on macOS.
@simonmaurer do you think it makes sense to enables this for the benchmarks for LCE as well?

yes, totally agree. according to these comments and evaluations this can further improve latency (even more than xnnpack) depending on the model. could we control this somehow via an additional cmdline parameter when executing the lce_benchmark_model binary? if not this can just be activated with the mentioned build flags

[lgeiger]

could we control this somehow via an additional cmdline parameter when executing the lce_benchmark_model binary

This seems to be a build flag, so I don't think we can have a commandline parameter to toggle this behaviour in the benchmark binary.