swift code being compiled 3 times

[omarzl]

Hello @thii!

We migrated all our project and its dependencies to bazel but we can't reduce the build time in incremental builds for swift targets.

I added the profiling flag to bazel build command and here are the results loaded in chrome, as you can see there are three "Compiling Swift module Apollo" steps in different threads.

And if I use the official Bazel rule there is only one "Compiling Swift module Apollo" step

So I think that the swift module is being compile 3 times, I tried to modify the swift_static_framework.bzl without success, as far as I understand I guess that it is compiling 3 times because of the 3 header maps (public, private, private angled), each one compiles a swift module?

Do you have the same behavior with Line's dependencies?

Here is the target definition:

load("@rules_apple_line//apple:apple_library.bzl", "swift_library")

swift_library(
    name = "Apollo",
    srcs = glob(["Apollo/Sources/Apollo/*.swift"]),
    module_name = "Apollo",
    visibility = ["//visibility:public"],
)

And the trace files:

traces.zip

[thii]

By "incremental builds", do you mean that it doesn't happen in a full build?

It shouldn't compile 3 times, unless you build your target for more than one platform (for instance, iOS and watchOS). You can pass -s to your bazel build command to see all the subcommands it actually invokes during the build. That way you can tell if it rebuilds the same thing more than once or not.

[omarzl]

I was doing tests with incremental builds but it happens in clean/full builds too, sorry for that.

i am compiling only for one platform (iOS) and it happens to all my swift targets, I did an small example reproducing it using one of our dependencies (Apollo) and here is the output of bazel build with --subcommands flag, check out that the "Compiling Swift module Apollo" action is executed 3 times

bazel build //:ApolloFramework --subcommands
...
...
SUBCOMMAND: # //:Apollo_swift [action 'Compiling Swift module Apollo', configuration: 131f8533d08a1fa057f9227cecdb0d1b210d4ae31ca79bf3a5e735583a6b1974, execution platform: @local_config_platform//:host]
(cd /private/var/tmp/_bazel_omarzl/7dc0343477432e36b5861789c0d91fa5/execroot/__main__ && \
  exec env - \
    APPLE_SDK_PLATFORM=iPhoneSimulator \
    APPLE_SDK_VERSION_OVERRIDE=14.1 \
    XCODE_VERSION_OVERRIDE=12.1.0.12A7403 \
  bazel-out/host/bin/external/build_bazel_rules_swift/tools/worker/worker swiftc @bazel-out/applebin_ios-ios_x86_64-fastbuild-ST-1665503f7bef/bin/Apollo.swiftmodule-0.params)
SUBCOMMAND: # //:Apollo_swift [action 'Compiling Swift module Apollo', configuration: 8eaf4483c69e18e1711a6aa123998417f8eabcd8ff41968bfcf1ed027e775076, execution platform: @local_config_platform//:host]
(cd /private/var/tmp/_bazel_omarzl/7dc0343477432e36b5861789c0d91fa5/execroot/__main__ && \
  exec env - \
    APPLE_SDK_PLATFORM=iPhoneSimulator \
    APPLE_SDK_VERSION_OVERRIDE=14.1 \
    XCODE_VERSION_OVERRIDE=12.1.0.12A7403 \
  bazel-out/host/bin/external/build_bazel_rules_swift/tools/worker/worker swiftc @bazel-out/ios-x86_64-min11.0-applebin_ios-ios_x86_64-fastbuild/bin/Apollo.swiftmodule-0.params)
SUBCOMMAND: # //:Apollo_swift [action 'Compiling Swift module Apollo', configuration: fe6fd5eeb435b0e61960bc3d2d9747398c1cd0057e054cbddbccec978de023ec, execution platform: @local_config_platform//:host]
(cd /private/var/tmp/_bazel_omarzl/7dc0343477432e36b5861789c0d91fa5/execroot/__main__ && \
  exec env - \
    APPLE_SDK_PLATFORM=iPhoneSimulator \
    APPLE_SDK_VERSION_OVERRIDE=14.1 \
    XCODE_VERSION_OVERRIDE=12.1.0.12A7403 \
  bazel-out/host/bin/external/build_bazel_rules_swift/tools/worker/worker swiftc @bazel-out/ios-x86_64-min11.0-applebin_ios-ios_x86_64-fastbuild-ST-e2dec09bbcae/bin/Apollo.swiftmodule-0.params)
SUBCOMMAND: # //:Apollo_swift [action 'Linking libApollo_swift.a', configuration: fe6fd5eeb435b0e61960bc3d2d9747398c1cd0057e054cbddbccec978de023ec, execution platform: @local_config_platform//:host]
(cd /private/var/tmp/_bazel_omarzl/7dc0343477432e36b5861789c0d91fa5/execroot/__main__ && \
  exec env - \
    APPLE_SDK_PLATFORM=iPhoneSimulator \
    APPLE_SDK_VERSION_OVERRIDE=14.1 \
    XCODE_VERSION_OVERRIDE=12.1.0.12A7403 \
  external/local_config_cc/libtool -D -no_warning_for_no_symbols -static -o bazel-out/ios-x86_64-min11.0-applebin_ios-ios_x86_64-fastbuild-ST-e2dec09bbcae/bin/libApollo_swift.a -filelist bazel-out/ios-x86_64-min11.0-applebin_ios-ios_x86_64-fastbuild-ST-e2dec09bbcae/bin/libApollo_swift.a-0.params)
SUBCOMMAND: # //:Apollo.intermediate.apple_static_library [action 'Linking Apollo.intermediate.apple_static_library-fl.a', configuration: fbf3f564826cec64317ce9897de2cb07673daba6dd49f3e0b9851cb9476b47c0, execution platform: @local_config_platform//:host]
(cd /private/var/tmp/_bazel_omarzl/7dc0343477432e36b5861789c0d91fa5/execroot/__main__ && \
  exec env - \
    APPLE_SDK_PLATFORM=iPhoneSimulator \
    APPLE_SDK_VERSION_OVERRIDE=14.1 \
    PATH=/Users/omarzl/Library/Caches/bazelisk/downloads/bazelbuild/bazel-3.7.1-darwin-x86_64/bin:/usr/local/opt/ruby/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin \
    XCODE_VERSION_OVERRIDE=12.1.0.12A7403 \
  external/local_config_cc/libtool -D -no_warning_for_no_symbols -static -arch_only x86_64 -syslibroot __BAZEL_XCODE_SDKROOT__ -o bazel-out/ios-x86_64-min11.0-applebin_ios-ios_x86_64-fastbuild-ST-e2dec09bbcae/bin/Apollo.intermediate.apple_static_library-fl.a bazel-out/ios-x86_64-min11.0-applebin_ios-ios_x86_64-fastbuild-ST-e2dec09bbcae/bin/libApollo_swift.a)
SUBCOMMAND: # //:Apollo.intermediate [action 'Bundling Apollo.intermediate', configuration: 131f8533d08a1fa057f9227cecdb0d1b210d4ae31ca79bf3a5e735583a6b1974, execution platform: @local_config_platform//:host]
(cd /private/var/tmp/_bazel_omarzl/7dc0343477432e36b5861789c0d91fa5/execroot/__main__ && \
  exec env - \
  bazel-out/host/bin/external/build_bazel_rules_apple/tools/bundletool/bundletool bazel-out/applebin_ios-ios_x86_64-fastbuild-ST-1665503f7bef/bin/Apollo.intermediate-intermediates/bundletool_control.json)
SUBCOMMAND: # //:Apollo.intermediate [action 'Processing and signing Apollo.intermediate', configuration: 131f8533d08a1fa057f9227cecdb0d1b210d4ae31ca79bf3a5e735583a6b1974, execution platform: @local_config_platform//:host]
(cd /private/var/tmp/_bazel_omarzl/7dc0343477432e36b5861789c0d91fa5/execroot/__main__ && \
  exec env - \
  /bin/bash -c 'cp -p '\''bazel-out/applebin_ios-ios_x86_64-fastbuild-ST-1665503f7bef/bin/Apollo.intermediate-intermediates/unprocessed_archive.zip'\'' '\''bazel-out/applebin_ios-ios_x86_64-fastbuild-ST-1665503f7bef/bin/Apollo.intermediate.zip'\''')
SUBCOMMAND: # //:ApolloFramework [action 'Processing and bundling Apollo.framework', configuration: 63cc040ed2b86a512099924e698df6e0b9848625e6ca33d9556c5993dccbc2fb, execution platform: @local_config_platform//:host]
(cd /private/var/tmp/_bazel_omarzl/7dc0343477432e36b5861789c0d91fa5/execroot/__main__ && \
  exec env - \
  /bin/bash -c '
        external/bazel_tools/tools/zip/zipper/zipper x bazel-out/applebin_ios-ios_x86_64-fastbuild-ST-1665503f7bef/bin/Apollo.intermediate.zip
        external/bazel_tools/tools/zip/zipper/zipper c bazel-out/darwin-fastbuild/bin/ApolloFramework.zip $(find Apollo.framework -type f) $@
        rm -rf bazel-out/applebin_ios-ios_x86_64-fastbuild-ST-1665503f7bef/bin/Apollo.intermediate.zip
    ' '' 'Apollo.framework/Modules/Apollo.swiftmodule/x86_64.swiftmodule=bazel-out/ios-x86_64-min11.0-applebin_ios-ios_x86_64-fastbuild/bin/Apollo.swiftmodule' 'Apollo.framework/Modules/Apollo.swiftmodule/x86_64.swiftdoc=bazel-out/ios-x86_64-min11.0-applebin_ios-ios_x86_64-fastbuild/bin/Apollo.swiftdoc')
Target //:ApolloFramework up-to-date:
  bazel-bin/ApolloFramework.zip
INFO: Elapsed time: 54.475s, Critical Path: 26.61s
INFO: 249 processes: 26 internal, 220 darwin-sandbox, 3 worker.
INFO: Build completed successfully, 249 total actions

Here is the example project ready to run: bazel build //:ApolloFramework --subcommands
bazel_duplicated_example.zip

[thii]

Interesting find - thanks for the report. This is probably related to this bug in Bazel. I'll try to find a way to work around it.

We build the ios_application target directly so we don't have this problem.

[thii]

@omarzl Can you try again with the latest version of swift_static_framework rule?