Update Protobuf to include privacy manifest

[maurovc]

Describe the bug

As of Spring 2024, the SDKs listed in https://developer.apple.com/support/third-party-SDK-requirements need to provide a Privacy Manifest. While RecaptchaEnterprise is not listed, Protobuf which is pulled transitively is.

While Protobuf has an open issue we wanted to make sure to leave note here as this repository will also be transitively affected so that it's also under your radar.

Integration Method

Select the method used to integrate with reCAPTCHA Mobile.

• SPM (iOS)
• Cocoapods (iOS)
• Direct Download (iOS)
• GMaven (Android)

Note: Any on iOS

SDK Version (e.g. 18.1.0):

All

To Reproduce

Steps to reproduce the behavior:

1. Create an app having RecaptchaEnterprise as dependency
2. Archive the app
3. Check if Protobuf includes the privacy manifest file (.xcprivacy)

Expected behavior

Steps 1 and 2 from the section above and have an ipa including the .xcprivacy file within the Protobuf framework

Xcode version for iOS (please complete the following information):

• All

Device (please complete the following information):
N/A

Additional context
N/A

[mcorner]

Yes we are aware, thank you. We are working on it and will have it done before Apple enforces this in the AppStore process.

If you are using the preview versions of the SDK (v16/v17) those will not be updated and you will need to migrate to v18.

[maurovc]

Yes we are aware, thank you. We are working on it and will have it done before Apple enforces this in the AppStore process.

If you are using the preview versions of the SDK (v16/v17) those will not be updated and you will need to migrate to v18.

Amazing, thanks for the context @mcorner

[jpeckner]

@mcorner Could you confirm if the privacy manifest has now been added to RecaptchaEnterprise v18?

[mcorner]

Not yet. We are hoping for the first week of April. There is the additional requirement of signing the xcframeworks as well.

[jpeckner]

Not yet. We are hoping for the first week of April. There is the additional requirement of signing the xcframeworks as well.

@mcorner Thanks, appreciate the quick reply!

[mcorner]

I recognize the timeline here is far from ideal. It is hard (sometimes even for me) to see why something that looks simple takes this long. But it is actually quite tricky given the number of SDKs and apps we produce and we are supporting direct download, pods, and SPM. (There are also a few bugs on Apple's side that we are waiting on).

If you have already upgraded to 18.x then this will be a drop in, API compatible replacement.

[Nathan-Molby]

@mcorner is the timeline still to release the new version with a privacy manifest this week?

[mcorner]

Not this week for sure, perhaps next week. There is a lot of this that is simply out of our control at the moment. We have the privacy manifest, but SDKs are also supposed to be signed which we are working on.

Have you gotten any warnings while submitting to the AppStore?

[Nathan-Molby]

Yeah we've gotten just about every warning possible, but the warnings don't provide any information about which SDK actually generated the warning.

[Vitaliy-Sh]

@mcorner Hello! Are there any news?

[mcorner]

As of last week there was still a bug on Apple's side that meant that static xcframeworks wouldn't validate properly. They are fixing it. We are checking again this week.

[CoderPug]

thanks for the update @mcorner !, would you mind also sharing the public statement related to that bug on Apple's side with static xcframeworks? this is valuable as we are also facing some similar issues 🙏

[mcorner]

I believe this is it: firebase/firebase-ios-sdk#12557

So one option is we can release with what we have, but you would have to do the merging yourself.

I am sorry about this, it is far from ideal, but we are all just waiting to see what apple does.

[jpeckner]

@mcorner FWIW, my app appears to be passing all of Apple's checks (see caveats below), and the top-level Google dependencies it includes are:

• RecaptchaEnterprise 18.4.2, installed via SPM
• FirebaseCrashlytics 10.22.0, installed via CocoaPods using pod 'Firebase/Crashlytics'. What's strange is that Firebase 10.24.0 was supposed to be first Firebase version that was fully codesigned and therefore capable of passing Apple's checks.

I'm saying "appears to be passing", because I haven't gotten a warning email from Apple since submitting a build to them this morning (but I did get a warning email from them yesterday, within five minutes, when I temporarily deleted part of my app's PrivacyInfo.xcprivacy file, just to verify that the warning system was still active 😅).

It does seem odd that we're using Recaptcha and Firebase versions that purportedly aren't code-signed, and yet not getting an email. I wish I could say with confidence that "no warning email == app will accepted after May 1st", and it seems likely, but not an iron-clad guarantee. I'm still planning to update asap once Recaptcha v18.5 is available.

tl;dr - Curious to see if anyone else is also not getting an email after submitting a build for review, with RecaptchaEnterprise less than or equal to 18.4.2, and/or Firebase less than 10.24.0.

EDIT: I'm also very curious to know if anyone has found anywhere on App Store Connect, or another official channel from Apple, that confirms if a given build has passed or failed their checks. I know there are unofficial tools going around such as https://github.com/Wooder/ios_17_required_reason_api_scanner, but I'm looking for something from Apple specifically.

[mcorner]

The latest is that we don't think we need to code sign reCAPTCHA ATM because it isn't on the list. There are a large number of gotchas in using signed xcframeworks, so we will skip it for now. A beta release with the privacy manifest will be out really soon, probably this week.

As for apple review...not sure I think it is changing a lot as bugs come up.

[jpeckner]

Yep, it's definitely a confusing situation. I emailed Apple today and asked them to confirm if "no warning email == app will be accepted after May 1st"; will post back here with details on any response they send.

[walterjgsp]

Our iOS SDK version 18.5.0-beta03 is now available, featuring a Privacy.xcprivacy file that can be used for privacy report purposes. To successfully generate a privacy report during archiving, please ensure you're using:

• Xcode 15.3 if integrating with SPM
• Cocoapods 1.12
• For direct downloads, remember to set any of the Embed * options when archiving. This is crucial for Xcode to generate the report.

We'd love to hear about your testing experience and answer any questions you may have.

[Nathan-Molby]

@walterjgsp can you explain the requirement for Xcode 15.3 for SPM?

[walterjgsp]

Sure @Nathan-Molby. Starting in version 15.3, Xcode will embed SPM binary targets that package static xcframeworks. In previous versions of Xcode this doesn't happen automatically and there's no easy way to tell Xcode to Embed * when using SPM like we can do with the direct download method.

You can test that by generating the privacy report. When you try to generate the privacy report using Xcode 15.2 for example as described in https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_data_use_in_privacy_manifests#4239187 it will not work. When using 15.3 it will work.

[mcorner]

A "subtle undocumented change" firebase/firebase-ios-sdk#12470 (comment)

[Nathan-Molby]

I've downloaded Xcode 15.3.0 and created a new project where I added the ReCaptcha SDK version 18.5.0-beta03, but the resulting privacy report is empty. Is there something special I have to do to get Xcode to see the privacy manifest?

[Nathan-Molby]

The binaryTarget URL didn't get correctly bumped to v18.5.0-beta03 in the Package.swift file. It is still pointing to v18.5.0-beta02, which doesn't contain the privacy manifest. So even though the SPM version is 18.5.0-beta03, the binary target it is downloading is 18.5.0-beta02.

When I download the binary target with the url with 18.5.0-beta03, it correctly includes the privacy manifest. Please create a new tag or release which correctly bumps the URL path

[walterjgsp]

The tag was pointing to the wrong branch, please try again and let me know if it's working now.

Maybe it will be necessary to clean the project cache, that can be done by removing the reCaptcha entry on the folder ~/Library/Developer/Xcode/DerivedData/<your project>/SourcePackages/checkouts/

[Nathan-Molby]

Great, that worked, thank you!

[mcorner]

Keep in mind that as of now, Apple is not incorporating privacy manifests in static frameworks in the review process. So you may still get a warning. We have been told they will fix this, but don't have an ETA.

[Nathan-Molby]

Unfortunately my app received the following error when submitting to Apple:

Asset validation failed Invalid Bundle. The bundle {My App}/Frameworks/RecaptchaEnterprise.framework does not support the minimum OS Version specified in the Info.plist.

Based on this thread, it appears this is a bug that Apple introduced with Xcode 15.3. Other SDKs are apparently getting around it by raising the MinimumOSVersion to 100 to allow it to pass this check. See this thread:
https://developer.apple.com/forums/thread/749554

Obviously this isn't ReCaptcha's fault, but at the moment we can't actually submit an app with ReCaptcha to the app store because min Xcode version for privacy manifest is 15.3 but 15.3 is broken.

Could you potentially implement the hacky fix described in the thread?

[walterjgsp]

Thanks for reporting that @Nathan-Molby, we can be done from our side.

[mcorner]

@Nathan-Molby Also try XCode 15.4 beta. We think this issue has been fixed there. However, this also opens the possibility that setting the minimum OS to 100 will actually break the SDK in 15.4 even if it fixes it in 15.3.....

[jpeckner]

Yep, it's definitely a confusing situation. I emailed Apple today and asked them to confirm if "no warning email == app will be accepted after May 1st"; will post back here with details on any response they send.

Just a quick update here: I didn't get a response from Apple via email, so I called developer support just now. The service rep said it's probably true that "no warning email == app will be accepted after May 1st", but wouldn't commit to saying it's an ironclad guarantee. He said we'll just have to wait until after May 1st to know for sure 😐

[mcorner]

FYI: Read Apple's latest announcement: https://developer.apple.com/news/?id=pvszzano. Our understanding is that this statement: "The code is part of a dynamic framework embedded via the Embed Frameworks build phase" means we are exempt for now. This is likely due to the existing problems on Apple's side in processing static frameworks correctly. Will update as we learn more.

[walterjgsp]

@Nathan-Molby, please try the new 18.5.0-beta04 version. We changed the MinimumOsVersion in the Info.plist to 100.0 on that version. Please let me know if it works for you after this work around applied.

[walterjgsp]

Friendly ping to @Nathan-Molby, did 18.5.0-beta04 worked with the hack?

[walterjgsp]

We are including privacy manifest in our SDK and also implemented the work around for the MinimumOsVersion to make it work with Xcode 15.3. Closing due to the lack of response.