Align how we enrich the event with device context on Hybrid SDKs

[marandaneto]

The way how we enrich the event with device context on Hybrid SDKs varies depending on the OS.

iOS: The device context lives on the scope, so we call the native bridge, serialize the scope, return a Map and build the event with all the device context, all the device context is available in the beforeSend callback.
https://github.com/getsentry/sentry-dart/blob/912b9205691837abdd546c62844bc9568b908495/flutter/ios/Classes/SentryFlutterPluginApple.swift#L84-L132
Once the event is built, we call captureEnvelope https://github.com/getsentry/sentry-dart/blob/912b9205691837abdd546c62844bc9568b908495/flutter/ios/Classes/SentryFlutterPluginApple.swift#L340

Android: The Hybrid SDK writes the envelope into the disk and a File observer that lives in the Android SDK picks it up, deserializes, and enriches the event with the device context via Event processors, all the device context is not available in the beforeSend callback.
https://github.com/getsentry/sentry-dart/blob/912b9205691837abdd546c62844bc9568b908495/flutter/android/src/main/kotlin/io/sentry/flutter/SentryFlutterPlugin.kt#L89

Both implementations have a trade-off but I believe we should follow the iOS way or find a better alternative.
For that, we'd need to change how the Android SDK enriches the event with device context.

What we want:
The device context should be visible in the beforeSend callback for Android and iOS, you should also be able to filter out the event.
We want to unify this behavior since this leads to different problems depending on the platform.

Ps: We cannot get rid of the File observer on Android due to sentry-native that cannot call the JNI bridge within the signal handler.

Hacks like this should go away.

Android SDK

Preview Give feedback

1. Move Session Health logic from captureEvent to captureEnvelope to keep the feature working for Hybrid SDKs
2. Provide a way to retrieve the scope data (pull based approach). Have a ScopeUtils.serialize(scope) class
3. Add API to retrieve Android scope data sentry-java#2814
   +0
4. For event enriching: For collections (e.g. breadcrumbs) stop trying to merge values (use hybrid hint)
5. Provide a way to retrieve the sdk name/version
6. Provide a way to retrieve dist + environment from options

Hybrid SDKs

Preview Give feedback

1. Retrieve above data
2. Use HubAdapter.getInstance().captureEnvelope() for capturing the envelope
3. Consider if moving to captureEnvelope is not breaking feature (event won't be in the native beforeSend anymore)
4. Add SentrySerialized to ReactNativeWritable convertor, load device context from sentry-java scope sentry-react-native#3170
   +0
5. Enrich event with Android context on the Dart layer sentry-dart#1556
   Platform: Dart +1
   
6. Refactor Breadcrumb filters sentry-capacitor#263
   Android Platform: Capacitor Status: Backlog +3
   

[marandaneto]

Related issue getsentry/sentry-java#2063

[philipphofmann]

I guess the main reason for going with the iOS approach is having all the data available on beforeSend, right, @marandaneto?

[marandaneto]

I guess the main reason for going with the iOS approach is having all the data available on beforeSend, right, @marandaneto?

Yes.

[adinauer]

Android needs to provide a way of retrieving the info (eg. as a map), then hybrid SDKs can pick that up and create the event before passing it to the Android SDK.

[bruno-garcia]

Ps: We cannot get rid of the File observer on Android due to sentry-native that cannot call the JNI bridge within the signal handler.

In the case of the signal handler, we will read the envelope after the app restarts. Do we have any other case to have the file observer? Maybe it could go otherwise?

[marandaneto]

@bruno-garcia if the C layer calls Java back via JNI when it's not a signal handler to capture the event, the file watcher could go.

[marandaneto]

There's another edge case that causes a bug on React Native, on Android, we remove all the breadcrumbs because the Native layer adds them back (this is done to avoid duplication).
If the device is offline, the Android SDK sends the event on restart but does not add the breadcrumbs back because the event is cached and doesn't have the crumbs anymore.
Also getsentry/sentry-react-native#2146

[krystofwoldrich]

Assuming we are going the iOS way, we need

• captureEnvelope, which exists on the hub in java already,
• storeEnvelope for hard crashes,
• and create SentryEnvelope.from(byteArray), I thought something like that might exist but I could not find it.

Is that right?

[marandaneto]

storeEnvelope can be implemented in the plugin as well, no need to come from the Android SDK, this is technically already there.
SentryEnvelope.from(byteArray) you can do EnvelopeReader.read(stream), but yeah exposing a method that takes a byte array would be better.

[markushi]

We could refactor the DefaultAndroidEventProcessor to be able to provide the generated data to the hybrid SDKs as well. This is already done on iOS. ANRv2 already contains some ground work for this, as the scope serialization is done here already.

For View Hierarchies this is causing issues right now as well, as events and attachments are sent separated.

[markushi]

Lets wait for ANRv2 before starting work on this.

[marandaneto]

Another one getsentry/sentry-react-native#2896

[krystofwoldrich]

Linking another related issue getsentry/sentry-react-native#3090

[krystofwoldrich]

What will hybrid SDKs need from SentryAndroid?

• Methods to get currently initialized SDK name and version. Hybrid SDKs can't save the information from the init method as this would not work for manually initialized apps.
• Get native SDK data about user, device and app, contexts, tags, extra, fingerprint, severityLevel, environment and breadcrumbs. These data we currently retrieve from sentry-cocoa. We might be able to use Sentry.configureScope as in sentry-cocoa.
• Way to store final envelope bytes (only sent_at should be updated) during app crash.
• Way to send final envelope bytes (only send_at should be updated) during non-crash events.

Some of the mentioned items are possible today, this is meant to capture everything that needs to be available to the hybrid SDKs.

[marandaneto]

I'd avoid the Sentry.configureScope approach, I'd rather expose an internal API that we have access to.

[markushi]

Required work:

Moved to the issue description.

Android:

• [x] For event enriching: For collections (e.g. breadcrumbs) stop trying to merge values (use hybrid hint)
• [ ] Provide a way to retrieve the scope data (pull based approach). Have a ScopeUtils.serialize(scope) class
• [x] Provide a way to retrieve the sdk name/version
• [x] Provide a way to retrieve dist + environment from options

RN:

• [ ] Retrieve above data
• [ ] Use HubAdapter.getInstance().captureEnvelope() for capturing the envelope

[krystofwoldrich]

@romtsn @markushi For these two points, will we just adjust the captureEnvelope for hybrid SDKs or would it make sense to create a new method that just updates the sent_at but otherwise sends as is?

HubAdapter.getInstance().captureEnvelope() Does not enrich data apart from sent_at.

• Way to store final envelope bytes (only sent_at should be updated) during app crash.
• Way to send final envelope bytes (only send_at should be updated) during non-crash events.

[krystofwoldrich]

Because of the changes in Profiles ingestion, getsentry/sentry-react-native#3057 is blocked by this.

[krystofwoldrich]

I've checked how to call captureEnvelope, and it works but I think we could have a simpler interface for creating envelopes from bytes.

        final HubAdapter hubAdapter = HubAdapter.getInstance();
        IEnvelopeReader envelopeReader = hubAdapter.getOptions().getEnvelopeReader();
        final InputStream byteStream = new ByteArrayInputStream(bytes)) {
        final SentryEnvelope sentryEnvelope = envelopeReader.read(byteStream);
        if (sentryEnvelope != null) {
            hubAdapter.captureEnvelope(sentryEnvelope);
        }

@markushi Would something like SentryEnvelope.from(bytes) make sense?

        byte[] bytes = new byte[rawBytes.size()];
        final SentryEnvelope sentryEnvelope = SentryEnvelope.from(bytes);
        if (sentryEnvelope != null) {
            hubAdapter.captureEnvelope(sentryEnvelope);
        }

@marandaneto I'm not sure what about Flutter, would it make sense to have a constructor from(bytes)?

[marandaneto]

Dart issue getsentry/sentry-dart#1556

[marandaneto]

@krystofwoldrich keep the meta issue open till all the SDKs implement it :)

[kahest]

closing as completed - last remaining work item is tracked on capacitor repo