[vm/ffi] Implement TypedData unwrapping to Pointers in Leaf calls

[dcharkes]

After implementing leaf calls (#36707), we should support passing typed data into FFI trampolines.

Leaf calls will not permit the garbage collector to run, which means that it is safe to pass a pointer to underlying data to C, provided that C does not hold on to that pointer after the FFI call returns. This removes the need to copy the data before passing it to C.

Pointer<NativeFunction<Void Function(Pointer<Int8>)>> functionPointer;

final myFunction = functionPointer.asFunction<void Function(Uint8List)>(isLeaf: true);
// or
final myFunction = functionPointer.asFunction<void Function(ByteBuffer)>(isLeaf: true);

Note that the first option needs to take Uint8List.offsetInBytes into account, so that is a double unwrapping.

[blaugold]

Some APIs use slices to pass strings or raw bytes:

struct Slice {
  void *buf;
  size_t size;
};

Maybe this pattern can be considered when designing this feature.

[dcharkes]

Pointer<NativeFunction<Void Function(Pointer<Int8>)>> functionPointer;

final myFunction = functionPointer.asFunction<void Function(Uint8List)>(isLeaf: true);
// or
final myFunction = functionPointer.asFunction<void Function(ByteBuffer)>(isLeaf: true);

Note that the first option needs to take Uint8List.offsetInBytes into account, so that is a double unwrapping.

We should probably implement both.

[mraleph]

Some additional interesting questions here is whether we want to allow unwrapping of external typed data even on non-leaf calls and whether we want to allow long term sharing of typed data backing buffer with native code.

When embedder creates external typed data through Dart_NewExternalTypedDataWithFinalizer they essentially pass ownership of the buffer to Dart - so it should be feasible to create an API that can facilitate sharing of the ownership with native.

One possibility here could be to allow unwrapping external typed data into a reference counted slice, like so:

typedef struct _Dart_SliceVtbl {
  void (*acquire)(struct _Dart_Slice* slice);
  void (*release)(struct _Dart_Slice* slice);
} Dart_SliceVtbl;

typedef struct _Dart_Slice {
  Dart_SliceVtbl* vtbl;
  void* peer;
  void* buffer;
  size_t size;
} Dart_Slice;

I have not entirely decided whether it should be passed by value or by pointer to the native side.

If the native code wants to keep the slice around it can do slice.vtbl->acquire(&slice) and the slice.buffer and .peer will remain valid even after the FFI call returns back to Dart. Later when native side is done it can call slice.vtbl->release(&slice). At this point buffer might be released if there are no other references to it.

[haberman]

This would greatly accelerate protobuf parsing using https://github.com/protocolbuffers/upb. Profiles currently show ~40% of the overall parsing cost is just copying the data into an FFI buffer.

A memory copy function a la #43968 would also help greatly, but avoiding the malloc()+free()+copy() will be ideal in cases where the input buffer is small enough to be parsed as a leaf function.

[dcharkes]

Some additional interesting questions here is whether we want to allow unwrapping of external typed data even on non-leaf calls and whether we want to allow long term sharing of typed data backing buffer with native code.

@mraleph If we need to do some kind of memory management for non-leaf functions and we're not passing the actual buffer, then that means we can't share the API between leaf and non-leaf functions.

If we can't share the API between leaf and non-leaf functions, we should go and implement at the leaf functions (and not have it blocked on an API design for non-leaf functions).

[dcharkes]

This would also be useful for ICU4X https://github.com/unicode-org/icu4x/blob/1c0b95ef95d869841fec45ed0e7b107b47c8afad/ffi/capi/dart/package/lib/src/GraphemeClusterSegmenter.g.dart#L69-L88

The code would then look as follows:

  GraphemeClusterBreakIteratorUtf16 segmentUtf16(Uint16List input) {
    final alloc = ffi2.Arena();

    final result = _ICU4XGraphemeClusterSegmenter_segment_utf16(
        _underlying, input, input._length);
    alloc.releaseAll();
    return GraphemeClusterBreakIteratorUtf16._(result);
  }

  // ignore: non_constant_identifier_names
  static final _ICU4XGraphemeClusterSegmenter_segment_utf16 = _capi<
          ffi.NativeFunction<
              ffi.Pointer<ffi.Opaque> Function(
                  ffi.Pointer<ffi.Opaque>,
                  ffi.Pointer<ffi.Uint16>,
                  ffi.Size)>>('ICU4XGraphemeClusterSegmenter_segment_utf16')
      .asFunction<
          ffi.Pointer<ffi.Opaque> Function(ffi.Pointer<ffi.Opaque>,
              Uint16List, int)>(isLeaf: true);

Thanks @robertbastian for reporting!

Because the length would be in elements (not in bytes), it would make more sense to go for Uint8List and friends rather than ByteBuffer.

Some APIs use slices to pass strings or raw bytes

Hm, just passing a struct by value doesn't work, because while populating the struct Dart might do a GC. So the unwrapping can only happen in the FfiCall itself. A workaround would be to wrap the native code, which should become much easier with the native assets feature. @blaugold Any specific use case in mind here? Would wrapping the native code work?

[blaugold]

For my specific case, I have to pass around this memory slice struct to make use of the API of Fleece. Currently, I'm allocating native memory and copy to it from Dart before passing it to Fleece, but with this issue resolved that would not be necessary anymore. 💪 I already need to use some native code to glue things together, so wrapping would work without much pain and be even easier once native assets become stable. 🔜🤞🤓

[dcharkes]

We want to support this use case with a different API, see the discussions in #54739.

[dcharkes]

The next question is whether we should do out of bounds checks:

void testAddressOfInt8ListElementAtOutOfBounds() {
  const length = 20;
  const dontCare = -1;
  final typedData = makeInt8List(length);
  Expect.throws(() {
    takeInt8Pointer(typedData[30].address, dontCare);
  });
  Expect.throws(() {
    takeInt8Pointer(typedData[-10].address, dontCare);
  });
}

@Native<Int8 Function(Pointer<Int8>, Size)>(
    symbol: 'TakeInt8Pointer', isLeaf: true)
external int takeInt8Pointer(Pointer<Int8> pointer, int length);

Int8List makeInt8List(int length) {
  final typedData = Int8List(length);
  for (int i = 0; i < length; i++) {
    final value = i % 2 == 0 ? i : -i;
    typedData[i] = value;
  }
  return typedData;
}

Both for TypedData and Array (#54739).

Pros:

• More safety, these are Dart objects and we know the bounds.

Cons:

• Slower & increased code size.
• Fake safety, if a Pointer is passed, we're likely iterating over it but we have no way to check whether the length argument in the example is within bounds.

[greenrobot]

Safe by default, unsafe if the API user indicates they know what they are doing (e.g. special function)? Essentially the reserve of what C++ is doing; which is usually a good guideline. 🙃