NativeAOT: Export native methods from referenced assemblies.

[josephmoresena]

According with dotnet/corert#7215 it is possible to export native methods declared in other assemblies by passing them as input to ILCompiler.
However the way shown there is not very friendly. Would be possible to mark the referenced packages or projects in order to include them as ILCompiler input?

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[agocke]

@josephmoresena Could you elaborate on what kind of experience you want to see here?

[josephmoresena]

Hi @agocke! Some simple way to export methods from a referenced package/assembly in a NativeAOT-compiled class library project would be desirable.
e.g.:
I have a package A (a class library) which is the base for a certain family of class library projects that are compiled with NativeAOT.
Any native library in that family must export an I method that is used to initialize certain resources. A library exposes an E event that is rised when the native interface calls to I.
I want to create a class library project B that uses A library. B doesn't require to have an I implementation since A handles the basic initialization for any library. If B really needs to handle something when I is called, it can subscribe to the E event using an initializer module.

[ivanpovazan]

@agocke we have encountered a very similar scenario while trying to integrate NativeAOT with Xamarin framework.

---

Scenario

• managed framework assembly Microsoft.iOS.dll - initializes ObjCRuntime.Runtime

[UnmanagedCallersOnly (EntryPoint = "xamarin_objcruntime_runtime_nativeaotinitialize")]
unsafe static void SafeInitialize (InitializationOptions* options, IntPtr* exception_gchandle)
{
    ...
    Initialize (options);
    ...
}

• managed app MySingleView.dll (references Microsoft.iOS.dll) - managed Main entry point

using UIKit;
...
static void Main (string[] args)
{
    UIApplication.Main (args, null, typeof (AppDelegate));
}

• native hosting app - xamarin bootstrap (simplified)

...
xamarin_objcruntime_runtime_nativeaotinitialize(options, gchandle);
__managed_Main(argc, argv);
...

In the build process ILC is invoked to build MySingleView.dll as a static library which is later linked with the native hosting piece producing the final binary.

NOTE: There are other UnmanagedCallersOnly (UCO further in the text) methods which are used for managed-to-Objective-C interop that need to be preserved from Microsoft.iOS assembly, but for simplicity I left them out.

Workaround

When compiling MySingleView.dll the only way to preserve UCO methods defined in Microsoft.iOS.dll is to include the referenced framework assembly in IlcCompileInput as described here: dotnet/corert#7215 (comment)

Proposal

After a discussion with @MichalStrehovsky, one idea is to:

• introduce a formal ItemGroup to define the list of referenced assemblies which expose UCO methods that the application needs at runtime
• introduce a new ILC compiler switch to pass in the ItemGroup elements
• adapt compilation roots resolution of the referenced assemblies with ExportedMethodsRootProvider

---

@josephmoresena, would this work for you as well?

[agocke]

Just to make sure I'm understanding -- is the basic problem here that AOT library compilation only includes the "current project" assembly as a root? And you want to have more libraries referenced as roots?

[MichalStrehovsky]

The issue is that the compiler currently looks for UnmanagedCallersOnly with a named entrypoint in the input assembly (and CoreLib), but not in the referenced assemblies. We often don't even open the referenced assembly files if nothing needs them, so digging for attributes in them would be a compiler throughput regression (file IO is not cheap). It's also questionable whether that would be the right behavior.

This proposes adding functionality to make the entrypoint rooting more controllable. The only question is whether we would want more granularity (ability to select individual named entrypoints vs "all named entrypoint in the assembly"). Probably all named entrypoints is sufficient.

[josephmoresena]

@josephmoresena, would this work for you as well?

Partially. In that test, the JNI library that I was using did not require its own implementation of JNI_OnLoad or JNI_OnUnload, but this may not be the majority, especially if we refer to Android, where many things may have to be done through JNI.

I wonder... what if (several?) different assemblies require this type of initialization on a single exported EntryPoint, and additionally if the invocation of that EntryPoint returns a result?

For example, in the case of JNI_OnLoad, the minimum version of the JVM that guarantees the operation of the library. It may be one for the functionality initialized in the framework, but for the functionality implemented in the output assembly it is a higher one.

[josephmoresena]

This proposes adding functionality to make the entrypoint rooting more controllable. The only question is whether we would want more granularity (ability to select individual named entrypoints vs "all named entrypoint in the assembly"). Probably all named entrypoints is sufficient.

I agree with you, however I think that this type of method would go far beyond the definition of UCO.
In a way I see them more as a type of subscription rather than a declaration, although what worries me is everything related to the hierarchy and the final return value.

[ivanpovazan]

@agocke I might not have been clear enough with the intention, Michal gave a more thorough explanation above

@josephmoresena, would this work for you as well?

Partially. In that test, the JNI library that I was using did not require its own implementation of JNI_OnLoad or JNI_OnUnload, but this may not be the majority, especially if we refer to Android, where many things may have to be done through JNI.

I wonder... what if (several?) different assemblies require this type of initialization on a single exported EntryPoint, and additionally if the invocation of that EntryPoint returns a result?

For example, in the case of JNI_OnLoad, the minimum version of the JVM that guarantees the operation of the library. It may be one for the functionality initialized in the framework, but for the functionality implemented in the output assembly it is a higher one.

@josephmoresena, I am not sure I understand the use case you are referring to.

ILCompiler exports managed methods decorated with UnmanagedCallersOnlyAttribute which have EntryPoint specified for them to the native world: https://github.com/dotnet/runtime/blob/main/src/coreclr/nativeaot/docs/interop.md#native-exports

As Michal stated, there is an optimization in ILCompiler, which will not export these kind of methods from referenced libraries, but only from the assemblies passed as inputs to the compiler. The example I gave above tries to express the need for handling this in more formal way.

If you think this is not the same, as the issue you are describing, could you please provide more context and possibly a code example? Moreover, if there is a discrepancy between the two I can open a separate GH issue, regarding my comment above.

[ivanpovazan]

This proposes adding functionality to make the entrypoint rooting more controllable. The only question is whether we would want more granularity (ability to select individual named entrypoints vs "all named entrypoint in the assembly"). Probably all named entrypoints is sufficient.

Maybe this can be handled in the similar fashion as direct PInvoke calls are supported?

[josephmoresena]

@josephmoresena, I am not sure I understand the use case you are referring to.

This thread is for exactly what you're referring to: Exporting symbols of assemblies referenced from the library project.
But this is really just a particular case where only one of the assemblies in the project should export a symbol.

Exploring that idea in #72556, I found that exporting those kinds of symbols would be problematic if more than one assembly needs to export the same symbol, and not just because of the 'redefinition' issue.

It will be especially problematic if those symbols are exported from Framework libraries or NuGet packages so that all this happens 'behind the scenes' of the project developer. So I think this kind of symbol export should be something that goes beyond the UCO functionality.

[jkotas]

I think this kind of symbol export should be something that goes beyond the UCO functionality.

This is a problem for domain-specific interop libraries to solve. For example, if multiple assemblies need subscribe to JNI_OnLoad in solutions that interop with JVM, the Java interop library should own this subscription and allow individual assemblies to subscribe to it.