DynamoDB doesn't work with native AOT

[branog68]

I wrote a lambda function with .net 7 native AOT. Without a database access, it worked properly. I tried to code access to DynamoDB table. Here is my code:

internal class DbRepository : IDbRepository
{
    private static readonly IAmazonDynamoDB _dbClient = new AmazonDynamoDBClient();

    public DbRepository()
    {
        DbTable = Table.LoadTable(_dbClient, Utils.GetRequiredEnvVariable("DDB_TABLE"));
    }

    public Table DbTable { get; }

In the constructor for the statement Table.LoadTable I got following exception:

---> System.TypeInitializationException: A type initializer threw an exception. To determine which type, inspect the InnerException's StackTrace property. 
---> System.NotSupportedException: 'System.Nullable`1[System.Byte]' is missing native code or metadata. This can happen for code that is not compatible with trimming or AOT. Inspect and fix trimming and AOT related warnings that were generated when the app was published. For more information see https://aka.ms/nativeaot-compatibility
at System.Reflection.Runtime.General.TypeUnifier.WithVerifiedTypeHandle(RuntimeConstructedGenericTypeInfo, RuntimeTypeInfo[]) + 0x98 
at Amazon.DynamoDBv2.Converter`1.<GetTargetTypes>d__0.MoveNext() + 0xdf
at Amazon.DynamoDBv2.ConverterCache.AddConverter(Converter converter, DynamoDBEntryConversion conversion) + 0x99 
at Amazon.DynamoDBv2.DynamoDBEntryConversion.AddConverters(String suffix) + 0x1ca
at Amazon.DynamoDBv2.DynamoDBEntryConversion..ctor(ConversionSchema schema, Boolean isImmutable) + 0xa1
at Amazon.DynamoDBv2.DynamoDBEntryConversion..cctor() + 0x27
at System.Runtime.CompilerServices.ClassConstructorRunner.EnsureClassConstructorRun(StaticClassConstructionContext*) + 0xb9

I'm using the latest version of AWSSDK.DynamoDBv2 (3.7.102.5). I added the AWSSDK.DynamoDBv2 in the rd-config file too:

<Directives xmlns="http://schemas.microsoft.com/netfx/2013/01/metadata">
  <Application>
    <Assembly Name="bootstrap" Dynamic="Required All"></Assembly>
	<Assembly Name="AWSSDK.SimpleSystemsManagement" Dynamic="Required All"></Assembly>
	<Assembly Name="AWSSDK.Core" Dynamic="Required All"></Assembly>
	<Assembly Name="AWSSDK.DynamoDBv2" Dynamic="Required All"></Assembly>
  </Application>
</Directives>

I'm using Windows 11 Pro 10.0.22621 with dotnet version 7.0.100. And here is my project file:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net7.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <AWSProjectType>Lambda</AWSProjectType>
    <AssemblyName>bootstrap</AssemblyName>
    <!-- This property makes the build directory similar to a publish directory and helps the AWS .NET Lambda Mock Test Tool find project dependencies. -->
    <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
    <!-- Generate Native AOT image during publishing to improve cold start time. -->
    <PublishAot>true</PublishAot>
    <!-- StripSymbols tells the compiler to strip debugging symbols from the final executable if we're on Linux and put them into their own file. 
    This will greatly reduce the final executable's size.-->
    <StripSymbols>true</StripSymbols>
  </PropertyGroup>
  <!-- 
  When publishing Lambda functions for ARM64 to the provided.al2 runtime a newer version of libicu needs to be included
  in the deployment bundle because .NET requires a newer version of libicu then is preinstalled with Amazon Linux 2.
  -->
  <ItemGroup Condition="'$(RuntimeIdentifier)' == 'linux-arm64'">
    <RuntimeHostConfigurationOption Include="System.Globalization.AppLocalIcu" Value="68.2.0.9" />
    <PackageReference Include="Microsoft.ICU.ICU4C.Runtime" Version="68.2.0.9" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="Amazon.Lambda.APIGatewayEvents" Version="2.6.0" />
    <PackageReference Include="Amazon.Lambda.RuntimeSupport" Version="1.8.2" />
    <PackageReference Include="Amazon.Lambda.Core" Version="2.1.0" />
    <PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" Version="2.3.0" />
    <PackageReference Include="AWSSDK.DynamoDBv2" Version="3.7.102.5" />
    <PackageReference Include="AWSSDK.SimpleSystemsManagement" Version="3.7.103.37" />
    <PackageReference Include="LazyCache.AspNetCore" Version="2.4.0" />
  </ItemGroup>
   
  <ItemGroup>
    <!--The runtime directives file allows the compiler to know what types and assemblies to not trim out of the final binary, even if they don't appear to be used.-->
    <RdXmlFile Include="rd.xml" />
  </ItemGroup>
</Project>

[3Scribe]

This is the big problem we have with AOT. I don't think it's ready for production use yet as too many of the libraries either contain reflection and/or lack the information for serialization. We backed off from switching to AOT for our Lambdas for now.

To get around your problem I think you'll have to add this directive to your Serializer Context function:

[JsonSerializable(typeof(System.Byte))]

It may be the first of many you'll have to add.

[branog68]

Thank you for the hint. I tried it, but it didn't help. Here is the new code:

[JsonSerializable(typeof(System.Byte))]
internal class DbRepository : IDbRepository
{
    private static readonly IAmazonDynamoDB _dbClient = new AmazonDynamoDBClient();

    public DbRepository()
    {
        DbTable = Table.LoadTable(_dbClient, Utils.GetRequiredEnvVariable("DDB_TABLE"));
    }

    public Table DbTable { get; }

The exception was the same.

[3Scribe]

Apologies, I might have been a bit unclear. When you created your Lambda you would have a Main function and a Serializer Context function (these are the boilerplate ones you get when you create in Visual Studio using the AOT template)

private static async Task Main()
{
    Func<string, ILambdaContext, string> handler = FunctionHandler;
    await LambdaBootstrapBuilder.Create(handler, new SourceGeneratorLambdaJsonSerializer<LambdaFunctionJsonSerializerContext>())
        .Build()
        .RunAsync();
}

[JsonSerializable(typeof(string))]
public partial class LambdaFunctionJsonSerializerContext : JsonSerializerContext
{
    // By using this partial class derived from JsonSerializerContext, we can generate reflection free JSON Serializer code at compile time
    // which can deserialize our class and properties. However, we must attribute this class to tell it what types to generate serialization code for.
    // See https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-source-generation
}

You need to add the [JsonSerializable(typeof(System.Byte))] to the Serializer context function like this:

[JsonSerializable(typeof(System.Byte))]
[JsonSerializable(typeof(string))]
public partial class LambdaFunctionJsonSerializerContext : JsonSerializerContext
{
    // By using this partial class derived from JsonSerializerContext, we can generate reflection free JSON Serializer code at compile time
    // which can deserialize our class and properties. However, we must attribute this class to tell it what types to generate serialization code for.
    // See https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-source-generation
}

Give that a try. You might get the same error but with a different type mentioned in this like of the stack trace:

---> System.NotSupportedException: 'System.Nullable`1[System.Byte]' is missing native code or metadata. This can happen for code that is not compatible with trimming or AOT. Inspect and fix trimming and AOT related warnings that were generated when the app was published. For more information see https://aka.ms/nativeaot-compatibility

which is another directive to add to that function. Our feeling is that libraries really need to have these built in before AOT can safely be used in production.

[branog68]

Thank you for the explanation. I added it to my LambdaFunctionJsonSerializerContext:

[JsonSerializable(typeof(System.Byte))]
[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyRequest))]
[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyResponse))]
public partial class LambdaFunctionJsonSerializerContext : JsonSerializerContext
{
    // By using this partial class derived from JsonSerializerContext, we can generate reflection free JSON Serializer code at compile time
    // which can deserialize our class and properties. However, we must attribute this class to tell it what types to generate serialization code for.
    // See https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-source-generation
}

but it throws the same exception. I tried it also with
[JsonSerializable(typeof(System.Nullable<System.Byte>))]
But the result/exception was the same.

[mrkdeng]

Thank you @branog68 for contacting the team and thank you @3Scribe for the info. We'll have the right person take a look at this issue.

[Beau-Gosse-dev]

Thanks for opening this issue @branog68 (and thanks for all the helpful details). And thanks for the suggestion @3Scribe. The serialize was a good place to look, but this seems to be related to custom code in the dynamoDb library that is using reflection.

It enters into the library here:
https://github.com/aws/aws-sdk-net/blob/master/sdk/src/Services/DynamoDBv2/Custom/DocumentModel/Table.cs#L427

And eventually throws here (because the type Nullable<Byte> was trimmed out):
https://github.com/aws/aws-sdk-net/blob/43d13ab087d3938cb9428b251f1c64f02019150a/sdk/src/Services/DynamoDBv2/Custom/Conversion/DynamoDBEntryConversion.cs#L643

Unfortunately, this seems to be an error that can't be fixed by telling the compiler not to trim the DynamoDb library since the thing actually being trimmed out is in a System library. I can see if it would be possible to exclude System from trimming, but you'd probably end up with a gigantic binary that wouldn't be usable. I can also see if using a TrimmerRootAssembly or other trimming option would work.

I'll also open this issue against the aws-sdk-net repo, so they can track it there.

The real fix will take a bit more effort, which is to actually remove these reflective calls from the DynamoDb library.

[Beau-Gosse-dev]

I opened the issue here. aws/aws-sdk-net#2574

Let's track it there going forward, so I'll close this issue. But please open another issue or reopen this one if needed.

For what it's worth, I think I was able to put together something that may eventually work, but it's very much a hacky solution. The below code uses the calls to DoNothing to force the compiler to keep those types around. But I only got through 3 types, I assume more and more would just keep throwing the error.

public static string FunctionHandler(string input, ILambdaContext context)
{
        DoNothing<System.Nullable<System.Byte>>();
        DoNothing<System.Nullable<System.SByte>>();
        DoNothing<System.Nullable<System.Int16>>();

        IAmazonDynamoDB _dbClient = new AmazonDynamoDBClient();
        var DbTable = Table.LoadTable(_dbClient, "MyFakeTable");

        return input.ToUpper();
}

public static void DoNothing<[DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.All)] T>()
{
        if (typeof(T) == null)
        {
            Console.WriteLine("T is null");
        }
}

[Beau-Gosse-dev]

Possible work-around for now, here: aws/aws-sdk-net#2542 (comment)