feat: Add Azure Event Hubs module

[rafek1241]

First version of PR was made by @WakaToa at #1183

What does this PR do?

Add support for Azure EventHubs emulator.

Info

You need to provide an Azurite (emulator) instance/endpoint to start the EventHub emulator. This can be done using the EventHubsConfiguration.

You need to specify the following configuration file when starting the emulator.

{
  "UserConfig": {
    "NamespaceConfig": [
      {
        "Type": "EventHub",
        "Name": "emulatorNs1",
        "Entities": [
          {
            "Name": "eh1",
            "PartitionCount": "2",
            "ConsumerGroups": [
              {
                "Name": "cg1"
              }
            ]
          }
        ]
      }
    ], 
    "LoggingConfig": {
      "Type": "File"
    }
  }
}

It will be dynamically mapped and i have built a floating builder so that the user can set the configuration as easily as possible.
The namespace "emulatorNs1" is mandatory and it is the only one currently supported by the emulator.

var configurationBuilder = ConfigurationBuilder
    .Create()
    .WithEventHub(EventHubName, "2", new[] { EventHubConsumerGroupName });

var builder = new EventHubsBuilder()
        .WithAcceptLicenseAgreement(true)
        .WithConfigurationBuilder(ConfigurationBuilder);

_eventHubsContainer = builder.Build();

await _eventHubsContainer.StartAsync();

Related issues

• Closes feat: Add Azure EventHubs module #1183
• Closes [Enhancement]: Add Azure EventHubs emulator support #1349

[netlify]

✅ Deploy Preview for testcontainers-dotnet ready!

Name	Link
🔨 Latest commit	aee785d
🔍 Latest deploy log	https://app.netlify.com/sites/testcontainers-dotnet/deploys/67a52f665ae514000867416d
😎 Deploy Preview	https://deploy-preview-1342--testcontainers-dotnet.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[rafek1241]

TODO:

• create test case for existing azurite externally added WithAzurite(...).
• Fix issues in tests

[krishankumar95]

@HofmeisterAn ; Can you please help here with assigning relevant maintainers for review. Thanks.

[krishankumar95]

Can we add tests for Kafka Send/Receive as well? Refer the Emulator Kafka sample for more details.

[rafek1241]

I can do it and also include in the documentation :)

[rafek1241]

I did the test & documentation but I need your help in there, its not working unfortunately. Maybe I did something wrong, but i'm not familiar with Kafka.

[HofmeisterAn]

I reviewed most of the PR and created a PR with a few suggestions. See: rafek1241#1.

[HofmeisterAn]

Thanks for the PR once again! I have a few small change requests. I'm happy to merge the PR afterward. Thanks 🙏.

[HofmeisterAn]

Is this additional wait strategy necessary for Event Hubs too? Is there an upstream issue we can refer to, similar to what we do for the Service Bus module?

[rafek1241]

Maybe its not needed, we need to test it out because I copied that from service bus emulator, actually.

[HofmeisterAn]

Maybe @krishankumar95 has some insights. A proper way to detect the readiness of the service is crucial for TC.

[rafek1241]

Tests are working without that so we have to wait and if there will be no response, I advice to revert that change (remove waiting strategy )

[HofmeisterAn]

This is due to the TFM. We can simply use IReadOnlyCollection and don't need the additional dependency.

namespace Testcontainers.EventHubs;

[PublicAPI]
public record RootConfiguration(UserConfig UserConfig)
{
    public UserConfig UserConfig { get; } = UserConfig;
}

[PublicAPI]
public record UserConfig(IReadOnlyCollection<NamespaceConfig> NamespaceConfig, LoggingConfig LoggingConfig)
{
    public IReadOnlyCollection<NamespaceConfig> NamespaceConfig { get; } = NamespaceConfig;

    public LoggingConfig LoggingConfig { get; } = LoggingConfig;
}

[PublicAPI]
public record NamespaceConfig(string Type, string Name, IReadOnlyCollection<Entity> Entities)
{
    public string Type { get; } = Type;

    public string Name { get; } = Name;

    public IReadOnlyCollection<Entity> Entities { get; } = Entities;
}

[PublicAPI]
public record Entity(string Name, int PartitionCount, IReadOnlyCollection<ConsumerGroup> ConsumerGroups)
{
    public string Name { get; } = Name;

    public int PartitionCount { get; } = PartitionCount;

    public IReadOnlyCollection<ConsumerGroup> ConsumerGroups { get; } = ConsumerGroups;
}

[PublicAPI]
public record ConsumerGroup(string Name)
{
    public string Name { get; } = Name;
}

[PublicAPI]
public record LoggingConfig(string Type)
{
    public string Type { get; } = Type;
}

[PublicAPI]
public sealed class EventHubsServiceConfiguration
{
    private readonly NamespaceConfig _namespaceConfig;

    private EventHubsServiceConfiguration(NamespaceConfig namespaceConfig)
    {
        _namespaceConfig = namespaceConfig;
    }

    public static EventHubsServiceConfiguration Create()
    {
        var namespaceConfig = new NamespaceConfig("EventHub", "ns-1", Array.Empty<Entity>());
        return new EventHubsServiceConfiguration(namespaceConfig);
    }

    public EventHubsServiceConfiguration WithEntity(string name, int partitionCount, params string[] consumerGroups)
    {
        return WithEntity(name, partitionCount, new ReadOnlyCollection<string>(consumerGroups));
    }

    public EventHubsServiceConfiguration WithEntity(string name, int partitionCount, IEnumerable<string> consumerGroups)
    {
        var entity = new Entity(name, partitionCount, new ReadOnlyCollection<ConsumerGroup>(consumerGroups.Select(consumerGroup => new ConsumerGroup(consumerGroup)).ToList()));
        var entities = new ReadOnlyCollection<Entity>(_namespaceConfig.Entities.Append(entity).ToList());
        return new EventHubsServiceConfiguration(new NamespaceConfig(_namespaceConfig.Type, _namespaceConfig.Name, entities));
    }

    public bool Validate()
    {
        return _namespaceConfig.Entities.All(entity => entity.PartitionCount is > 0 and <= 32 && entity.ConsumerGroups.Count is > 0 and <= 20);
    }

    public string Build()
    {
        var rootConfiguration = new RootConfiguration(new UserConfig([_namespaceConfig], new LoggingConfig("file")));
        return JsonSerializer.Serialize(rootConfiguration);
    }
}

[HofmeisterAn]

Thanks for the changes. I think we just need to address the remaining leftovers, and then we're good to merge.

[HofmeisterAn]

Shouldn't we move this to EventHubsContainer:

/// <summary>
/// Gets the broker address.
/// </summary>
/// <returns>The broker address.</returns>
public string GetBootstrapAddress()
{
    return new UriBuilder("PLAINTEXT", Hostname, GetMappedPublicPort(EventHubsBuilder.KafkaPort)).ToString();
}

[HofmeisterAn]

Suggested change

	/// <summary>
	/// Gets the Event Hubs connection string.
	/// </summary>
	/// <returns>The Event Hubs connection string.</returns>
	public string GetConnectionString()
	{
	var properties = new Dictionary<string, string>
	{
	{
	"Endpoint",
	new UriBuilder(Uri.UriSchemeHttp, Hostname, GetMappedPublicPort(EventHubsBuilder.EventHubsPort))
	.ToString()
	},
	{ "DefaultEndpointsProtocol", Uri.UriSchemeHttp },
	{ "SharedAccessKeyName", "RootManageSharedAccessKey" },
	{ "SharedAccessKey", "SAS_KEY_VALUE" },
	{ "UseDevelopmentEmulator", "true" },
	};
	return string.Join(";", properties.Select(property => string.Join("=", property.Key, property.Value)));
	}
	/// <summary>
	/// Gets the Event Hubs connection string.
	/// </summary>
	/// <returns>The Event Hubs connection string.</returns>
	public string GetConnectionString()
	{
	var properties = new Dictionary<string, string>();
	properties.Add("Endpoint", new UriBuilder(Uri.UriSchemeHttp, Hostname, GetMappedPublicPort(EventHubsBuilder.EventHubsPort)).ToString());
	properties.Add("DefaultEndpointsProtocol", Uri.UriSchemeHttp);
	properties.Add("SharedAccessKeyName", "RootManageSharedAccessKey");
	properties.Add("SharedAccessKey", "SAS_KEY_VALUE");
	properties.Add("UseDevelopmentEmulator", "true");
	return string.Join(";", properties.Select(property => string.Join("=", property.Key, property.Value)));
	}

Can we please keep it as it was? It is much easier for me to maintain the repository when the projects and modules are aligned. This helps me maintain an overview and refactor things more efficiently.