.NET Isolated Azure Durable Functions: Support for Durable Entities with Netherite Storage Provider

Finally, Durable Entities are now supported with Netherite Storage Provider in .NET Isolated Azure Durable Functions.

While durabletask-netherite: 1.5.1 says it added support for isolated entities, unfortunately, it does not work (microsoft/durabletask-netherite/issues/390: System.NotSupportedException: Durable entities are not supported by the current backend configuration). We need Microsoft.Azure.Functions.Worker.Extensions.DurableTask.Netherite version 1.5.3 (at least) for this to work.

Now let's look at how to configure Netherite Storage Provider in a .NET Isolated Azure Durable Function.

First, we need to install the following package: 
   Microsoft.Azure.Functions.Worker.Extensions.DurableTask.Netherite >= 1.5.3. 

Then we need to update the host.json as follows.

{
  "version": "2.0",
  "extensions": {
    "durableTask": {
      "storageProvider": {
        "type": "Netherite"
      }
    }
  },
  "logging": {
    ...
  }
}

And finally, for local development, we need to update the local.settings.json as follows.

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "EventHubsConnection": "SingleHost",
    "FUNCTIONS_WORKER_RUNTIME": "dotnet-isolated"
  }
}

You need to set up an Event Hubs namespace to run Netherite on Azure and I am not covering that in this post.

Now let's write a simple function to try out the Durable Entity functionality with Netherite.

I have the following simple CounterEntity.

using Microsoft.Azure.Functions.Worker;
using Microsoft.DurableTask.Entities;
using Microsoft.Extensions.Logging;
 
namespace AzureFunctionsDemo.Netherite;
 
public class CounterEntity : TaskEntity<int>
{
    readonly ILogger logger;
 
    public CounterEntity(ILogger<CounterEntity> logger)
    {
        this.logger = logger;
    }
 
    public void Add(int amount) => State += amount;
 
    public void Reset() => State = 0;
 
    public int Get() => State;
 
    [Function(nameof(CounterEntity))]
    public Task RunEntityAsync([EntityTrigger] TaskEntityDispatcher dispatcher)
    {
        return dispatcher.DispatchAsync(this);
    }
}

And I am invoking CounterEntity through the following HTTP function that calls a Durable Function.

using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.DurableTask;
using Microsoft.DurableTask.Client;
using Microsoft.DurableTask.Entities;
using Microsoft.Extensions.Logging;
 
namespace AzureFunctionsDemo.Netherite;
 
public static class Function1
{
    [Function(nameof(HttpStart))]
    public static async Task<HttpResponseData> HttpStart(
        [HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")] HttpRequestData req,
        [DurableClient] DurableTaskClient client,
        FunctionContext executionContext)
    {
        ILogger logger = executionContext.GetLogger(nameof(HttpStart));
 
        string instanceId = await client.ScheduleNewOrchestrationInstanceAsync(nameof(EntityOrchestrator));
 
        logger.LogInformation("Started orchestration with ID = '{instanceId}'.", instanceId);
 
        return await client.CreateCheckStatusResponseAsync(req, instanceId);
    }
 
    [Function(nameof(EntityOrchestrator))]
    public static async Task<int> EntityOrchestrator(
        [OrchestrationTrigger] TaskOrchestrationContext context)
    {
        ILogger logger = context.CreateReplaySafeLogger(nameof(EntityOrchestrator));
 
        var entityId = new EntityInstanceId(nameof(CounterEntity), "myCounter");
 
        await context.Entities.CallEntityAsync(entityId, nameof(CounterEntity.Add), 10);
 
        int currentValue = await context.Entities.CallEntityAsync<int>(entityId, nameof(CounterEntity.Get));
 
        return currentValue;
    }
}

After triggering the HTTP function, it will execute the Orchestration invoking the Durable Entity. Once the Orchestration is completed and when I check the Orchestration Status, I can see the current value of the CounterEntity.

Orchestration Status

Great to see it's working, was waiting for this. 

Hope this helps.

More read:
   Configure Durable Functions with the Netherite storage provider

Happy Coding.

Regards,

Jaliya

.NET 8: Container Braking Change: Multi-platform Container Tags are Linux-only

The .NET 8 multi-platform container tags have been updated to be Linux-only. This means that the latest, <major>.<minor>, and <major>.<minor>.<patch> tags are Linux-only going forward.

Previously if we pull a .NET image with a multi-platform tag from a Windows machine, we would get a Windows image (if the tag supports the Windows platform), but not anymore.

For example: if we run the following in a pipeline using windows-latest agent, we will get an error.

FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS base
 
# ...
 
FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
 
# ...

If we want to run it on a windows agent, then we need to explicitly update tag usage to indicate which Windows version we're targeting, something like the following.

FROM mcr.microsoft.com/dotnet/aspnet:8.0-nanoserver-ltsc2022 AS base
 
# ...
 
FROM mcr.microsoft.com/dotnet/sdk:8.0-nanoserver-ltsc2022 AS build
 
# ...

We can select a Windows image name based on our requirement: Windows Server Core vs Nanoserver.

Official announcement:
   Multi-platform container tags are Linux-only

Hope this helps.

Happy Coding.

Regards,
Jaliya

Webhook to Subscribe to Event Grid System Topics in Azure Storage Account

In this post, let's see how we can subscribe to Event Grid System Topics in Azure Storage Account. Event Grid System Topics are events published by Azure services. We can configure different subscribers, but in this case for the demo purposes, I am using a Web Hook, which will basically be an Azure Function running locally. 

The Azure Function is referencing the package: Microsoft.Azure.Functions.Worker.Extensions.EventGrid and has the following simple HTTP Trigger.

using Azure.Messaging.EventGrid;
using Azure.Messaging.EventGrid.SystemEvents;
using Microsoft.AspNetCore.Http;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Extensions.Logging;
using System.Net;
 
namespace FunctionApp1;
 
public class HttpFunction
{
    private readonly ILogger<HttpFunction> _logger;
 
    public HttpFunction(ILogger<HttpFunction> logger)
    {
        _logger = logger;
    }
 
    [Function(nameof(EventWebhook))]
    public async Task<HttpResponseData> EventWebhook(
        [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequestData req)
    {
        _logger.LogInformation("C# HTTP trigger function processed a request.");
 
        using var memoryStream = new MemoryStream();
        await req.Body.CopyToAsync(memoryStream);
 
        EventGridEvent[] eventGridEvents = EventGridEvent.ParseMany(BinaryData.FromBytes(memoryStream.ToArray()));
 
        foreach (EventGridEvent eventGridEvent in eventGridEvents)
        {
            // Handle system events
            if (eventGridEvent.TryGetSystemEventData(out object eventData)
                && eventData is SubscriptionValidationEventData subscriptionValidationEventData)
            {
                // Do any additional validation (as required) and then return back ValidationCode
                var responseContent = new
                {
                    ValidationResponse = subscriptionValidationEventData.ValidationCode
                };
 
                HttpResponseData responseData = req.CreateResponse(HttpStatusCode.OK);
                await responseData.WriteAsJsonAsync(responseContent);
                return responseData;
            }
        }
 
        // TODO: Handle custom events
 
        return req.CreateResponse(HttpStatusCode.Accepted);
    }
}

Note: I am running the Azure Function in a Dev Tunnel, so the locally running Azure Function can be accessed over the Internet (which is very important when creating an Event Subscription).

Azure Function Local Dev tunnel

Now let's create the Event Subscription in the Storage Account.

Storage Account: Events

Now I am selecting Web Hook Event Subscription.

Create Event Subscription

There I have given names for the Event Subscription and the Event Grid System Topic. Now I am configuring for which storage account events I need to subscribe to by selecting Event Types, 

Event Types

Here for the demo purposes, I am only interested in Blob Created and Blob Deleted.

Now lastly we need to configure the Web Hook endpoint.

Web Hook Configuration

Here I am providing the publicly accessible endpoint of the EventWebhook I have in Azure Function. 

Web Hook Configuration

Finally, I am clicking Create on Create Event Subscription page.

We can see Azure is sending a request to the Webhook to make sure it's reachable and valid. This is important, otherwise, event subscription deployment will fail.

System Event

I am handling the system event and returning the response. 

And now I can see, the Event Subscription is successfully deployed.

Event Subscription Deployed Successfully

Now to test this functionality, I am creating a new blob in my target storage account. I can see Azure is sending a BlobCreated event.

BlobCreated Event

And when a Blob is Deleted, Azure sends a BlobDeleted event.

BlobDeleted Event

Hope this helps.

Happy Coding.

Regards,

Jaliya

EF Core 8.0: Use of Arrays with Microsoft SQL Server

I haven't really used Arrays with EF Core and then saw this nice post: A beginner’s guide to mapping arrays in EF Core 8 and wanted to try it out on Microsoft SQL Server.

Consider the below Entity.

public record Post
{
    public int Id { get; set; }
 
    public string Title { get; set; }
 
    public string[] Tags { get; set; }
 
    public DateTime[] Visits { get; set; }
}

I have Tags and Visits as arrays.

When the table is created, EF Core uses the following query.

CREATE TABLE [Posts] (
    [Id] int NOT NULL IDENTITY,
    [Title] nvarchar(max) NOT NULL,
    [Tags] nvarchar(max) NOT NULL,
    [Visits] nvarchar(max) NOT NULL,
    CONSTRAINT [PK_Posts] PRIMARY KEY ([Id])
);

Tags are Visits are created as nvarchar(max) columns, interesting.

I have inserted some data, and this is what it looks like.

Table Data

So for Microsoft SQL Server that doesn't support array types natively, EF Core is using JSON arrays.

Now I have a simple query to retrieve posts with a given tag.

List<Post> postsWithTag = await context.Posts
    .Where(x => x.Tags.Contains("Tag 1"))
    .ToListAsync();

Let's Look at the query EF generated.

SELECT [p].[Id], [p].[Tags], [p].[Title], [p].[Visits]
FROM [Posts] AS [p]
WHERE N'Tag 1' IN (
    SELECT [t].[value]
    FROM OPENJSON([p].[Tags]) WITH ([value] nvarchar(max) '$') AS [t]
)

Since the querying field is a JSON array, EF Core uses SQL OPENJSON function to parse JSON text and creates a temporary table that contains nvarchar(max) column because Tags is a string[].

Now let's consider another query on DateTime[] Visits.

List<Post> postsVisitedToday = await context.Posts
    .Where(x => x.Visits.Any(x => x.Date == DateTime.Now.Date))
    .ToListAsync();

This generated the following query.

SELECT [p].[Id], [p].[Tags], [p].[Title], [p].[Visits]
FROM [Posts] AS [p]
WHERE EXISTS (
    SELECT 1
    FROM OPENJSON([p].[Visits]) WITH ([value] datetime2 '$') AS [v]
    WHERE CONVERT(date, [v].[value]) = CONVERT(date, GETDATE())
)

This time EF Core added a filter on the temporary table that contains datetime2 column, pretty smart!

More Read:
   What's New in EF Core 8: Primitive collections

Hope this helps! 

Happy Coding.

Regards,

Jaliya