Thursday, June 20, 2024

.NET Isolated Azure Durable Functions: Support for Durable Entities with Netherite Storage Provider

Finally, Durable Entities are now supported with Netherite Storage Provider in .NET Isolated Azure Durable Functions.

While durabletask-netherite: 1.5.1 says it added support for isolated entities, unfortunately, it does not work (microsoft/durabletask-netherite/issues/390: System.NotSupportedException: Durable entities are not supported by the current backend configuration). We need Microsoft.Azure.Functions.Worker.Extensions.DurableTask.Netherite version 1.5.3 (at least) for this to work.

Now let's look at how to configure Netherite Storage Provider in a .NET Isolated Azure Durable Function.

First, we need to install the following package: 
   Microsoft.Azure.Functions.Worker.Extensions.DurableTask.Netherite >= 1.5.3

Then we need to update the host.json as follows.

{
  "version""2.0",
  "extensions": {
    "durableTask": {
      "storageProvider": {
        "type""Netherite"
      }
    }
  },
  "logging": {
    
...
  }
}

And finally, for local development, we need to update the local.settings.json as follows.

{
  "IsEncrypted"false,
  "Values": {
    "AzureWebJobsStorage""UseDevelopmentStorage=true",
    "EventHubsConnection""SingleHost",
    "FUNCTIONS_WORKER_RUNTIME""dotnet-isolated"
  }
}

You need to set up an Event Hubs namespace to run Netherite on Azure and I am not covering that in this post.

Now let's write a simple function to try out the Durable Entity functionality with Netherite.

I have the following simple CounterEntity.

using Microsoft.Azure.Functions.Worker;
using Microsoft.DurableTask.Entities;
using Microsoft.Extensions.Logging;

namespace AzureFunctionsDemo.Netherite;

public class CounterEntity : TaskEntity<int>
{
    readonly ILogger logger;

    public CounterEntity(ILogger<CounterEntity> logger)
    {
        this.logger = logger;
    }

    public void Add(int amount) => State += amount;

    public void Reset() => State = 0;

    public int Get() => State;

    [Function(nameof(CounterEntity))]
    public Task RunEntityAsync([EntityTrigger] TaskEntityDispatcher dispatcher)
    {
        return dispatcher.DispatchAsync(this);
    }
}

And I am invoking CounterEntity through the following HTTP function that calls a Durable Function.

using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.DurableTask;
using Microsoft.DurableTask.Client;
using Microsoft.DurableTask.Entities;
using Microsoft.Extensions.Logging;

namespace AzureFunctionsDemo.Netherite;

public static class Function1
{
    [Function(nameof(HttpStart))]
    public static async Task<HttpResponseData> HttpStart(
        [HttpTrigger(AuthorizationLevel.Anonymous, "get""post")] HttpRequestData req,
        [DurableClient] DurableTaskClient client,
        FunctionContext executionContext)
    {
        ILogger logger = executionContext.GetLogger(nameof(HttpStart));

        string instanceId = await client.ScheduleNewOrchestrationInstanceAsync(nameof(EntityOrchestrator));

        logger.LogInformation("Started orchestration with ID = '{instanceId}'."instanceId);

        return await client.CreateCheckStatusResponseAsync(reqinstanceId);
    }

    [Function(nameof(EntityOrchestrator))]
    public static async Task<int> EntityOrchestrator(
        [OrchestrationTrigger] TaskOrchestrationContext context)
    {
        ILogger logger = context.CreateReplaySafeLogger(nameof(EntityOrchestrator));

        var entityId = new EntityInstanceId(nameof(CounterEntity)"myCounter");

        await context.Entities.CallEntityAsync(entityIdnameof(CounterEntity.Add), 10);

        int currentValue = await context.Entities.CallEntityAsync<int>(entityIdnameof(CounterEntity.Get));

        return currentValue;
    }
}

After triggering the HTTP function, it will execute the Orchestration invoking the Durable Entity. Once the Orchestration is completed and when I check the Orchestration Status, I can see the current value of the CounterEntity.

Orchestration Status
Great to see it's working, was waiting for this. 

Hope this helps.

More read:
   Configure Durable Functions with the Netherite storage provider

Happy Coding.

Regards,
Jaliya

Wednesday, June 19, 2024

.NET 8: Container Braking Change: Multi-platform Container Tags are Linux-only

The .NET 8 multi-platform container tags have been updated to be Linux-only. This means that the latest<major>.<minor>, and <major>.<minor>.<patch> tags are Linux-only going forward.

Previously if we pull a .NET image with a multi-platform tag from a Windows machine, we would get a Windows image (if the tag supports the Windows platform), but not anymore.

For example: if we run the following in a pipeline using windows-latest agent, we will get an error.

FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS base

# ...

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build

# ...

If we want to run it on a windows agent, then we need to explicitly update tag usage to indicate which Windows version we're targeting, something like the following.

FROM mcr.microsoft.com/dotnet/aspnet:8.0-nanoserver-ltsc2022 AS base

# ...

FROM mcr.microsoft.com/dotnet/sdk:8.0-nanoserver-ltsc2022 AS build

# ...

We can select a Windows image name based on our requirement: Windows Server Core vs Nanoserver.

Official announcement:
   Multi-platform container tags are Linux-only

Hope this helps.

Happy Coding.

Regards,
Jaliya

Monday, June 17, 2024

Webhook to Subscribe to Event Grid System Topics in Azure Storage Account

In this post, let's see how we can subscribe to Event Grid System Topics in Azure Storage AccountEvent Grid System Topics are events published by Azure services. We can configure different subscribers, but in this case for the demo purposes, I am using a Web Hook, which will basically be an Azure Function running locally. 

The Azure Function is referencing the package: Microsoft.Azure.Functions.Worker.Extensions.EventGrid and has the following simple HTTP Trigger.
using Azure.Messaging.EventGrid;
using Azure.Messaging.EventGrid.SystemEvents;
using Microsoft.AspNetCore.Http;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Extensions.Logging;
using System.Net;

namespace FunctionApp1;

public class HttpFunction
{
    private readonly ILogger<HttpFunction> _logger;

    public HttpFunction(ILogger<HttpFunctionlogger)
    {
        _logger = logger;
    }

    [Function(nameof(EventWebhook))]
    public async Task<HttpResponseDataEventWebhook(
        [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequestData req)
    {
        _logger.LogInformation("C# HTTP trigger function processed a request.");

        using var memoryStream = new MemoryStream();
        await req.Body.CopyToAsync(memoryStream);

        EventGridEvent[] eventGridEvents = EventGridEvent.ParseMany(BinaryData.FromBytes(memoryStream.ToArray()));

        foreach (EventGridEvent eventGridEvent in eventGridEvents)
        {
            // Handle system events
            if (eventGridEvent.TryGetSystemEventData(out object eventData)
                && eventData is SubscriptionValidationEventData subscriptionValidationEventData)
            {
                // Do any additional validation (as required) and then return back ValidationCode
                var responseContent = new
                {
                    ValidationResponse = subscriptionValidationEventData.ValidationCode
                };

                HttpResponseData responseData = req.CreateResponse(HttpStatusCode.OK);
                await responseData.WriteAsJsonAsync(responseContent);
                return responseData;
            }
        }

        // TODO: Handle custom events

        return req.CreateResponse(HttpStatusCode.Accepted);
    }
}
Note: I am running the Azure Function in a Dev Tunnel, so the locally running Azure Function can be accessed over the Internet (which is very important when creating an Event Subscription).
Azure Function Local Dev tunnel
Now let's create the Event Subscription in the Storage Account.
Storage Account: Events
Now I am selecting Web Hook Event Subscription.
Create Event Subscription
There I have given names for the Event Subscription and the Event Grid System Topic. Now I am configuring for which storage account events I need to subscribe to by selecting Event Types
Event Types
Here for the demo purposes, I am only interested in Blob Created and Blob Deleted.

Now lastly we need to configure the Web Hook endpoint.
Web Hook Configuration
Here I am providing the publicly accessible endpoint of the EventWebhook I have in Azure Function. 
Web Hook Configuration
Finally, I am clicking Create on Create Event Subscription page.

We can see Azure is sending a request to the Webhook to make sure it's reachable and valid. This is important, otherwise, event subscription deployment will fail.
System Event
I am handling the system event and returning the response. 

And now I can see, the Event Subscription is successfully deployed.
Event Subscription Deployed Successfully
Now to test this functionality, I am creating a new blob in my target storage account. I can see Azure is sending a BlobCreated event.
BlobCreated Event
And when a Blob is Deleted, Azure sends a BlobDeleted event.
BlobDeleted Event
Hope this helps.

Happy Coding.

Regards,
Jaliya

Friday, June 14, 2024

EF Core 8.0: Use of Arrays with Microsoft SQL Server

I haven't really used Arrays with EF Core and then saw this nice post: A beginner’s guide to mapping arrays in EF Core 8 and wanted to try it out on Microsoft SQL Server.

Consider the below Entity.
public record Post
{
    public int Id { getset}

    public string Title { getset}

    public string[] Tags { getset}

    public DateTime[] Visits { getset}
}
I have Tags and Visits as arrays.

When the table is created, EF Core uses the following query.
CREATE TABLE [Posts] (
    [Id] int NOT NULL IDENTITY,
    [Title] nvarchar(max) NOT NULL,
    [Tags] nvarchar(max) NOT NULL,
    [Visits] nvarchar(max) NOT NULL,
    CONSTRAINT [PK_Posts] PRIMARY KEY ([Id])
);
Tags are Visits are created as nvarchar(max) columns, interesting.

I have inserted some data, and this is what it looks like.
Table Data
So for Microsoft SQL Server that doesn't support array types natively, EF Core is using JSON arrays.

Now I have a simple query to retrieve posts with a given tag.
List<Post> postsWithTag = await context.Posts
    .Where(x => x.Tags.Contains("Tag 1"))
    .ToListAsync();
Let's Look at the query EF generated.
SELECT [p].[Id], [p].[Tags], [p].[Title], [p].[Visits]
FROM [Posts] AS [p]
WHERE N'Tag 1' IN (
    SELECT [t].[value]
    FROM OPENJSON([p].[Tags]) WITH ([value] nvarchar(max) '$') AS [t]
)
Since the querying field is a JSON array, EF Core uses SQL OPENJSON function to parse JSON text and creates a temporary table that contains nvarchar(max) column because Tags is a string[].

Now let's consider another query on DateTime[] Visits.
List<Post> postsVisitedToday = await context.Posts
    .Where(x => x.Visits.Any(x => x.Date == DateTime.Now.Date))
    .ToListAsync();
This generated the following query.
SELECT [p].[Id], [p].[Tags], [p].[Title], [p].[Visits]
FROM [Posts] AS [p]
WHERE EXISTS (
    SELECT 1
    FROM OPENJSON([p].[Visits]) WITH ([value] datetime2 '$') AS [v]
    WHERE CONVERT(date, [v].[value]) = CONVERT(date, GETDATE())
)
This time EF Core added a filter on the temporary table that contains datetime2 column, pretty smart!
Hope this helps! 

Happy Coding.

Regards,
Jaliya

Monday, May 27, 2024

.NET 9 and ASP.NET Core: Built-in Support for OpenAPI Document Generation

With .NET 9, ASP.NET Core now has built-in support for OpenAPI document generation in both controller-based and minimal APIs. For as long as I can remember, ASP.NET Core has been using Swagger to generate the Open API document. Now we have Microsoft.AspNetCore.OpenApi package, and technically we can get rid of using Swagger. The new package still doesn't support a rich UI like Swagger UI, and that's something to look forward to.

Now let's see how this works.

Install the package: Microsoft.AspNetCore.OpenApi, note: it has to be the latest preview as of today and that is 9.0.0-preview.4.24267.6 (or any newer version than this).
<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>net9.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.OpenApi" Version="9.0.0-preview.4.24267.6" />
  </ItemGroup>

</Project>
Now we can create a simple API something like follows:
WebApplicationBuilder builder = WebApplication.CreateBuilder(args);

builder.Services.AddOpenApi();

WebApplication app = builder.Build();

if (app.Environment.IsDevelopment())
{
    app.MapOpenApi();
}

app.MapGet("/hello", () => "Hello world!")
    .WithDescription("Returns Hello");

app.Run();
You can access the OpenAPI document at: https://localhost:<port>/openapi/v1.json
OpenAPI Document
Read the following documentation to learn all the different customization options:

Hope this helps.

Happy Coding.

Regards,
Jaliya

Wednesday, May 22, 2024

C# 13.0: params Improvements

It's another exciting time of the year when Microsoft Build is happening, a lot of exciting announcements.

In this post, let's have a look at a C# 13.0 feature that is now available with the latest Visual Studio 2022 Version 17.11 Preview 1.

C# 13.0 is supported on .NET 9. To try this feature, make sure you are using the preview language version.
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net9.0</TargetFramework>
    <LangVersion>preview</LangVersion>
  </PropertyGroup>

</Project>
Prior to C# 13.0, when we are using params, the parameter must be a single-dimensional array, something like follows:
void DoSomething(params string[] items)
{
    
// do something with items
}
And we can call it as follows:
DoSomething("apple", "orange"); // comma separated list
DoSomething(["apple", "orange"]); // array
DoSomething(); // empty, the length of the params list is zero
With C# 13.0, params parameter type can be any recognized collection type, like List<T>, Span<T>, IEnumerable<T>, etc. You can even use your own collection types if they follow special rules.

So from C# 13.0, the following is totally valid.
void DoSomething(params IEnumerable<string> items)
{
    
// do something with items
}
That's pretty cool.

If you still haven't registered for Microsoft Build 2024, you are still not late. Register now for free and access all the exciting content.

Happy Coding.

Regards,
Jaliya

Tuesday, May 14, 2024

C# 12.0: .. Spread Element or Spread Operator

C# 12.0 introduced .. and its terminology is a bit confusing, some call it Spread Element and some call it Spread Operator.

Before going through the correct terminology, first, let's see what it does.

IEnumerable<string> sweetFruits = ["Apple", "Banana", "Mango", "Pineapple"];
IEnumerable<string> sourFruits = ["Orange", "Grapefruit", "Lemon", "Lime"];

IEnumerable<string> fruits = [.. sweetFruits, .. sourFruits];
// Output: Apple, Banana, Mango, Pineapple, Orange, Grapefruit, Lemon, Lime

Here .. is spreading the elements in a collection. We are spreading the sweetFruits and sourFruits, and then combining those to create fruits.

And we can use this feature in different ways.

For an example consider this.

IEnumerable<Employee> employees =
[
    new Employee("John Doe", "Contract"),
    new Employee("Jane Doe", "Permanent")
];

List<Employee> permanentEmployees = employees
    .Where(e => e.Type == "Permanent")
    .ToList();

We can use .. and filter permanentEmployees as follows and not do .ToList().

List<Employee> permanentEmployees =
[
    .. employees.Where(e => e.Type == "Permanent")
];

Now what do we call it? 

There is a nice explanation given in this post: .NET Blog: Refactor your code with C# collection expressions

Spread Element
I personally agree with the explanation and even the feature specification uses the term Spread Element. But there are some places in official .NET documentation (like here: C# 12: Collection Expressions) that refer .. as Spread Operator.

Hopefully, we can get this terminology consistent across.

Hope this helps.

Happy Coding.

Regards,
Jaliya


Update 15/05/2024:

Reached out to .NET team and they are already in the process of addressing inconsistencies, both in the feature spec and the docs. It is going to be called the Spread Element and not Spread Operator.

Also, note that .. is used in three different places in the language: in collection expressions to indicate a spread element, in list patterns to indicate a slice pattern and as the range operator. The only location where it’s an operator is the range operator.

Wednesday, May 1, 2024

Visual Studio: New Durable Functions Project: System.InvalidOperationException: Synchronous operations are disallowed

When creating a new Azure Function App Project and selecting Durable Functions Orchestration, right now the basic functionality is erroring out when the default Http Function is triggered.

Error: System.InvalidOperationException: Synchronous operations are disallowed. Call WriteAsync or set AllowSynchronousIO to true instead.


Unfortunately, this is a known issue (https://github.com/Azure/azure-functions-dotnet-worker/issues/2425) with the templates and hopefully, the templates will get updated soon.

For the time being, you can update the code in the HttpStart function to use CreateCheckStatusResponseAsync as follows.
[Function("Function1_HttpStart")]
public static async Task<HttpResponseData> HttpStart(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")] HttpRequestData req,
    [DurableClient] DurableTaskClient client,
    FunctionContext executionContext)
{
    ILogger logger = executionContext.GetLogger("Function1_HttpStart");

    // Function input comes from the request content.
    string instanceId = await client.ScheduleNewOrchestrationInstanceAsync(
        nameof(Function1));

    logger.LogInformation("Started orchestration with ID = '{instanceId}'.", instanceId);

    
// Returns an HTTP 202 response with an instance management payload.
    // See https://learn.microsoft.com/azure/azure-functions/durable/durable-functions-http-api#start-orchestration

    return await client.CreateCheckStatusResponseAsync(req, instanceId);
}
Hope this helps.

Happy Coding.

Regards,
Jaliya

Thursday, April 18, 2024

Session: App Service Networking Features for Developers at 2024 Global Azure, Auckland

Delivered a session today at 2024 Global Azure, Auckland, the title of the talk was App Service Networking Features for Developers.
 App Service Networking Features for Developers at 2024 Global Azure, Auckland
In the session, I demoed and talked through the following.
  • How we can add Private Endpoints to Azure Services (Azure Storage in this case)
  • How we can integrate an App Service to a VNET and verify it's communicating with a private endpoint-enabled Azure Service via internal IP
  • How App Service in another VNET can communicate with the above private endpoint-enabled Azure Service (via network peering)
Basically how to migrate from something like the below,
Public
To something like the below.
Restricted
Happy Coding.

Regards,
Jaliya

Wednesday, April 17, 2024

Azure API Management: Replace Backend Service URLs in Response Body

In this post, let's see how we can replace backend API URLs in the response body from an Azure API Management (APIM) policy.

Say, we have a backend API endpoint that has the following code.

WebApplicationBuilder builder = WebApplication.CreateBuilder(args);

WebApplication app = builder.Build();

app.UseHttpsRedirection();

app.MapGet("/endpoints", (HttpContext httpContext) =>
{
    string baseUrl = $"{httpContext.Request.Scheme}://{httpContext.Request.Host}";

    return new
    {
        StatusQueryUri = $"{baseUrl}/status",
        HealthQueryUri = $"{baseUrl}/health",
    };
})
.WithName("GetEndpoints")
.WithOpenApi(); // other endpoints

And it would work as follows.

Response from Backend API
Now if we are exposing this API via Azure APIM, we can't be returning internal endpoints. We need to replace the Base URL with the corresponding APIM API endpoints.

To achieve that we can use set-body policy and do something like the following.

<policies>
  <inbound>
    <base />
  </inbound>
  <backend>
    <base />
  </backend>
  <outbound>
    <base />
    <set-body>
    @{
        string urlToReplace = context.Request.Url.Scheme + "://" + context.Request.Url.Host;
        string urlToReplaceWith = context.Request.OriginalUrl.Scheme
          + "://" + context.Request.OriginalUrl.Host 
          + context.Api.Path;
          
        string response = context.Response.Body.As<string>();
        return response.Replace(urlToReplace, urlToReplaceWith);
    }
    </set-body>
  </outbound>
  <on-error>
    <base />
  </on-error>
</policies>

Here,

The output is as follows:

Response from APIM
Hope this helps.

Happy Coding.

Regards,
Jaliya

Tuesday, April 16, 2024

.NET Isolated Azure Durable Functions: Wait for Any Event and Wait for All the Events

In this post, let's see how we can wait for any event and wait for all the events in .NET Isolated Azure Durable Functions.

Let's consider the below code.

using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.DurableTask;
using Microsoft.DurableTask.Client;
using Microsoft.Extensions.Logging;
using System.Text.Json.Nodes;

namespace FunctionApp1;

public static class Function1
{
    [Function(nameof(HttpStart))]
    public static async Task<HttpResponseData> HttpStart(
        [HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")] HttpRequestData req,
        [DurableClient] DurableTaskClient client,
        FunctionContext executionContext)
    {
        ILogger logger = executionContext.GetLogger(nameof(HttpStart));

        string instanceId = await client.ScheduleNewOrchestrationInstanceAsync(nameof(Orchestrator));

        return client.CreateCheckStatusResponse(req, instanceId);
    }

    [Function(nameof(Orchestrator))]
    public static async Task<Dictionary<string, JsonObject>> Orchestrator(
        [OrchestrationTrigger] TaskOrchestrationContext context)
    {
        ILogger logger = context.CreateReplaySafeLogger(nameof(Orchestrator));

        List<string> events = [
            "Event1",
            "Event2",
            "Event3"
        ];

        Dictionary<string, Task<JsonObject>> tasks = [];

        foreach (string eventName in events)
        {
            logger.LogInformation($"http://localhost:7137/runtime/webhooks/durabletask/instances/{context.InstanceId}/raiseEvent/{eventName}");

            Task<JsonObject> task = context.WaitForExternalEvent<JsonObject>(eventName);
            tasks.Add(eventName, task);
        }

        // TODO: Handle WhenAny

        // TODO: Handle WhenAll
    }
}

Here I have a list of WaitForExternalEvent tasks, that we are going to listen to. The first scenario is we are going to resume execution when any of the events occur. The second scenario is we are going to resume execution when all of the events have occurred.

WhenAny

We can use the Task Parallel Libraries' (TPL) Task.WhenAny for this scenario.

// Handling WhenAny
Task<JsonObject> result = await Task.WhenAny(tasks.Values);

KeyValuePair<string, Task<JsonObject>> winner = tasks.Single(x => x.Value == result);
return new Dictionary<string, JsonObject>
{
    { winner.Key, await result }
};

Say for example, we are triggering "Event2", at that time the execution will resume.

WhenAny

WhenAll

We can use the Task Parallel Libraries' (TPL) Task.WhenAll for this scenario.

// Handling WhenAll
JsonObject[] results = await Task.WhenAll(tasks.Values);

Dictionary<string, JsonObject> resultsMap = tasks
    .Zip(results, (x, y) => new { x.Key, y })
    .ToDictionary(x => x.Key, x => x.y);

return resultsMap;
In this case, the execution won't resume until all the events are triggered.
WhenAll
Hope this helps.

Happy Coding.

Regards,
Jaliya