Thursday, April 18, 2024

Session: App Service Networking Features for Developers at 2024 Global Azure, Auckland

Delivered a session today at 2024 Global Azure, Auckland, the title of the talk was App Service Networking Features for Developers.
 App Service Networking Features for Developers at 2024 Global Azure, Auckland
In the session, I demoed and talked through the following.
  • How we can add Private Endpoints to Azure Services (Azure Storage in this case)
  • How we can integrate an App Service to a VNET and verify it's communicating with a private endpoint-enabled Azure Service via internal IP
  • How App Service in another VNET can communicate with the above private endpoint-enabled Azure Service (via network peering)
Basically how to migrate from something like the below,
Public
To something like the below.
Restricted
Happy Coding.

Regards,
Jaliya

Wednesday, April 17, 2024

Azure API Management: Replace Backend Service URLs in Response Body

In this post, let's see how we can replace backend API URLs in the response body from an Azure API Management (APIM) policy.

Say, we have a backend API endpoint that has the following code.

WebApplicationBuilder builder = WebApplication.CreateBuilder(args);

WebApplication app = builder.Build();

app.UseHttpsRedirection();

app.MapGet("/endpoints", (HttpContext httpContext) =>
{
    string baseUrl = $"{httpContext.Request.Scheme}://{httpContext.Request.Host}";

    return new
    {
        StatusQueryUri = $"{baseUrl}/status",
        HealthQueryUri = $"{baseUrl}/health",
    };
})
.WithName("GetEndpoints")
.WithOpenApi(); // other endpoints

And it would work as follows.

Response from Backend API
Now if we are exposing this API via Azure APIM, we can't be returning internal endpoints. We need to replace the Base URL with the corresponding APIM API endpoints.

To achieve that we can use set-body policy and do something like the following.

<policies>
  <inbound>
    <base />
  </inbound>
  <backend>
    <base />
  </backend>
  <outbound>
    <base />
    <set-body>
    @{
        string urlToReplace = context.Request.Url.Scheme + "://" + context.Request.Url.Host;
        string urlToReplaceWith = context.Request.OriginalUrl.Scheme
          + "://" + context.Request.OriginalUrl.Host 
          + context.Api.Path;
          
        string response = context.Response.Body.As<string>();
        return response.Replace(urlToReplace, urlToReplaceWith);
    }
    </set-body>
  </outbound>
  <on-error>
    <base />
  </on-error>
</policies>

Here,

The output is as follows:

Response from APIM
Hope this helps.

Happy Coding.

Regards,
Jaliya

Tuesday, April 16, 2024

.NET Isolated Azure Durable Functions: Wait for Any Event and Wait for All the Events

In this post, let's see how we can wait for any event and wait for all the events in .NET Isolated Azure Durable Functions.

Let's consider the below code.

using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.DurableTask;
using Microsoft.DurableTask.Client;
using Microsoft.Extensions.Logging;
using System.Text.Json.Nodes;

namespace FunctionApp1;

public static class Function1
{
    [Function(nameof(HttpStart))]
    public static async Task<HttpResponseData> HttpStart(
        [HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")] HttpRequestData req,
        [DurableClient] DurableTaskClient client,
        FunctionContext executionContext)
    {
        ILogger logger = executionContext.GetLogger(nameof(HttpStart));

        string instanceId = await client.ScheduleNewOrchestrationInstanceAsync(nameof(Orchestrator));

        return client.CreateCheckStatusResponse(req, instanceId);
    }

    [Function(nameof(Orchestrator))]
    public static async Task<Dictionary<string, JsonObject>> Orchestrator(
        [OrchestrationTrigger] TaskOrchestrationContext context)
    {
        ILogger logger = context.CreateReplaySafeLogger(nameof(Orchestrator));

        List<string> events = [
            "Event1",
            "Event2",
            "Event3"
        ];

        Dictionary<string, Task<JsonObject>> tasks = [];

        foreach (string eventName in events)
        {
            logger.LogInformation($"http://localhost:7137/runtime/webhooks/durabletask/instances/{context.InstanceId}/raiseEvent/{eventName}");

            Task<JsonObject> task = context.WaitForExternalEvent<JsonObject>(eventName);
            tasks.Add(eventName, task);
        }

        // TODO: Handle WhenAny

        // TODO: Handle WhenAll
    }
}

Here I have a list of WaitForExternalEvent tasks, that we are going to listen to. The first scenario is we are going to resume execution when any of the events occur. The second scenario is we are going to resume execution when all of the events have occurred.

WhenAny

We can use the Task Parallel Libraries' (TPL) Task.WhenAny for this scenario.

// Handling WhenAny
Task<JsonObject> result = await Task.WhenAny(tasks.Values);

KeyValuePair<string, Task<JsonObject>> winner = tasks.Single(x => x.Value == result);
return new Dictionary<string, JsonObject>
{
    { winner.Key, await result }
};

Say for example, we are triggering "Event2", at that time the execution will resume.

WhenAny

WhenAll

We can use the Task Parallel Libraries' (TPL) Task.WhenAll for this scenario.

// Handling WhenAll
JsonObject[] results = await Task.WhenAll(tasks.Values);

Dictionary<string, JsonObject> resultsMap = tasks
    .Zip(results, (x, y) => new { x.Key, y })
    .ToDictionary(x => x.Key, x => x.y);

return resultsMap;
In this case, the execution won't resume until all the events are triggered.
WhenAll
Hope this helps.

Happy Coding.

Regards,
Jaliya

Sunday, April 14, 2024

.NET Isolated Azure Durable Functions: Specifying SubOrchestrator Instance Id

In Azure Durable Functions, there are times we need to run SubOrchestrations using a specific Instance Id

As far as I can recall in In Process durable functions, when calling a SubOrchestrator there was a specific overload to specify the instance Id. With Isolated Durable Functions, we do have an overload, but I feel it's not that intuitive, hence this small post.

In .NET Isolated Azure Durable Functions, this is how we can specify SubOrchestrator InstanceId.

Passing SubOrchestratorOptions
Here, for TaskOptions, we can pass SubOrchestrationOptions, a derived type of TaskOptions as follows.

await context.CallSubOrchestratorAsync("SubOrchestrator1", new SubOrchestrationOptions
{
    InstanceId = "SomeInstanceId",
});

Hope this helps.

Happy Coding.

Regards,
Jaliya

Monday, April 8, 2024

Middleware in .NET Isolated Azure Functions

In this post, let's have a look at Middleware in .NET Isolated Azure Functions. .NET Isolated functions supports middleware registration, following a similar model as in ASP.NET Core. With middleware, we can inject logic into the invocation pipeline, and before and after functions execute.

The ConfigureFunctionsWorkerDefaults method has an overload that we can register middleware as follows.
IHost host = new HostBuilder()
    .ConfigureFunctionsWorkerDefaults((context, builder) =>
    {
        // Register Middleware

        // This middleware is applied to all functions
        builder.UseMiddleware<MyCustomMiddleware>();

        // This middleware is only applied when running integration tests
        IConfiguration configuration = builder.Services.BuildServiceProvider().GetService<IConfiguration>();
        bool isRunningIntegrationTests = configuration.GetValue<bool>("IsRunningIntegrationTests");
         builder.UseWhen<IntegrationTestsFunctionMiddleware>((context) => isRunningIntegrationTests);
    })
    .Build();
To implement a middleware, you need to implement the interface IFunctionsWorkerMiddleware
internal sealed class IntegrationTestsFunctionMiddleware : IFunctionsWorkerMiddleware
{
    internal static class FunctionBindingTypes
    {
        public const string ServiceBus = "serviceBus";
    }

    public async Task Invoke(FunctionContext context, FunctionExecutionDelegate next)
    {
        // If the function has a service bus output binding, short-circuit the execution of the function
        if (context.FunctionDefinition.OutputBindings.Values.Any(a => a.Type == FunctionBindingTypes.ServiceBus))
        {
            return;
        }

        await next(context);
    }
}
Above IntegrationTestsFunctionMiddleware short-circuits the execution of the function if the function has a serviceBus output binding.

Hope this helps.

Happy Coding.

Regards,
Jaliya

Tuesday, March 19, 2024

App Service Outbound Traffic through VNet Gets 403 When Trying to Access Another App Service with Public Network Access Enabled, but has a Private Endpoint

In this post, let's go through an interesting scenario related to App Service networking.

- App A: is integrated into VNet A.
App A: Networking
- App B: has Public network access enabled with no access restrictions. But it has a Private Endpoint in VNet B.
App B: Networking
- App C: has Public network access enabled with no access restrictions. No private endpoints.
App C: Networking
Now the interesting part. I am seeing the following access behavior.
  • App A -> App B: 403 Forbidden. From anywhere else -> App B: 200.
  • App A -> App C: 200
I was scratching my head for a couple of days trying to understand why is that App A -> App B: 403 Forbidden. Because App B has Public network access enabled and also a private endpoint and public access can co-exist on an app.

I can fix this by peering VNet A and VNet B. But I still needed to figure out why App A isn't reaching App B on the default endpoint (not on the private endpoint) as it's like App B is receiving traffic from any other source.

Finally, I got an explanation from Mads Wolff Damg√•rd (a Principal Product Manager at Microsoft).

This is happening because I had Service Endpoint registered to Microsoft.Web in VNet As' integration subnet for App A
VNet As' integration subnet for App A
When we have a service endpoint registered, the traffic is sent over the public channel, but as service endpoint traffic. This uses the same protocol as the private endpoint and will then try to parse it as a private endpoint call, but since VNet A has no knowledge about the private endpoint, the traffic fails with 403.

Once I removed the service endpoint registration, App A was able to reach App B without any issues.

So hope someone finds this helpful!

Happy Coding.

Regards,
Jaliya

Friday, March 15, 2024

Read TLS/SSL Certificate in Azure App Service from C# Code

Recently I was updating an old .NET Core web application to .NET 8 and the code was reading a certificate as follows.

private X509Certificate2 GetCertificateByThumbprint(string thumbprint)
{
    X509Store store = new (StoreName.My, StoreLocation.CurrentUser);
    store.Open(OpenFlags.ReadOnly);
    X509Certificate2Collection certificateCollection =  store.Certificates.Find(X509FindType.FindByThumbprint, thumbprint, true);
    return certificateCollection.OfType<X509Certificate2>().SingleOrDefault();
}

This piece of code wasn't working once the application is deployed to Azure App Service (Windows). The certificate is set up in App Service, but the code wasn't picking it up. As usual, QAs were insisting it used to work.

It seems I needed to add an app setting WEBSITE_LOAD_CERTIFICATES with the value of comma-separated certificate thumbprints in order for them be loaded and accessible from App Service code.

{
  "name""WEBSITE_LOAD_CERTIFICATES",
  "value""<comma-separated-certificate-thumbprints>",
  "slotSetting"false
}

You can read more on Use a TLS/SSL certificate in your code in Azure App Service. It contains instructions for other scenarios like loading a certificate from a file and loading a certificate in Linux/Windows containers.

Hope this helps.

Happy Coding.

Regards,
Jaliya

Monday, March 11, 2024

Azure AD B2C: Call an External API Using Client Credentials in an User Journey

In this post, let's see how to call an external API using Client Credentials in an Azure AD B2C User Journey.

I am assuming Azure AD B2C App Registration is already set up for the client app with the necessary permission  (scope access) to call the protected API and you have noted down the Client ID, Client Secret, and the Scope.

Note: There are no additional actions to enable the client credentials for user flows or custom policies. Both Azure AD B2C user flows and custom policies support the client credentials flow by default. But of course, you can create a custom policy to customize the user journey of the OAuth 2.0 Client credentials and extend the token issuance process.

First, you can test that everything is set up correctly using the following Powershell script.

$clientId = "<clientId>"
$clientSecret = "<clientSecret>"
$endpoint = "https://<tenant-name>.b2clogin.com/<tenant-name>.onmicrosoft.com/<policy>/oauth2/v2.0/token"
$scope = "<scope>"
$body = "grant_type=client_credentials&scope=" + $scope + "&client_id=" + $clientId + "&client_secret=" + $clientSecret

$token = Invoke-RestMethod -Method Post -Uri $endpoint -Body $body
$token | ConvertTo-Json

Here the scope is something like follows:

$scope = "https://<tenant-name>.onmicrosoft.com/45a2252d-099a-4c6a-9c57-66eac05e2693/.default"
The script should output something like below.
Test Client Credentials
Now let's see how we can use this in an Azure AD B2C User Journey.

1. Define a ClaimType for access_token.

<BuildingBlocks>
  <ClaimsSchema> ...
    <ClaimType Id="access_token">
      <DisplayName>Access Token</DisplayName>
      <DataType>string</DataType>
    </ClaimType>
  </ClaimsSchema> ... </BuildingBlocks>

2. Define TechnicalProfiles to retrieve access_token and to call the external API using the retrieved access_token.

<ClaimsProvider>
  ...
  <TechnicalProfiles>
    <TechnicalProfile Id="REST-GetClientCredentials">
      <DisplayName>Get Client Credentials</DisplayName>
      <Protocol Name="Proprietary" Handler="Web.TPEngine.Providers.RestfulProvider, Web.TPEngine, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>
      <Metadata>
        <Item Key="ServiceUrl">
          https://<tenant-name>.b2clogin.com/<tenant-name>.onmicrosoft.com/<policy>/oauth2/v2.0/token?grant_type=client_credentials&amp;scope=<scope>&amp;client_id=<clientId>&amp;client_secret=<clientSecret>
        </Item>
        <Item Key="SendClaimsIn">Body</Item>
        <Item Key="AuthenticationType">None</Item>
        <Item Key="AllowInsecureAuthInProduction">true</Item>
      </Metadata>
      <OutputClaims>
        <OutputClaim ClaimTypeReferenceId="access_token"/>
      </OutputClaims>
      <UseTechnicalProfileForSessionManagement ReferenceId="SM-Noop"/>
    </TechnicalProfile>
    <TechnicalProfile Id="REST-CallApiUsingClientCredentials">
      <DisplayName>Call an External API using Client Credentials</DisplayName>
      <Protocol Name="Proprietary" Handler="Web.TPEngine.Providers.RestfulProvider, Web.TPEngine, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />
      <Metadata>
        <Item Key="ServiceUrl"><Endpoint to call></Item>
        <Item Key="SendClaimsIn">Header</Item>
        <Item Key="AuthenticationType">Bearer</Item>
        <Item Key="UseClaimAsBearerToken">access_token</Item>
        <Item Key="AllowInsecureAuthInProduction">true</Item>
        <Item Key="IncludeClaimResolvingInClaimsHandling">true</Item>
      </Metadata>
      <InputClaims>
        <InputClaim ClaimTypeReferenceId="access_token"/>
      </InputClaims>
      <OutputClaims>
        <!-- Output Claims from Calling the API -->
      </OutputClaims>
      <UseTechnicalProfileForSessionManagement ReferenceId="SM-Noop" />
    </TechnicalProfile> ...
  </TechnicalProfiles>
</ClaimsProvider>

3. Finally, introduce additional OrchestrationSteps to your UserJourney to use the above TechnicalProfiles.

<UserJourneys>
  <UserJourney Id="<UserJourneyId>">
    <OrchestrationSteps>
      ...
      <OrchestrationStep Order="7" Type="ClaimsExchange">
        <ClaimsExchanges>
          <ClaimsExchange Id="RESTGetClientCredentials" TechnicalProfileReferenceId="REST-GetClientCredentials" />
        </ClaimsExchanges>
      </OrchestrationStep>
      <OrchestrationStep Order="8" Type="ClaimsExchange">
        <ClaimsExchanges>
          <ClaimsExchange Id="RESTCallApiUsingClientCredentials" TechnicalProfileReferenceId="REST-CallApiUsingClientCredentials" />
        </ClaimsExchanges>
      </OrchestrationStep> ...
      <OrchestrationStep Order="11" Type="SendClaims" CpimIssuerTechnicalProfileReferenceId="JwtIssuer" />
    </OrchestrationSteps>
  </UserJourney>

Now that should be it.

Hope this helps.

Happy Coding.

Regards,
Jaliya

Friday, March 1, 2024

Creating Integration Tests for Azure Functions

I wanted to have some integration tests for Azure Functions, especially for some complex durable functions. When you have durable functions and when you want to make sure that the orchestrations are behaving as expected, having integration tests is the only way to ensure that. And another important thing is I needed to be able to run these tests not just locally, but in a CI pipeline (GitHub workflows, Azure DevOps Pipeline, etc) as well.

Unfortunately as of today, there is no proper integration test mechanism for Azure Durable Functions (or Azure Functions) like we have for ASP.NET Core applications.

I came up with the following approach after gathering inputs from GitHub issues and other related posts on the subject.

The basic concept is as follows. Note I am using XUnit.net as my testing Framework.

1. Create a fixture class that implements IDisposable and on the constructor, I am spinning up the Function Application to test using func start. And doing the cleanup on Dispose().

2. Create an XUnit Collection Fixture using the above fixture. So basically my single test context (the function application) will get shared among different tests in several test classes, and it will get cleaned up after all the tests in the test classes have finished.

My fixture looks like something below.
using Polly;
using System.Diagnostics;
using System.Runtime.InteropServices;

namespace HelloAzureFunctions.Tests.Integration.Fixtures;

public class AzureFunctionFixture : IDisposable
{
    private readonly string _path = Directory.GetCurrentDirectory();
    private readonly string _testOutputPath = Path.Combine(Directory.GetCurrentDirectory(), "integration-test-output.log");
    private readonly int _port = 7071;
    private readonly string _baseUrl;
    private readonly Process _process;

    public readonly HttpClient HttpClient;

    public AzureFunctionFixture()
    {
        _baseUrl = $"http://localhost:{_port}";

        HttpClient = new HttpClient()
        {
            BaseAddress = new Uri(_baseUrl)
        };

        if (File.Exists(_testOutputPath))
        {
            File.Delete(_testOutputPath);
        }

        DirectoryInfo directoryInfo = new(_path);
        _process = StartProcess(_port, directoryInfo);
        _process.OutputDataReceived += (sender, args) =>
        {
            File.AppendAllLines(_testOutputPath, [args.Data]);
        };
        _process.BeginOutputReadLine();
    }

    public void Dispose()
    {
        if (!_process.HasExited)
        {
            _process.Kill(entireProcessTree: true);
        }

        _process.Dispose();
        HttpClient.Dispose();
    }

    public async Task WaitUntilFunctionsAreRunning()
    {
        PolicyResult<HttpResponseMessage> result =
            await Policy.TimeoutAsync(TimeSpan.FromSeconds(30))
                .WrapAsync(Policy.Handle<Exception>().WaitAndRetryForeverAsync(index => TimeSpan.FromMilliseconds(500)))
                .ExecuteAndCaptureAsync(() => HttpClient.GetAsync(""));

        if (result.Outcome != OutcomeType.Successful)
        {
            throw new InvalidOperationException("The Azure Functions project doesn't seem to be running.");
        }
    }

    private static Process StartProcess(int port, DirectoryInfo workingDirectory)
    {
        string fileName = "func";
        string arguments = $"start --port {port} --verbose";

        if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
        {
            fileName = "powershell.exe";
            arguments = $"func start --port {port} --verbose";
        }

        ProcessStartInfo processStartInfo = new(fileName, arguments)
        {
            UseShellExecute = false,
            CreateNoWindow = true,
            RedirectStandardOutput = true,
            WorkingDirectory = workingDirectory.FullName,
            EnvironmentVariables =
            { 
                // Passing an additional environment variable to the application,
                // So it can control the behavior when running for Integration Tests

                [ApplicationConstants.IsRunningIntegrationTests] = "true"
            }
        };

        Process process = new() { StartInfo = processStartInfo };
        process.Start();

        return process;
    }
}
I can use this fixture for my tests and it will work fine for running integration tests locally.

Now we need to be able to run these tests in a CI pipeline. I am using the following GitHub workflow.
name: Run Integration Tests

on:
  push:
    branches: ["main"]
    paths-ignore:
      - '**.md'

env:
  DOTNET_VERSION: '8.0.x'

jobs:
  build-and-test:
    strategy:
      matrix:
        os: [ubuntu-latestwindows-latest]
    runs-on: ${{ matrix.os }}
    env:
      INTEGRATION_TEST_EXECUTION_DIRECTORY: ./tests/HelloAzureFunctions.Tests.Integration/bin/Debug/net8.0

    steps:
    - name: 'Checkout GitHub Action'
      uses: actions/checkout@v3

    - name: Setup .NET ${{ env.DOTNET_VERSION }} Environment
      uses: actions/setup-dotnet@v3
      with:
        dotnet-version: ${{ env.DOTNET_VERSION }}

    - name: Build
      run: dotnet build

    # Install Azure Functions Core Tools in the runner, 
    # so we have access to 'func.exe' to spin up the Azure Functions app in integration tests
    - name: Install Azure Functions Core Tools 
      run: |
        npm install -g azure-functions-core-tools@4 --unsafe-perm true

    # Setup Azurite in the runner, 
    # so the Azure Functions app we are going to spin up, can use azurite as it's Storage Provider
    - name: Setup Azurite 
      shell: bash
      run: |
        npm install -g azurite
        azurite --silent &

    - name: Run Integration Tests
      # If there are any errors executing integration tests, uncomment the following line to continue the workflow, so you can look at integration-test-output.log
      # continue-on-error: true 
      run: dotnet test ${{ env.INTEGRATION_TEST_EXECUTION_DIRECTORY }}/HelloAzureFunctions.Tests.Integration.dll

    - name: Upload Integration Tests Execution Log
      uses: actions/upload-artifact@v4
      with:
        name: artifact-${{ matrix.os }}
        path: ${{ env.INTEGRATION_TEST_EXECUTION_DIRECTORY }}/integration-test-output.log
When the workflow runs, the output is as follows.
Build and Test: ubuntu-latest
Build and Test: windows-latest
You can find the full sample code here on this repo:

Happy Coding.

Regards,
Jaliya

Tuesday, February 20, 2024

.NET 8.0 Isolated Azure Functions: Binding Expressions that uses Azure App Configuration

In this post let's see how we can use binding expressions in an .NET 8.0 Isolated Azure Function and how to consume the binding expression values from Azure App Configuration (AAC).

Binding expressions are basically something like this. Let's take a simple ServiceBus trigger function. 
[Function(nameof(ServiceBusTrigger))]
public static void ServiceBusTrigger( [ServiceBusTrigger("%Messaging:Topic%", "%Messaging:Subscription%")] ServiceBusReceivedMessage serviceBusReceivedMessage)
{
    // TODO: Process the received message
}
Here the  %Messaging:Topic% and %Messaging:Subscription% are binding expressions and its value doesn't have to be a compile time constant.

In In-Process Azure functions, it's pretty straightforward, you can just add Azure App Configuration as another configuration provider in the Startup, and it will work.

But in Isolated functions at least as of today (20th February 2024), you can't do that (Support expression resolution from configuration sources registered by the worker #1253). While it's a bit disappointing (after having Isolated functions available for a couple of years), you can use the following workaround.

Let's say I have the following values in my Azure App Configuration.
Azure App Configuration
Azure App Configuration Values
I can use the following notation to access AAC values.
@Microsoft.AppConfiguration(Endpoint=https://aac-temp-001.azconfig.io; Key=<key>)
// if you want to choose a particular Label
@Microsoft.AppConfiguration(Endpoint=https://aac-temp-001.azconfig.io; Key=<key>; Label=<label>)
So I can update Function App settings in Azure as follows. Make sure the identity of the function app (system-assigned managed identity or user-assigned managed identity) can read the configuration from AAC.
Function App Configuration
More read:
   Use App Configuration references for App Service and Azure Functions (preview)