Tuesday, February 23, 2021

EF Core: Using HiLo Algorithm to Generate IDs

In this post, let's see how we can use Hi/Lo algorithm as the key generation strategy in EF Core. This can be achieved using UseHiLo method.

As always let's go by an example.

Consider the below code.
public class Category
{
    public int Id { getset; }

    public string Name { getset; }

    public ICollection<Product> Products { getset; }
}

public class Product
{
    public int Id { getset; }

    public string Name { getset; }

    public int CategoryId { getset; }
}
And MyDbContext is setup like this.
public class MyDbContext : DbContext
{
    public DbSet<Category> Categories { getset; }

    public DbSet<Product> Products { getset; }

    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {
        optionsBuilder
            .UseSqlServer("ConnectionString")
            .LogTo(Console.WriteLineLogLevel.Information);
    }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<Category>(builder =>
        {
            builder
                .HasMany(x => x.Products)
                .WithOne()
                .HasForeignKey(x => x.CategoryId);
        });
    }
}
And now I am adding a Category and a Product.
using var context = new MyDbContext();
await context.Database.EnsureDeletedAsync();
await context.Database.EnsureCreatedAsync();

var category = new Category() { Name = "Some Category" };
context.Categories.Add(category);

var product = new Product { Name =
"Some Product", CategoryId = category.Id };
context.Products.Add(product);

await context.SaveChangesAsync()
The SaveChanges here will fail here with the following error: "The INSERT statement conflicted with the FOREIGN KEY constraint "FK_Products_Categories_CategoryId"". The reason is, we are setting the CategoryId of the Product, but it's still 0 because the Category record is still not saved to the database.

In this kind of a scenario, using HiLo algorithm to determine the Id before the record is actually saved in the database is quite handy.

Enabling HiLo is pretty easy. You can either enable it for all the entities or for a specific entity.
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    // HiLo Id generation For all the entities
    //modelBuilder.UseHiLo();

    modelBuilder.Entity<Category>(builder =>
    {
        builder
            .HasMany(x => x.Products)
            .WithOne()
            .HasForeignKey(x => x.CategoryId);

        // HiLo Id generation only for Category
        builder
            .Property(x => x.Id)
            .UseHiLo("categoryseq");
    });
}
After this change, if you run the example, you are not going to see the previous error, instead, you should see Category now has a valid Id.
Valid CategoryId
If you have a second category added, it will have an Id of value 2.

Here something important to note is if you do UseHilo() for all the entities, the Ids are going to be kind of messed up, if you take the above example, the Id of the Product here will be 2.

Now let's try 2 simultaneous adding. While the control is waiting on SaveChanges, I am starting another instance of the application. 
Another Instance
Here the new category has the Id of value 11.

Now how is this working?

If you add in database migration, you will see EF is creating a Sequence, something like below.
migrationBuilder.CreateSequence(
    name"categoryseq",
    incrementBy: 10);
And once the database is updated, you can see the created sequence here.
Sequences
categoryseq
Now let's imagine so far we have only created the database and no data is inserted.

Now we are creating a new database context and adding a category to context (context.Categories.Add(category)), EF will run a query like this.
SELECT NEXT VALUE FOR [categoryseq]
This will return 1, and the category will be assigned the value of 1 and it has now blocked Ids from 1 to 10. If we add in another category in the same database context, EF won't run the above query again, it will increment the Id by 1 and when we have added 10 categories, EF will run the above query again to get the next value, which will be 11. But imagine while we are adding these 10 categories, some other user creates a new database context and starts adding categories, then that process will also call the above query, so this time he will get 11 (because so far we have only blocked from 1 to 10). And when we call for next value, we will get 21, because that process has now blocked 11 to 20.

Isn't it nice? But there is a kind of downside here. This can cause missing Ids (if you are worried about the order of the Ids), but generally, it shouldn't be an issue.

Hope this helps!

Happy Coding.

Regards,
Jaliya

Wednesday, February 10, 2021

EF Core: Owned Entity Types

In this post let's see what is Owned Entity Types in Entity Framework Core. This is a very nice feature when it comes to DDD.

Owned Entities are entities that can be only appeared on navigation properties of other entity types. Basically, they can't exist without the Owner.  

Let's go by an example. Consider the following two entities.
public class User
{
    public int Id { getset; }

    public string Name { getset; }

    public Address Address { getset; }
}
public class Address
{
    public string Street { getset; }

    public string City { getset; }
}
And my DbContext looks like this.
public class MyDbContext : DbContext
{
    public DbSet<User> Users { getset; }

    //...
}
So here, we can use OwnsOne to configure the Address Owned Entity to the Owner User as below (I am using Fluent API instead of annotations, you can annotations if you prefer, but I prefer Fluent API all the time).
public class UserEntityConfiguration : IEntityTypeConfiguration<User>
{
    public void Configure(EntityTypeBuilder<Userbuilder)
    {
        builder.ToTable($"{nameof(User)}");

        builder.OwnsOne(x => x.Address);
    }
}
And this will create something like this.
dbo.User
If you don't like the default naming convention, you can always override the behavior like below.
public class UserEntityConfiguration : IEntityTypeConfiguration<User>
{
    public void Configure(EntityTypeBuilder<Userbuilder)
    {
        builder.ToTable($"{nameof(User)}");

        builder.OwnsOne(x => x.Address, y =>
        {
            y.Property(y => y.City)
                    .HasColumnName("City");

            y.Property(y => y.Street)
                    .HasColumnName("Street");
        });
    }
}
And this would give something like below.
dbo.User

Now let's see consider the following code to insert some data and retrieval.
using var context = new MyDbContext();

User user = new User
{
    Name = "John Doe",
    Address = new Address { Street = "Some Street1", City = "Some City1" }
};

await context.Users.AddAsync(user);
await context.SaveChangesAsync();
using var anotherContext = new OrderDbContext();

user = await anotherContext.Users.FirstOrDefaultAsync();
Here for selection, EF Core will generate a query as below.
SELECT TOP(1) [u].[Id], [u].[Name], [u].[City], [u].[Street]
FROM [User] AS [u]
So here you should be able to see a very important thing. When we are getting User, we didn't have to explicitly Include Address to load Address details, it's loading its Owned Entities by default. This might not be a good example, since it's in the same table, we will see a clearer example when we are going through OwnsMany later in this post. 

Note: I am using anotherContext here for retrieval, because EF Core will automatically fix-up navigation properties to any other entities that were previously loaded into the context instance. So even if you don't explicitly include the data for a navigation property, the property may still be populated if some or all of the related entities were previously loaded.

Above was kind of a one-to-one relationship. Now let's have a look at one-to-many type of relationship.
public class User
{
    public int Id { getset; }

    public string Name { getset; }

    public ICollection<Address> Addresses { getset; }
}
public class Address
{
    public string Street { getset; }

    public string City { getset; }

    public string Type { getset; }
}
Now I can use OwnsMany as below.
public class UserEntityConfiguration : IEntityTypeConfiguration<User>
{
    public void Configure(EntityTypeBuilder<Userbuilder)
    {
        builder.ToTable($"{nameof(User)}");

        builder.OwnsMany(x => x.Addresses, y =>
        {
            y.ToTable("UserAddress");

            y.Property(y => y.City)
                .HasColumnName("City");

            y.Property(y => y.Street)
                .HasColumnName("Type");
            y.Property(y => y.Street)
                .HasColumnName("Type");
        });
    }
}
And this would give something like this.
dbo.User and dbo.UserAddress
Now let's have a look at below code to insert some data and retrieval.
using var context = new MyDbContext();
await context.Database.EnsureDeletedAsync();
await context.Database.EnsureCreatedAsync();

var user = new User
{
    Name = "John Doe",
    Addresses = new List<Address>
    {
        new Address { Street = "Some Street1", City = "Some City1", Type= "Shipping" },
        new Address { Street = "Some Street2", City = "Some City2", Type= "Mailing" }
    }
};

await context.Users.AddAsync(user);
await context.SaveChangesAsync();

using var anotherContext = new MyDbContext();

user = await anotherContext.Users.FirstOrDefaultAsync();
Here for selection, the generated query would be the following. (you can basically ignore the subquery, that's because I am getting FirstOrDefault),
SELECT [t].[Id], [t].[Name], [u0].[UserId], [u0].[Id], [u0].[City], [u0].[Street], [u0].[Type]
FROM (
    SELECT TOP(1) [u].[Id], [u].[Name]
    FROM [User] AS [u]
) AS [t]
LEFT JOIN [UserAddress] AS [u0] ON [t].[Id] = [u0].[UserId]
ORDER BY [t].[Id], [u0].[UserId], [u0].[Id]
And here also, without doing .Include(x => x.Addresses), the addresses are being returned.

So that's about it. You can read more on Owned Entity Types by going to the below link.
There are some restrictions when it comes to Owned Entities which makes perfect sense.
  • You cannot create a DbSet<T> for an owned type.
  • You cannot call Entity<T>() with an owned type on ModelBuilder.
  • Instances of owned entity types cannot be shared by multiple owners (this is a well-known scenario for value objects that cannot be implemented using owned entity types).
Hope this helps.

Happy Coding.

Regards,
Jaliya

Wednesday, January 27, 2021

Azure Pipelines: Create/Update Azure API Management APIs

In this post, let's see how we can create/update Azure API Management APIs from both classic release pipelines and YAML based pipelines.

Here I am assuming, We have already created Azure API Management Service in Azure and we have set up Products.

The process is actually quite easy, thanks to this great free extension API Management Suite by Stephane Eyskens

First, let's have a look at how we can use this in classic release pipelines.

Once this extension is installed, you should be able to see a bunch of tasks starting with API Management - *. From there what we need is API Management - Create or Update API task. We just need to add this task after the deployment of the API.

API Management - Create or Update API

Then it's just a matter of configuring the task.
API Management - Create or Update API Settings
Most of the things here are pretty obvious, so I am not going to explain each one of them. The important ones are,
  • Products
    • Since I have already created a product in the APIM, I can just enter the name of the Product.
    • If you are creating the Product as part of the pipeline (using API Management - Create or update product task), then you need to select Product created by previous task.
  • OpenAPI Specification
    • The APIM needs to have access to OpenAPI specification in order to create the API. You have several options here. You need to select the version, it's format (json or yaml), any authorization needed to access the specification and the location (URL, Code or Build Artifact)
  • Configuration of the API
    • Here, the nice thing is you can specify the Policy. I prefer maintaining the policy in the code, so here I have just selected my policy by providing it's path.
So that's basically it. Once the API is deployed, this task will run, it will pull down the OpenAPI specification, create/update APIM API, apply the policy and it's all good to go.

If you prefer having deployment steps as part of YAML pipeline, we can easily set up this task there as well. Basically something like this.
task: stephane-eyskens.apim.apim.apim@5
    displayName: 'API Management - Create/Update API '
    inputs:
      ConnectedServiceNameARM: ''
      ResourceGroupName: ''
      ApiPortalName: ''
      UseProductCreatedByPreviousTask: false
      product1: ''
      OpenAPISpec: 'v3'
      swaggerlocation: ''
      targetapi: ''
      DisplayName: ''
      pathapi: ''
      subscriptionRequired: false
      TemplateSelector: Artifact
      policyArtifact: '$(Pipeline.Workspace)\scripts\apim\apim-policy.xml'
      MicrosoftApiManagementAPIVersion: '2018-01-01'
Don't worry, you have the UI to select the options just like classic releases.
API Management - Create or Update API Settings
So hope this helps. 

Huge thanks to Stephane Eyskens for coming up with this nice set of tasks.

Happy Coding.

Regards,
Jaliya

Thursday, January 14, 2021

Azure Pipelines: Passing Variables Between Jobs and Stages

In this post, let's see how we can pass variables between Stages and Jobs in Azure Pipelines..

Consider the following simple pipeline.

stages:
stage: Build
  jobs:
  - job: FirstJob
    pool:
      vmImage: 'windows-latest'
    steps:
     - bash: echo "##vso[task.setvariable variable=myVariable;isOutput=true]Hello World"
       name: stepSetVariable
       
  - job: SecondJob
    dependsOn: FirstJob
    pool:
      vmImage: 'ubuntu-latest'
    variables:
      # Map variable
      # syntax: $[ dependencies.{JobName}.outputs['{stepName}.{variableName}'] ]
      myJobVariable: $[ dependencies.FirstJob.outputs['stepSetVariable.myVariable'] ]  
    steps:
    # Echos Hello World
    - script: echo $(myJobVariable)
      name: stepEchoVariable

stage: Deployment
  dependsOn: Build
  pool:
    vmImage: 'macOS-latest'
  variables:
    # Map variable
    # syntax: $[ stageDependencies.{BuildName}.{JobName}.outputs['{stepName}.{variableName}'] ]
    myStageVariable: $[ stageDependencies.Build.FirstJob.outputs['stepSetVariable.myVariable'] ]
  jobs:
  - job: DeploymentJob
    steps:
    # Echos Hello World
    - script: echo $(myStageVariable)
      name: stepEchoVariable

Here,

  • I have 2 Stages, Build and Deployment
  • Build stage has 2 jobs, 
    • In the FirstJob, I am creating a variable called myVariable. I am making it an output variable and setting its value to Hello World.
    • In the SecondJob
      • The SecondJob is depending on FirstJob because the variable needs to be created first. 
      • Then I am declaring a variable called myJobVariable and mapping it with myVariable created in FirstJob. Since it's within the same Stage, I can map the variable using this syntax. 
$[ dependencies.{JobName}.outputs['{stepName}.{variableName}'] ]
      • Then I am just printing the value of myJobVariable
  • In Deployment stage,
    • This stage is depending on Build stage, again because we are trying to access an output variable from Build stage.
    • Then I am declaring a variable called myStageVariable and mapping it with myVariable created in Build stages' FirstJob. Since it's in a different Stage, I can reference the variable using stageDependencies
$[ stageDependencies.{BuildName}.{JobName}.outputs['{stepName}.{variableName}'] ]

    • Then I have just a single job to print the value of myStageVariable

Note: I have used different images for two jobs in Build stage and for the Deployment stage, so the concept is more clear.

Hope this helps.

Happy Coding.

Regards,
Jaliya

Thursday, January 7, 2021

EF Core 5.0: Excluding Tables from Migrations

In this post, let's see how we can exclude Tables from Migrations in EF Core 5.0. It's actually pretty easy.

Consider the below use case.

public class IdentityDbContext : DbContext
{
    public DbSet<ApplicationUser> ApplicationUsers { getset; }

    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {
        // Configuring
    }
}

public class OrderDbContext : DbContext
{
    public DbSet<ApplicationUser> ApplicationUsers { getset; }

    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {
        // Configuring
    }
}

public class ApplicationUser
{
    public int Id { getset; }

    public string Name { getset; }
}

Here the ApplicationUser is shared between two DbContexts (IdentityDbContext and OrderDbContext). And now let's say between these two DbContexts, we are only going to maintain full ApplicationUser information in only IdentityDbContext. In that case, we don't want further changes on ApplicationUser to be applied on OrderDbContext.

To achieve that, we can exclude the ApplicationUser in migrations for OrderDbContext as below.

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    modelBuilder.Entity<ApplicationUser>().ToTable("ApplicationUsers", t => t.ExcludeFromMigrations());
}

Unfortunately, we need to call ToTable and from there customize the TableBuilder to ExcludeFromMigrations.

Hope this helps.

Happy Coding.

Regards,
Jaliya