Click here to Skip to main content
14,331,865 members

Speed up ASP.NET Core WEB API application. Part 3

Rate this:
4.97 (11 votes)
Please Sign up or sign in to vote.
4.97 (11 votes)
2 Oct 2019CPOL
Deep refactoring and refinement of ASP.NET Core WEB API application code

Deep refactoring and refinement of ASP.NET Core WEB API application code

Introduction

Part 1. Creating a test RESTful WEB API application.

Part 2. Increasing ASP.NET Core WEB API application's productivity.

In Part 3 we will review the following:

Why do we need to refactoring and refinement the code?

The goal of Part1 was to create a really simple basic application we can start from. The main focus was on how to make it easier to apply and examine different approaches, to modify code and check results.

Part2 was dedicated to productivity. A variety approaches were realized. And the code became more complicated compared to Part1.

Now, after making a choice of approaches and implementing them, we can consider our application as a whole. It becomes evident that the code requires deep refactoring and refinement so that it satisfies various principles of good programming style.

The Don't Repeat Yourself (DRY) principle

According to the DRY principle, we should eliminate duplication of code. Therefore, let us examine the ProductsService code to see whether it has any repetition. We can see at once, that the following fragment is repeated several times in all the methods which return the ProductViewModel or IEnumerable<productviewmodel> typed value:

new ProductViewModel()
    {
        Id = p.ProductId,
        Sku = p.Sku,
        Name = p.Name
    }
…

We have always created a ProductViewModel type object from a Product type object. It is logical, to move field initialization of the ProductViewModels object into its constructor. Let us create a constructor method in the ProductViewModel class. In the constructor we fill the object’s field values with appropriate values of the Product parameter:

public ProductViewModel(Product product)
{
    Id = product.ProductId;
    Sku = product.Sku;
    Name = product.Name;
}

Now we can rewrite the duplicated code in the FindProductsAsync and GetAllProductsAsync methods of the ProductsService:

return new OkObjectResult(products.Select(p => new ProductViewModel()
                    {
                        Id = p.ProductId,
                        Sku = p.Sku,
                        Name = p.Name
                    }));
                    
    return new OkObjectResult(products.Select(p => new ProductViewModel(p)));                    
…                    

And change GetProductAsync and DeleteProductAsync methods of the ProductsService class:

return new OkObjectResult(new ProductViewModel()
                    {
                        Id = product.ProductId,
                        Sku = product.Sku,
                        Name = product.Name
                    });
                    
    return new OkObjectResult(new ProductViewModel(product));
…                    

And repeat the same for the PriceViewModel class.

new PriceViewModel()
                    {
                        Price = p.Value,
                        Supplier = p.Supplier
                    }
…                    

Although we use the fragment only once in the PricesService, it is better to encapsulate the PriceViewModel’s fields initialization inside the class in its constructor.

Let us create a PriceViewModel class constructor

public PriceViewModel(Price price)
{
    Price = price.Value;
    Supplier = price.Supplier;
}
…                    

And change the fragment:

return new OkObjectResult(pricess.Select(p => new PriceViewModel()
                    {
                        Price = p.Value,
                        Supplier = p.Supplier
                    })
                    .OrderBy(p => p.Price)
                    .ThenBy(p => p.Supplier));
                    
    return new OkObjectResult(pricess.Select(p => new PriceViewModel(p))
                    .OrderBy(p => p.Price)
                    .ThenBy(p => p.Supplier));
                    
… 

Exceptions handling in try/catch/finally blocks

The next problem that should be solved is the exception handling. Throughout the application all operations that can cause exception have been called inside a try-catch construction. This approach is very convenient during the debugging process, because it allows us to examine an exception at the particular place it occurs. But this approach also has a disadvantage of code repetition. A better way of exception handling in ASP.NET Core is to handle them globally in middleware or in Exception filters.

We will create Exception handling middleware to centralize exceptions handling with logging and generating user friendly error messages.

Requirements for our Exception handling middleware

  • Logging detailed information to a log file;
  • Detailed error message in debug mode and friendly message in production;
  • Unified error message format

Logging to a file in .Net Core

At start of .NET Core application in Main method we have created and run the web server.

…    
BuildWebHost(args).Run();
… 

At this moment, an instance of ILoggerFactory is created automatically. Now it can be accessed via dependency injection and perform logging anywhere in the code. However, with the standard ILoggerFactory we cannot log to a file. To overcome this limitation, we will use the Serilog library, that extends the ILoggerFactory and allows logging to a file."

Let us install the Serilog.Extensions.Logging.File NuGet package first:

Image 1

We should add using Microsoft.Extensions.Logging; statement modules in which we are going to apply logging.

The Serilog library can be configured in different ways. In our simple example, to setup logging rules for Serilog, we should add the next code in the Startup class in the Configure method

public void Configure(IApplicationBuilder app, IHostingEnvironment env,
                          ILoggerFactory loggerFactory)
        {
            loggerFactory.AddFile("Logs/log.txt");
… 

This means, that the logger will write to relative \Logs directory and log files’ name format will be: log-yyyymmdd.txt

Unified exception message format

During its work our application can generate different types of exception messages. Our aim is to unify the format of these messages so that they could be processed by some universal method of a client application.

Let all messages have the following format:

{
    "message": "Product not found"
}    

The format is really very simple. It is acceptable for a simple application, like ours. But we should foresee the opportunity to expand it and to do this centralized in one place. For this, we will create an ExceptionMessage class, which will encapsulate message formatting procedures. And we will use this class wherever we need to generate exception messages.

Let us create a folder Exceptions in our project and add there a class ExceptionMessage:>

using Newtonsoft.Json;

namespace SpeedUpCoreAPIExample.Exceptions
{
    public class ExceptionMessage
    {
        public string Message { get; set; }

        public ExceptionMessage() {}

        public ExceptionMessage(string message)
        {
            Message = message;
        }

        public override string ToString()
        {
            return JsonConvert.SerializeObject(new { message = new string(Message) });
        }
    }
}

Now we can create our ExceptionsHandlingMiddleware

.NET Core exception handling middleware implementation

In the Exceptions folder create a class ExceptionsHandlingMiddleware:

using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using System;
using System.Net;
using System.Threading.Tasks;

namespace SpeedUpCoreAPIExample.Exceptions
{
    public class ExceptionsHandlingMiddleware
    {
        private readonly RequestDelegate _next;
        private readonly ILogger<ExceptionsHandlingMiddleware> _logger;

        public ExceptionsHandlingMiddleware(RequestDelegate next, ILogger<ExceptionsHandlingMiddleware> logger)
        {
            _next = next;
            _logger = logger;
        }

        public async Task InvokeAsync(HttpContext httpContext)
        {
            try
            {
                await _next(httpContext);
            }
            catch (Exception ex)
            {
                await HandleUnhandledExceptionAsync(httpContext, ex);
            }
        }

        private async Task HandleUnhandledExceptionAsync(HttpContext context,
                                Exception exception)
        {
            _logger.LogError(exception, exception.Message);

            if (!context.Response.HasStarted)
            {
                int statusCode = (int)HttpStatusCode.InternalServerError; // 500
                string message = string.Empty;
#if DEBUG
                message = exception.Message;
#else
                message = "An unhandled exception has occurred";
#endif
                context.Response.Clear();
                context.Response.ContentType = "application/json";
                context.Response.StatusCode = statusCode;

                var result = new ExceptionMessage(message).ToString();
                await context.Response.WriteAsync(result);
            }
        }
    }
}

This middleware intercepts unhandled exceptions, logs exceptions’ details and emits detailed messages while debugging (#if DEBUG) or user-friendly messages without debugging.

Note, how we use ExceptionMessage class, to format the result.

Now, we should add this middleware in the application HTTP request pipeline in the Startup.Configure method before app.UseMvc(); statement.

app.UseMiddleware<ExceptionsHandlingMiddleware>();;
…
app.UseMvc();

Let us check, how it works. For this, we will change a stored procedure name in the ProductsRepository.FindProductsAsync method for a nonexistent method GetProductsBySKUError.

public async Task<IEnumerable<product>> FindProductsAsync(string sku)
{
    return await _context.Products.AsNoTracking().FromSql("[dbo].GetProductsBySKUError @sku = {0}", sku).ToListAsync();
}

And remove Try-Catch block from the ProductsService.FindProductsAsync method

public async Task<IActionResult> FindProductsAsync(string sku)
{
    try
    {
        IEnumerabler<Product> products = await _productsRepository.FindProductsAsync(sku);}
    catch
    {
        return new ConflictResult();
    }
    …
}

Let us run our application and check the results

Call http://localhost:49858/api/products/find/aa with Swagger

We will have 500 Http Response code and a message:

Image 2

And let us check log files

Image 3

Now we have Logs folder with a file

Image 4

Inside the file we have detailed exception description:

""[dbo].GetProductsBySKUError @sku = @p0" (627a98df)
System.Data.SqlClient.SqlException (0x80131904): Could not find stored procedure 'dbo.GetProductsBySKUError'.
…

We have claimed, that our Exception Handling Middleware should generate detailed error message in debug mode and friendly message in production. Let us check it. For this, we will change Active solution configuration for Release in the toolbar:

Image 5

or in the Configuration manager:

Image 6

Then call incorrect API ones again. The result, as we expected, will be:

Image 7

So, our exception handler works as we expected.

Note! If we did not remove Try-Catch block we would never let this handler work, because the unhandled exemption would be processed by the code inside Catch statement.

Do not forget to restore the correct stored procedure name GetProductsBySKU!

Now we can remove all Try-Catch block in the ProductsService and PricesService clacces.

Note! We omit code of removing Try-Catch blocks implementation for brevity.

The only places we still need Try-Catch blocks are ProductsService.PreparePricesAsync and PricesService.PreparePricesAsync methods. We do not want to breake the application workflow in those places, as we discussed in Part 2

After removing Try-Catch block the code became much simpler and straightforward. But we still have some repetition in most services’ method, when we return

return new NotFoundResult();

Let us improve this too.

In all methods, that find a collection of values, such as ProductsService.GetAllProductsAsync, ProductsService.FindProductsAsync and PricesService.GetPricesAsync we have two problems.

The first one is in checking, whether a collection, received from a repository, is not empty. For this we have used а statement

if (products != null) 
…

But a collection will never be null in our case (except if a handled exception happens in a repository). Since all exceptions are handled now in a dedicated middleware outside services and repositories, we will always receive a collection of values (empty, if nothing was found). So, the proper way to check the results will be

if (products.Any())

or

(products.Count() > 0)

and the same for PricesService class in GetPricesAsync method: change

if (pricess != null)
    
    if (pricess.Any())

The second problem is what result we should return for empty collections. So far, we have returned NotFoundResult(), but it is also not really correct. For example, if we create another API that should return a value composed of a Product and its Prices, an empty prices collection will be represented in a JSON structure as an empty massive and StatusCode will be 200 - OK. So, to be consistent, we should rewrite the code of the abovementioned methods to remove NotFoundResult for empty collections:

public async Task<IActionResult> FindProductsAsync(string sku)
{
    IEnumerable<Product> products = await _productsRepository.FindProductsAsync(sku);

    if (products.Count() == 1)
    {
        //only one record found - prepare prices beforehand
        ThreadPool.QueueUserWorkItem(delegate
        {
            PreparePricesAsync(products.FirstOrDefault().ProductId);
        });
    };

    return new OkObjectResult(products.Select(p => new ProductViewModel(p)));
}    

public async Task<IActionResult> GetAllProductsAsync()
{
    IEnumerable<Product> products = await _productsRepository.GetAllProductsAsync();

    return new OkObjectResult(products.Select(p => new ProductViewModel(p)));
}

And in PricesService

public async Task<IActionResult> GetPricesAsync(int productId)
{
    IEnumerable<Price> pricess = await _pricesRepository.GetPricesAsync(productId);

    return new OkObjectResult(pricess.Select(p => new PriceViewModel(p))
                            .OrderBy(p => p.Price)
                            .ThenBy(p => p.Supplier));
}

The code becomes really straightforward, but another problem still remains: is this a correct solution to return IActionResult from Services.

What type of result should the business logic layer return to a controller?

Classically, the business layer’s methods return a POCO (Plain old CLR object) typed value to a controller and then the controller forms a proper response with an appropriate StatusCode. For example, the ProductsService.GetProductAsync method should return either a ProductViewModel object or null (if a product is not found). And the Controller should generate OkObjectResult(ProductViewModel) or NotFound() response respectively.

But this approach is not always possible. Actually, we can have different reasons to return null from a Service. For example, let us imagine an application in which a user can access some content. This content can be either public, private or prepaid. When a user requests some content, an ISomeContentService can return either an ISomeContent or null. There are some possible reasons for this null:

401 Unauthorized
402 Payment Required
403 Forbidden
404 Not Found
…

The reason becomes clear inside the Service. How can the Service notify a Controller about this reason, if a method returns just the null? This is not enough for a controller to create a proper response. To solve this issue, we have used IActionResult type as a return type from Services – business layer. This approach is really flexible, as with IActionResult result we can pass everything to a controller. But should a business layer form an API’s response, performing a controller’s job? Will it not break the separation of concerns design principal?

One possible way to get rid of IActionResult in a business layer is using custom exceptions to control the application’s workflow and generate proper Responses. To provide this we will enhance our Exception handling middleware to make it able to process custom exceptions.

Custom exceptions handling middleware

Let us create a simple HttpException class, inherited from Exception. And enhance out exception handler middleware to process exceptions of HttpException type.

In the HttpException folder add class HttpException

using System;
using System.Net;

namespace SpeedUpCoreAPIExample.Exceptions
{
    // Custom Http Exception
    public class HttpException : Exception
    {
        // Holds Http status code: 404 NotFound, 400 BadRequest, ...
        public int StatusCode { get; }
        public string MessageDetail { get; set; }

        public HttpException(HttpStatusCode statusCode, string message = null, string messageDetail = null) : base(message)
        {
            StatusCode = (int)statusCode;
            MessageDetail = messageDetail;
        }
    }
}

And change the ExceptionsHandlingMiddleware class code

public async Task InvokeAsync(HttpContext httpContext)
    {
        try
        {
            await _next(httpContext);
        }
        catch (HttpException ex)
        {
            await HandleHttpExceptionAsync(httpContext, ex);
        }
        catch (Exception ex)
        {
            await HandleUnhandledExceptionAsync(httpContext, ex);
        }
    }
…
…
private async Task HandleHttpExceptionAsync(HttpContext context, HttpException exception)
{
    _logger.LogError(exception, exception.MessageDetail);

    if (!context.Response.HasStarted)
    {
        int statusCode = exception.StatusCode;
        string message = exception.Message;

        context.Response.Clear();

        context.Response.ContentType = "application/json";
        context.Response.StatusCode = statusCode;

        var result = new ExceptionMessage(message).ToString();
        await context.Response.WriteAsync(result);
    }
}    

In the middleware, we process HttpException type exception before general Exception type, invoking the HandleHttpExceptionAsync method. And we log detailed exception message, if provided.

Now, we can rewrite ProductsService.GetProductAsync and ProductsService.DeleteProductAsync

public async Task<IActionResult> GetProductAsync(int productId)
{
    Product product = await _productsRepository.GetProductAsync(productId);

    if (product == null)
        throw new HttpException(HttpStatusCode.NotFound, "Product not found",  $"Product Id: {productId}");

    ThreadPool.QueueUserWorkItem(delegate
    {
        PreparePricesAsync(productId);
    });

    return new OkObjectResult(new ProductViewModel(product));
}

public async Task<IActionResult> DeleteProductAsync(int productId)
{
    Product product = await _productsRepository.DeleteProductAsync(productId);

    if (product == null)
        throw new HttpException(HttpStatusCode.NotFound, "Product not found",  $"Product Id: {productId}");

    return new OkObjectResult(new ProductViewModel(product));
}
…

In this version, instead of returning 404 Not Found from the services with IActionResult, we are throwing a custom HttpException and the exceptions handle middleware returns a proper response to a user. Let us check how it works by calling an API with a productid, that is evidently not in Products table:

http://localhost:49858/api/products/100

Image 8

Our universal Exceptions Handling Middleware works fine.

Since we have created an alternative way to pass any StatucCode and message from the business layer, we can easily change a return value type from IActionResult to a proper POCO type. For this we have to rewrite the following interfaces:

public interface IProductsService
{
    Task<IActionResult> GetAllProductsAsync();
    Task<IActionResult> GetProductAsync(int productId);
    Task<IActionResult> FindProductsAsync(string sku);
    Task<IActionResult> DeleteProductAsync(int productId);
    
    Task<IEnumerable<ProductViewModel>> GetAllProductsAsync();
    Task<ProductViewModel> GetProductAsync(int productId);
    Task<IEnumerable<ProductViewModel>> FindProductsAsync(string sku);
    Task<ProductViewModel> DeleteProductAsync(int productId);
}    

And change

public interface IPricesService
{
    Task<IEnumerable<Price>> GetPricesAsync(int productId);
    
    Task<IEnumerable<PriceViewModel>> GetPricesAsync(int productId);    
…
}

We should also redeclare appropriate methods in the ProductsService and PricesService classes, by changing IActionResult type to a type from the interfaces. And also change their return statements, by removing OkObjectResult statement. For example, in the ProductsService.GetAllProductsAsync method:

the new version will be:

public async Task<IEnumerable<ProductViewModel>> GetAllProductsAsync()
{
    IEnumerable<Product> products = await _productsRepository.GetAllProductsAsync();

    return products.Select(p => new ProductViewModel(p));
}

The final task is to change the controllers’ actions so that they create an OK response. It will always be 200 OK, because NotFound will be returned by the ExceptionsHandlingMiddleware

For example, for the ProductsService.GetAllProductsAsync the return statement should be changed from:

// GET /api/products
[HttpGet]
public async Task<IActionResult> GetAllProductsAsync()
{
    return await _productsService.GetAllProductsAsync();
}

to:

// GET /api/products
[HttpGet]
public async Task<IActionResult> GetAllProductsAsync()
{
    return new OkObjectResult(await _productsService.GetAllProductsAsync());
} 

You do this in all the ProductsController’s actions and in the PricesService.GetPricesAsync action.

Using Typed Clients with HttpClientFactory

Our previous implementation of HttpClient has some issues, we can improve. First of all, we have to inject IHttpContextAccessor to use it in the GetFullyQualifiedApiUrl method. Both IHttpContextAccessor and GetFullyQualifiedApiUrl method are dedicated only to HttpClient and never used in other places of ProductsService. If we want to apply the same functionality in another services, we will have to write almost the same code. So, it is better, to create a separate helper class – wrapper around HttpClient and encapsulate all the necessary HttpClient calling business logic inside this class.

We will use another way of working with the HttpClientFactory - Typed Clients class.

In the Interfaces folder create an ISelfHttpClient intetface:

using System.Threading.Tasks;

namespace SpeedUpCoreAPIExample.Interfaces
{
    public interface ISelfHttpClient
    {
        Task PostIdAsync(string apiRoute, string id);
    }
}

We have declared only one method, that calls any controller's action with HttpPost method and Id parameter

Let us create a Helpers folder and add there a new class SelfHttpClient inherited from the ISelfHttpClient interface:

using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Options;
using SpeedUpCoreAPIExample.Interfaces;
using SpeedUpCoreAPIExample.Models;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net.Http;
using System.Threading.Tasks;

namespace SpeedUpCoreAPIExample.Helpers
{
    // HttpClient for application's own controllers access 
    public class SelfHttpClient : ISelfHttpClient
    {
        private readonly HttpClient _client;
        
        public SelfHttpClient(HttpClient httpClient, IHttpContextAccessor httpContextAccessor)
        {
            string baseAddress = string.Format("{0}://{1}/api/",
                                httpContextAccessor.HttpContext.Request.Scheme,
                                httpContextAccessor.HttpContext.Request.Host);

            _client = httpClient;
            _client.BaseAddress = new Uri(baseAddress);
        }

        // Call any controller's action with HttpPost method and Id parameter.
        // apiRoute - Relative API route.
        // id - The parameter.
        public async Task PostIdAsync(string apiRoute, string id)
        {
            try
            {
                var result = await _client.PostAsync(string.Format("{0}/{1}", apiRoute, Id), null).ConfigureAwait(false);
            }
            catch (Exception ex)
            {
                //ignore errors
            }
        }
    }
}

In this class we obtain a baseAddress of API to be called in the class constructor. In PostIdAsync method we call the API with the HttpPost method by its relative apiRoute route and passing Id as a response parameter. Note, that instead of creating an empty HttpContent we just send null

We should declare this class in the Startup.ConfigureServices method:

services.AddHttpClient();
        services.AddHttpClient<ISelfHttpClient, SelfHttpClient>();

Now we can use in any place of the application. In ProductsService service we should inject it at the class constructor. And we can remove both IHttpContextAccessor and IHttpClientFactory as we do not use them anymore and we can remove the GetFullyQualifiedApiUrl method.

New version of ProductsService constructor will be:

public class ProductsService : IProductsService
{
    private readonly IProductsRepository _productsRepository;
    private readonly ISelfHttpClient _selfHttpClient;

    public ProductsService(IProductsRepository productsRepository, ISelfHttpClient selfHttpClient)
    {
        _productsRepository = productsRepository;
        _selfHttpClient = selfHttpClient;
    }
}    

Let us change the PreparePricesAsync method. First of all, we rename it CallPreparePricesApiAsync as this name is more informative and the method:

private async void CallPreparePricesApiAsync(string productId)
{
    await _selfHttpClient.PostIdAsync("prices/prepare", productId);
}

Do not forget to change PreparePricesAsync for CallPreparePricesApiAsync everywhere when we call this method in the ProductsService. Also take into account, that in CallPreparePricesApiAsync we use type of string productId parameter

You can see, that we pass a tailing part of the API URL as a PostIdAsync parameter. The new SelfHttpClient is really reusable. For example, if we had an API /products/prepare, we could call the API like this:

private async void CallPrepareProductAPIAsync(string productId)
{
    await _selfHttpClient.PostIdAsync("products/prepare", productId);
}

Handling the application's settings

In previous parts, we accessed the application’s settings, by injecting IConfiguration. Then, in class constructers we created a Settings class, in which we parsed appropriate settings variables and applied default values. This approach is good for debugging, but after debugging, using simple POCO classes to access the application’s settings seems to be more preferable. So, let us slightly change our appsettings.json. We will form two sections with settings for the Products and Prices services:

  "Caching": {
    "PricesExpirationPeriod": 15
  }

  "Products": {
    "CachingExpirationPeriod": 15,
    "DefaultPageSize": 20
  },
  "Prices": {
    "CachingExpirationPeriod": 15,
    "DefaultPageSize": 20
  },
…    

Note! We will use DefaultPageSize values letter in this article.

Let us create settings POCO classes. Create a Settings folder with the following files:

namespace SpeedUpCoreAPIExample.Settings
{
    public class ProductsSettings
    {
        public int CachingExpirationPeriod { get; set; }
        public int DefaultPageSize { get; set; }
    }
}

and

namespace SpeedUpCoreAPIExample.Settings
{
    public class PricesSettings
    {
        public int CachingExpirationPeriod { get; set; }
        public int DefaultPageSize { get; set; }
    }
}

Although the classes are still similar, in a real application the setting of different services can vary significantly. So, we will use both classes in order to not divide them later.

Now, all we need for using these classes is to declare them in Startup.ConfigureServices:

//Settings
    services.Configure<ProductsSettings>(Configuration.GetSection("Products"));
    services.Configure<PricesSettings>(Configuration.GetSection("Prices"));

    //Repositories

After that, we can inject settings classes anywhere in our application, as we will demonstrate in following sections

Caching concern separation

In the PricesRepository we have implemented caching with an IDistributedCache cache. Caching in a repository based on the idea to entirely close from the business layer details of data storage sources. In this case, it is not known for a Service whether the data passes the caching stage. Is this solution really good?

Repositories are responsible for working with the DbContext, i.e. getting the data from or saving to a database. But caching is definitely out of this concern. In addition, in more complex systems, after receiving the raw data from the database, the data may need to be modified before it is delivered to the user. And it is reasonable to cache the data in the final state. According to this it is better to apply caching at the business logic layer – in services.

Note! In the PricesRepository.GetPricesAsync and PricesRepository.PreparePricesAsync methods the code for caching is almost the same. Logically we should move this code to a separate class to avoid duplication.

Generic Asynchronous DistributedCache repository

The idea is to create a repository that will encapsulate IDistributedCache business logic. The repository will be generic and be able to cache any type of objects. Here is its Interface

using Microsoft.Extensions.Caching.Distributed;
using System;
using System.Threading.Tasks;

namespace SpeedUpCoreAPIExample.Interfaces
{
    public interface IDistributedCacheRepository<T>
    {
        Task<T> GetOrSetValueAsync(string key, Func<Task<T>> valueDelegate, DistributedCacheEntryOptions options);
        Task<bool> IsValueCachedAsync(string key);
        Task<T> GetValueAsync(string key);
        Task SetValueAsync(string key, T value, DistributedCacheEntryOptions options);
        Task RemoveValueAsync(string key);
    }
}

The only interesting place here is an asynchronous delegate as a second parameter of the GetOrSetValueAsync method. It will be discussed in the implementation section. In the Repositories folder create a new class DistributedCacheRepository:

using Microsoft.Extensions.Caching.Distributed;
using Newtonsoft.Json;
using SpeedUpCoreAPIExample.Interfaces;
using System;
using System.Threading.Tasks;

namespace SpeedUpCoreAPIExample.Repositories
{
    public abstract class DistributedCacheRepository<T> : IDistributedCacheRepository<T> where T : class
    {
        private readonly IDistributedCache _distributedCache;
        private readonly string _keyPrefix;

        protected DistributedCacheRepository(IDistributedCache distributedCache, string keyPrefix)
        {
            _distributedCache = distributedCache;
            _keyPrefix = keyPrefix;
        }

        public virtual async Task<T> GetOrSetValueAsync(string key, Func<Task<T>> valueDelegate, DistributedCacheEntryOptions options)
        {
            var value = await GetValueAsync(key);
            if (value == null)
            {
                value = await valueDelegate();
                if (value != null)
                    await SetValueAsync(key, value, options ?? GetDefaultOptions());
            }

            return null;
        }

        public async Task<bool> IsValueCachedAsync(string key)
        {
            var value = await _distributedCache.GetStringAsync(_keyPrefix + key);

            return value != null;
        }

        public async Task<T> GetValueAsync(string key)
        {
            var value = await _distributedCache.GetStringAsync(_keyPrefix + key);
            
            return value != null ? JsonConvert.DeserializeObject<T>(value) : null;
        }

        public async Task SetValueAsync(string key, T value, DistributedCacheEntryOptions options)
        {
            await _distributedCache.SetStringAsync(_keyPrefix + key, JsonConvert.SerializeObject(value), options ?? GetDefaultOptions());
        }

        public async Task RemoveValueAsync(string key)
        {
            await _distributedCache.RemoveAsync(_keyPrefix + key);
        }
        
        protected abstract DistributedCacheEntryOptions GetDefaultOptions();        
    }
}

The class is abstract as we are not going to create its instances directly. Instead, it will be a base class for the PricesCacheRepository and ProductsCacheRepository classes. Note, that the GetOrSetValueAsync has a virtual modifier – we will override this method in inherited classes. The same is true with the GetDefaultOptions method, in which case it is declared as abstract, so it will have its implementation in derived classes. And when it is called within the parent DistributedCacheRepository class, inherited methods from derived classes will be called.

The second parameter of the GetOrSetValueAsync method is declared as an asynchronous delegate: Func<Task<T>> valueDelegate. In the GetOrSetValueAsync method we are first trying to get a value from the Cache. If it is not already cached, we get it by calling the valueDelegate function and then cache the value.

Let us create inherited classes of definite types from the DistributedCacheRepository.

using Microsoft.Extensions.Caching.Distributed;
using SpeedUpCoreAPIExample.Models;
using System;
using System.Collections.Generic;
using System.Threading.Tasks;

namespace SpeedUpCoreAPIExample.Interfaces
{
    public interface IPricesCacheRepository
    {
        Task<IEnumerable<Price>> GetOrSetValueAsync(string key, Func<Task<IEnumerable<Price>>> valueDelegate, DistributedCacheEntryOptions options = null);
        Task<bool> IsValueCachedAsync(string key);
        Task RemoveValueAsync(string key);
    }
}
using Microsoft.Extensions.Caching.Distributed;
using SpeedUpCoreAPIExample.Models;
using System;
using System.Threading.Tasks;

namespace SpeedUpCoreAPIExample.Interfaces
{
    public interface IProductCacheRepository
    {
        Task<Product> GetOrSetValueAsync(string key, Func<Task<Product>> valueDelegate, DistributedCacheEntryOptions options = null);
        Task<bool> IsValueCachedAsync(string key);
        Task RemoveValueAsync(string key);
        Task SetValueAsync(string key, Product value, DistributedCacheEntryOptions options = null);
    }
}

Then we will create two classes in the Repositories folder

using Microsoft.Extensions.Caching.Distributed;
using Microsoft.Extensions.Options;
using SpeedUpCoreAPIExample.Interfaces;
using SpeedUpCoreAPIExample.Models;
using SpeedUpCoreAPIExample.Settings;
using System;
using System.Collections.Generic;
using System.Threading.Tasks;

namespace SpeedUpCoreAPIExample.Repositories
{
    public class PricesCacheRepository : DistributedCacheRepository<IEnumerable<Price>>, IPricesCacheRepository
    {
        private const string KeyPrefix = "Prices: ";
        private readonly PricesSettings _settings;

        public PricesCacheRepository(IDistributedCache distributedCache, IOptions<PricesSettings> settings)
                      : base(distributedCache, KeyPrefix)
        {
            _settings = settings.Value;
        }

        public override async Task<IEnumerable<Price>> GetOrSetValueAsync(string key, Func<Task<IEnumerable<Price>>> valueDelegate, DistributedCacheEntryOptions options = null)
        {
            return base.GetOrSetValueAsync(key, valueDelegate, options);
        }
        
        protected override DistributedCacheEntryOptions GetDefaultOptions()
        {
            //use default caching options for the class if they are not defined in options parameter
            return new DistributedCacheEntryOptions()
                          {
                              AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(_settings.CachingExpirationPeriod)
                          };
        }
    }
}

And

using Microsoft.Extensions.Caching.Distributed;
using Microsoft.Extensions.Options;
using SpeedUpCoreAPIExample.Interfaces;
using SpeedUpCoreAPIExample.Models;
using SpeedUpCoreAPIExample.Settings;
using System;
using System.Threading.Tasks;

namespace SpeedUpCoreAPIExample.Repositories
{
    public class ProductCacheRepository : DistributedCacheRepository<Product>, IProductCacheRepository
    {
        private const string KeyPrefix = "Product: ";
        private readonly ProductsSettings _settings;

        public ProductCacheRepository(IDistributedCache distributedCache, IOptions<ProductsSettings> settings) : base(distributedCache, KeyPrefix)
        {
            _settings = settings.Value;
        }

        public override async Task<Product> GetOrSetValueAsync(string key, Func<Task<Product>> valueDelegate, DistributedCacheEntryOptions options = null)
        {
            return await base.GetOrSetValueAsync(key, valueDelegate, options);
        }
        
        protected override DistributedCacheEntryOptions GetDefaultOptions()
        {
            //use default caching options for the class if they are not defined in options parameter
            return new DistributedCacheEntryOptions()
                          {
                              AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(_settings.CachingExpirationPeriod)
                          };
        }        
    }
}

Note! Implementation of GetDefaultOptions is equal in both the ProductCacheRepository and the PricesCacheRepository classes and, it seems, could be moved to the base class. But in a real application, the caching policy can vary for different objects and if we move some universal implementation of GetDefaultOptions to the base class, we will have to change the base class when the caching logic of a derived class changes. This will violate the "Open-Close" design principle. That is why, we have implemented GetDefaultOptions method in derived classes.

Declare the repositories in the Startup class

…    
    services.AddScoped<IPricesCacheRepository, PricesCacheRepository>();
    services.AddScoped<IProductCacheRepository, ProductCacheRepository>();
…    

Now, we can remove caching from PricesRepository and make it as simple as possible:

using Microsoft.EntityFrameworkCore;
using SpeedUpCoreAPIExample.Contexts;
using SpeedUpCoreAPIExample.Interfaces;
using SpeedUpCoreAPIExample.Models;
using System.Collections.Generic;
using System.Threading.Tasks;

namespace SpeedUpCoreAPIExample.Repositories
{
    public class PricesRepository : IPricesRepository
    {
        private readonly DefaultContext _context;

        public PricesRepository(DefaultContext context)
        {
            _context = context;
        }

        public async Task<IEnumerable<Price>> GetPricesAsync(int productId)
        {
            return await _context.Prices.AsNoTracking().FromSql("[dbo].GetPricesByProductId @productId = {0}", productId).ToListAsync();
        }
    }
}

We can also rewrite the PricesService class. Instead of IDistributedCache we have injected IPricesCacheRepository.

using SpeedUpCoreAPIExample.Interfaces;
using SpeedUpCoreAPIExample.Models;
using SpeedUpCoreAPIExample.ViewModels;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace SpeedUpCoreAPIExample.Services
{
    public class PricesService : IPricesService
    {
        private readonly IPricesRepository _pricesRepository;
        private readonly IPricesCacheRepository _pricesCacheRepository;

        public PricesService(IPricesRepository pricesRepository, IPricesCacheRepository pricesCacheRepository)
        {
            _pricesRepository = pricesRepository;
            _pricesCacheRepository = pricesCacheRepository;
        }

        public async Task<IEnumerable<PriceViewModel>> GetPricesAsync(int productId)
        {
            IEnumerable<Price> pricess = await _pricesCacheRepository.GetOrSetValueAsync(productId.ToString(), async () =>
                await _pricesRepository.GetPricesAsync(productId));
                
            return pricess.Select(p => new PriceViewModel(p))
                                .OrderBy(p => p.Price)
                                .ThenBy(p => p.Supplier);
        }

        public async Task<bool> IsPriceCachedAsync(int productId)
        {
            return await _pricesCacheRepository.IsValueCachedAsync(productId.ToString());
        }

        public async Task RemovePriceAsync(int productId)
        {
            await _pricesCacheRepository.RemoveValueAsync(productId.ToString());
        }

        public async Task PreparePricesAsync(int productId)
        {
            try
            {
                await _pricesCacheRepository.GetOrSetValueAsync(productId.ToString(), async () => await _pricesRepository.GetPricesAsync(productId));
            }
            catch
            {
            }
        }
    }
}

In the GetPricesAsync and PreparePricesAsync methods we have used the GetOrSetValueAsync method of the PricesCacheRepository. If a desired value is not in the cache, the asynchronous method GetPricesAsync is called.

We have also created IsPriceCachedAsync and RemovePriceAsync methods which will be used later. Do not forget to declare them in the IPricesService interface:

using SpeedUpCoreAPIExample.ViewModels;
using System.Collections.Generic;
using System.Threading.Tasks;

namespace SpeedUpCoreAPIExample.Interfaces
{
    public interface IPricesService
    {
        Task<IEnumerable<PriceViewModel>> GetPricesAsync(int productId);
        Task<bool> IsPriceCachedAsync(int productId);
        Task RemovePriceAsync(int productId);
        Task PreparePricesAsync(int productId);
    }
}

Let us check how the new caching approach works. For this, set a breakpoint inside the GetPricesAsync method:

Image 9

And call http://localhost:49858/api/prices/1 API with the Swagger Inspector Extension two times:

Image 10

During the first call, the debugger reaches the breakpoint. This means, that the GetOrSetValueAsync method cannot find a result in the cache and has to call the _pricesRepository.GetPricesAsync(productId) method, passed to the GetOrSetValueAsync as a delegate. But at the second call, the application workflow does not stop at the breakpoint, because it takes a value from the cache.

Now we can use our universal caching mechanism in the ProductService

namespace SpeedUpCoreAPIExample.Services
{
    public class ProductsService : IProductsService
    {
        private readonly IProductsRepository _productsRepository;
        private readonly ISelfHttpClient _selfHttpClient;
        private readonly IPricesCacheRepository _pricesCacheRepository;
        private readonly IProductCacheRepository _productCacheRepository;
        private readonly ProductsSettings _settings;

        public ProductsService(IProductsRepository productsRepository, IPricesCacheRepository pricesCacheRepository,
            IProductCacheRepository productCacheRepository, IOptions<ProductsSettings> settings, ISelfHttpClient selfHttpClient)
        {
            _productsRepository = productsRepository;
            _selfHttpClient = selfHttpClient;
            _pricesCacheRepository = pricesCacheRepository;
            _productCacheRepository = productCacheRepository;
            _settings = settings.Value;
        }
        
        public async Task<ProductsPageViewModel> FindProductsAsync(string sku)
        {
            IEnumerable<product> products = await _productsRepository.FindProductsAsync(sku);

            if (products.Count() == 1)
            {
                //only one record found
                Product product = products.FirstOrDefault();
                string productId = product.ProductId.ToString();

                //cache a product if not in cache yet
                if (!await _productCacheRepository.IsValueCachedAsync(productId))
                {
                    await _productCacheRepository.SetValueAsync(productId, product);
                }

                //prepare prices
                if (!await _pricesCacheRepository.IsValueCachedAsync(productId))
                {
                    //prepare prices beforehand
                    ThreadPool.QueueUserWorkItem(delegate
                    {
                        CallPreparePricesApiAsync(productId);
                    });
                }
            };

            return new OkObjectResult(products.Select(p => new ProductViewModel(p)));
        }
        
        …
        public async Task<ProductViewModel> GetProductAsync(int productId)
        {
            Product product = await _productCacheRepository.GetOrSetValueAsync(productId.ToString(), async () => await _productsRepository.GetProductAsync(productId));

            if (product == null)
            {
                throw new HttpException(HttpStatusCode.NotFound, "Product not found",  $"Product Id: {productId}");
            }

            //prepare prices
            if (!await _pricesCacheRepository.IsValueCachedAsync(productId.ToString()))
            {
                //prepare prices beforehand
                ThreadPool.QueueUserWorkItem(delegate
                {
                    CallPreparePricesApiAsync(productId.ToString());
                });
            }

            return new ProductViewModel(product);
        }

        …
        public async Task<ProductViewModel> DeleteProductAsync(int productId)
        {
            Product product = await _productsRepository.DeleteProductAsync(productId);

            if (product == null)
            {
                throw new HttpException(HttpStatusCode.NotFound, "Product not found",  $"Product Id: {productId}");
            }
                
            //remove product and its prices from cache
            await _productCacheRepository.RemoveValueAsync(productId.ToString());
            await _pricesCacheRepository.RemoveValueAsync(productId.ToString());

            return new OkObjectResult(new ProductViewModel(product));
        }        
        …

In-Memory and In-Database pagination in Entity Framework

You may have noticed, that the ProductsController’s methods GetAllProductsAsync and FindProductsAsync and the PricesController’s GetPricesAsync method, return collections of products and prices, which have no limitation according to the size of the collections. This mean, that in real application with huge database, responses of some API can return such a large amount of data, that a client application will not be able to process this data or even receive it in a reasonable period of time. To avoid this issue, a good practice is to establish pagination of the API’s results.

There two ways of organizing pagination: in the memory and in the database. For example, when we receive prices for some product, we cache the result in Redis cache. So, we already have available the whole set of the prices and can establish in-memory pagination, which is the fasters approach.

On the other hand, using in-memory pagination in the GetAllProductsAsync method is not a good idea, because to do pagination in memory, we should read the entire Products collection to memory from a database. It is a really slow operation, which consumes a lot of resources. So, in this case, it is better to filter necessary set of data in the database, according to page size and index.

For pagination, we will create a universal PaginatedList class, that will be able to work with collections of any data type and support both in-memory and in-database pagination approaches.

Let us create a generic PaginatedList <T>, inherited from List <T> in the Helpers folder

using Microsoft.EntityFrameworkCore;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace SpeedUpCoreAPIExample.Helpers
{
    public class PaginatedList<T> : List<T>
    {
        public int PageIndex { get; private set; }
        public int PageSize { get; private set; }
        public int TotalCount { get; private set; }
        public int TotalPages { get; private set; }

        public PaginatedList(IEnumerable<T> source, int pageSize, int pageIndex = 1)
        {
            TotalCount = source.Count();

            PageIndex = pageIndex;
            PageSize = pageSize == 0 ? TotalCount : pageSize;
            TotalPages = (int)Math.Ceiling(TotalCount / (double)PageSize);

            this.AddRange(source.Skip((PageIndex - 1) * PageSize).Take(PageSize));
        }

        private PaginatedList(IEnumerable<T> source, int pageSize, int pageIndex, int totalCount) : base(source)
        {
            PageIndex = pageIndex;
            PageSize = pageSize;
            TotalCount = totalCount;
            TotalPages = (int)Math.Ceiling(TotalCount / (double)PageSize);
        }

        public static async Task<PaginatedList<T>> FromIQueryable(IQueryable<T> source, int pageSize, int pageIndex = 1)
        {
            int totalCount = await source.CountAsync();
            pageSize = pageSize == 0 ? totalCount : pageSize;

            int totalPages = (int)Math.Ceiling(totalCount / (double)pageSize);

            if (pageIndex > totalPages)
            {
                //return empty list
                return new PaginatedList<T>(new List<T>(), pageSize, pageIndex, totalCount);
            }

            if (pageIndex == 1 && pageSize == totalCount)
            {
                //no paging needed
            }
            else
            {
                source = source.Skip((pageIndex - 1) * pageSize).Take(pageSize);
            };

            List<T> sourceList = await source.ToListAsync();
            return new PaginatedList<T>(sourceList, pageSize, pageIndex, totalCount);
        }
    }
}

We need the first constructor, to work with in-memory data collection of any type. The second constructor is also being used with collections in memory, but when the page size and the number of pages are already known. We mark it private, as it's being used only in the PaginatedList class itself in the FromIQueryable method.

FromIQueryable is used to establish in-database pagination. The source parameter has IQueryable type. With IQueryable we do not work with physical data until we execute a real request to the database, like source.CountAsync() or source.ToListAsync(). So, we are able to format a proper pagination query and receive only a small set of filtered data in one request.

Let us also adjust the ProductsRepository.GetAllProductsAsync and ProductsRepository.FindProductsAsync methods so that they can work with in-database pagination. Now they should return IQueryable, but not IEnumerable as before.

namespace SpeedUpCoreAPIExample.Interfaces
{
    public interface IProductsRepository
    {
…  
        Task<IEnumerable<Product>> GetAllProductsAsync();
        Task<IEnumerable<Product>> FindProductsAsync(string sku);

        IQueryable<Product> GetAllProductsAsync();
        IQueryable<Product> FindProductsAsync(string sku);
 …  
   }
}

Correct methods’ code in ProductsRepository class

public async Task<IEnumerable<Product>> GetAllProductsAsync()
        {
            return await _context.Products.AsNoTracking().ToListAsync();
        }

        public IQueryable<Product> GetAllProductsAsync()
        {
            return  _context.Products.AsNoTracking();
        }

        public async Task<IEnumerable<Product>> FindProductsAsync(string sku)
        {
            return await _context.Products.AsNoTracking().FromSql("[dbo].GetProductsBySKU @sku = {0}", sku).ToListAsync();
        }

        public IQueryable<Product> FindProductsAsync(string sku)
        {
            return _context.Products.AsNoTracking().FromSql("[dbo].GetProductsBySKU @sku = {0}", sku);
        }
…  

Let us define the classes, in which we will return pagination results to users. In the ViewModels folder create PageViewModel – a base class

namespace SpeedUpCoreAPIExample.ViewModels
{
    public class PageViewModel
    {
        public int PageIndex { get; private set; }
        public int PageSize { get; private set; }
        public int TotalPages { get; private set; }
        public int TotalCount { get; private set; }

        public bool HasPreviousPage => PageIndex > 1;
        public bool HasNextPage => PageIndex < TotalPages;

        public PageViewModel(int pageIndex, int pageSize, int totalPages, int totalCount)
        {
            PageIndex = pageIndex;
            PageSize = pageSize;
            TotalPages = totalPages;
            TotalCount = totalCount;
        }
    }
}

And ProductsPageViewModel and PricesPageViewModel classes, inherited from PageViewModel

using SpeedUpCoreAPIExample.Helpers;
using SpeedUpCoreAPIExample.Models;
using System.Collections.Generic;
using System.Linq;

namespace SpeedUpCoreAPIExample.ViewModels
{
    public class ProductsPageViewModel : PageViewModel
    {
        public IList<ProductViewModel> Items;

        public ProductsPageViewModel(PaginatedList<Product> paginatedList) :
                base(paginatedList.PageIndex, paginatedList.PageSize, paginatedList.TotalPages, paginatedList.TotalCount)
        {
            this.Items = paginatedList.Select(p => new ProductViewModel(p)).ToList();
        }
    }
}
using SpeedUpCoreAPIExample.Helpers;
using SpeedUpCoreAPIExample.Models;
using System.Collections.Generic;
using System.Linq;

namespace SpeedUpCoreAPIExample.ViewModels
{
    public class PricesPageViewModel : PageViewModel
    {
        public IList<PriceViewModel> Items;

        public PricesPageViewModel(PaginatedList<Price> paginatedList) :
                base(paginatedList.PageIndex, paginatedList.PageSize, paginatedList.TotalPages, paginatedList.TotalCount)
        {
            this.Items = paginatedList.Select(p => new PriceViewModel(p))
                .OrderBy(p => p.Price)
                .ThenBy(p => p.Supplier)
                .ToList();
        }
    }
}

In PricesPageViewModel we applied additional sorting to the paginated list of PriceViewModel

Now we should change ProductsService.GetAllProductsAsync and ProductsService.FindProductsAsync so that they return ProductsPageViewMode

public interface IProductsService
…
    Task<IEnumerable<ProductViewModel>> GetAllProductsAsync();
    Task<IEnumerable<ProductViewModel>> FindProductsAsync(string sku);

    Task<ProductsPageViewModel> GetAllProductsAsync(int pageIndex, int pageSize);
    Task<ProductsPageViewModel> FindProductsAsync(string sku, int pageIndex, int pageSize);
    public class ProductsService : IProductsService
    {
        private readonly IProductsRepository _productsRepository;
        private readonly ISelfHttpClient _selfHttpClient;
        private readonly IPricesCacheRepository _pricesCacheRepository;
        private readonly IProductCacheRepository _productCacheRepository;
        private readonly ProductsSettings _settings;

        public ProductsService(IProductsRepository productsRepository, IPricesCacheRepository pricesCacheRepository,
            IProductCacheRepository productCacheRepository, IOptions<ProductsSettings> settings, ISelfHttpClient selfHttpClient)
        {
            _productsRepository = productsRepository;
            _selfHttpClient = selfHttpClient;
            _pricesCacheRepository = pricesCacheRepository;
            _productCacheRepository = productCacheRepository;
            _settings = settings.Value;
        }

        public async Task<ProductsPageViewModel> FindProductsAsync(string sku, int pageIndex, int pageSize)
        {
            pageSize = pageSize == 0 ? _settings.DefaultPageSize : pageSize;
            PaginatedList<Product> products = await PaginatedList<Product>
                                            .FromIQueryable(_productsRepository.FindProductsAsync(sku), pageIndex, pageSize);

            if (products.Count() == 1)
            {
                //only one record found
                Product product = products.FirstOrDefault();
                string productId = product.ProductId.ToString();

                //cache a product if not in cache yet
                if (!await _productCacheRepository.IsValueCachedAsync(productId))
                {
                    await _productCacheRepository.SetValueAsync(productId, product);
                }

                //prepare prices
                if (!await _pricesCacheRepository.IsValueCachedAsync(productId))
                {
                    //prepare prices beforehand
                    ThreadPool.QueueUserWorkItem(delegate
                    {
                        CallPreparePricesApiAsync(productId);
                    });
                }
            };

            return new ProductsPageViewModel(products);
        }

        public async Task<ProductsPageViewModel> GetAllProductsAsync(int pageIndex, int pageSize)
        {
            pageSize = pageSize == 0 ? _settings.DefaultPageSize : pageSize;
            PaginatedList<Product> products = await PaginatedList<Product>
                                            .FromIQueryable(_productsRepository.GetAllProductsAsync(), pageIndex, pageSize);

            return new ProductsPageViewModel(products);
        }
…        

Note, that if no valid parameters PageIndex and PageSize were passed to a PaginatedList constractor, default values – PageIndex = 1 and PageSize = whole datatable size are used. To avoid returning all records of Products and Prices tables, we will use default values DefaultPageSize taken from ProductsSettings and PricesSettings accordingly.

And change PricesServicePricesAsync to return PricesPageViewModel

public interface IPricesService
…
    Task<IEnumerable<PriceViewModel> GetPricesAsync(int productId);

    Task<PricesPageViewModel> GetPricesAsync(int productId, int pageIndex, int pageSize);
…    
    public class PricesService : IPricesService
    {
        private readonly IPricesRepository _pricesRepository;
        private readonly IPricesCacheRepository _pricesCacheRepository;
        private readonly PricesSettings _settings;

        public PricesService(IPricesRepository pricesRepository, IPricesCacheRepository pricesCacheRepository, IOptions<PricesSettings> settings)
        {
            _pricesRepository = pricesRepository;
            _pricesCacheRepository = pricesCacheRepository;
            _settings = settings.Value;
        }

        public async Task<PricesPageViewModel> GetPricesAsync(int productId, int pageIndex, int pageSize)
        {
            IEnumerable<Price> prices = await _pricesCacheRepository.GetOrSetValueAsync(productId.ToString(), async () =>
                    await _pricesRepository.GetPricesAsync(productId));            

            pageSize = pageSize == 0 ? _settings.DefaultPageSize : pageSize;
            return new PricesPageViewModel(new PaginatedList<Price>(prices, pageIndex, pageSize));
        }
…    

Now we can rewrite ProductsController and PricesController so that they can work with the new pagination mechanism

Let us change the ProductsController.GetAllProductsAsync and ProductsController.FindProductsAsync methods. The new versions will be:

[HttpGet]
public async Task<IActionResult> GetAllProductsAsync(int pageIndex, int pageSize)
{
    ProductsPageViewModel productsPageViewModel = await _productsService.GetAllProductsAsync(pageIndex, pageSize);

    return new OkObjectResult(productsPageViewModel);
}

[HttpGet("find/{sku}")]
public async Task<IActionResult> FindProductsAsync(string sku, int pageIndex, int pageSize)
{
    ProductsPageViewModel productsPageViewModel = await _productsService.FindProductsAsync(sku, pageIndex, pageSize);

    return new OkObjectResult(productsPageViewModel);
}

And PricesController.GetPricesAsync method:

[HttpGet("{Id:int}")]
public async Task<IActionResult> GetPricesAsync(int id, int pageIndex, int pageSize)
{
    PricesPageViewModel pricesPageViewModel = await _pricesService.GetPricesAsync(id, pageIndex, pageSize);

    return new OkObjectResult(pricesPageViewModel);
}

If we had some client that worked with an old version of our APIs, it could still work with the new version because, if we miss the pageIndex or pageSize parameter or both, their value will be 0, and our pagination mechanism can correctly process cases with pageIndex=0 and/or pageSize=0.

Since we have reached controllers in our code refactoring, let us stay here and sort out all the initial mess.

Controller vs ControllerBase

You might have noticed, that in our solution ProductsController inherited from the Controller class, and PricesController inherited from the ControllerBase class. Both controllers work well, so which version should we use? Controller class supports Views and so it should be used for creating web-sites that use views. For a WEB API service, ControllerBase is preferable, because it is more lightweight as it does not have features that we do not need in WEB API.

So, we will inherit both our controllers from ControllerBase and use the attribute [ApiController] that enables such useful features as automatic model validation, attribute Routing and others

So, change declaration of ProductsController for:

…
[Route("api/[controller]")]
[ApiController]
public class ProductsController : ControllerBase
{
…

Let us examine, how model validation works with the ApiController attribute. For this, we will call some APIs with invalid parameters. For instance, the following action expects integer Id, but we send a string instead:

http://localhost:49858/api/products/aa

The result will be:

Status: 400 Bad Request

{
    "id": [
        "The value 'aa' is not valid."
    ]
}

In case, when we have intentionally declared type of parameter [HttpGet("{Id:int}")] things are even worse:

http://localhost:49858/api/prices/aa

Status: 404 Not Found without any message about incorrect type of Id parameter.

So, firstly, we will remove Id type declaration from the HttpGet attribute in the PricesController.GetPricesAsync method:

[HttpGet("{Id:int}")]
[HttpGet("{id}")]

This will give us a standard 400 Bad Request and a type mismatch message.

Another problem that directly concerns application productivity is eliminating senseless job. For instance, http://localhost:49858/api/prices/-1 API will evidently return 404 Not Found, as our database will never have any negative Id value.

And we use positive integer Id parameter several times in our application. So, the idea is to create an Id validation filter and use it whenever we have an Id parameter.

Custom Id parameter validation filter and Attribute

In your solution create a Filters folder and a new class ValidateIdAsyncActionFilter in it:

using Microsoft.AspNetCore.Mvc.Filters;
using SpeedUpCoreAPIExample.Exceptions;
using System.Linq;
using System.Threading.Tasks;

namespace SpeedUpCoreAPIExample.Filters
{
    // Validating Id request parameter ActionFilter. Id is required and must be a positive integer
    public class ValidateIdAsyncActionFilter : IAsyncActionFilter
    {
        public async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
        {
            ValidateParameter(context, "id");

            await next();
        }
        
        private void ValidateParameter(ActionExecutingContext context, string paramName)
        {
            string message = $"'{paramName.ToLower()}' must be a positive integer.";

            var param = context.ActionArguments.SingleOrDefault(p => p.Key == paramName);
            if (param.Value == null)
            {
                throw new HttpException(System.Net.HttpStatusCode.BadRequest, message, $"'{paramName.ToLower()}' is empty.");
            }

            var id = param.Value as int?;
            if (!id.HasValue || id < 1)
            {
                throw new HttpException(System.Net.HttpStatusCode.BadRequest, message,
                                        param.Value != null ? $"{paramName}: {param.Value}" : null);
            }
    }
}    

In the filter we check whether a request has only one Id parameter. If the Id parameter is missed or does not have positive integer value, the filter generates BadRequest HttpException. Throwing an HttpException involves our ExceptionsHandlingMiddleware in the process, with all its benefits like logging, unified message format and so on.

To be able to apply this filter in any place of our controllers, we will create a ValidateIdAttribute in the same Filters folder:

using Microsoft.AspNetCore.Mvc;

namespace SpeedUpCoreAPIExample.Filters
{
    public class ValidateIdAttribute : ServiceFilterAttribute
    {
        public ValidateIdAttribute() : base(typeof(ValidateIdAsyncActionFilter))
        {
        }
    }
}    

In ProductsController add reference filters classes namespace

using SpeedUpCoreAPIExample.Filters;
…

and add the [ValidateId] attribute to all the GetProductAsync and DeleteProductAsync actions that need an Id parameter:

…
    [HttpGet("{id}")]
    [ValidateId]
    public async Task<IActionResult> GetProductAsync(int id)
    {
…
    [HttpDelete("{id}")]
    [ValidateId]
    public async Task<IActionResult> DeleteProductAsync(int id)
    {
…

And we can apply the ValidateId attribute to the whole PricesController controller as all its actions need an Id parameter. In addition, we need to correct inaccuracies in the PricesController class namespace – it should obviously be namespace SpeedUpCoreAPIExample.Controllers, but not namespace SpeedUpCoreAPIExample.Contexts

using Microsoft.AspNetCore.Mvc;
using SpeedUpCoreAPIExample.Filters;
using SpeedUpCoreAPIExample.Interfaces;
using SpeedUpCoreAPIExample.ViewModels;
using System.Threading.Tasks;

namespace SpeedUpCoreAPIExample.Contexts
namespace SpeedUpCoreAPIExample.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    
    public class PricesController : ControllerBase
    {    
…

The last step is to declare the filter in Startup.cs

using SpeedUpCoreAPIExample.Filters;
…
public void ConfigureServices(IServiceCollection services)
…
services.AddSingleton<ValidateIdAsyncActionFilter>();
…

Let us check how the new filter works. For this we will again call incorrectly API http://localhost:49858/api/prices/-1. The result will be exactly as we desired:

Status: 400 Bad Request

{
    "message": "'Id' must be a positive integer."
}

Note! We have used the ExceptionMessage class and now Messages usually satisfy our format conventions, but not always! If we try the http://localhost:49858/api/prices/aa ones again, we will still have a standard 400 Bad Request message. This happens, because of the [ApiController] attribute. When it is applied, the framework automatically registers a ModelStateInvalidFilter, which will work before our ValidateIdAsyncActionFilter filter and will generate a message of its own format.

We can suppress this behavior in the ConfigureServices method of the Startup class:

…    
services.AddMvc();
services.AddApiVersioning();
…
services.Configure<ApiBehaviorOptions>(options =>
{
    options.SuppressModelStateInvalidFilter = true;
});

After that, only our filter is working and we can control the model validation messages format. But now we are obligated to organize explicit validation for all parameters of controller actions.

Pagination parameters custom model validation filter

We have used pagination tree times in our simple application. Let us examine what will happen with incorrect parameters. For this we will call http://localhost:49858/api/products?pageindex=-1

The result will be:

Status: 500 Internal Server Error

{
    "message": "The offset specified in a OFFSET clause may not be negative."
}

This message is really confusing, because there was not a Server Error, it was a pure BadRequest. And the text itself is mysterious if you do not know that it is about pagination.

We would prefer to have a response:

Status: 400 Bad Request

{
    "message": "'pageindex' must be 0 or a positive integer."
}

Another question is where to apply parameter checking. Note, that our pagination mechanism works well if any or both parameters are omitted – it uses default values. We should control only negative parameters. Throwing HttpException at PaginatedList level id not a good idea, as code should be reusable without changing it, and next time a PaginatedList will not necessarily be used in ASP.NET applications. Checking parameters at the Services level is better, but will demand duplication of the validation code or creating other public helper classes with validation methods.

As far as pagination parameters come from outside,better places to organize their checking are in controllers before passing to a pagination procedure.

So, we have to create another model validation filter, that will validate the PageIndex and PageSize parameters. The idea of validation is slightly different – any or both parameters can be omitted, can be equal zero or an integer greater than zero.

In the same Filters folder create a new class ValidatePagingAsyncActionFilter:

using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.Filters;
using Newtonsoft.Json.Linq;
using System.Linq;
using System.Threading.Tasks;

namespace SpeedUpCoreAPIExample.Filters
{
    // Validating PageIndex and PageSize request parameters ActionFilter. If exist, must be 0 or a positive integer
    public class ValidatePagingAsyncActionFilter : IAsyncActionFilter
    {
        public async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
        {
            ValidateParameter(context, "pageIndex");
            ValidateParameter(context, "pageSize");

            await next();
        }

        private void ValidateParameter(ActionExecutingContext context, string paramName)
        {
            var param = context.ActionArguments.SingleOrDefault(p => p.Key == paramName);
            if (param.Value != null)
            {
                var id = param.Value as int?;
                if (!id.HasValue || id < 0)
                {
                    string message = $"'{paramName.ToLower()}' must be 0 or a positive integer.";
                    throw new HttpException(System.Net.HttpStatusCode.BadRequest, message,
                                            param.Value != null ? $"{paramName}: {param.Value}" : null);
                }
            }
        }
    }
}

Then create ValidatePagingAttribute class:

using Microsoft.AspNetCore.Mvc;

namespace SpeedUpCoreAPIExample.Filters
{
    public class ValidatePagingAttribute : ServiceFilterAttribute
    {
        public ValidatePagingAttribute() : base(typeof(ValidatePagingAsyncActionFilter))
        {
        }
    }
}

Then declare the filter in Startup.cs

public void ConfigureServices(IServiceCollection services)
…
services.AddSingleton<ValidatePagingAsyncActionFilter>();
…

And finally, add [ValidatePaging] attribute to ProductsController.GetAllProductsAsync, ProductsController.FindProductsAsync methods:

…
    [HttpGet]
    [ValidatePaging]
    public async Task<IActionResult> GetAllProductsAsync(int pageIndex, int pageSize)
    {
…    
    [HttpGet("find/{sku}")]
    [ValidatePaging]
    public async Task<IActionResult> FindProductsAsync(string sku, int pageIndex, int pageSize)
    {
…    

and PricesController.GetPricesAsync method:

…
    [HttpGet("{id}")]
    [ValidatePaging]
    public async Task<IActionResult> GetPricesAsync(int id, int pageIndex, int pageSize)
    {
…    

Now we have an auto validation mechanism for all the sensitive parameters, and our application works correctly (at least locally)

Cross-origin resource sharing (CORS)

In a real application we will bind some domain name to our web-service and its URL will look like http://mydomainname.com/api/

At the same time, a client application that consumes APIs of our service can host on a different domain. If a client, a web-site for example, uses AJAX for API requests, and the response does not contain Access-Control-Allow-Origin header with value = * (all domains allowed), or with the same host as origin (client's host), browsers that support CORS, will block the response for safety reasons.

Let us make sure. Build and publish our application to IIS, bind it with a test URL (mydomainname.com in our example), and call any API with https://resttesttest.com/ - on-line tool for API checking:

Image 11

Enable CORS ASP.NET Core

To enforce our application sending the right header, we should enable CORS. For this install the Microsoft.AspNetCore.Cors NuGet package (if you still do not have it installed with another package like Microsoft.AspNetCore.MVC or Microsoft.AspNetCore.All)

The simplest way to enable CORS is to add to the Startup.cs the following code:

public void Configure(
…
app.UseCors(builder => builder
    .AllowAnyOrigin()
    .AllowAnyMethod()
    .AllowAnyHeader());
…
app.UseMvc();
…

This way we have allowed access to our API from any host. We could also add the .AllowCredentials() option, but it is not secure to use it with AllowAnyOrigin.

After that, rebuild, republish the application to IIS and test it with resttesttest.com or another tool. At first glance, everything works fine - the CORS error message disappeared. But this works only until our ExceptionsHandlingMiddleware enters the game.

No CORS headers sent in case of HTTP error

This happens because in fact, the response headers collection is empty, when an HttpException or any other Exception occurs and the middleware processes it. This means, that no Access-Control-Allow-Origin header is passed to a client application and CORS issue arises.

How to send HTTP 4xx-5xx response with CORS headers in an ASPNET.Core web app

To overcome this problem, we should enable CORS slightly differently. In Startup.ConfigureServices enter the following code:

public void ConfigureServices(IServiceCollection services)
{
    services.AddCors(options =>
    {
        options.AddPolicy("Default", builder =>
        {
            builder.AllowAnyOrigin();
            builder.AllowAnyMethod();
            builder.AllowAnyHeader();
        });
    });
…

And in Startup.Configure:

public void Configure(
…
    app.UseCors("Default");
…
    app.UseMvc();
…

Enabling CORS this way, gives us access to CorsOptions in any place of our application via dependency injection. And the idea is to repopulate the response header in ExceptionsHandlingMiddleware with the CORS policy, taken from CorsOptions.

Correct code of the ExceptionsHandlingMiddleware class:

using Microsoft.AspNetCore.Cors.Infrastructure;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using Newtonsoft.Json;
using System;
using System.Net;
using System.Threading.Tasks;

namespace SCARWebService.Exceptions
{
    public class ExceptionsHandlingMiddleware
    {
        private readonly RequestDelegate _next;
        private readonly ILogger<ExceptionsHandlingMiddleware> _logger;
        private readonly ICorsService _corsService;
        private readonly CorsOptions _corsOptions;

        public ExceptionsHandlingMiddleware(RequestDelegate next, ILogger<ExceptionsHandlingMiddleware> logger,
                                                ICorsService corsService, IOptions<CorsOptions> corsOptions)
        {
            _next = next;
            _logger = logger;
            _corsService = corsService;
            _corsOptions = corsOptions.Value;
        }

…
        private async Task HandleHttpExceptionAsync(HttpContext context, HttpException exception)
        {
            _logger.LogError(exception, exception.MessageDetail);

            if (!context.Response.HasStarted)
            {
                int statusCode = exception.StatusCode;
                string message = exception.Message;

                context.Response.Clear();

                //repopulate Response header with CORS policy
                _corsService.ApplyResult(_corsService.EvaluatePolicy(context, _corsOptions.GetPolicy("Default")), context.Response);
    
                context.Response.ContentType = "application/json";
                context.Response.StatusCode = statusCode;

                var result = new ExceptionMessage(message).ToString();
                await context.Response.WriteAsync(result);
            }
        }

        private async Task HandleUnhandledExceptionAsync(HttpContext context, Exception exception)
        {

            _logger.LogError(exception, exception.Message);

            if (!context.Response.HasStarted)
            {
                int statusCode = (int)HttpStatusCode.InternalServerError; // 500
                string message = string.Empty;
#if DEBUG
                message = exception.Message;
#else
                message = "An unhandled exception has occurred";
#endif
                context.Response.Clear();
                
                //repopulate Response header with CORS policy
                _corsService.ApplyResult(_corsService.EvaluatePolicy(context, _corsOptions.GetPolicy("Default")), context.Response);
                    
                context.Response.ContentType = "application/json";
                context.Response.StatusCode = statusCode;

                var result = new ExceptionMessage(message).ToString();
                await context.Response.WriteAsync(result);
            }
        }
…

If we rebuild and republish our application, it will work fine without any CORS issue, when its APIs are being called from any host.

API versioning

Before making our application public, we must consider how its APIs will be consumed. After a certain period of time the requirements might be changed and we will have to rewrite the application so that its API will return different sets of data. If we publish web-service with new changes, but do not update the client’s applications that consume the APIs we will have big problems with client-server compatibility.

To avoid these problems, we should establish API versioning. For instance, an old version of Products API will have a route:

http://mydomainname.com/api/v1.0/products/

and a new version will have a route

http://mydomainname.com/api/v2.0/products/

In this case, even old client applications will continue working fine, until they are updated for a release that can work correctly with version v2.0

In our application we will realize URL Path Based Versioning, where a version number is a part of the APIs URL, like in the above-mentioned example.

In .NET Core Microsoft.AspNetCore.Mvc.Versioning package is responsible for Versioning. So, we should install the package first:

Image 12

Image 13

Then add services.AddApiVersioning() to the Startup's class ConfigureServices method:

…
    services.AddMvc();
    services.AddApiVersioning();

And finally, add ApiVersion and correct Route attributes to both controllers:

[ApiVersion("1.0")]
    [Route("/api/v{version:apiVersion}/[controller]/")]

Now we have versioning. Having done that, if we want to enhance the application for a version 2.0, for example, we can add the [ApiVersion("2.0")] attribute to a controller:

[ApiVersion("1.0")]
    [ApiVersion("2.0")]

then create an action, we want to be working only with v2.0 and add add [MapToApiVersion("2.0")] attribute to the action.

The versioning mechanism works perfectly out of the box almost without any coding but, as usual, with a fly in the ointment: if we have accidentally used a wrong version in the API URL (http://localhost:49858/api/v10.0/prices/1), we will have an error message in the following format:

Status: 400 Bad Request

{
    "error": {
        "code": "UnsupportedApiVersion",
        "message": "The HTTP resource that matches the request URI 'http://localhost:49858/api/v10.0/prices/1' does not support the API version '10.0'.",
        "innerError": null
    }
}

This is the standard error response format. It is much more informative, but absolutely far from our desired format. So, if we want to use unified format for all type of messages, we have to make a choice between the detailed standard error response format and the simple one, we have designed for our application.

To apply the standard error response format, we could just extend our ExceptionMessage class. Fortunately, we have foreseen this opportunity and it would not be difficult. But in this format messages are even more detailed, than we want to pass to users. Such detalization is probably not really relevant in a simple application. So, as far as we are not going to complicate things, we will use our simple format.

Controlling API versioning error message format

Let us create a VersioningErrorResponseProvider class in the Exceptions folder:

using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.Versioning;

namespace SpeedUpCoreAPIExample.Exceptions
{
    public class VersioningErrorResponseProvider : DefaultErrorResponseProvider
    {
        public override IActionResult CreateResponse(ErrorResponseContext context)
        {
            string message = string.Empty;
            switch (context.ErrorCode)
            {
                case "ApiVersionUnspecified":
                    message = "An API version is required, but was not specified.";

                    break;

                case "UnsupportedApiVersion":
                    message = "The specified API version is not supported.";

                    break;

                case "InvalidApiVersion":
                    message = "An API version was specified, but it is invalid.";

                    break;

                case "AmbiguousApiVersion":
                    message = "An API version was specified multiple times with different values.";

                    break;

                default:
                    message = context.ErrorCode;

                    break;
            }

            throw new HttpException(System.Net.HttpStatusCode.BadRequest, message, context.MessageDetail);
        }
    }
}

The class inherits from DefaultErrorResponseProvider. It just formats a friendly message, according to an ErrorCode (list of codes) and throws out HttpException BadRequest exception. Then the exception is processed by our ExceptionHandlerMiddleware with logging, unified error message formatting, etc.

The last step is to register the VersioningErrorResponseProvider class as versioning HTTP error response generator. In the Startup class, in the ConfigureServices method add options at API versioning service registration:

…    
    services.AddMvc();
    services.AddApiVersioning(options =>
    {
        options.ErrorResponses = new VersioningErrorResponseProvider();
    });
…    

Thus, we have changed the standard error response behavior to our desired one.

Versioning in inner HTTP invocations

We also have to apply versioning in the SelfHttpClient class. In the class we set the BaseAddress property of HttpClient to call API. We should consider versioning when building the base address.

To avoid hard coding of the APIs version we are going to invoke, we create a settings class for API versioning. In the appsettings.json file create an API section:"

…
 ,
 "Api": {
    "Version": "1.0"
  }
…

Then in the Settings folder create ApiSettings.cs file:

namespace SpeedUpCoreAPIExample.Settings
{
    public class ApiSettings
    {
        public string Version { get; set; }
    }
}

Declare the class in the Startup's ConfigureServices method:

public void ConfigureServices(IServiceCollection services)
…  
    //Settings
    services.Configure<ApiSettings>(Configuration.GetSection("Api"));
…

And, finally, change the SelfHttpClient's constructor:

public SelfHttpClient(HttpClient httpClient, IHttpContextAccessor httpContextAccessor, IOptions<ApiSettings> settings)
{
    string baseAddress = string.Format("{0}://{1}/api/v{2}/",
                                        httpContextAccessor.HttpContext.Request.Scheme,
                                        httpContextAccessor.HttpContext.Request.Host,
                                        settings.Value.Version);

    _client = httpClient;
    _client.BaseAddress = new Uri(baseAddress);
}

DNS name resolving locally

Let us finish with the SelfHttpClient class. We use it to call our own API for data preparation in advance. In the class contractor we build the base address of our API, using HttpContextAccessor. As far as we have started publishing our application on the internet, the base address will be like http://mydomainname.com/api/v1.0/. When we invoke an API, the HttpClient in background appeals to a DNS server to resolve this mydomainname.com host name into the IP of the web server where the application runs and then goes to this IP. But we know the IP - it is the IP of our own server. So, to avoid this senseless trip to a DNS server, we should resolve the host name locally, by adding it in the hosts file on our server.

Path to the hosts file is C:\Windows\System32\drivers\etc\

You should add the next entries:

192.168.1.1 mydomainname.com
192.168.1.1 www.mydomainname.com

where 192.168.1.1 - is the IP of our web-server in a local network

After this improvement HTTP response will not even leave the boundaries of our server and, thus, will be executed much faster.

Documenting .NET Core API application

We can consider two aspects of documenting the application:

  • XML documentation of code - actually, the code should be self-documented. However, sometimes we still need to give an extra explanation about the details of some methods and their parameters. We will document our code with XML comments;
  • OpenAPI documentation - documenting APIs so that developers of the client’s application could be able to apply to this document in an OpenAPI Specification format and receive the comprehensive information that reflects all the API's details.

XML comments

To enable XML comments, open project properties and select Build tab:

Image 14

Here we should check the XML documentation file checkbox and leave the default value. We should also add 1591 warning numbers into the Suppress warnings textbox to prevent compiler warnings if we omit XML comments for some public classes, properties, methods etc.

Now we can comment our code like this:

/// <summary>
/// Call any controller's action with HttpPost method and Id parameter.
/// </summary>
/// <param name="apiRoute">Relative API route.</param>
/// <param name="id">The parameter.</param>
public async Task PostIdAsync(string apiRoute, string id)
…  

Here you can find detailed information about Documenting code with XML comments.

An XML file with a name, specified in the XML documentation file textbox, will be created. We will need this file later.

OpenAPI documentation for RESTful APIs with Swagger

Requirements to API documentation mechanism:

The documentation should be generated automatically;

API versioning should be supported and autodiscovered;

Documentation from the XML comments file should also be used;

The mechanism should provide the UI with the documentation where users are able to test the APIs without writing a real client application;

The documentation should include examples of using.

We will use Swagger to fulfill all these requirements. Let us install the necessary NuGet packages. In the NuGet package manager install:

Swashbuckle.AspNetCore (4.0.1),
Swashbuckle.AspNetCore.Examples (2.9.0),
Swashbuckle.AspNetCore.Filters (4.5.5),
Microsoft.AspNetCore.Mvc.Versioning.ApiExplorer (3.2.1)

Note! We need the ApiExplorer package to discover all API versions automatically and generate descriptions and endpoint for each discovered version.

After installation our Dependencies - NuGet list will also include:

Image 15

Note! Although at the time of writing this article Swashbuckle.AspNetCore and Swashbuckle.AspNetCore.Filters version 5.0.0-rc8 were available, we used lower versions. The reason for this was some compatibility issues between versions 2.9.0 and 5.0.0-rc8. So, the proven stable combination of NuGet packages was selected. Hopefully, in new releases, Swagger developers will resolve all the compatibility issues.

Let us create a Swagger folder in our application and then a SwaggerServiceExtensions class in it. This static Swagger extensions class will encapsulate all the logic concerning service setup. We will call methods of this class from the Startup's ConfigureServices and Configure methods, and thus making the Startup class shorter and readable.

Here is the entire class with the following explanations:

using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Mvc.ApiExplorer;
using Microsoft.Extensions.DependencyInjection;
using Swashbuckle.AspNetCore.Examples;
using Swashbuckle.AspNetCore.Swagger;
using Swashbuckle.AspNetCore.SwaggerUI;
using System;
using System.IO;
using System.Reflection;

namespace SpeedUpCoreAPIExample.Swagger
{
    public static class SwaggerServiceExtensions
    {
        public static IServiceCollection AddSwaggerDocumentation(this IServiceCollection services)
        {
            services.AddVersionedApiExplorer(options =>
            {
                //The format of the version added to the route URL (VV = <major>.<minor>) 
                options.GroupNameFormat = "'v'VV";

                //Order API explorer to change /api/v{version}/ to /api/v1/  
                options.SubstituteApiVersionInUrl = true;
            });

            // Get IApiVersionDescriptionProvider service
            IApiVersionDescriptionProvider provider = services.BuildServiceProvider().GetRequiredService<IApiVersionDescriptionProvider>();

            services.AddSwaggerGen(options =>
            {
                //Create description for each discovered API version
                foreach (ApiVersionDescription description in provider.ApiVersionDescriptions)
                {
                    options.SwaggerDoc(description.GroupName, 
                        new  Info()
                        {
                            Title = $"Speed Up ASP.NET Core WEB API Application {description.ApiVersion}",
                            Version = description.ApiVersion.ToString(),
                            Description = "Using various approaches to increase .Net Core RESTful WEB API productivity.",
                            TermsOfService = "None",
                            Contact = new Contact
                            {
                                Name = "Silantiev Eduard",
                                Email = "",
                                Url = "https://www.codeproject.com/Members/EduardSilantiev"
                            },
                            License = new License
                            {
                                Name = "The Code Project Open License (CPOL)",
                                Url = "https://www.codeproject.com/info/cpol10.aspx"
                            }
                        });
                }

                //Extend Swagger for using examples
                options.OperationFilter<ExamplesOperationFilter>();

                //Get XML comments file path and include it to Swagger for the JSON documentation and UI.
                string xmlCommentsPath = Assembly.GetExecutingAssembly().Location.Replace("dll", "xml");
                options.IncludeXmlComments(xmlCommentsPath);
             });

            return services;
        }

        public static IApplicationBuilder UseSwaggerDocumentation(this IApplicationBuilder app,
                                        IApiVersionDescriptionProvider provider)
        {
            app.UseSwagger();
            app.UseSwaggerUI(options =>
            {
                //Build a swagger endpoint for each discovered API version  
                foreach (ApiVersionDescription description in provider.ApiVersionDescriptions)
                {
                    options.SwaggerEndpoint($"/swagger/{description.GroupName}/swagger.json", description.GroupName.ToUpperInvariant());
                    options.RoutePrefix = string.Empty;
                    options.DocumentTitle = "SCAR store API documentation";
                    options.DocExpansion(DocExpansion.None);
                }
            });

            return app;
        }
    }
}

In the AddSwaggerDocumentation method we add VersionedApiExplorer with options, that allows ApiExplorer to understand the format of our versioning in API's routes and automatically change /v{version:apiVersion}/ to /v1.1/ in OpenApi documentation.

Note! The "'v'VV" pattern fits our versioning considerations: <major>.<minor> i.e. v1.0. But Swagger will turn v1.0 to v1 and v1.1 will stay as it is. Nevertheless, APIs will work fine with both v1.0 and v1 notations. Here you can find detailed information about Custom API Version Format Strings

Then we instantiate ApiVersionDescriptionProvider. We need this service to obtain a list of versions and generate description for each discovered version. In services.AddSwaggerGen command we generate these descriptions.

Here you can find details about OpenAPI Specification.

In the next line we extend Swagger Generator so that it will be able to add response example (and request example, although not in our case) to OpenApi documentation:

…      
    options.OperationFilter<ExamplesOperationFilter>();
…

The final stage of the AddSwaggerDocumentation method is to let Swagger know the path to the XML comments file. Thus, Swagger will include XML comments in its json OpenApi file and UI.

In the UseSwaggerDocumentation method we enable Swagger and build Swagger UA endpoints for all API versions. We use IApiVersionDescriptionProvider again to discover all APIs, but this time we pass the provider as a parameter of the method, because we call the UseSwaggerDocumentation method from the Startup.Configure method, where we are already able to get the provider reference via dependency injection.

RoutePrefix = string.Empty option means that the Swagger UI will be available at the root URL of our application, i.e. http://mydomainname.com or http://mydomainname.com/index.html

DocExpansion(DocExpansion.None) means that request bodies in the Swagger UI will all be collapsed at opening.

Swagger response examples

We have already extended Swagger for using examples in the AddSwaggerDocumentation method. Let us create example data classes. In the Swagger folder create a file SwaggerExamples.cs that will consist of all example classes:

using SpeedUpCoreAPIExample.Exceptions;
using SpeedUpCoreAPIExample.ViewModels;
using Swashbuckle.AspNetCore.Examples;
using System.Collections.Generic;

namespace SpeedUpCoreAPIExample.Swagger
{
    public class ProductExample : IExamplesProvider
    {
        public object GetExamples()
        {
            return new ProductViewModel(1, "aaa", "Product1");
        }
    }

    public class ProductsExample : IExamplesProvider
    {
        public object GetExamples()
        {
            return new ProductsPageViewModel()
            {
                PageIndex = 1,
                PageSize = 20,
                TotalPages = 1,
                TotalCount = 3,
                Items = new List<ProductViewModel>()
                {
                    new ProductViewModel(1, "aaa", "Product1"),
                    new ProductViewModel(2, "aab", "Product2"),
                    new ProductViewModel(3, "abc", "Product3")
                }
            };
        }
    }

    public class PricesExamples : IExamplesProvider
    {
        public object GetExamples()
        {
            return new PricesPageViewModel()
            {
                PageIndex = 1,
                PageSize = 20,
                TotalPages = 1,
                TotalCount = 3,
                Items = new List<PriceViewModel>()
                { 
                    new PriceViewModel(100, "Bosch"),
                    new PriceViewModel(125, "LG"),
                    new PriceViewModel(130, "Garmin")
                }
            };
        }
    }

    public class ProductNotFoundExample : IExamplesProvider
    {
        public object GetExamples()
        {
            return new ExceptionMessage("Product not found");
        }
    }

    public class InternalServerErrorExample : IExamplesProvider
    {
        public object GetExamples()
        {
            return new ExceptionMessage("An unhandled exception has occurred");
        }
    }
}

The classes are really simple, they just return ViewModels with example data or error message examples in our unified messages format. Then we will link the API's response code with an appropriate example.

Now we add the Swagger service in the Startup.ConfigureServices method:

public void ConfigureServices(IServiceCollection services)
…          
        services.AddSwaggerDocumentation();
…

and add Swagger middleware in the Startup.Configure method:

public void Configure(IApplicationBuilder app, IHostingEnvironment env,
                      ILoggerFactory loggerFactory, IApiVersionDescriptionProvider provider)
…          
    app.UseSwaggerDocumentation(provider);

    app.UseCors("Default");
    app.UseMvc();
…

Note! We get IApiVersionDescriptionProvider via dependency injection and pass it to UseSwaggerDocumentation as a parameter.

Tags and attributes to form OpenApi documentation

Swagger understands most XML comments tags and has a variety of its own attributes. We have chosen only a small part of them, but quite enough for generating brief and clear documentation.

We should apply these tags and attributes in controllers at actions declaration. Here are some examples for ProductsController with explanations:

/// <summary>
    /// Gets all Products with pagination.
    /// </summary>
    /// <remarks>GET /api/v1/products/?pageIndex=1&pageSize=20</remarks>
    /// <param name="pageIndex">Index of page to display (if not set, defauld value = 1 - first page is used).</param>
    /// <param name="pageSize">Size of page (if not set, defauld value is used).</param>
    /// <returns>List of product swith pagination state</returns>
    /// <response code="200">Products found and returned successfully.</response>
    [ProducesResponseType(typeof(ProductsPageViewModel), StatusCodes.Status200OK)]
    [SwaggerResponseExample(StatusCodes.Status200OK, typeof(ProductsExample))]
    [HttpGet]
    [ValidatePaging]
    public async Task<IActionResult> GetAllProductsAsync(int pageIndex, int pageSize)
…
/// <summary>
    /// Gets a Product by Id.
    /// </summary>
    /// <remarks>GET /api/v1/products/1</remarks>
    /// <param name="id">Product's Id.</param>
    /// <returns>A Product information</returns>
    /// <response code="200">Product found and returned successfully.</response>
    /// <response code="404">Product was not found.</response>
    [ProducesResponseType(typeof(ProductViewModel), StatusCodes.Status200OK)]
    [ProducesResponseType(typeof(string), StatusCodes.Status404NotFound)]
    [SwaggerResponseExample(StatusCodes.Status200OK, typeof(ProductExample))]
    [SwaggerResponseExample(StatusCodes.Status404NotFound, typeof(ProductNotFoundExample))]
    [HttpGet("{id}")]
    [ValidateId]
    public async Task<IActionResult> GetProductAsync(int id)
…

The tags are clearly self-explanatory. Let us review the attributes:

[ProducesResponseType(typeof(ProductViewModel), StatusCodes.Status200OK)]

We state here, that the type of return value will be ProductViewModel if the operation is successful: Response code = 200 OK)

[SwaggerResponseExample(StatusCodes.Status200OK, typeof(ProductExample))]

Here we link the StatusCodes.Status200OK and ProductExample class, that we have created and filled with demo data.

Note! Swagger has automatically recognized the id parameter as required from the [HttpGet("{id}")] attribute.

The response codes list of out APIs is not really full. The exception handling middleware can also return Status500InternalServerError (internal server error) for any API. Instead of adding a description for the Response code = 500 code for each action we can declare this once for the entire controller:

…
    [ApiVersion("1.0")]
    [Route("/api/v{version:apiVersion}/[controller]/")]
    [ApiController]
    [ProducesResponseType(typeof(string), StatusCodes.Status500InternalServerError)]
    [SwaggerResponseExample(StatusCodes.Status500InternalServerError, typeof(InternalServerErrorExample))]
    public class ProductsController : ControllerBase
    {
…

Note! We do not want to expose our inner API api/v1/prices/prepare of PricesController so that it is visible to the client’s app developers. That's why we attributed the action with IgnoreApi = true:

[ApiExplorerSettings(IgnoreApi = true)]
    [HttpPost("prepare/{id}")]
    public async Task<IActionResult> PreparePricesAsync(int id)
    {
…

If we start our application and go to its root URL, we will find the Swagger UI that was formed according to the provided options, XML comments and Attributes:

Image 16

In the right-top corner we can see the "Select a spec" session, which is a version selector. If we add an [ApiVersion("2.0")] attribute in some controller, the 2.0 version will be discovered automatically and will appear in this dropdownlist:

Image 17

The Swagger UI is really simple. We can expand/collapse each API and observe its description, parameter, examples, etc. If we want to test the API, we should click the "TryItOut" button:

Image 18

Then enter a value, you want to examine in the appropriate parameter's input box and click Examine:

Image 19

The result in this case will be as expected:

Image 20

For developers of the client’s apps, an OpenApi json file is available for downloading:

Image 21

It can be used for autogenerating code of client application with NSwagStudio, for example, or imported into some testing frameworks, like Postman, to establish automatic testing of APIs.

Getting rid of unused or duplicated NuGet packages

Code refactoring and refinement seems an endless process. So, we have to stop here. However, you can continue with a useful tool, such as ReSharper, to get new ideas about how to improve your code quality.

Since the code will not be changed any more, at least in the boundaries of this article, we can revise the NuGet packages that we have at this moment. It now becomes evident that we have some packages duplication and a real mess in their versioning.

At the moment our dependencies structure looks like this:

Image 22

Actually, the Microsoft.AspNetCore.All package includes all four of these selected packages, so we can easily remove them from the application.

Image 23

But when removing these packages, we should take into account version compatibility. For example, the Microsoft.AspNetCore.All (2.0.5) package includes Microsoft.AspNetCore.Mvc (2.0.2). This means, that we will have problems with the ApiController attribute we are using in our controllers and which is available since MVC version 2.1.

So, after removing extra packages, we should also upgrade Microsoft.AspNetCore.All to the latest stable version. First, we should install the new version of SDK on our development machine (if we still have not). As we have already installed version 2.2, we will just change the Target framework of our application to .NET Core 2.2. For this, right click the project, go to the Properties menu and change the Target framework to 2.2.

Image 24

Then upgrade Microsoft.AspNetCore.All package. In the NuGet package manager choose Microsoft.AspNetCore.All from among installed packages and install new version:

Image 25

If we try to rebuild our solution with new dependencies, it will be built successfully but with the following warning:

warning NETSDK1071: A PackageReference to 'Microsoft.AspNetCore.All' specified a Version of `2.2.6`. Specifying the version of this package is not recommended. For more information, see https://aka.ms/sdkimplicitrefs    

To put it simply, we should remove the explicit version specification of Microsoft.AspNetCore.All in the CSPROJ file. For this, Right click the project and select the Upload Project menu. When unloading is completed, right click the project again and select:

Image 26

Just remove Version="2.2.6" from the PackageReference for the Microsoft.AspNetCore.All. The result should be:

<Project Sdk="Microsoft.NET.Sdk.Web">
…
  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.All" />
    <PackageReference Include="Serilog.Extensions.Logging.File" Version="1.1.0" />
  </ItemGroup>

Reload the project again

Image 27

Note, that after removing explicit version specification we can see Microsoft.AspNetCore.All both under NuGet and SDK sections (and still with its version).

Image 28

But if we rebuild the solution again, it will be built successfully without any warnings. We can start the application and test APIs with the Swagger or any other tool. It works fine.

The Microsoft.AspNetCore.All and Microsoft.AspNetCore.App metapackages

Even in such a small and simple application like ours, we had the beginning of NuGet and Version Hell. We have easily solved these issues by using Microsoft.AspNetCore.All.

Another benefit of using a metapackage is the size of our application. It becomes smaller, because metapackages follow the Shared Framework concept. With the Shared Framework, all the Dll files that make up the metapackage are being installed in a shared folder and can also be used by another applications. In our application, we have just links to Dll in this folder. When we build the application, all these Dll are not being copied into the application’s folder. This means that, to work properly, .NET Core 2.0 (or a higher version) runtime must be installed on a target machine.

When we containerize our application, the benefits of Shared Framework concept are even greater. The metapackage will be a part of the ASP.NET Core Runtime Docker Image. The application image will include only packages that are not parts of the metapackage and, thus, the application image will be smaller and can be deployed faster.

The last wonder to be uncovered - is implicit versioning. Since we have removed the exact metapackage version in the CSPROJ file, our application will work with any version of .NET Core runtime, installed on the target machine, if the runtime has an equal or higher version than the metapackage we have referenced to. This makes it easier to deploy our application in another environment and update .NET Core runtime without needing to rebuild the application.

Note, that implicit versioning works only if our project uses <Project Sdk="Microsoft.NET.Sdk.Web">

Migration from ASP.NET Core 2.2 to 3.0

The code for this article is written with ASP.NET Core 2.2. While preparing the article, a new version 3.0 was released. If you want to examine the code with the ASP.NET Core 3.0 , consider migrating from ASP.NET Core 2.2 to 3.0

Points of Interest

Even after such a significant improvement, our application is still not ready for production. It lacks HTTPS support, auto testing, keeping connection strings safe, etc., etc. These will probably be the focus of forthcoming articles.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Eduard Silantiev
Software Developer (Senior)
Ukraine Ukraine
.NET solution architect and developer since 2003.

C#, .NET Core, RestFul API Web-Service, MSSQL Server, IoT, Azure

Comments and Discussions

 
Questiontools? Pin
kiquenet.com12-Oct-19 0:34
professionalkiquenet.com12-Oct-19 0:34 
PraiseGood job Pin
kruppe8-Oct-19 13:20
memberkruppe8-Oct-19 13:20 
GeneralRe: Good job Pin
Eduard Silantiev9-Oct-19 0:35
memberEduard Silantiev9-Oct-19 0:35 
GeneralMy vote of 5 Pin
Robert_Dyball6-Oct-19 21:34
professionalRobert_Dyball6-Oct-19 21:34 
GeneralRe: My vote of 5 Pin
Eduard Silantiev9-Oct-19 0:38
memberEduard Silantiev9-Oct-19 0:38 
QuestionVery nice article Pin
harrybowles5-Oct-19 1:21
memberharrybowles5-Oct-19 1:21 
AnswerRe: Very nice article Pin
Eduard Silantiev9-Oct-19 0:45
memberEduard Silantiev9-Oct-19 0:45 
QuestionMessage Closed Pin
3-Oct-19 4:12
memberAccflex ERP3-Oct-19 4:12 
QuestionMany thanks for Sharing your knowledge Pin
zaktecs3-Oct-19 3:54
memberzaktecs3-Oct-19 3:54 
AnswerRe: Many thanks for Sharing your knowledge Pin
Eduard Silantiev9-Oct-19 0:38
memberEduard Silantiev9-Oct-19 0:38 
You are welcome

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Article
Posted 1 Oct 2019

Stats

6.9K views
19 bookmarked