core monitoring readiness

Monitoring: Readiness Check

Unlike the health-check which can indicate a possible unrecoverable microservice, a readiness check indicates the Microserivce is healthy but should not currently have any requests routed. Container orchestrators such as Kubernetes distinguishes between health and readiness checks since an unhealthy microservice will be recreated after a specific number of negative health checks. If a given microservice needs to perform some dedicated processing or needs to have maintenance completed, the service can report that it is not ready so requests will be routed to other services within the cluster.

The current readiness status of a microservice can be changed by calling the method named ToggleReadyStatus on the ICompositeApp contract. This example will simply expose an API that will delegate to this method to illustrate how the current readiness state can be queried.

Add the following method to the ExampleController located within the Examples.Monitoring.WebApi/Controllers directory:

using Examples.Monitoring.App.Plugin;
using Examples.Monitoring.Infra.Plugin;
using Microsoft.AspNetCore.Mvc;
using NetFusion.Core.Bootstrap.Container;

namespace Examples.Monitoring.WebApi.Controllers;

[ApiController, Route("api/[controller]")]
public class ExamplesController : ControllerBase
{
    private readonly ICompositeApp _compositeApp;
    private readonly IPendingRequestsMonitor _pendingRequests;
    private readonly ICachedItemsMonitor _cachedItems;

    public ExamplesController(
        ICompositeApp compositeApp,
        IPendingRequestsMonitor pendingRequests,
        ICachedItemsMonitor cachedItems)
    {
        _compositeApp = compositeApp;
        _pendingRequests = pendingRequests;
        _cachedItems = cachedItems;
    }

    // ...
    
    [HttpPut("ready-status/toggle")]
    public IActionResult ToggleReadinessState() => Ok(_compositeApp.ToggleReadyStatus());
}

Next, update the service's Program.cs file to expose and API used to query the current readiness status:

var app = builder.Build();

app.UseSerilogRequestLogging();
app.UseRouting();

app.MapHealthCheck();
app.MapStartupCheck();
app.MapReadinessCheck();  // <-- Add this line

app.MapControllers();

Request the current readiness status by running the service and making the following HTTP requests:

curl http://localhost:5007/mgt/ready-check --verbose

Sense there is a 20 second delay in the service module, the service will return an HTTP 503 status code indicating that it is not ready until after the service fully starts. Once started, the service will return a HTTP 200 status code. After the service is started and returns a ready status, execute the following to toggle the current status of the service:

curl -X PUT http://localhost:5007/api/examples/ready-status/toggle
curl http://localhost:5007/mgt/ready-check --verbose
curl -X PUT http://localhost:5007/api/examples/ready-status/toggle
curl http://localhost:5007/mgt/ready-check --verbose