Here is the official guideline from Microsoft covering WebAPI and MVC cases for all versions of .NET.
For Web API it suggests redirecting to a dedicated controller end-point to return ProblemDetails. As it may lead to potential exposure in the OpenAPI spec of end-points that aren't meant to be called directly, I'd suggest a simpler solution:
public void Configure(IApplicationBuilder app, IWebHostEnvironment env){ ... app.UseExceptionHandler(a => a.Run(async context => { var error = context.Features.Get<IExceptionHandlerFeature>().Error; var problem = new ProblemDetails { Title = "Critical Error"}; if (error != null) { if (env.IsDevelopment()) { problem.Title = error.Message; problem.Detail = error.StackTrace; } else problem.Detail = error.Message; } await context.Response.WriteAsJsonAsync(problem); })); ...}In this case, we leverage a standard middleware that returns custom details (with a stack trace for dev mode) and avoid creating 'internal' end-points.
P.S. Note that the official guideline relies on IExceptionHandlerPathFeature before .NET v3 and since then (up to v5 as for now) - on IExceptionHandlerFeature.