Rick Strahl's Weblog  

Wind, waves, code and everything in between...
.NET • C# • Markdown • WPF • All Things Web
Contact   •   Articles   •   Products   •   Support   •   Advertise
Sponsored by:
West Wind WebSurge - Rest Client and Http Load Testing for Windows

Upgrading my AlbumViewer Sample Application to ASP.NET Core 3.0


:P
On this page:

ASP.NET Core 3.0 shipped yesterday and it's a big release with many new features and a huge batch of performance updates. Although this is a major release that touches a lot of the existing framework code, a lot of the big changes happen under the covers. For the most part this is an incremental update and doesn't require monumental changes to upgrade. Some changes are required to be sure - mainly in the Startup class configuration - but considering the feature changes and enhancements they are pretty tame.

To give an idea of what's involved in updating an application - even if a very small one - I'll describe the steps I went through to upgrade my AlbumViewer Angular ASP.NET Core application from .NET Core 2.2 to 3.0 as I've done for most of the previous major releases.

To give you an idea, here's the application:

AlbumViewer Sample Application

You can find the source code at:

Upgrading .NET Core/ASP.NET Core Versions

Every time an update rolls around there are things that change and it's not immediately obvious what has to be changed. But the good news is that most of the changes are isolated and related to how the application is configured.

Most of the changes in these updates have to do with Startup configuration changes. To help with this, the first thing I do when updating to a new version is to create a separate, new project to use as a template to see how the base application features are supposed to be configured and then match those changes into my existing project. Since this is an ASP.NET Core backend for an Angular application, I use the stock API template as a starting point - pick whatever most closely matches your current application.

Then it's time to compare what the new template generates and what your code does. If you're doing this from scratch be prepared to do a little research on the changed options as often not only does the syntax change, but also the behavior (as you'll see later).

.NET Core First Impressions

There's tons of new stuff in .NET Core 3.0, but the really good news is that most of the new features really are behind the scenes so they don't require massive reworking of existing applications. To be honest in the 3 applications I've done this with to date, the vast majority of changes need to be applied in the configuration of the application, and almost none inside of my own actual application code. Other than a few interface type name changes that can ripple through an application, actual application changes are minimal which means the changes are really isolated in the configuration portion of each application.

The second thing that is very noticable is that performance of .NET Core 3.0 applications seems to be drastically improved. Startup speed feels dramatically faster, as does the build tooling. It seems that the build tooling is much better at taking advantage of incrmental builds so full builds of the entire project are much rarer.

I tend to run projects with dotnet watch run from the command line and the auto-restarts now seem considerably quicker than in 2.x.

.NET Core 3.0 Updates

Ok, so now let's take a look through a few of the changes I had to make to the AlbumViewer application in order to make it work with .NET Core 3.0. All of these are configuration changes that are isolated to Startup.cs, which is a testament to the nice organization of configurability in ASP.NET Core. While configuration can become quite complex in large projects, at least it's easy to find where you need to look for the configuration options.

You can see most of the updates in this commit on GitHub.

EndPoint Routing

One of the new features of ASP.NET Core that was already introduced in 2.2 and has been moved front and center in 3.0 is EndPoint Routing.

This new routing mechanism is global to the ASP.NET hosting infrastructure, rather than directly tied to the ASP.NET MVC framework as it was before. Previously routing was closely tied to MVC controllers, whereas now routing is managed at the host endpoint level.

This addresses a number of common use cases that makes it now possible to access routes as part of the middleware pipeline without having to use MVC specific mechanisms to access routing info. This was very difficult previously, but with EndPoint routing route data is now available as part of the pipeline.

In addition the same top level routing mechanism is used for MVC, SignalR and gRPC and any other framework that requires routing. The all can take advantage of this functionality without having to create their own framework specific routing semantics.

That's a nice enhancement, but it requires a few changes in the way routing and MVC is set up in the application.

In ConfigureServices() and the old code I explicitly used AddMvc() to set up MVC controller handling as well as configuring JSON API options:

old code

services
    .AddMvc(options =>
    {
        // options.Filters.Add(new ApiExceptionFilter());
    })
    .SetCompatibilityVersion(Microsoft.AspNetCore.Mvc.CompatibilityVersion.Version_2_2)
    .AddJsonOptions(opt =>
    {
        var resolver = opt.SerializerSettings.ContractResolver;
        if (resolver != null)
        {
	        var res = resolver as DefaultContractResolver;
	        res.NamingStrategy = null;
        }

        if (HostingEnvironment.IsDevelopment())
            opt.SerializerSettings.Formatting = Newtonsoft.Json.Formatting.Indented;
    });

The updated version uses AddControllers():

new code

services.AddControllers()
     // Use classic JSON 
     .AddNewtonsoftJson(opt =>
     {
         var resolver = opt.SerializerSettings.ContractResolver;
         if (resolver != null)
         {
             var res = resolver as DefaultContractResolver;
             res.NamingStrategy = null;
         }

         if (HostingEnvironment.IsDevelopment())
             opt.SerializerSettings.Formatting = Newtonsoft.Json.Formatting.Indented;
     });

In Configure() method then the hookup for using MVC and Controllers has also changed. Rather than specifying:

old code

app.UseMvcWithDefaultRoute();

you now have to add UseRouting() - which is the endpoint routing feature - plus you set up an endpoint for the MVC application:

new code

app.UseRouting();

// authentication / CORS have to follow .UseRouting! but before endpoints

app.UseEndpoints(endpoints =>
{
    endpoints.MapControllers();
});

Gotcha: Order Matters!

This new routing configuration is a little more finicky and I ran into a big Gotcha here with Authentication due to the order of the middleware components where Authentication/Authorization was not working if the order is not correct.

Very specifically it's important that UseRouting() is applied before you apply behavior that relies on routing such as authentication or authorization that is tied to specific routes on a controller or other endpoint.

The proper order for Routing, Auth/CORS and Endpoint mapping is:

// First
app.UseRouting();

app.UseCors("CorsPolicy");

app.UseAuthentication();
app.UseAuthorization();

app.UseStatusCodePages();
app.UseDefaultFiles(); 
app.UseStaticFiles();

// put last so header configs like CORS or Cookies etc can fire
app.UseEndpoints(endpoints =>
{
  endpoints.MapControllers();
});

JSON Changes

As you can see in the code above JSON configuration is also changing. The JSON configuration used above is for controller JsonResult returns.

ASP.NET Core 3.0 by default now uses a new set of high peroformance, built-in JSON parsing support classes including a new JSON parser and serializer that is higher performance and less featured than the old JSON.NET parser. Microsoft decided that such a core component should be part of the core framework rather than requiring a separate library that has to be added to each project.

The new parser is very fast and memory efficient as it is built from scratch using the new low level memory features like Span<T> and Memory<T> to take maximum advantage of these new features at the framework level.

If you're using basic JSON results and always receive return strongly typed results then the new Json parser features work great and give you an extra performance boost.

However, if you also return things like dynamic data, anonymous types or using the LINQ to JSON features for JSON.NET in the past, then you likely have to stick with the JSON.NET parser. I for one frequently return anonymous objects to clients in order to combine multiple resultsets into a single result object and for that I need to use the old JSON parser.

To do this you have to add an extra assembly now, as JSON.NET is no longer included in the framework dependency list by default:

<PackageReference Include="Microsoft.AspNetCore.Mvc.NewtonsoftJson" Version="3.0.0" />

And then explicitly configure it as part of the AddControllers() definition:

services.AddControllers()
    // Use classic JSON 
    .AddNewtonsoftJson(opt =>
    {
        var resolver = opt.SerializerSettings.ContractResolver;
        if (resolver != null)
        {
            var res = resolver as DefaultContractResolver;
            res.NamingStrategy = null;
        }

        if (HostingEnvironment.IsDevelopment())
            opt.SerializerSettings.Formatting = Newtonsoft.Json.Formatting.Indented;
    });

Note the additional configuration is optional, but in this particular application I needed to make sure I use PascalCase naming (the default had changed in .NET Core 2.0 to camelCase) in order to keep the original application's JSON client naming requirement intact. The default for .NET Core 2.x and later has been to use CamelCase and the code above removes the CamelCase contract resolver that's used by default.

Similar options are available for the new JSON parser, but the configuration syntax is somewhat different.

CORS Changes

I also ran into issues with CORS - specifically in HTTP DELETE requests, not working. In the AlbumViewer when I delete an Artist I get this HTTP response:

Even though AllowAnyMethod() is set on the CORS configuration it looks like DELETE is not working.

After a bit of experimenting (and remembering something similar with AllowAnyOrigin() in 2.2) I had to explicited specify the supported methods using WithMethods():

services.AddCors(options =>
{
    options.AddPolicy("CorsPolicy",
        builder => builder
            //.AllowAnyOrigin() // doesn't work
            .SetIsOriginAllowed(s=> true)
            //.AllowAnyMethod()  // doesn't work for DELETE!
            .WithMethods("GET","POST","DELETE","OPTIONS","PUT")
            .AllowAnyHeader()
            .AllowCredentials()
        );
});

And that works.

It would be nice if AllowAnyOrigin() and AllowAnyMethod() don't support any Origin and any Method, they should be obsoleted or at the very least there should be some information in the help text to specify how it doesn't work - neither of those worked as what their naming advertises.

It also looks like you're supposed to configure CORS slightly different in 3.0.

In 2.2 I used the above Policy definition in ConfigureServices() and then apply the policy globally.

old

 app.UseCors("CorsPolicy");

In 3.0 it looks like you're supposed to explicitly attach the policy to an endpoint. You still need to specify app.UseCors() (without a policy just to enable it), and then add it to the controller mapping:

new (but doesn't work!)

app.UseCors(); // enable CORS

...

app.UseEndpoints(endpoints =>
{
    endpoints.MapControllers()
        .RequireCors("CorsPolicy");   // set a specific policy
});

However I had no luck getting that to work. Using the above combination netted me no CORS headers when accessing the app from a cross-domain (Angular Dev Server) site.

I had to go back to the original app.UseCors("CorsPolicy") to get CORS work…

This is a bit frustrating. The CORS documentation is very complex mainly because there are a number of combinations, and the fact that there seems to be some duplication and non-functionality of features seems odd.,

IHostingEnvironment to IWebHostEnvironment

Another minor modification due to the endpoint routing changes is that the Startup class now by default injects an IWebHostEnvironment instead of a more generic IHostEnvironment. The naming reflects Web specific features as well as some new functionality that's more specific to Web applications.

The change is a search and replace for IHostingEnvironment to IWebHostingEnvironment. This affects the startup class and any other places where you might be injecting the hosting environment. In my AlbumViewer I use the Hosting Environment to extract some status information about the application to be displayed in the about page as well as finding the Content and Web roots for the application for reading the intial album data file for import.

Entity Framework 3.0 Fail

The final migration bits are related to Entity Framework and here I ran into some serious issues that I couldn't resolve. In fact I ended up going back to EF Core 2.2 because EF Core 3.0 just didn't work correctly with the data I was working with even for a really simple use case.

The very specific scenario that was giving me trouble had to do with a simple list query that returns Albums and artists as a list of nested objects. Specifically in .NET Core 3.0 the list would fail to return the Artist record for any album where there are multiple albums for an Artist. First albumn for the artist shows, any subsequent Artist for albums by the same artist would return empty objects.

This is a pretty basic query over data:

public async Task<List<Album>> GetAllAlbums(int page = 0, int pageSize = 15)
{
    IQueryable<Album> albums = Context.Albums
        .Include(ctx => ctx.Tracks)
        .Include(ctx => ctx.Artist)
        .OrderBy(alb => alb.Title);

    if (page > 0)
    {
        albums = albums
                        .Skip((page - 1) * pageSize)
                        .Take(pageSize);
    }

    return await albums.ToListAsync();
}

Yet it fails. I couldn't find a workaround and an issue on GitHub is pending for this.

There were also changes in the way that child entities are handled in one to many relationships - whereas in previous versions you didn't have to explicitly have to Context.Add() new or Context.Update() existing entities when you added them to a loaded parent property collection, you now always have to explicit add, update and remove entities. This is perhaps more consistent as there can be scenarios where your entity may not have been loaded from the DB yet, and so adding wouldn't do anything, but still this is a breaking change that is likely to affect a lot of people because it worked previously. In my scenario I didn't have any places where this was an issue but in several other applications I did. This can be problematic because you won't see those breaks show up until runtime.

Roll back to Entity Framework 2.2

The first problem above was a show stopper for me. I couldn't get past that issue, so I ended up rolling back to EF Core 2.2 which just worked properly returning the result set with its children as expected.

The good news is that EF Core 2.2.6 works just fine in .NET Core 3.0 applications - I didn't see any side effects due to old dependencies and existing functionality worked just fine with the 2.2.6 libraries. For now and for me at least I can't justify dealing with the issues in EF Core 3.0.

This is obvious not ideal, but until EF Core 3.0 can iron out some of these issues using 2.2 is the only way I can move forward for the moment.

EF Core Problems Continue

It's disheartening to see where EF Core is headed. This and previous releases removed features, changed a number of core behaviors, and apparently has a number of show stopper issues that as far as I can tell don't have a workaround. While I'm sure these initial release issues will get addressed quickly in patch releases, I do wonder how a bug like the child entity loading ever made it into production - this has to be a very common scenario that just about every application uses.

Database applications are at the core of the vast majority of applications that are built today, and having a solid data stack is a vital component of any application stack. EF Core seems barely able to fill that role. While there are better alternatives, they are not an easy sell with many customers due to the perception that EF comes from Microsoft and so that has to be the good enough solution. Only it shouldn't be the good enough data solution, it should be as kick ass as the rest of the .NET stack. It is not - it's been a real struggle to work with EF Core since the beginning and it's not really getting better as new versions are coming out.

I realize that it's difficult to build a reliable, consistent and performant data access layer/platform, and that it takes time to get that right. But at the same time the EF Core team has had the benefit of the doubt for 3 major versions (with several major sub-versions) to build out a mature, stable and reliable platform. Instead EF Core still feels like a V1 product with all the inherent inconsistencies and behavior and API changes. It's one thing if you see this from a vendor or third party but quite another when it's the ‘official data platform’ for the entire .NET platform.

Summary

Overall the upgrade process for ASP.NET Core 3.0 was pretty painless. The bulk of it - sans the debugging of the EF Core data issue and the CORS config issue - took around 20 minutes. The data issue took a while to track down and then some additional time going back to EF Core 2.2 and re-testing. Plus time to report and follow up on it with Microsoft. But even so the process is relatively minor as there are just a few places that need to be updated.

At the end of the day make sure you double check each of the Startup class configuration section for things like Authentication, CORS, Routing, Logging and see if the syntax or behavior has changed.

Once that's done the rest is very likely to just work. And be noticably faster.

this post created and published with Markdown Monster
Posted in ASP.NET  

The Voices of Reason


 

Jens
September 24, 2019

# re: Upgrading my AlbumViewer Sample Application to ASP.NET Core 3.0

It is unbelievable that they put out a good enough ORM with Linq2SQL, and then abandoned that in favour of EF. We' still use L2S in around 95% of our application (stored procedures and ADO.Net code make up for the other 5% because we couldn't get the needed speed out of L2S). This regrettably keeps us from moving to .Net Core as there is as far as I know no migration path from L2S to EF (any version). So even if EF Core was good enough, without a migration path I'm not sure that we'll ever want to do the effort of migrating.


JB
September 25, 2019

# re: Upgrading my AlbumViewer Sample Application to ASP.NET Core 3.0

somebody in the EF Core team is gunna be in a whole-lotta-trouble...


Lee
September 25, 2019

# re: Upgrading my AlbumViewer Sample Application to ASP.NET Core 3.0

Just go with NHibernate. Entity Framework wrote all this magic to fall back to querying in memory when it could not be translated to SQL. It sounds great but without profiling the SQL, you never knew what limitations it hit. You now have to call ToList() or AsEnumerable() to force the query to be executed and then you can add any further LINQ expressions against the data in memory.

This is exactly what I've been doing in NHibernate for years and I can't believe they wasted time building such a stupid feature. This unforunately is just one example of a feature they have wasted time on when there's plenty of things people need they have tip toed around or taken ages to implement.


Chris
September 26, 2019

# re: Upgrading my AlbumViewer Sample Application to ASP.NET Core 3.0

Really useful post as always Rick. We find it difficult to comprehend why Microsoft seem to be struggling with Entity Framework 3.0. We are also still using Linq-to-Sql in all our applications. Trying to convince the developers here to ditch this proven ORM in favour of Entity Framework 3.0 that is not yet "production ready" is a non-starter. Going further, it would be irresponsible of us to write a business application for a Client using this technology given these concerns with Entity Framework 3.0. Surely the fact that Microsoft have also updated Entity Framework 6 to run on .NET Core so we effectively have two ORMs to choose from seems to be an admission from Microsoft that Entity Framework 3.0 is not yet up to scratch ?


Allen Newton
September 26, 2019

# re: Upgrading my AlbumViewer Sample Application to ASP.NET Core 3.0

Great post. I ran into these same change requirements in a 2.2 web app. I tried the conversion from 2.2 to 3-preview a couple of weeks ago. Which was good practice but there were still some minor changes like the UseRouting location that I either missed when working with the 3-preview or it was somehow new the 3.0 released (no matter). What I wanted to say is that I also have .Include in use and after the upgrade to EF 3.0, it seems to be working fine. I might run into this bug later because it is going to take specific integration testing to ferret it out. It's a runtime bug that requires resolving data to see it. But you might try one thing. You might try using the scaffolding tool to see how your models and DbContext are generated. There is a new tool for v3 that you need to install manually. I took a quick look at your source code and I don't see the same patterns that the scaffolding tool would generate. For example, the IList property might be generated as public virtual ICollection Tracks {get;set;} instead. There is also an instantiation in the construction of Albums for this property that would be a new HashSet. Anyway, if you have not already tried this, it might be worth a look. And thanks for pointing out this bug in EF Core 3. I'll be wary of it as I write more integration tests for my project.


Jon
September 27, 2019

# re: Upgrading my AlbumViewer Sample Application to ASP.NET Core 3.0

I always liked Rick. Now, I see that in addition to being a great developer, he also has good taste in music. I thought I was the only one who knew of and still listened to Attitude Adjustment, American Paranoia. Great album to code to.


James Hancock
May 04, 2020

# re: Upgrading my AlbumViewer Sample Application to ASP.NET Core 3.0

Now we're at .NET Core 3.1 and EF Core 3 is still AWEFUL. Just horrific, with multiple runtime breaks that make upgrading disasterous at best.

3 Rewrites of EF Core, and it still hasn't caught up with EF 6.3 and is barely faster. I mean single digits faster in almost all cases.

And we're still missing patch (from x in y where x.Id = 5 update { field1, field2, field 3}), delete with where (i.e. from x in y where x.z = 5 delete should be a thing a decade or more ago), a sane left join syntax (why leftjoin wasn't added long ago to do this I don't know)

And the In Memory driver has different bugs than the SQL Driver by a lot to the point where most complex queries won't work on both, so you can't do proper integration tests of your code that is even remotely reflective of what you'll get in SQL and you can't even share queries between the two often. (in memory has a bug where it evaluates the back side of ? : notations even if the first option was true.)

The first 3 pages of bugs on the github repository are all show stoppers, and there are hundreds of these. Meanwhile the level of arrogance and dismissiveness of the EF team on github should embarrass Microsoft, but no, they literally are bragging about runtime breaks on Twitter and Damian Edwards et. al. are piling on!

And then there's the "we do null checking for you so you don't have to worry about it." That's true as long as it isn't doing a final in memory processing. Anything that executes in SQL Server handles nulls. Anything that executes in .NET Core doesn't anymore. So you get null reference exceptions randomly with no indication of the field that it has a null reference on (it has to be possible to pass the field that you couldn't get properties on!) EF 6.3 handles this just fine thanks.

Meanwhile the EF team won't fix show stopper bugs between major versions apparently unless some large multi-national pays them to put the fix in and EF Core 5.0 has different bugs, and is no better. (And the promised useful error messages are still useless with NullReferenceException errors on property bindings still not telling you what field it failed on.

EF Core is the canary in the coal mine for .NET and the statistics about use prove it. Someone on the .NET leadership team needs to take a hint or .NET is DONE and EF Core was just the warning that it was going to happen. (and I'm not the only one complaining about the elitism and exclusionary behavior of the .NET Core leadership, it's all over twitter in the past few months.)


West Wind  © Rick Strahl, West Wind Technologies, 2005 - 2024