Weirdness with EF Core 2 (SQLite), .NET Standard, and .NET Framework

I've run into some issues referencing .NET Standard projects from a .NET Framework project -- specifically in regard to Entity Framework Core 2 and SQLite. I encountered this when moving some of my demo code to newer projects.

Here's the error that I get:

This is a missing DLL. I've managed to come up with 2 workarounds -- neither of which I like -- but they get the projects working.

Let's take a look at the project.

This article is technically *not* part of the More DI series of articles. However, the issue is present in the code described in that series. The code is available on GitHub: https://github.com/jeremybytes/di-decorators.

The Projects
The solution consists of 12 projects which are a mix of project types. Here are the full-framework projects:

The PeopleViewer project is a WPF application using .NET Framework 4.7.2.

The test projects are also .NET Framework due to current requirements of NUnit 3.x.

All of the "library" projects are .NET Standard 2.0:

The .NET Standard projects include the Presentation (which includes the view model), the Readers (including the CSV data reader, the SQL data reader, the Service data reader, and all of the decorators), and the Shared projects.

There is one .NET Core project:

This is the WebAPI service that provides data for the Service data reader.

Project References
Project references can get a bit weird when using .NET Standard with .NET Framework. When I was first having issues, I came across Scott Hanselman's article: Referencing .NET Standard Assemblies from both .NET Core and .NET Framework, and this helped. Let's see how.

Here are the dependencies from the "PersonReader.SQL" project (listed in the PersonReader.SQL.csproj file):

We can see the references to Entity Framework Core and SQLite.

The "PeopleViewer" project does not reference either of these NuGet packages directly:

Based on the advice provided in Scott Hanselman's article, I manually edited the "PeopleViewer.csproj" file to add a "RestoreProjectStyle" value of "PackageReference":

This took care of some of my issues: the ServiceReader works. But the SQLReader is still broken.

The Service Data Reader Works
The Service data reader project has a reference to NewtonSoft.Json (which is probably not a surprise):

By using the "PackageReference" setting, the correct version of the Newtonsoft.Json DLLs made it to the output folder of the "PeopleViewer" project.

You can look at the article "Adding Retry with the Decorator Pattern" for more information on running this code.

Be sure to read Scott Hanselman's article for the details on this setting and why it's needed.

The SQL Data Reader is Broken
But even with this setting, the SQL Data Reader is still broken. Let's take a closer look.

First, we'll update the "ComposeObjects" method to use the SQLReader object. This is in the "PeopleViewer" project, App.xaml.cs file -- although the code in the GitHub project is more complex than what we have here since it uses all of the decorators.

This builds a MainWindow that uses a PeopleReaderViewModel that uses a SQLReader.

When we run the application and click the "Refresh People" button, we get the error:

This indicates that we're missing a DLL that's needed for SQLite functionality.

If we check the output folder, we see that we have many Entity Framework DLLs, including one that references SQLite:

But "e_sqlite3.dll" is nowhere to be found.

As a side note, I tried this with the most recent version of EF Core/SQLite (version 2.2.1 at the time of writing) and had the same results.

I managed to work around this in 2 different ways. If there's a better solution, *PLEASE* let me know about it in the comments. I do not like either of these options.

Workaround #1: Include the NuGet Package in the .NET Framework Project
The first workaround I found was to include the Entity Framework Core / SQLite NuGet packages in the "PeopleViewer" project (the WPF application).

This works because the application now has all of the DLLs that it needs:

The "e_sqlite3.dll" is in the "x64" folder (and the "x86" folder as well).

Why I Don't Like This
I don't like this solution because it creates a dependency on a NuGet package that the .NET Framework project (the WPF application) does not explicitly need. The .NET Standard library (the SQL data reader) is the project that needs the package in order to work. This solution seems to mess up the dependency graph.

It can get a bit worse. If the version of the NuGet package referenced by the .NET Framework project is different from the version referenced in the .NET Standard project, there will be runtime errors. (I've run into this with Newtonsoft.Json references.)

Workaround #2: Copying the Needed Files to the Output Folder
The second workaround I came up with is to copy the missing files into the output folder. Since the bulk of the DLLs are making it to the output folder, I figured that just including the "e_sqlite3.dll" that's missing might be the best way to go.

To do this, I grabbed the "x64" and "x86" folders above, and put them into an "AdditionalFiles" folder at the root of the solution.

This folder already exists: it has the text file used by the CSV data reader (People.txt) as well as the SQLite database used by the SQL data reader (People.db). So adding a couple more files doesn't seem like too much of a problem.

The files are copied to the output folder using a post-build event on the "PeopleViewer" project:

This copies the contents of the "AdditionalFiles" folder to the output folder. For more information on Visual Studio build events, refer to "Using Build Events in Visual Studio to Make Life Easier".

The other thing I had to do was explicitly add these files to the Git repository. The .gitignore file doesn't usually include DLLs, so I had to add these two files manually. You can see them in the GitHub repository: https://github.com/jeremybytes/di-decorators/tree/master/AdditionalFiles/x64.

This has the same effect as Workaround #1: the files are added to the output folder:

Why I Don't Like This
The reason I don't like this workaround is the same as above: There's a good chance that the versions will get out of sync in the future. In addition, I don't like the idea of adding executable files to the Git repository.

Frustration
There is a bug somewhere. I'm not sure if it's a bug in the Entity Framework Core packages (specifically the SQLite package). I'm not sure if it's a conflict between the package management schemes in the .NET Standard project vs. the .NET Framework project. And I have to admit that this is a pretty big frustration for me.

The frustration is 2-fold -- first for myself. I've been coding in the .NET framework for close to 15 years. I really would not expect to have these types of difficulties, even with things that are relatively new. But as a developer, I deal with this type of issue all the time.

The bigger frustration is when I think about other developers. Many developers in the .NET space do not have the type of experience that I have. If I am having difficulty, then that means that there are a large number of other developers who will also have difficulty.

This is even a bigger concern since the bulk of my effort is spent helping developers learn more about C# and .NET -- in particular, I work with the "dark matter developers" described by Scott Hanselman . I have a fear that if we make things too complex in the environment, we will have difficulty getting new developers on board.

This is just one example of what is causing me concerns. The other items are a topic for another day.

Wrap Up
If you have any advice on handling this issue, please let me know. I would file a bug report, but as noted, I'm not exactly sure where the bug is.

If you have run into this same issue, understand that I do not like either of these workarounds. They add concerns about versioning. But they do get the code working, and that's always the first step.

Happy Coding!

More DI: The Real Power of Decorators -- Stacking Functionality

Over the last several articles, we have looked at a decorator to add retry functionality, a decorator to log exceptions, and a decorator to add a client-side cache. We've seen that decorators let us easily add functionality without creating subclasses and without modifying our existing objects.

But the real power of decorators is that we can stack the functionality one piece at a time. (Okay, so there are several compelling reasons to use the decorator pattern even if we are not stacking functionality; but the stacking is really cool.)

Small pieces of functionality are easier to test. By stacking decorators, we can mix and match the pieces of functionality that we need.

This article is one of a series about dependency injection. The articles can be found here: More DI and the code is available on GitHub: https://github.com/jeremybytes/di-decorators.

The Data Reader Decorators
The decorators in the application operate on the "IPersonReader" interface (from the "Common" project, IPersonReader.cs file):

All of the decorators wrap the interface "IPersonReader" and also implement the interface "IPersonReader". This means that the decorators can wrap other decorators.

Let's see how this works step-by-step.

Behavior Without Decorators
For this article, we'll modify the "ComposeObjects" method in the startup of our application. This is in the "PeopleViewer" project, App.xaml.cs file. We'll start with just the behavior without decorators: using the web service data reader.

As a reminder, the code shown here is different from the final code in the GitHub project. If you'd like to follow along, just replace the code in the "ComposeObjects" method with what we have here.

This method puts together 3 objects. Starting from the last line, (1) we create a MainWindow instance; this is the main form of the application. The MainWindow needs a view model, so (2) we create a PeopleReaderViewModel instance to pass to the MainWindow. The view model needs an IPersonReader, so (3) we create a ServiceReader instance (to get data from a web service) and pass that to the view model.

A Nested Composition
As an alternative to this syntax, we can also use a "nested" syntax that gets rid of the intermediate variables. This is in the "AlternateComposeObjects" method in the same file.

This is a little easier to read since we can see what is nested in what. The "ServiceReader" is nested in the "PeopleReaderViewModel" which is nested in the "MainWindow".

I'll show both of these syntaxes along the way. They both have their pros and cons when it comes to readability.

Running the Application
As a reminder, to run the application we need to start the web service. This is in the "People.Service" project. The easiest way to start the service is to navigate a command window to the project folder, and type "dotnet run".

This shows the service listening on "http://localhost:9874", and you can navigate to "http://localhost:9874/api/people" to see the service in action.

Then we can run the application in Visual Studio. When we click the "Refresh People" button, we get data back.

Now we'll go back to the command window and stop the service with "Ctrl-C".

With the service stopped, click "Clear Data" and "Refresh People". Then we see the following error:

As a reminder, if the code stops in the debugger, just click "Continue" or press "F5" to run past the exception.

Now let's see what happens when we start adding functionality.

Adding a Retry Decorator
The first step is to add the decorator to retry the call if it fails. This described here: More DI: Adding Retry with the Decorator Pattern.

Here is the updated "ComposeObjects" method:

In this code, we wrap the "ServiceReader" in a "RetryReader". The RetryReader also needs a delay, so we pass in a TimeSpan of 3 seconds. When a call fails, the RetryReader will wait 3 seconds and try again, for a total of 3 attempts. After 3 attempts, it will return the exception from the wrapped data reader.

The nested syntax shows the relationships a bit more clearly:

We can see that the RetryReader wraps the ServiceReader.

Now if we run the application and click "Refresh People" (without re-starting the service), we wait 6 seconds before the failure (due to the delay and retry). If we restart the service after clicking the button, then the retry will work, and the application shows the data. See the original article for more details.

Stacking Decorators
To see the how we can stack functionality with decorators, we will add the exception logging decorator to our composition. We do this by wrapping the "RetryReader" in the "ExceptionLoggingReader":

Here's the nested syntax:

I'm a bit torn between these syntaxes, which is why I am showing them together. The nested syntax shows the relationships very easily. Here we can see that the ServiceReader is wrapped in the RetryReader which is wrapped in the ExceptionLoggingReader.

But some of the other parameters are a bit more difficult to follow. If we use the original syntax, it is easier to tell that the "TimeSpan" is the retry delay since we've given that intermediate variable a name. I experimented with mixing the syntaxes, but did not like the results.

Logging Behavior
If we run the application again (without restarting the service), we get the same behavior as above. But now there is a log file in the executable folder: ExceptionLog.txt. For more details on the ExceptionLoggingDecorator, see the original article on that decorator.

Here is the flow:

• The view model calls the ExceptionLoggingReader.
• The ExceptionLoggingReader calls the RetryReader.
• The RetryReader calls the ServiceReader.
• The ServiceReader throws an exception.
• The RetryReader retries the call. 
• The ServiceReader throws an exception (again).
• The RetryReader retries the call.
• The ServiceReader throws the exception (again).
• The RetryReader rethrows the exception.
• The ExceptionLoggingReader takes the exception and logs it.
• The ExceptionLoggingReader rethrows the exception.
• The exception bubbles up through the view model and eventually hits the application's global exception handler (that's where the pop-up comes from).

Here is the log file:

The log only has 1 exception for this process. If we look in the exception log, we see the "RetryReader.<GetPeople>" call. In fact, there are 3 calls in the call stack. This is because our RetryReader uses a recursive call for the retry functionality (see the original article for details).

The result is that when we assemble our objects this way, we get the functionality of *both* the retry decorator *and* the exception logging decorator.

But notice that we only have 1 exception logged. That's probably what we want, but we can change that by stacking things in a different order.

Stacking Order Matters
Something that we need to be aware of is that the order that we stack our decorators is important. Let's flip the order of our decorators. Instead of the ExceptionLoggingReader wrapping the RetryReader, let's have the RetryReader wrap the ExceptionLoggingReader.

Here's the code:

And the nested syntax:

The application behaves differently now.

• The view model calls the RetryReader.
• The RetryReader calls the ExceptionLoggingReader.
• The Exception LoggingReader calls the ServiceReader.
• The ServiceReader throws an exception.
• The ExceptionLoggingReader logs the exception and rethrows it.
• The RetryReader retries the call to the ExceptionLoggingReader. 
• The ExceptionLoggingReader calls the ServiceReader.
• The ServiceReader throws an exception (again).
• The ExceptionLoggingReader logs the exception and rethrows it.
• The RetryReader tries the call to the ExceptionLoggingReader.
• The ExceptionLoggingReader calls the ServiceReader.
• The ServiceReader throws an exception (again).
• The ExceptionLoggingReader logs the exception and rethrows it.
• The RetryReader rethrows the exception.
• The exception bubbles up through the view model and eventually hits the application's global exception handler.

We can see the difference by looking at the ExceptionLog.txt file:

Here we can see the first exception logged. Then 3 seconds later (approximately), the second exception is logged. Further down in the file, the third exception is logged.

So we do need to be aware of the order we use when we stack decorators.

Stacking All of the Decorators
We can also stack all of the decorators. The final step will add a client-side cache to the application. For details, see the original article on the caching decorator. This is shown in the final App.xaml.cs file in the GitHub project:

And the nested syntax:

This takes the ServiceReader, wraps it in a RetryReader, wraps that in an ExceptionLoggingReader, and wraps that in a CachingReader. The CachingReader is given to the PeopleReaderViewModel. This gives us all 3 pieces of functionality. (For details on caching, see the original article on the caching decorator.)

Decorators and Transparent Functionality
The view model only cares that it gets an "IPersonReader". The view model itself does not need to know anything about the decorators or if they wrap other decorators. All the view model cares about is that there is a "GetPeople" method that returns data; the details don't matter. That's the magic of using the decorator pattern with dependency injection.

Decorators and Unit Testing
Each decorator has a single function. This makes them easy to unit test. In our tests (retry tests, exception logging tests, and caching tests) we are able to focus on one function at a time. So our tests can be straightforward and clear.

For this project, we can also add integration tests to see how the decorators behave together in different combinations (and that's something I'll be adding in the future).

Wrap Up
Dependency Injection and the Decorator Pattern are a powerful combination. We can mix-and-match pieces of functionality depending on our specific needs. Each piece of functionality is isolated so that tests can be focused. And we can get these benefits without modifying our existing classes. We just snap the pieces together in a different order.

Coming up, we'll look at some other aspects of this project, including how .NET Standard projects interact with .NET Framework projects. We'll also look at using dependency injection containers to compose our objects and manage the lifetime. And we'll look at a more general-purpose interface that allows us to use the decorators with a variety of underlying objects.

Happy Coding!

More DI: Adding a Client-Side Cache with the Decorator Pattern

The Decorator Pattern lets us add functionality without changing existing classes. Previously, we looked at adding retry functionality and exception logging to a data reader by using decorators. This time, we'll add a client-side cache to our data reader. (For more details on how the Decorator Pattern works, see the prior article More DI: Adding Retry with the Decorator Pattern.)

Network calls, whether to a web service or a database, are often the most slowest parts of our application due latency and bandwidth limitations. To minimize these types of calls, we want to add a client-side cache to the application. This cache will provide data for a specified amount of time and then hit the real data reader when it's time to refresh the cache. This will keep the network calls to a minimum.

The Decorator Pattern lets us add functionality without changing existing classes.

This article is one of a series about dependency injection. The articles can be found here: More DI and the code is available on GitHub: https://github.com/jeremybytes/di-decorators.

Behavior Without a Cache
Before adding the cache, let's see how the application behaves. For this, we'll remove the previous decorators and go back to the base state.

Composing the Objects
Here's the composition root (from the "PeopleViewer" project, App.xaml.cs file):

The code here is much different from what is in the GitHub project. The GitHub project uses all of the decorators, and we'll look at that in a later article. To follow along with this article, you can replace the code in the "ComposeObjects" method with what we have here.

As a reminder, the "ComposeObjects" method has 3 steps. (1) It creates a ServiceReader; this is a data reader that gets data from a web service. (2) It uses the ServiceReader as a parameter to create a PeopleReaderViewModel; this view model has the presentation logic for the form. (3) It uses the view model as a parameter to create the MainWindow; this is the view that has the UI elements for the form.

Running the Application (Success)
We'll start by running the application. Since we are using the ServiceReader, we also need to start the web service. This is done from the command line with "dotnet run". (See this article for more information on how I quickly open a command prompt to the correct folder.)

Once the service is running, we'll start the application and click the "Refresh People" button.

This gives us the data from the service.

Running the Application (Failure)
Now let's stop the service and try again. To stop the service, just press "Ctrl-C" in the command window.

Then in the application, click "Clear Data" then "Refresh People". The call will fail since the service is no longer running. When the debugger stops in the code, just click "Continue" or "F5" to keep running. The application will give us the generic error message.

This shows that each time we click the "Refresh People" button, the application will make a call to the web service. This is the behavior that we'd like to change with the client-side cache.

A Client-Side Cache
To add the cache, we will decorate the data reader with caching functionality. To do this, we wrap a current data reader, add the cache, and then expose the same interface to the outside world. (For more details on using the Decorator pattern, see More DI: Adding Retry with the Decorator Pattern.)

We need to wrap and implement the IPersonReader interface (from the "Common" project, IPersonReader.cs file):

Here is the code for the CachingReader (from the "PersonReader.Decorators" project, CachingReader.cs file):

The first thing to note is that the CachingReader implements the "IPersonReader" interface, so it has the 2 methods "GetPeople" and "GetPerson".

At the top of the class, we have a field for the data reader that will be wrapped ("_wrappedReader"). This is set from an incoming constructor parameter. In addition, the constructor has a TimeSpan parameter for the duration of our client-side cache. This is also stored in a private field ("_cacheDuration").

The other 2 fields manage the cache. The "_cachedItems" field holds the data. The "_dataDateTime" is the time the data was last refreshed. This is used to see if the cache has expired.

The "GetPeople" method validates the cache and then returns the cached items.

Validating the Cache
The "ValidateCache" method ensures that we have current data. Here's the code for that:

The "if" statement checks to see if the cache is current. (We'll look at "IsCacheValid" in just a moment.) If the cache is populated and valid, then the method returns immediately, and the current cache is used.

If the cache is not populated or if it is expired, then the "GetPeople" method is called on the wrapped data reader. The items returned are put into the cache and the data date/time is updated.

If the call to "GetPeople" fails for some reason, then the CachingReader will return a default record that simply states "No Data Available". This will give us a value that we can display in the UI (and this works for this specific application). Another option would be to let the exception bubble up.

After setting the default record, a call to "InvalidateCache" ensures that the data will be refreshed with the next call. (We'll see "InvalidateCache" in just a moment.)

Checking for an Expired Cache
The "IsCacheValid" property will check to see if the cache is populated and not expired. Here's the code for that:

First, this checks to see if the cached items field is populated. If it is null, then it returns "false" immediately.

If the cache is populated, then we calculate the age of the cache by getting the difference between the data date/time and the current time. Then we check to see if the difference is less than the cache duration value.

This returns "true" if the cache is still valid.

Invalidating the Cache
We may want to invalidate the cache manually. For this, we have the "InvalidateCache" method:

This sets the data date/time to the minimum value for DateTime. This will make sure that the cache will be refreshed on the next call (unless the cache duration value is set to a *very* large value).

The implementation of this cache is fairly naive. This assumes that the data coming back from the "GetPeople" method is small enough to be stored in memory. I've used this implementation for small data sets for a while now. But it would need to be beefed up for larger data sets.

Using the Caching Decorator
Just as we did with the retry decorator and the exception logging decorator, to use the caching decorator, we snap the pieces together in a different order.

Here is the code with the CachingReader added, from the "PeopleViewer" project, App.xaml.cs file. (Reminder, this code is different from what is in the final GitHub project.)

This adds the CachingReader to the object composition. Let's walk the method from the bottom up. (1) We create a MainWindow instance. This needs an "IPeopleViewModel" as a parameter. (2) So above that, we create a PeopleReaderViewModel to pass to the MainWindow. The PeopleReaderViewModel needs an "IPersonReader". (3) So above that we create the CachingReader to pass to the view model. The CachingReader has 2 parameters: an "IPersonReader" to wrap and a TimeSpan for the duration. (4) So above that we create a ServiceReader to get data from a web service and also create a TimeSpan of 10 seconds.

Running the Application (Success)
So let's see the cache in action. First, start the web service like we did above:

Then run the application and click "Refresh People". This gives us the expected data:

Now we'll check the cache. Keep the application running for the next steps.

Using the Cache
With the application still running, stop the web service using "Ctrl-C" as we did above.

Now click the "Clear Data" and "Refresh People" buttons in the application.

We still have data!

Now if we continue to click "Clear Data" and "Refresh Data", the cache will eventually expire. After the cache is expired, the CachingReader tries to get data from the service (which is no longer running). That call fails, so the CachingReader returns the default record:

This shows the client-side cache in action. Even though the web service is no longer running, we can still get data from the client-side cache until the data gets too old.

Note: if the cache is expiring too quickly to see these results, just set the "duration" to a larger value in the "ComposeObjects" method above.

This code works with the CSVReader and SQLReader as well. I use the ServiceReader here since it is the easiest to demonstrate by stopping the service.

Unit Testing the Caching Decorator
Just like with the other decorators, we want to unit test the caching decorator. The process is very similar to the previous tests, so we'll go pretty quickly through this. For more details, see More DI: Unit Testing Async Methods which shows the tests for the retry decorator.

These tests are in the "PersonReader.Decorator.Tests" project, CachingReaderTests.cs file.

Testing That the Cache is *Not* Used
Our first test will check to make sure that the cache is not used on the first call to the "GetPeople" method. Here's the code for that:

Just like with our other decorators, we want to use a fake data reader for testing. In this case, we have a "CountedReader" that keeps track of how many times the "GetPeople" method is called.

Let's look at the fake reader and then come back to this test.

Counting Method Calls
The fake data reader is in the same testing project, CountedReader.cs file:

The CountedReader implements "IPersonReader" and so has the "GetPeople" and "GetPerson" methods. The class has a public property to keep track of how many times the methods are called. This property is public, so it is available to the tests.

The "GetPeople" method increments the call count and then returns an empty collection of Person objects. The "GetPerson" method does something similar. We don't need any fake data here, so the empty results are fine.

Back to the Test
Let's go back to the first test.

Here, we create a CountedReader and a duration (a TimeSpan of 1 second). Then we use these values to create the CachingReader.

Next, we call the "GetPeople" method.

Finally, we assert that the CallCount on the CountedReader is 1. This lets us know that the wrapped data reader was called one time for the test. And this is exactly what we expect.

Testing the Cache
The next test will see if the cache is working. Here's the code:

The name of the test tells how we're going to verify that the cache is being used. When the "GetPeople" method is called 2 times, we expect that the wrapped reader is only called 1 time.

The setup for this test is the same as the prior one. We create a caching reader with a duration of 1 second.

For the action, we call "GetPeople" twice. We expect that the second call should use the cache.

Then we check to see how many times the CountedReader is called. We expect that it will be called the first time but not the second time. So, the CallCount should be 1.

Verifying an Expired Cache
For the last test, we want to make sure that the reader still works after the cache has expired.

This test has the same setup: we create a CachingReader with a 1 second cache.

In the action section, we call the "GetPeople" method. This should populate the cache.

Then we wait for 2 seconds. (For more information on how "Task.Delay" works, take a look at the article for the Retry Decorator.)

After waiting 2 seconds, we call the "GetPeople" method again. The cache should be expired, so we expect that the wrapped data reader will be called to refresh the cache.

The result is that we expect the CountedReader to be called twice. And that is what we check with the assertion.

Note: This test takes at least 2 seconds to complete. This is a slooow test, so I've marked it with the "Slow" category. It is possible to create shorter durations for the cache and for the delay, but if we cut things too fine, then we may end up with timing issues in the test.

Test Results
When we run the tests, they all pass:

And we can see that the third tests did take 2 seconds to complete.

Dependency Injection and the Decorator Pattern
Dependency Injection and the Decorator Pattern work really well together. We can easily add retry functionality, exception logging, or a client-side cache to our application. We just need to take the loosely coupled pieces and snap them together in a different order.

We did *not* need to change the view (MainWindow). We did *not* need to change the view model (PeopleReaderViewModel). We did *not* need to change the existing data reader (ServiceReader). We just snap our pieces together in a different order.

It seems like I'm repeating this quite a bit. And that's because it is so important. Decorators let us add functionality in pieces. These atomic units are easier to test, and we can compose them in whatever order we like.

Since our decorators wrap "IPersonReader" objects, and our decorators are "IPersonReader" objects, we can have our decorators wrap other decorators. This will stack the functionality so that we get all of the features that we want. That's what we'll look at next time. So be sure to check back.

Happy Coding!