Tuesday, January 6, 2015

Rewriting a Legacy App - Production Tests and Updates

A couple months back, I came up with a minimum viable product to replace an existing application (articles: Rewriting a Legacy Application). Even though it only took a few days to rewrite the application, I didn't put it into production right away.

Since there are some unknowns regarding the computer and hardware interaction, I wanted to make sure that I was around to watch the application once it was in production. Since I did a lot of traveling in November and we had holidays in December, I decided to hold off.

I finally put it into production late last week. In addition, I uploaded the project to GitHub and started tracking updates (GitHub project: https://github.com/jeremybytes/house-control). Time for a status update.

Commit Log
Here's the commit log for the changes that were made since the last article:

Let's walk through these (starting with the oldest item at the bottom).

Production Runner
The first thing I needed was an application that I could put into production. So far in the project, I had the libraries and a test console application (called "X10Test") but no production application.

I created a new console application called "HouseControlAgent". This is pretty simple. In fact, the Main method is simpler than our test application:

When we new up the "HouseController" object, it loads up the schedule and starts the timer. At some point in the future, I might want to add "Start" and "Stop" methods to this class. But for now, it will be tied to the console application.

This gave me something that I could run on the production machine. I copied files over, stopped the old system, and started up the new one.

Current Status
So far, things have been running pretty well. I haven't noticed any drop outs, and the schedule items seem to be activating at the right time.

Remove Test Records
The next thing I had to do was remove the test records that we had in the system. As mentioned in Part 4, testing things that are scheduled is a bit tricky. So we had some test records in place:

This is part of the "Schedule" object, which means that we don't have any external control over it. So the first thing I did was remove the test records from this class.

Moving the Test Records?
Then I thought about adding them back in somewhere else -- like in the console application. This would give us a quick way to verify that the system was working once we started it up in production. We would not have to wait for a schedule item to fire.

But there's a problem with this. The schedules are based only on the time portion of the DateTime object. That meant that if we added a schedule item at application start up, it would fire *every day* at that time. Probably not what we want.

So, I started thinking about what to do next. We'll loop back around to this, but first...

Removing the One Minute Window
Another thing that was mentioned in Part 4 is that we have a 1 minute window where we process scheduled items. Since our timer fires on 30 second intervals, this means that we end up sending each command 3 times.

This is the code that was in the legacy application. This application was in production for many years and ran on different types of hardware during that time. Sometimes the commands would not register with the X10 system, so I added this window that would send the command multiple times.

To see whether this was still necessary, I removed the window. I changed the timer to run on a 1 minute interval (instead of a 30 second interval), and I also compressed the window to 30 seconds.

Timer with 1 minute interval

30 second window

The result is that each command is now only sent 1 time.

Current Status
This has seemed to be working okay over the last few days. Unfortunately, I ran into a bit of a snag earlier today (just a couple hours ago). An item was scheduled for 4:30 p.m., but it didn't happen.

I checked the application, and it showed the command was sent at 4:30 p.m., but it was not registered by the system for some reason. I'll need to keep an eye on this. If things are not reliable, then I might need to go back to the expanded window.

Or I might come up with an entirely different solution where each command is sent more than one time. We'll have to keep an eye on things and see. That's one of the joys of working with this type of hardware. Since it's communicating with RF and through the power lines, there are multiple opportunities for interference.

One-Time Events
The idea of keeping the ability to put in test records turned into something a little bit bigger. I wanted a way to put in events that only fire one time -- this might be a good way to do a "status check" when the system starts up. We may even want to reset all items to a known state.

But as mentioned, the problem with the current schedule is that it was based on the time only (not the date). If I wanted to support these one-time only events, I would need to make a few updates.

First, I created a new enum "ScheduleTypes" that we could use:

Then I added this to the "ScheduleItem" object that we have. For anything coming from the "Schedule.txt" file, I just set things to "Daily" (but I'll want to be able to use the other options in the future).

In order to support adding these one-time events, I added a new method to the "HouseController" object:

This makes it easy for us to add one-time items in our console application.

Using the Date
But there's another consideration. How do I know when it's safe to remove a one-time event? If we're only basing things on time (and not date), then I don't know if this event has already passed or if it's still 23 hours in the future.

So I made some modifications to the system so that it takes into account both the time *and* date of a ScheduleItem. And we also have to treat our schedule a bit different.

For example, consider the items in our "Schedule.txt" file:

The dates for all of our items is January 1st, 2000. If the date portion is now important, we need to deal with this somehow.

And that's when I came up with the concept of "rolling" the schedule.

Rolling the Schedule
"Rolling the schedule" means that we take any items that are in the past, and re-schedule them for the next day. This way, all of our schedule items will have DateTimes that are in the future.

The method that does this is a little wonky right now, but here's what we have:

This loops through all of the items in our schedule, then looks for any items that are in the past.

From there, we figure out what to do with the item based on the schedule type. For example, if something is listed as "Daily", we want to roll the schedule forward to the next day. And we have a helper method set up to do this:

This will take today's date, add one day (to make it tomorrow) and then append the time from the event. So if an item was scheduled for "January 6th, 2015 at 3:15 p.m." (which is in the past at the time of this writing), it will be rolled forward to "January 7th, 2015 at 3:15 p.m." (which is in the future at the time of this writing).

I think this may run into some problems with midnight schedules; I'll need to do a bit more testing with this.

One-Time Events
But the important part of rolling the schedule is that is also lets us have our one-time events. Notice this code from the "RollSchedule" method:

This will remove an event that is marked as "Once" and is already in the past.

As a side note, since we're removing items from our schedule, we can't use a standard iterator and "foreach" loop. (We're not allowed to add or remove items in an active iterator.) So instead, we have to use a "for" loop.

And one other thing to note about our "for" loop is that we are counting backwards. We start at the bottom of the list and work our way up. The reason we do this is that we are potentially removing items. By going backward through the index, we make sure that we don't end up with an "index out of bounds" error.

Unit Testing
As you might imagine, there are a lot of things that can go wrong when we roll the schedule. Because of this, I put some unit tests into place. This will let us test the helper methods that we have to roll the dates and also confirm that the schedule object itself is working as intended.

ScheduleTestsGitHub: HouseControl.Library.Test/ScheduleTests.cs
ScheduleHelperTestsGitHub: HouseControl.Library.Test/ScheduleHelperTests.cs

I found these tests extremely helpful. As I was writing the tests and helper methods, I found a lot of problems with my code. And having the tests made it easier for me to figure out what I needed to do to get the behavior that I was expecting.

There's still more work to be done with the tests. I'll talk about this a bit more in the "Open Issues" section.

Current Status
This functionality is working fine so far. I can add one-time events that are scheduled at start up. Then after the events run, they are removed from the schedule.

Update 1: Just after completing this article, I discovered that the schedule items are not rolling properly on application start up. This is a problem with the "RollForwardToNextDay" method above. Things are getting rolled to "tomorrow" even though the initial schedule should be "today". This is now listed as "Bug" in the GitHub Issues. It looks like I have some more work to do on this...

Update 2: Issue is fixed with commit 956aab2. Solution was to use the current date when loading the schedule items from disk. Then the initial schedule roll works as expected. Details in the logged issue.

This functionality will also be good for future use. If we want to send one-time commands to the system (for example, we want to manually turn on a light), we can use the one-time event feature. This puts us in a good position with regard to that.

Open Issues
Since I have the project on GitHub, I thought that I would try out some of the tracking features for enhancements. I've opened three issues so far:

These items are there primarily to make testing easier. In addition, I know that there are some things that need to be improved with my handling of DateTime (thanks to the master of all things time-related: Matt Johnson (@mj1856 on Twitter).

The biggest reason I opened these issues is that I took an initial stab at this functionality and managed to get things twisted up pretty badly. This is where I was thankful for having a branch in Git that I could simply abandon to get back to my working code. These issues are my way of approaching things in smaller chunks.

Centralize "DateTime.Now"
Having DateTime.Now in the code makes things especially challenging to test. I've looked into several solutions, and I think the best one is to abstract out the calls to DateTime.Now so that they can be replaced for testing.

The first step for this centralize any calls to "DateTime.Now". I already have plans to create some helper methods (in the "ScheduleHelper" class) to do things like "check if this date is in the past" and "calculate how far away this event is from right now".

This doesn't immediately make things more testable, but it does set up the next step.

Abstract calls to "DateTime.Now"
Once things are centralized, it's easier to set up an interface for a TimeProvider that will give us the current time. With this in place, we can replace our centralized calls to "DateTime.Now" with calls into our TimeProvider.

Once we have this abstraction, we can create a mock or fake object that we can use for testing. The advantage here is that we can choose whatever time we want to represent the "current time". This lets us create tests for a lot of scenarios that would be difficult to test otherwise.

After this is in place, I expect the number of unit tests to balloon.

Update to UTC times
This last issue is for stability purposes. I've had enough conversations with Matt to know that dealing with local times is not the best way of doing things. There will be issues around Daylight Saving Time and other things that could come up.

If you want to take advantage of Matt's knowledge, you can check out his Pluralsight course "Date and Time Fundamentals". Unless you take a look at this, you may never realize just how complicated it is to deal with DateTime properly (especially when globalizing applications).

Wrap Up
I managed to get the MVP into production (which is good). I also got the project uploaded to GitHub. And I've already started working on improvements to the application.

Applications evolve over time based on the needs of the users. In this case, I have a pretty easy job: I already know what the user needs (because that's me). But as mentioned before, there are a few more features that I would like to add -- hopefully before the summer time when I need the air conditioner to go on/off on a reasonable schedule.

Happy Coding!

No comments:

Post a Comment