Okay, so last time I said Jeremy was on .NET Rocks!, Richard just read an email that I sent in. This time, I'm actually on the show (at least for a little bit). For .NET Rocks! #1000, Carl and Richard asked folks to tell them how the show had impacted their career. I sent in my story, and they were nice enough to give me a few minutes to tell my story on the show. You can hear me around the 16:30 mark.
I've been telling folks for a long time how a particular episode gave me the kick in the butt that I needed to get started as a speaker. And after that first time out, I was totally hooked. If you don't believe me, check this article that I wrote in 2011: Meet the Next Code Camp Speaker: You!
I took that first step and spoke at the So Cal Code Camp in January 2010. Since then, I've given over 120 presentations. And I'll keep speaking as long as people keep coming to see me. I love to teach, and I love to watch people learn.
Lots of great things have happened along the way. Microsoft recognized me with an MVP award for Visual C# (2 years so far). I got involved with Pluralsight and produced 6 courses (with more on the way). I've met tons of great people (many of whom I now consider friends), and I've received email from developers around the world.
A Funny Story From the Early Days
Getting started as a speaker isn't easy. You can't just sit around and wait for people to ask you. After my first Code Camp, I was looking for user groups and other places to speak. I was attending my local user group (in Southern California), and the speaker that night ran another group in the area.
During the break, I went up to him and asked if I could get on the speaking calendar for his group. He didn't seem very excited about it, and I can't really blame him -- he didn't know anything about me. But he was nice enough to put me on the calendar 6 months out.
About 3 months later, I was at the Desert Code Camp in Chandler, Arizona. I'd done 3 talks that day, and they had all gone well. At the attendee dinner that night, the user group leader was there, and he had attended one of my presentations. He said, "You're from Southern California, right? Would you be interested in speaking at my group sometime?"
A bit sheepishly, I said, "I'm already on your schedule." And I reminded him that we had met a few months earlier.
So, at first I felt a bit pushy, but then I was invited to speak. That made me feel a lot better.
Try It At Least Once
And if you're interested in speaking, just get started. Code Camps are generally easy to sign up for, and most of them like to give new speakers an opportunity to show their stuff. And if you like it, just talk to people. Now I know most of the user group leaders in my area (and several from outside my area), and it's pretty easy for me to get on someone's schedule. I'm still trying to break into the conference world, but I'm well on my way.
I think that every developer should try it at least once. We all have unique experiences to share with the developer community. And if you find that it's not your thing, at least you tried. But if you do find out you like it, then you might find yourself on an amazing journey.
Happy Coding!
Wednesday, June 25, 2014
Sunday, June 22, 2014
Coding Practice: Displaying Bitmaps from Pixel Data
I'm not sure what you do when you wake up early and can't get back to sleep. For some reason, I decided to read about machine learning, and I ended up somewhere I didn't expect.
I went from this:
To building an application that does this:
Along the way, I watched the trial-and-error approach I took to solve a problem. So, here it is with the mistakes (and still not quite good code) and the working solution.
You can download the completed code here: http://www.jeremybytes.com/downloads.aspx#CPDB.
[Update: 7/11/2015: Project has been uploaded to GitHub: https://github.com/jeremybytes/digit-display]
[Update 06/27/2016: This project has been expanded beyond what's shown in this article. To see the code here, check the "DigitDisplay" branch on GitHub: jeremybytes/digit-display Branch DigitDisplay.]
How I Got There
So, I started by running a search for articles on machine learning with F#. And I came across lots of links from my friend Mathias Brandewinder (http://www.clear-lines.com/blog/). I guess I'll really have to attend one of his machine learning workshops the next time we're both in the same place.
After bouncing through a few links, I ended up looking at a Kaggle competition (https://www.kaggle.com/c/digit-recognizer/data). The challenge is to build a system that can recognize hand-written digits. Now, I knew that I was not up to that challenge (complex algorithms are something I need to work up to). But I looked at the sample data files, and I got a bit intrigued.
The sample data was in .csv format. And it was basically a comma-separated collection of values between 0 and 255 that represented the darkness of a pixel. The data sets were there to help you train and test your system. But I thought it might be interesting to try to display the images in an application.
Here's what a record looks like:
1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,188,255,94,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,191,250,253,93,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,123,248,253,167,10,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,80,247,253,208,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,29,207,253,235,77,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,54,209,253,253,88,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,93,254,253,238,170,17,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,23,210,254,253,159,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,16,209,253,254,240,81,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,27,253,253,254,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,20,206,254,254,198,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,168,253,253,196,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,20,203,253,248,76,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,22,188,253,245,93,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,103,253,253,191,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,89,240,253,195,25,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,15,220,253,253,80,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,94,253,253,253,94,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,89,251,253,250,131,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,214,218,95,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
For the training data, the first value (1 in this case) is the number represented by the data. Then there are 784 values that represent a 28 by 28 pixel bitmap. And there were lots of records in the file.
Step 1: Build a Parser
I created my WPF shell application, and then created a library to handle loading the file and parsing the strings into integer arrays and turning the arrays into bitmaps.
I knew I could parse the file pretty easily; I've done that lots of times before. But I wasn't quite as sure about turning a big string of characters into a bitmap. So, that's where I started.
Here was my first shot at it:
I started by splitting the CSV string into an array. But this is an array of strings; I needed integers. So, I looped through the strings and parsed them into integers. Note that I don't have any error handling here. If there is bad data, this whole thing will blow up. But the dataset I was working with was "clean", so I didn't worry about it. Then I took the flat integer array and parsed it into a 28x28 array.
But I wasn't at all sure that I got this right. In fact, I'm famous for being "off by one". Since I knew that I was still a long way off from creating a bitmap, I figured it would be best to create some unit tests to at least make sure that I'm parsing the array correctly.
This was a bit tedious because I had to take my sample input (just part of it here):
And turn it into 28 arrays of 28 elements each (what my expected output was):
This was a lot of counting, copying, and pasting. But I eventually got the data set up.
After that, the tests were pretty easy (there are 2 other tests that have the other "chunks" of arrays):
And what I found is that I was off by one. In the code that sets the "pixelIndex", I was subtracting one (to account for the first value that I wanted to discard). Instead, I should have been adding one. Here's the corrected code:
Since I knew that my 28 x 28 array was correct, I moved on to creating the bitmap image. As a side note, my unit testing pretty much stopped here. The rest of the code deals with images -- and even better, I had no idea what these images were supposed to look like (as we'll see). So, I didn't have much that I could test.
Step 2: Creating the Bitmap
I now had an integer array that I could work with: int[28,28]. This represented the pixels that I needed to create the bitmap. Here was my first stab at that:
I'm not a big fan of nested "for" statements. But I was just trying to get the job done here. I used this to copy the values from the arrays into the pixels from the bitmap image. Since these are grayscale, I applied the value to the RGB values. So if the value was 238, I would get an RGB value of "238, 238, 238".
I really didn't know if this code was right or not. I needed to display this somehow to find out. But first, I had to load the data.
Step 3: Parse the File
I created another class that would load the data from the CSV file. I just created a static method that would parse the file and generate an array of strings (that I could then pass to the methods I already created).
This checks the config file for the file name, and then looks for the file in the same folder as the executable. Then it just loops through the file and creates an array of strings where each string represents a separate bitmap/digit.
There's a note that we're skipping the first line. That's because the training file (the one I'm using here) has a header row that enumerates the columns.
Now it was time to start putting things together.
Step 4: Displaying a Bitmap
I flipped over to my WPF application, added an <Image> element to the markup, and then flipped to the code behind to populate that image.
I was doing pretty good for a while...
But now what? There's no way to directly assign as System.Drawing.Bitmap (the type that I had) to a WPF Image control.
Fortunately, the developer's friend, StackOverflow, had my solution: http://stackoverflow.com/a/6775114. So, I created a static class for this code and completed the method.
Now it was time for the moment of truth -- running the application:
At least I got an image. But it doesn't look quite right. Time to read the instructions (from the Kaggle site):
And see how that looks:
Much better. And I know from looking at the data, that this should be a number "1".
Step 5: Displaying All the Digits
So, I successfully parsed and displayed a single digit image. Next it was time to try to display a lot of them. I swapped out my <Image> for a <ListBox> and then created the image items to put into the list in code:
This just get the first 10 items, loops through them, and generates bitmaps. Then it creates WPF Image controls for each one and puts it into the ListBox. Here's the output:
Not quite what I wanted. It needs some size constraints:
I set the height and width to 28 since that matched the pixel dimensions. The output was much better:
So, I increased to 100 items:
Then 1000:
And this is where I start to look at the data. Something's not right.
Step 6: Fixing the Image
These numbers don't look quite right. It almost looks like some of them are upside down and some of them are backwards. I thought at first that this might have been intentional. But after staring at it for a bit more, I knew things weren't right on my end.
This probably had something to do with the X-Y coordinate system. Some systems use the top left corner as 0,0. Some systems use the bottom left corner as 0,0. Instead of trying to change my arrays, I figured it would be easier to manipulate the bitmap after it's created. So, I played around a bit with the "RotateFlip" method until I found a setting that worked:
That seems to take care of things:
Much better.
Step 7: Final Image Fix
There was one last thing that bothered me. The digits looked blurry. It made sense that they would be blurry when they were stretched out (like in the initial display), but they should be at the native sizes now.
Or should they?
That's when I remembered how WPF worked. When you set the height and width of controls, you don't actually set pixel sizes; you set device-independent units. This is so that WPF can scale appropriately on different DPI devices. As a side note, I really appreciate the scaling now that I'm using a high-resolution display.
Instead of setting the height and width to "28", let's set them to the size of the image itself:
This gives us more digits per line (they are a bit smaller). But none of them are blurry now.
You can click on the image to see the full-resolution version.
Step 8: Functional Style
Since I started out this experiment looking for functional programming ideas, I figured it would be good to incorporate a few in the application.
I made "GenerateDigitArray" and "GetBitmapFromDigitArray" into static methods. And the "LoadDataStrings" was already static.
What's important about these methods is that they are atomic -- they do not depend on any external state or the state of the class that they are in. They do not make any modifications to existing objects. They take parameters and return separate objects. That means they can operate completely independently and have no side effects.
By making them static, they also become easier to use in our code:
This code shows that we do not need to create instances of the FileLoader or the DigitBitmap objects. We just use the static methods that are on those classes.
We can definitely take this further, and the functions are fairly specific by only dealing with 28x28 arrays. But this is a really good place to start. And again, one of the important things is that we consciously think about these functional concepts.
Step 9: General Clean-Up
That's the working application that I wanted. I know, it's not a very exciting application. But it's kind of cool to think that I'm displaying all of these hand-written numbers based only on comma-separate values.
I did a little bit of clean-up after this. I won't go into all the details (you can check the code download if you're interested). I made a change to the "LoadDataStrings" method. In my main code, I wanted to get rid of the "for" loop in the main block of code and make it a "foreach" so I could process however many records were in the file. You can see this by looking at the previous code sample.
[Update: 7/11/2015: Project has been uploaded to GitHub: https://github.com/jeremybytes/digit-display]
[Update 06/27/2016: This project has been expanded beyond what's shown in this article. To see the code here, check the "DigitDisplay" branch on GitHub: jeremybytes/digit-display Branch DigitDisplay.]
The problem is the first time I did this, my computer just started spinning. That's when I checked the file and found that there were over 40,000 items. That's a bit much to process into images all at once. So, I added a "threshold" parameter to the "LoadDataStrings" method. That way I could say, "Just give me the first 1,000 values". And it would be easier to experiment with. It needs a bit of optimization to work with that many records, and the WPF ListBox may not be the best choice of controls for displaying that many items. It's something to think about further.
Wrap Up
You never know where coding explorations will take you. I found this to be a good exercise for me -- a way to do a few things that I've never done before. And hopefully you've gotten a bit of insight by seeing my thought process. I don't always get things right the first time. And that's perfectly okay. But by working in small steps, it's really easy to keep making progress.
Happy Coding!
I went from this:
To building an application that does this:
Along the way, I watched the trial-and-error approach I took to solve a problem. So, here it is with the mistakes (and still not quite good code) and the working solution.
You can download the completed code here: http://www.jeremybytes.com/downloads.aspx#CPDB.
[Update: 7/11/2015: Project has been uploaded to GitHub: https://github.com/jeremybytes/digit-display]
[Update 06/27/2016: This project has been expanded beyond what's shown in this article. To see the code here, check the "DigitDisplay" branch on GitHub: jeremybytes/digit-display Branch DigitDisplay.]
How I Got There
So, I started by running a search for articles on machine learning with F#. And I came across lots of links from my friend Mathias Brandewinder (http://www.clear-lines.com/blog/). I guess I'll really have to attend one of his machine learning workshops the next time we're both in the same place.
After bouncing through a few links, I ended up looking at a Kaggle competition (https://www.kaggle.com/c/digit-recognizer/data). The challenge is to build a system that can recognize hand-written digits. Now, I knew that I was not up to that challenge (complex algorithms are something I need to work up to). But I looked at the sample data files, and I got a bit intrigued.
The sample data was in .csv format. And it was basically a comma-separated collection of values between 0 and 255 that represented the darkness of a pixel. The data sets were there to help you train and test your system. But I thought it might be interesting to try to display the images in an application.
Here's what a record looks like:
1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,188,255,94,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,191,250,253,93,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,123,248,253,167,10,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,80,247,253,208,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,29,207,253,235,77,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,54,209,253,253,88,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,93,254,253,238,170,17,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,23,210,254,253,159,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,16,209,253,254,240,81,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,27,253,253,254,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,20,206,254,254,198,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,168,253,253,196,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,20,203,253,248,76,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,22,188,253,245,93,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,103,253,253,191,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,89,240,253,195,25,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,15,220,253,253,80,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,94,253,253,253,94,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,89,251,253,250,131,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,214,218,95,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
For the training data, the first value (1 in this case) is the number represented by the data. Then there are 784 values that represent a 28 by 28 pixel bitmap. And there were lots of records in the file.
Step 1: Build a Parser
I created my WPF shell application, and then created a library to handle loading the file and parsing the strings into integer arrays and turning the arrays into bitmaps.
I knew I could parse the file pretty easily; I've done that lots of times before. But I wasn't quite as sure about turning a big string of characters into a bitmap. So, that's where I started.
Here was my first shot at it:
I started by splitting the CSV string into an array. But this is an array of strings; I needed integers. So, I looped through the strings and parsed them into integers. Note that I don't have any error handling here. If there is bad data, this whole thing will blow up. But the dataset I was working with was "clean", so I didn't worry about it. Then I took the flat integer array and parsed it into a 28x28 array.
But I wasn't at all sure that I got this right. In fact, I'm famous for being "off by one". Since I knew that I was still a long way off from creating a bitmap, I figured it would be best to create some unit tests to at least make sure that I'm parsing the array correctly.
This was a bit tedious because I had to take my sample input (just part of it here):
And turn it into 28 arrays of 28 elements each (what my expected output was):
This was a lot of counting, copying, and pasting. But I eventually got the data set up.
After that, the tests were pretty easy (there are 2 other tests that have the other "chunks" of arrays):
And what I found is that I was off by one. In the code that sets the "pixelIndex", I was subtracting one (to account for the first value that I wanted to discard). Instead, I should have been adding one. Here's the corrected code:
Since I knew that my 28 x 28 array was correct, I moved on to creating the bitmap image. As a side note, my unit testing pretty much stopped here. The rest of the code deals with images -- and even better, I had no idea what these images were supposed to look like (as we'll see). So, I didn't have much that I could test.
Step 2: Creating the Bitmap
I now had an integer array that I could work with: int[28,28]. This represented the pixels that I needed to create the bitmap. Here was my first stab at that:
I'm not a big fan of nested "for" statements. But I was just trying to get the job done here. I used this to copy the values from the arrays into the pixels from the bitmap image. Since these are grayscale, I applied the value to the RGB values. So if the value was 238, I would get an RGB value of "238, 238, 238".
I really didn't know if this code was right or not. I needed to display this somehow to find out. But first, I had to load the data.
Step 3: Parse the File
I created another class that would load the data from the CSV file. I just created a static method that would parse the file and generate an array of strings (that I could then pass to the methods I already created).
This checks the config file for the file name, and then looks for the file in the same folder as the executable. Then it just loops through the file and creates an array of strings where each string represents a separate bitmap/digit.
There's a note that we're skipping the first line. That's because the training file (the one I'm using here) has a header row that enumerates the columns.
Now it was time to start putting things together.
Step 4: Displaying a Bitmap
I flipped over to my WPF application, added an <Image> element to the markup, and then flipped to the code behind to populate that image.
I was doing pretty good for a while...
But now what? There's no way to directly assign as System.Drawing.Bitmap (the type that I had) to a WPF Image control.
Fortunately, the developer's friend, StackOverflow, had my solution: http://stackoverflow.com/a/6775114. So, I created a static class for this code and completed the method.
Now it was time for the moment of truth -- running the application:
At least I got an image. But it doesn't look quite right. Time to read the instructions (from the Kaggle site):
Each image is 28 pixels in height and 28 pixels in width, for a total of 784 pixels in total. Each pixel has a single pixel-value associated with it, indicating the lightness or darkness of that pixel, with higher numbers meaning darker. This pixel-value is an integer between 0 and 255, inclusive.Okay, so I got my light/dark reversed. In my image, the higher numbers represent lighter values. Let's fix the method by subtracting the value from 255 (to invert it):
And see how that looks:
Much better. And I know from looking at the data, that this should be a number "1".
Step 5: Displaying All the Digits
So, I successfully parsed and displayed a single digit image. Next it was time to try to display a lot of them. I swapped out my <Image> for a <ListBox> and then created the image items to put into the list in code:
This just get the first 10 items, loops through them, and generates bitmaps. Then it creates WPF Image controls for each one and puts it into the ListBox. Here's the output:
Not quite what I wanted. It needs some size constraints:
I set the height and width to 28 since that matched the pixel dimensions. The output was much better:
So, I increased to 100 items:
Then 1000:
And this is where I start to look at the data. Something's not right.
Step 6: Fixing the Image
These numbers don't look quite right. It almost looks like some of them are upside down and some of them are backwards. I thought at first that this might have been intentional. But after staring at it for a bit more, I knew things weren't right on my end.
This probably had something to do with the X-Y coordinate system. Some systems use the top left corner as 0,0. Some systems use the bottom left corner as 0,0. Instead of trying to change my arrays, I figured it would be easier to manipulate the bitmap after it's created. So, I played around a bit with the "RotateFlip" method until I found a setting that worked:
That seems to take care of things:
Much better.
Step 7: Final Image Fix
There was one last thing that bothered me. The digits looked blurry. It made sense that they would be blurry when they were stretched out (like in the initial display), but they should be at the native sizes now.
Or should they?
That's when I remembered how WPF worked. When you set the height and width of controls, you don't actually set pixel sizes; you set device-independent units. This is so that WPF can scale appropriately on different DPI devices. As a side note, I really appreciate the scaling now that I'm using a high-resolution display.
Instead of setting the height and width to "28", let's set them to the size of the image itself:
This gives us more digits per line (they are a bit smaller). But none of them are blurry now.
You can click on the image to see the full-resolution version.
Step 8: Functional Style
Since I started out this experiment looking for functional programming ideas, I figured it would be good to incorporate a few in the application.
I made "GenerateDigitArray" and "GetBitmapFromDigitArray" into static methods. And the "LoadDataStrings" was already static.
What's important about these methods is that they are atomic -- they do not depend on any external state or the state of the class that they are in. They do not make any modifications to existing objects. They take parameters and return separate objects. That means they can operate completely independently and have no side effects.
By making them static, they also become easier to use in our code:
This code shows that we do not need to create instances of the FileLoader or the DigitBitmap objects. We just use the static methods that are on those classes.
We can definitely take this further, and the functions are fairly specific by only dealing with 28x28 arrays. But this is a really good place to start. And again, one of the important things is that we consciously think about these functional concepts.
Step 9: General Clean-Up
That's the working application that I wanted. I know, it's not a very exciting application. But it's kind of cool to think that I'm displaying all of these hand-written numbers based only on comma-separate values.
I did a little bit of clean-up after this. I won't go into all the details (you can check the code download if you're interested). I made a change to the "LoadDataStrings" method. In my main code, I wanted to get rid of the "for" loop in the main block of code and make it a "foreach" so I could process however many records were in the file. You can see this by looking at the previous code sample.
[Update: 7/11/2015: Project has been uploaded to GitHub: https://github.com/jeremybytes/digit-display]
[Update 06/27/2016: This project has been expanded beyond what's shown in this article. To see the code here, check the "DigitDisplay" branch on GitHub: jeremybytes/digit-display Branch DigitDisplay.]
The problem is the first time I did this, my computer just started spinning. That's when I checked the file and found that there were over 40,000 items. That's a bit much to process into images all at once. So, I added a "threshold" parameter to the "LoadDataStrings" method. That way I could say, "Just give me the first 1,000 values". And it would be easier to experiment with. It needs a bit of optimization to work with that many records, and the WPF ListBox may not be the best choice of controls for displaying that many items. It's something to think about further.
Wrap Up
You never know where coding explorations will take you. I found this to be a good exercise for me -- a way to do a few things that I've never done before. And hopefully you've gotten a bit of insight by seeing my thought process. I don't always get things right the first time. And that's perfectly okay. But by working in small steps, it's really easy to keep making progress.
Happy Coding!
Wednesday, June 18, 2014
New Video: Generics in Methods & Delegates
Part 3 of the C# Generics video series is now available. This time we're talking about generics in methods and delegates: C# Generics - Part 3: Methods & Delegates.
Generics are extremely useful in consolidating code. In this demo, we combine 2 methods into a single one by adding a generic type parameter. This allows the method to work with a variety of types rather than being specialized for a single interface or class.
The series will continue with upcoming videos on nested generics and generic constraints.
C# Generics (Playlist)
Happy Coding!
Generics are extremely useful in consolidating code. In this demo, we combine 2 methods into a single one by adding a generic type parameter. This allows the method to work with a variety of types rather than being specialized for a single interface or class.
The series will continue with upcoming videos on nested generics and generic constraints.
C# Generics (Playlist)
Happy Coding!
Thursday, June 12, 2014
Upcoming Pluralsight Course -- Abstract Art: Getting Things "Just Right"
I'm happy to announce that I've just started working on a new course for Pluralsight -- Abstract Art: Getting Things "Just Right". Most of the topics that I talk about are things that took me a long time to really understand well. This is no different.
Abstraction is awesome: it can help make our applications easier to maintain, test, and extend. Abstraction is also awful: it can make our code difficult to understand, navigate, and debug. And there's the problem. If we have too little, our applications can be rigid and difficult to maintain. If we have too much, our applications can be confusing and difficult to maintain. We need to find just the right amount of abstraction for a particular application and environment.
Why is it so hard to get abstraction "just right"? Because it goes against our nature as developers. We have natural tendencies that put us into one of two groups: under-abstractors and over-abstractors. Under-abstractors tend to shy away from abstractions: "We'll just hard-code this and copy/paste that, and it will be fine." Over-abstractors tend to add more abstraction than is actually useful: "We'd better put in these 3 layers and add a plug-in architecture because we might need them later." And even though no one likes to identify with either of these categories, we need to admit to who we are deep down.
The good news is that we can overcome our natural tendencies to find the right balance for our applications. And that's what this course is all about. We'll take a look at some of the mistakes and missteps I've made in building applications over the years, how there's a tendency to over-correct when adjusting our approach, and how I worked with another developer to come up with really good techniques to find the right balance.
There will be lots of practical advice and guidance to help us along the path getting abstraction "just right".
So, stay tuned for progress reports, and let me know if you have any suggestions on things you'd like to see.
Happy Coding!
Abstraction is awesome: it can help make our applications easier to maintain, test, and extend. Abstraction is also awful: it can make our code difficult to understand, navigate, and debug. And there's the problem. If we have too little, our applications can be rigid and difficult to maintain. If we have too much, our applications can be confusing and difficult to maintain. We need to find just the right amount of abstraction for a particular application and environment.
Why is it so hard to get abstraction "just right"? Because it goes against our nature as developers. We have natural tendencies that put us into one of two groups: under-abstractors and over-abstractors. Under-abstractors tend to shy away from abstractions: "We'll just hard-code this and copy/paste that, and it will be fine." Over-abstractors tend to add more abstraction than is actually useful: "We'd better put in these 3 layers and add a plug-in architecture because we might need them later." And even though no one likes to identify with either of these categories, we need to admit to who we are deep down.
The good news is that we can overcome our natural tendencies to find the right balance for our applications. And that's what this course is all about. We'll take a look at some of the mistakes and missteps I've made in building applications over the years, how there's a tendency to over-correct when adjusting our approach, and how I worked with another developer to come up with really good techniques to find the right balance.
There will be lots of practical advice and guidance to help us along the path getting abstraction "just right".
So, stay tuned for progress reports, and let me know if you have any suggestions on things you'd like to see.
Happy Coding!
Tuesday, June 10, 2014
June 2014 Speaking Engagements
At the end of the month, I'll be at the So Cal Code Camp. Always lots of fun: great people, great content.
Saturday & Sunday, June 28 & 29, 2014
So Cal Code Camp
San Diego, CA
http://www.socalcodecamp.com/
Topics:
o Abstract Art: Getting Things "Just Right" (more)
o Clean Code: Homicidal Maniacs Read Code, Too! (more)
o IEnumerable, ISaveable, IDontGetIt: Understanding .NET Interfaces (more)
o Learn to Love Lambdas (more)
It's also a great excuse to spend some time in beautiful San Diego. Hope to see you there.
Happy Coding!
Saturday & Sunday, June 28 & 29, 2014
So Cal Code Camp
San Diego, CA
http://www.socalcodecamp.com/
Topics:
o Abstract Art: Getting Things "Just Right" (more)
o Clean Code: Homicidal Maniacs Read Code, Too! (more)
o IEnumerable, ISaveable, IDontGetIt: Understanding .NET Interfaces (more)
o Learn to Love Lambdas (more)
It's also a great excuse to spend some time in beautiful San Diego. Hope to see you there.
Happy Coding!
Monday, June 9, 2014
Recognizing Higher-Order Functions in C#
I've been working more and more with functional programming in one form or another. This started a while back. My primary goal has been to look for ways to bring functional concepts into my current main language (which happens to be C#). What I have found is that I have been using functional concepts rather naturally -- as mentioned when I talked about LINQ and the Functional Approach.
So, here's another revelation: I've been using higher-order functions without realizing it.
What is a Higher Order Function?
The concept of a higher-order function is quite simple, but can also be a bit confusing:
But we can also pass functions as parameters. This is done using delegates or lambda expressions. Let's take a look at some examples.
Injecting Behavior with Delegates
Delegates are an important part of the C# language. And they are important to understand when looking at LINQ and lambda expressions (two of my favorite things in C#). In my presentation, "Get Func<>-y: Delegates in .NET", I show how to use delegates to inject functionality into a class.
Here's the class:
Let's take a closer look at the ToString method:
This is a function that takes another function as a parameter (Func<Person, string>). That makes this a higher-order function.
With this method in place, we can choose exactly what we want the ToString method to do when we execute it. One of the easiest things to do is use a lambda expression to represent the parameter, so that's what I normally do. Here's the code that uses this method:
The first method call is to the ToString function that takes no parameters. We just have this here for reference. But all of the other calls use the ToString method that takes a function as a parameter.
We won't go into the details of how delegates and Func<T> work; that's covered in the presentation materials. The short version is that our parameter type is Func<Person, string>. What this means is that our parameter needs to be a function that takes a "Person" as a parameter and returns a "string".
So, that's what each of these do. The "p" parameter of the lambda expression represents the current "Person" object. Then the body of the lambda expression performs the work to return a string. The first example simply calls the no-parameter ToString method. But from there, we take a little more control.
In one method, we output the LastName property in upper case. In another, we output the FirstName property as lower case. And in another, we use a "LastName comma FirstName" format.
And here's out output:
Why do we do this? Several reasons go hand-in-hand. The Single Responsibility Principle (the S in the SOLID principles) tells us that our class should only have one reason to change. With regard to our "Person" class, its primary responsibility should not be formatting output strings. So, we moved that responsibility out of the class so someone else can be responsible for it.
We also have the Open Close Principle (the O in the SOLID principles) that tells us that a class should be open to extension but closed to modification. We can add custom string outputs to the Person class (extending it) without making any changes to the class itself.
And bringing things back around, a higher-order function (ToString(Func<Person, string>)) makes this possible.
LINQ and Where
I love LINQ. It's one of my favorite tools. And I also love lambda expressions. In my presentation "Learn to Love Lambdas", I show plenty of both.
And I've recently put the pieces together that LINQ methods are good examples of higher-order functions. Let's specifically look at "Where". First, here's the declaration (from MSDN):
This shows us that "Where" takes a "Func<TSource, bool>" as a parameter. This is a delegate (which is a function), meaning that Where is a higher-order function. We won't go into all the other details of this declaration (like the extension method syntax or the generic types); these are covered in the presentation.
But let's look at the Func<TSource, bool> parameter. Like our other example, this needs to be a function that takes a TSource as a parameter (where TSource is simply the type of the objects in our collection that we're working with) and returns a true/false value.
We have a simple example of filtering with "Where" using a text box. Here's the UI:
And here's the code that uses this:
In this method, "data" is a collection of Person objects (similar to the Person class above). This code checks to see if the Name Filter checkbox is checked. If so, then it puts a Where condition on our data. And just like above, "p" represents our Person object. And we can see the comparison of the FirstName property of our Person to whatever is in the text box. This gives us a true/false value.
Here's the resulting output:
This really shows us the usefulness of a higher-order function. The "Where" function lets us pass in whatever functionality we want to determine which items we want to include in the output and which items we want to exclude from the output.
Now, "Where" is much more complicated than the "ToString" method that we showed above. "ToString" simply ran whatever function was passed in. But "Where" does quite a bit of other things. It's involved in iterating through a sequence of some sort (IEnumerable). And it does this in a lazy way -- it only returns items as we ask for them. So, there's a lot more going on behind the scenes than simply running the function that we pass as a parameter. But that function plays an important role.
As a side note, this application has a lot of code-behind (since I want to concentrate on LINQ and lambdas during the presentation and not get mired into the application too deeply). But I've also created a version that uses a view model to separate the UI from the presentation logic (also included in the "Learn to Love Lambdas" materials).
Here's what the code from the view model looks like for the name filter:
We can see that this is very similar to the previous code. The primary difference is that it does not interact directly with UI elements. It uses properties instead. So, instead of looking at the state of the "NameFilterCheckBox", it looks at the "NameFilterChecked" property on the view model (which is databound to the checkbox). And instead of looking at the "NameTextBox" UI element, it looks at the "nameFilterValue" field on the view model (which again is set through databinding to the text box in the UI).
Why Do I Care?
So, I haven't shown any new code here. This is all code that I've been presenting for several years and based on real applications that I've built.
Now I can start to use these constructs more intentionally in my code. When I think about needing the Strategy pattern or I find similar functionality that can be combined, I can remember how higher-order functions can help with these.
So, even though I've been a fan of delegates, lambda expressions, and LINQ for a long time. I've been able to add another label to them: higher-order functions. It's now a new tool in my toolbox. I can learn the advantages and disadvantages of them. And I can better recognize where I should be using them.
And this is why we want to keep learning.
Happy Coding!
So, here's another revelation: I've been using higher-order functions without realizing it.
What is a Higher Order Function?
The concept of a higher-order function is quite simple, but can also be a bit confusing:
A higher-order function is a function that takes a function as a parameter.We're used to passing objects as parameters. Sometimes they are simple objects like an integer or a string. Sometimes they are complex objects like collections or custom classes.
But we can also pass functions as parameters. This is done using delegates or lambda expressions. Let's take a look at some examples.
Injecting Behavior with Delegates
Delegates are an important part of the C# language. And they are important to understand when looking at LINQ and lambda expressions (two of my favorite things in C#). In my presentation, "Get Func<>-y: Delegates in .NET", I show how to use delegates to inject functionality into a class.
Here's the class:
Let's take a closer look at the ToString method:
This is a function that takes another function as a parameter (Func<Person, string>). That makes this a higher-order function.
With this method in place, we can choose exactly what we want the ToString method to do when we execute it. One of the easiest things to do is use a lambda expression to represent the parameter, so that's what I normally do. Here's the code that uses this method:
The first method call is to the ToString function that takes no parameters. We just have this here for reference. But all of the other calls use the ToString method that takes a function as a parameter.
We won't go into the details of how delegates and Func<T> work; that's covered in the presentation materials. The short version is that our parameter type is Func<Person, string>. What this means is that our parameter needs to be a function that takes a "Person" as a parameter and returns a "string".
So, that's what each of these do. The "p" parameter of the lambda expression represents the current "Person" object. Then the body of the lambda expression performs the work to return a string. The first example simply calls the no-parameter ToString method. But from there, we take a little more control.
In one method, we output the LastName property in upper case. In another, we output the FirstName property as lower case. And in another, we use a "LastName comma FirstName" format.
And here's out output:
Why do we do this? Several reasons go hand-in-hand. The Single Responsibility Principle (the S in the SOLID principles) tells us that our class should only have one reason to change. With regard to our "Person" class, its primary responsibility should not be formatting output strings. So, we moved that responsibility out of the class so someone else can be responsible for it.
We also have the Open Close Principle (the O in the SOLID principles) that tells us that a class should be open to extension but closed to modification. We can add custom string outputs to the Person class (extending it) without making any changes to the class itself.
And bringing things back around, a higher-order function (ToString(Func<Person, string>)) makes this possible.
LINQ and Where
I love LINQ. It's one of my favorite tools. And I also love lambda expressions. In my presentation "Learn to Love Lambdas", I show plenty of both.
And I've recently put the pieces together that LINQ methods are good examples of higher-order functions. Let's specifically look at "Where". First, here's the declaration (from MSDN):
This shows us that "Where" takes a "Func<TSource, bool>" as a parameter. This is a delegate (which is a function), meaning that Where is a higher-order function. We won't go into all the other details of this declaration (like the extension method syntax or the generic types); these are covered in the presentation.
But let's look at the Func<TSource, bool> parameter. Like our other example, this needs to be a function that takes a TSource as a parameter (where TSource is simply the type of the objects in our collection that we're working with) and returns a true/false value.
We have a simple example of filtering with "Where" using a text box. Here's the UI:
And here's the code that uses this:
In this method, "data" is a collection of Person objects (similar to the Person class above). This code checks to see if the Name Filter checkbox is checked. If so, then it puts a Where condition on our data. And just like above, "p" represents our Person object. And we can see the comparison of the FirstName property of our Person to whatever is in the text box. This gives us a true/false value.
Here's the resulting output:
This really shows us the usefulness of a higher-order function. The "Where" function lets us pass in whatever functionality we want to determine which items we want to include in the output and which items we want to exclude from the output.
Now, "Where" is much more complicated than the "ToString" method that we showed above. "ToString" simply ran whatever function was passed in. But "Where" does quite a bit of other things. It's involved in iterating through a sequence of some sort (IEnumerable). And it does this in a lazy way -- it only returns items as we ask for them. So, there's a lot more going on behind the scenes than simply running the function that we pass as a parameter. But that function plays an important role.
As a side note, this application has a lot of code-behind (since I want to concentrate on LINQ and lambdas during the presentation and not get mired into the application too deeply). But I've also created a version that uses a view model to separate the UI from the presentation logic (also included in the "Learn to Love Lambdas" materials).
Here's what the code from the view model looks like for the name filter:
We can see that this is very similar to the previous code. The primary difference is that it does not interact directly with UI elements. It uses properties instead. So, instead of looking at the state of the "NameFilterCheckBox", it looks at the "NameFilterChecked" property on the view model (which is databound to the checkbox). And instead of looking at the "NameTextBox" UI element, it looks at the "nameFilterValue" field on the view model (which again is set through databinding to the text box in the UI).
Why Do I Care?
So, I haven't shown any new code here. This is all code that I've been presenting for several years and based on real applications that I've built.
Is knowing that these are higher-order functions really important?The answer is YES. Now that I recognize these as higher-order functions, I can start to analyze them in that context. Higher-order functions have certain advantages, such as making the Strategy pattern easier to implement. And there are certain situations where we want to use higher-order functions, such as where we have similar functionality but we want to abstract out some of the details.
Now I can start to use these constructs more intentionally in my code. When I think about needing the Strategy pattern or I find similar functionality that can be combined, I can remember how higher-order functions can help with these.
So, even though I've been a fan of delegates, lambda expressions, and LINQ for a long time. I've been able to add another label to them: higher-order functions. It's now a new tool in my toolbox. I can learn the advantages and disadvantages of them. And I can better recognize where I should be using them.
And this is why we want to keep learning.
Happy Coding!
Subscribe to:
Posts (Atom)