It looks like March will be pretty busy for me. I have four events scheduled so far:
Wednesday, March 5, 2014
SoCal .NET Developers Group
Buena Park, CA
http://www.socaldotnet.com/
Topic: Dependency Injection: A Practical Introduction
Saturday, March 15, 2014
Utah Code Camp
Salt Lake City, UT
http://www.utahgeekevents.com
Topics:
Learn to Love Lambdas
Clean Code: Homicidal Maniacs Read Code, Too!
Tuesday, March 18, 2014
Disney .NET Developers Group
Burbank, CA
Topic: Keep Your UI Responsive with the BackgroundWorker Component
Thursday, March 20, 2014
SoCal .NET Architecture
Newport Beach, CA
http://www.scdna.org/
Topic: Abstract Art: Getting Things "Just Right"
Tuesday, March 25, 2014
San Diego .NET User Group
San Diego, CA
http://www.sandiegodotnet.com/
Topic: IEnumerable, ISaveable, IDontGetIt: Understanding .NET Interfaces
Thursday, March 27, 2014
dotNetGroup.org
Las Vegas, NV
http://dotnetgroup.org/
Topic: Dependency Injection: A Practical Introduction
Dependency Injection continues to be a popular topic. DI is extremely useful for creating loosely coupled applications. Unfortunately, most devs are introduced to DI by being thrown into code that uses a Dependency Injection container (such as Unity, Ninject, or StructureMap). We start learning DI by implementing it manually. That way, we understand the concepts and the steps involved. After we have a good handle on that, we look at how containers can help take care of the tedious bits and also provide extra functionality.
Stay tuned: I have a number of event scheduled at the beginning of April. So, if you're in Phoenix AZ, Los Angeles CA, San Bernardino CA, or the San Francisco Bay area, you'll have a chance to come see me soon.
Hope to see you at an upcoming event. And if you want me to come to your event, just drop me a note.
Happy Coding!
Thursday, February 27, 2014
Friday, February 21, 2014
Demystifying the "var" Keyword in C#
There are some misconceptions about the "var" keyword in C#. The "var" keyword was added to the C# language to solve a specific problem, but developers use it in other parts of their code as well (and there's nothing wrong with that). We'll take a look at what "var" is (and what it isn't), look at the reason it was created, and then I'll show where I use it and my reasoning behind that.
This whole discussion has been spurred by some questions that I've received from a few developers recently:
Does using "var" affect performance?
Does "var" indicate a variant type that we have in other languages?
Let's answer these questions by looking at how "var" works.
"var" Is Just a Shortcut
(The way that most developers use it)
Where we most often see "var", it is just a shortcut to writing out the type for a variable. The idea is that the compiler can figure out the actual type based on what we are assigning to the variable, so we don't have to be explicit about the type. Let's start with a simple example:
These lines of code are 100% equivalent from a technical perspective. (We'll talk about the readability perspective a bit later.) Both the "message1" and "message2" variables of are type "string". And they are both strongly typed.
We can see that "message1" is strongly typed by putting our mouse over the "var" keyword:
This clearly shows that "message1" is a string. Since the compiler can figure out the type based on the assignment, we can use "var" as a shortcut and let the compiler fill in the rest.
The variable is strongly typed. This is not a variant or dynamic variable that we might see in other languages. When "message1" is declared and initialized, it is a string, and it will always be a string. If we try to assign an integer to "message1" later in our code, we will get a compiler error.
There is no performance difference between these two lines of code. We can look at the IL (the intermediate language produced by the compiler) to verify this:
This shows the local variables for a particular method. We can see that "var message1" and "string message2" compile down to exactly the same code. This means that there is no performance difference between these 2 statements. All of the work is done at compile-time.
Let's look at an example with a more complex type:
As above, these two lines of code are equivalent. both "persons1" and "persons2" are of type "List<Person>". If we look at the IL that is generated, we see that both variables have the same type:
This type looks a little strange because it has a generic parameter and the assembly and namespaces are listed. But this just means "List<Person>".
Why "var" Exists
So, we've seen how most developers use "var" -- it's a way for us to create variables without having to write out the specific type. But, this isn't a good reason for adding a new keyword to a language; "var" was actually added for a completely different purpose.
With C# 3.0 (.NET 3.5), we got LINQ, Language INtegrated Query, and it's one of the most awesome things that's been added to C#. One thing that was added to make LINQ work well was the concept of anonymous types.
An anonymous type is a new type that is created "on-the-fly". And just like an anonymous delegate, an anonymous type does not have a name (well, it actually does, we just can't see it). Here's an example:
This declares a variable called "anonPerson" and assigns an anonymous type to it. We create an anonymous type by using the "new" keyword and then supplying property names and values inside curly braces. This example creates a type with 2 properties: FirstName and LastName.
This is a situation where we must use "var". Why? Well, think about the type of the "anonPerson" variable. What is it? By definition, it is an anonymous type which doesn't have a name, so we really have no way to refer to it. So instead, we use "var" and let the compiler deal with it. We can see how the compiler deals with it by hovering the mouse over the "var" keyword:
This shows us that "anonPerson" is an anonymous type, which has been designated 'a. That type consists of 2 properties: FirstName and LastName. The compiler has figured out these are both strings because we assigned strings to the properties.
Now, most of the time, we don't go around creating anonymous types. But, if we use projections with LINQ, we can easily end up with anonymous types. Let's take a look at some more code. First, we have some variables:
Again, these 2 lines of code are equivalent. The "People.GetPeople()" method returns a populated List consisting of 7 Person objects. The "Person" type is pretty simple; just 4 properties:
Next, let's query one of our variables using LINQ:
This will query the "people2" variable and find all of the items where the "FirstName" property is "John". Then in the "select" portion, it creates a new anonymous type based on 2 of the properties of the "Person" class. This is known as a projection, where we take the values of one type (or multiple types) and map them to a new type.
If we hover the mouse over the "var", we can see the actual type:
We get an "IEnumerable<T>" back from this query (which is pretty normal). And if we look, we see that the "T" is actually an anonymous type. This properties for this type happen to match the anonymous type that we created manually above: FirstName and LastName.
So, we can see that when we do projections in LINQ that result in anonymous types, we must use the "var" keyword since the anonymous type does not have a name.
Restrictions to Using "var"
Because of how "var" is implemented, there are a few restrictions on where we can use "var". These will result in compiler errors if we don't adhere to them.
First, we can only use "var" in a local scope where the variable is declared and initialized in a single statement. This makes sense if we think about it. Since the variable is strongly typed, the compiler needs to know what type it is when it is declared. It figured out the type by looking at the value that is being assigned. If the variable is not immediately initialized, then the compiler cannot determine the type.
Second, "var" cannot be used for class-level fields. This is something that people often try to do. "var" is pretty much limited to variables that are locally scoped to a method.
There are the restrictions that most people run into problems with. There are a few more restrictions, but we don't run across them as often. For more information, check out the MSDN article on Implicitly Typed Local Variables.
How I Use "var"
What we've seen is that "var" compiles to exactly the same IL as if we are explicit with our types. And we've also seen that there is no performance difference when we use "var". So, the question becomes, when should we use "var" and when should we be explicit?
Note: We're talking about the non-required use of "var" in declaring variables like in our first example. We've already seen that we must use "var" with anonymous types.
There are many schools of thought on this. Since there is no technical difference, it really comes down to a matter of preference. I've heard of some developers who say that we should use "var" wherever we can (meaning, wherever the restrictions allow us to use "var"). The thinking behind this is that if the types need to change in the future, then the code that declares the variables does not need to be updated. I understand the thinking behind this, but I don't necessarily agree with it.
Focus on Readability
Whenever I use "var", I think about the readability of my code. I'm a big proponent of making code that is easy to approach, follow, maintain, and debug (you can check out my materials on Clean Code if you need convincing of that).
So, let's go back to our examples from above, and I'll tell you where I do use "var" and where I don't.
For this example, I do use "var". For the "person1" variable, I can easily tell the type because it is stated in the assignment. This is a "List<Person>" variable.
In my view, "person2" is a bit harder to read (this is my subjective viewpoint, of course). Since "List<Person>" is repeated on both sides of the assignment operator, it just gets in the way of me picking out the important bits of this statement. This is even more pronounced when we have longer types, like "Dictionary<int, Person>".
Here's another example:
In this situation, I do not use "var". For the "people1" variable, it is not immediately clear what the type is. I need to know the return type of the "GetPeople()" method. It is easy to get that value, by hovering the mouse over either "GetPeople" or "var", but that's an extra step.
For the "people2" variable, I don't have to take any additional steps. I can see that the type is "List<Person>" right in the code.
These are just my views on the subject. If you are in a team environment, it is more important that everyone on the team takes the same approach. If we all do the same thing, then other developers know what to expect when they open up our code.
Wrap Up
I use "var" pretty unconsciously at this point. And that's probably why I get asked about it so much. When we're new to "var", we might think that it creates a variant or dynamic variable. Or we might think that there is some technical reason why we should or should not use it.
But the reality is that "var" does not have any effect on the compiled output. The variables are still strongly typed, and the IL that is generated is exactly the same as if we put in the explicit type. There are a few places where we must use "var" (such as with anonymous types), but most often "var" is simply used as a shortcut. Since the compiler can figure out what the actual type is, we don't have to put that in.
The primary thing we need to keep in mind when we're writing code is that someone else will have to read it. So, we want to make sure that when we do use these types of shortcuts, we don't diminish the readability of our code.
Happy Coding!
This whole discussion has been spurred by some questions that I've received from a few developers recently:
Does using "var" affect performance?
Does "var" indicate a variant type that we have in other languages?
Let's answer these questions by looking at how "var" works.
"var" Is Just a Shortcut
(The way that most developers use it)
Where we most often see "var", it is just a shortcut to writing out the type for a variable. The idea is that the compiler can figure out the actual type based on what we are assigning to the variable, so we don't have to be explicit about the type. Let's start with a simple example:
We can see that "message1" is strongly typed by putting our mouse over the "var" keyword:
The variable is strongly typed. This is not a variant or dynamic variable that we might see in other languages. When "message1" is declared and initialized, it is a string, and it will always be a string. If we try to assign an integer to "message1" later in our code, we will get a compiler error.
There is no performance difference between these two lines of code. We can look at the IL (the intermediate language produced by the compiler) to verify this:
Let's look at an example with a more complex type:
Why "var" Exists
So, we've seen how most developers use "var" -- it's a way for us to create variables without having to write out the specific type. But, this isn't a good reason for adding a new keyword to a language; "var" was actually added for a completely different purpose.
With C# 3.0 (.NET 3.5), we got LINQ, Language INtegrated Query, and it's one of the most awesome things that's been added to C#. One thing that was added to make LINQ work well was the concept of anonymous types.
An anonymous type is a new type that is created "on-the-fly". And just like an anonymous delegate, an anonymous type does not have a name (well, it actually does, we just can't see it). Here's an example:
This is a situation where we must use "var". Why? Well, think about the type of the "anonPerson" variable. What is it? By definition, it is an anonymous type which doesn't have a name, so we really have no way to refer to it. So instead, we use "var" and let the compiler deal with it. We can see how the compiler deals with it by hovering the mouse over the "var" keyword:
Now, most of the time, we don't go around creating anonymous types. But, if we use projections with LINQ, we can easily end up with anonymous types. Let's take a look at some more code. First, we have some variables:
Next, let's query one of our variables using LINQ:
If we hover the mouse over the "var", we can see the actual type:
So, we can see that when we do projections in LINQ that result in anonymous types, we must use the "var" keyword since the anonymous type does not have a name.
Restrictions to Using "var"
Because of how "var" is implemented, there are a few restrictions on where we can use "var". These will result in compiler errors if we don't adhere to them.
First, we can only use "var" in a local scope where the variable is declared and initialized in a single statement. This makes sense if we think about it. Since the variable is strongly typed, the compiler needs to know what type it is when it is declared. It figured out the type by looking at the value that is being assigned. If the variable is not immediately initialized, then the compiler cannot determine the type.
Second, "var" cannot be used for class-level fields. This is something that people often try to do. "var" is pretty much limited to variables that are locally scoped to a method.
There are the restrictions that most people run into problems with. There are a few more restrictions, but we don't run across them as often. For more information, check out the MSDN article on Implicitly Typed Local Variables.
How I Use "var"
What we've seen is that "var" compiles to exactly the same IL as if we are explicit with our types. And we've also seen that there is no performance difference when we use "var". So, the question becomes, when should we use "var" and when should we be explicit?
Note: We're talking about the non-required use of "var" in declaring variables like in our first example. We've already seen that we must use "var" with anonymous types.
There are many schools of thought on this. Since there is no technical difference, it really comes down to a matter of preference. I've heard of some developers who say that we should use "var" wherever we can (meaning, wherever the restrictions allow us to use "var"). The thinking behind this is that if the types need to change in the future, then the code that declares the variables does not need to be updated. I understand the thinking behind this, but I don't necessarily agree with it.
Focus on Readability
Whenever I use "var", I think about the readability of my code. I'm a big proponent of making code that is easy to approach, follow, maintain, and debug (you can check out my materials on Clean Code if you need convincing of that).
So, let's go back to our examples from above, and I'll tell you where I do use "var" and where I don't.
In my view, "person2" is a bit harder to read (this is my subjective viewpoint, of course). Since "List<Person>" is repeated on both sides of the assignment operator, it just gets in the way of me picking out the important bits of this statement. This is even more pronounced when we have longer types, like "Dictionary<int, Person>".
Here's another example:
For the "people2" variable, I don't have to take any additional steps. I can see that the type is "List<Person>" right in the code.
These are just my views on the subject. If you are in a team environment, it is more important that everyone on the team takes the same approach. If we all do the same thing, then other developers know what to expect when they open up our code.
Wrap Up
I use "var" pretty unconsciously at this point. And that's probably why I get asked about it so much. When we're new to "var", we might think that it creates a variant or dynamic variable. Or we might think that there is some technical reason why we should or should not use it.
But the reality is that "var" does not have any effect on the compiled output. The variables are still strongly typed, and the IL that is generated is exactly the same as if we put in the explicit type. There are a few places where we must use "var" (such as with anonymous types), but most often "var" is simply used as a shortcut. Since the compiler can figure out what the actual type is, we don't have to put that in.
The primary thing we need to keep in mind when we're writing code is that someone else will have to read it. So, we want to make sure that when we do use these types of shortcuts, we don't diminish the readability of our code.
Happy Coding!
Tuesday, February 18, 2014
1000 Ratings for C# Interfaces
My C# Interfaces course on Pluralsight hit a milestone today: 1000 ratings:
Thanks to everyone who took the time to watch the course and rate it. Be sure to check out my other courses as well: http://pluralsight.com/training/Authors/Details/jeremy-clark
Happy Coding!
Happy Coding!
Sunday, February 16, 2014
Jeremy's Screencast Production: Hardware, Software, and Process
Several people have asked about the screencast setup that I use for Pluralsight and YouTube: software, hardware, process. Just so I don't have to write the same email again, I'm going to put everything here. I've been using the same tools for the last 6 months or so, so I've had a chance to "dial in" what works for me.
Let's take a look at what I do and the tools I use. I'm not getting any kick-backs from the product vendors, and I'll provide Amazon links for the hardware just because it's easy (no kick-backs from Amazon, either).
Software
We'll start by looking at the software that I use and what I like about it.
Screen Capture
For screen capture, I use Camtasia Recorder 8 (which is part of Camtasia Studio from TechSmith). They have both a Windows and a Mac version. I use the Windows version, so I'll only speak to that.
Here's a screenshot of Camtasia Recorder:
Camtasia Recorder for screen capture and Camtasia Studio for editing go hand-in-hand. The features that are available in Recorder wouldn't be very useful to other video editing applications. You can see that there are options to determine which parts of the screen you will capture as well as whether you're capturing the webcam and audio.
[Note: I've since found out that some people crack open the .camrec file that is generated by Camtasia Recorder and use the individual elements with different editing tools.]
In addition, the audio options allow you to capture system sounds (or not), and the cursor is captured separately so that you can edit it later. All very useful.
I haven't used the webcam recording in a released video (yet). You can thank me later for not having to look at me.
Video Editing
As mentioned above, I use Camtasia Studio for editing. My needs are pretty simple, so I haven't found that I really wanted something that wasn't there (other than "automatically fix the audio" which I'll talk about below).
Here's a screenshot (which looks better on a big monitor with lots of stuff showing/zoomed in):
I won't go over all the features of Camtasia; you can learn about that from the website. But there are a few things that I really like and I use frequently.
As mentioned in the "Screen Capture" section, Camtasia Recorder captures the mouse as a separate object. This means that we can remove it or add effects during the editing process:
I don't generally do too much with the cursor. Primarily, I uncheck the "Mouse cursor visible" option during the slide portions of my videos. I really like that I don't have to think about this during recording, and I can easily fix it during editing.
You can also add "click" effects so that there is a visual change when the mouse is clicked. I don't generally use these, but I see how it could be very useful in a product tutorial where you're showing someone how to use an application.
I also spend quite a bit of time with the Audio tab:
I don't use the processing options (like the volume leveling or noise removal); I don't really have the problems that these options solve. But I do use the "Silence" button quite a bit as well as the "Volume down" button. I'll talk more about these when I go through the production process.
Finally, I really like the Callouts that are available:
I frequently use the "Spotlight" option (which is shown here). And I also use the arrows quite a bit. These are really useful for drawing the viewer's eye to just the right thing on the screen. I've also done a bit of animation with the arrows (to move an arrow around the screen to point to different things). But I usually don't get too complex with my video effects.
And there are lots of other features available as well, including zooming, scene transitions, and narration.
Video Rendering
Camtasia also offers quite a few options when rendering video. You can render to various formats, different bitrates and resolutions, and you can also add a watermark to the video during the rendering process (this is how I get the Pluralsight logo in the bottom right corner of those videos).
Camtasia will also let you do batch rendering of multiple projects. This is really good for something like Pluralsight where I usually have 7 to 10 separate clips that make up a module.
Hardware
Here's a picture of my "studio":
As you can see, things are pretty simple. The piece of hardware that makes the biggest difference to the quality of the screencast is the microphone.
Microphone
I use the RØde Podcaster microphone along with a swing arm and shock mount (Amazon link). This is a USB mic that plugs straight into my laptop, and I'll have to say that this is the best purchase that I've ever made. To demonstrate the difference that good audio makes, compare the following videos:
Laptop mic: C# Properties
Podcaster: Anatomy of a Lambda Expression
Now you might say that I should have used a headset mic instead of the built-in laptop mic, but it turned out that the mic built in to my MacBook Air was actually better quality that the Logitech headset that I had. Either way, the Podcaster is way ahead of the other options.
I also know a lot of screencasters who use the Blue Yeti (Amazon link) and are happy with the results. Again, you'll probably also want a swing arm (for convenience) and a shock mount to isolate the microphone from vibrations from the keyboard. You may also want a pop filter (the Podcaster has an integrated pop filter which works pretty well).
Swing Arm / Work Table
The microphone is attached to a table that is 40 inches tall. This is a good height for me. And because the microphone is mounted on a swing arm, I have the option of recording standing up or sitting down (on a barstool). Being able to change positions is especially welcome when doing all-day recording sessions.
The swing arm is also good because I can simply move the microphone out of the way when it comes time to edit. With my setup, I can move the microphone so that it sits behind the monitor.
Shuttle Control
My newest piece of hardware is a shuttle control. I have the ShuttlePRO V.2 from Contour Design (Amazon link). I've only used this on one video so far (C# Generics Part 1), but this has made a *huge* difference. One of my biggest frustrations in editing was getting exactly the right frame (especially when trying to adjust audio or edit portions of video). The mouse is too fiddly, and the keyboard behaves inconsistently depending on what has focus. The shuttle control eliminates that frustration.
[Update: I've used this shuttle control to edit about a dozen videos now. It's a lifesaver for navigating through the timeline.]
I picked this one based on a blog entry from Scott Hanselman (Human Computer Interaction Diversity - 7 Devices that AREN'T a mouse and keyboard). He points out that Tod created a Camtasia profile for this device (Camtasia Studio and the ShuttlePro V2). I'm sure that I'll be customizing the buttons for the things that work for me, but I've found these settings to be a really good starting point.
[Update 03/03/2014: It turns out that Tod is Tod Gentille, another Pluralsight author, and I got to meet him at the Pluralsight Author's Summit this weekend. He's a great guy, and I was happy to be able to thank him in person for creating these settings.]
Laptop
This stuff plugs into a 13" MacBook Air that's a couple years old (i7-2667M and 4GB RAM). I haven't had any problems with video capture, audio capture, or performance -- no stuttering or other craziness. One big advantage of this laptop is the SSD; I've been told that having an SSD (opposed to a spinning hard drive) makes a huge difference in overall editing and rendering experience.
Rendering is acceptable. I generally record and output at 1024x768, and rendering times are about 1 to 1 (meaning 1 minute of video takes about 1 minute to render). The CPUs max out during rendering, so make sure your computer is well-ventilated during this process.
Process
Finally, I'll talk about my process. This is what works for me. I have published about 20 hours of finished video over the last 6 months, and I've done a bit of tweaking of my process along the way.
Preparation
Preparing for a video takes a bit of time. This is the "creative" part of the process. And this is where I plan, create slides and put together the demo code.
I often get asked about whether I write a script. The answer is, I have a partial script. For the "slide" portion of the videos, I put together a script. I'm pretty good with live presentations, but for some reason, I trip over myself when I'm trying to talk slides during a screencast. So for this portion, I script out what I'm going to say.
For the "demo" portion of the videos, I have an outline of the points I want to hit along with any code snippets that I'm going to type. I'm really good at talking coherently while I'm coding (for some strange reason), so I don't need a full script. In addition, a full script would probably be distracting since I couldn't read from the script and type code at the same time.
Recording
It took me a while to "dial in" the settings for my microphone. I spent a bit of time playing with the recording volume settings and varying my distance from the mic. If it was too high, then I got a lot of background noise. If recording volume was too low, then I had to be really close to the mic to be picked up. I ended up with a setting of 90 and a distance of 2 inches from the microphone.
I got a really good tip from another screencaster. To help you keep a consistent distance from the microphone, attach a gauge of some sort. I have a chopstick rubber-banded to the side of the microphone that sticks out 2 inches. This helps me keep a consistent distance during the recording process (with the Podcaster, you speak into the end of the mic). Here's a picture of that:
As far as the recording process itself, I will do separate recordings for "slides" and "demos". And for Pluralsight where I may have multiple "slide" clips in a row, I record each clip separately.
When I'm recording, I don't stop recording if I make a mistake. I simply take a breath and try the line again. For slides, this isn't too difficult. Sometimes I need to backup a slide or hit the back arrow to reset an animation, but it's pretty easy to keep going.
For demos, it gets a little trickier. Since I started with live presentations before I did screencasts, I had a tendency to point to things with the mouse. But with screencasting, I "point" to things during the editing process by adding arrows and callouts. So, I generally try to stay "hands off the mouse" as much as possible during recording. This also makes it easier to do clean edits when I'm fixing mistakes.
Now, I'm pretty fortunate: I can talk and type at the same time. So, I record the audio and demo all at once. I do make mistakes (a lot). I'm often backing up to a previous point in the code to try a line again. If you don't let the mistakes frustrate you, it's pretty easy to keep going. That's what editing is for; it doesn't have to be a perfect recording.
If you can't talk and type at the same time, you can always add narration later on during the editing process. I know of a couple of screencasters who do this.
Editing
I spend the bulk of my time in editing. In order to try to streamline the process, I've adapted to making 3 editing passes.
Pass #1: Cut To Length
The first pass is cutting out all of the mistakes. This step takes a long time as I go through and cut out the "dead air" when I pause to get the right words, and also when I need to cut out the mistakes. I find that I cut out about 30% of the raw footage during this pass.
For example, the C# Generics Part 1 video was a little over 25 minutes of raw footage that I cut down to 14 minutes of completed video. For my Dependency Injection On-Ramp course for Pluralsight, I had 3 hours 48 minutes of raw video that was cut down to 2 hours and 37 minutes of final footage. I've found that this is pretty consistent, and I can estimate the final video length based on the raw footage. (This is all based on my recording quirks and mistakes, of course.)
Pass #2: Audio Adjustment
I would love to have a "fix audio" button. Like I said, I spent a bit of time dialing in the settings for my microphone to get the cleanest recording that I can, but I still need to do some audio editing.
I don't get any background noise in my setup, but I do have smacking noises that come from my lips and gums. This is generally when I open my mouth to start a phrase. I still haven't figured out what causes this, and I've tried a few different things (wet mouth, dry mouth), and a friend of mine even sent me an interesting link about it: http://filmsound.org/QA/mouthclick.htm.
I've learned to see these extraneous noises in the audio waveform, and I can select and silence them pretty easily. I'll also remove breathing sounds, but I don't need to do this very often. After enough experience recording, I find myself turning my head away from the mic if I need to take a big breath.
I don't have much of a problem with "pops" (hard "P" or "H" sounds). As mentioned above, the Podcaster has a built-in pop filter, and that works pretty well. I get an occasional "pop", and these are easy to fix by locating the sound (usually just 1 frame) and then clicking the "Volume down" button about 5 times. These are fairly rare in my recordings, though.
I have tried the noise removal tools in Camtasia, but they don't seem to pick up the noises that I'm concerned about. And if I change the settings so that they do get picked up, then it alters the rest of the audio as well. So for the time being, I'm sticking with the manual process even though it's a bit more work.
The reason the "Audio Adjustment" pass is separate from the "Cut To Length" pass is that I found I was spending time fixing audio on segments that I later cut out. That was a waste of time. And I also found that it was easier for me to concentrate on one thing at a time.
Pass #3: Video Effects
The last step of the process is adding the video effects. This includes simple things like transitions between "slides" and "demo" and also adding the callouts and animation. This is usually the easiest and fastest editing pass because I generally don't add effects to "slide" portions, and I limit my effects during "demo" portions to highlighting important elements.
Rendering
Rendering is pretty straight-forward. As mentioned above, my laptop renders at about a 1 to 1 ratio. And I have the common rendering settings saved in Camtasia. So, this step is usually just hitting the "Produce and share" button, picking the saved setting, and choosing an output location.
Rendering with Camtasia has been rock-solid. But I will give you a warning if you output to WMV files (which I do): Windows Media Player has a problem playing WMVs (yikes!). So, when you're watching the output in Media Player, you may see "smearing" of the cursor or multiple pointers on the screen at the same time. But don't worry, if you view the video with another player (like VLC) or upload it to YouTube, it will look just fine.
Wrap Up
I've actually been wanting to do screencasts for quite a while. I'm glad that I finally started in earnest last year. I'm hoping to fill up my YouTube channel with lots of good stuff this year (and I'll have some new Pluralsight courses as well).
So, now you know how I produce a screencast. These are the things that work for me. Hopefully, you'll be able to use some of these tips in your own productions.
Happy Coding!
Let's take a look at what I do and the tools I use. I'm not getting any kick-backs from the product vendors, and I'll provide Amazon links for the hardware just because it's easy (no kick-backs from Amazon, either).
Software
We'll start by looking at the software that I use and what I like about it.
Screen Capture
For screen capture, I use Camtasia Recorder 8 (which is part of Camtasia Studio from TechSmith). They have both a Windows and a Mac version. I use the Windows version, so I'll only speak to that.
Here's a screenshot of Camtasia Recorder:
Camtasia Recorder for screen capture and Camtasia Studio for editing go hand-in-hand. The features that are available in Recorder wouldn't be very useful to other video editing applications. You can see that there are options to determine which parts of the screen you will capture as well as whether you're capturing the webcam and audio.
[Note: I've since found out that some people crack open the .camrec file that is generated by Camtasia Recorder and use the individual elements with different editing tools.]
In addition, the audio options allow you to capture system sounds (or not), and the cursor is captured separately so that you can edit it later. All very useful.
I haven't used the webcam recording in a released video (yet). You can thank me later for not having to look at me.
Video Editing
As mentioned above, I use Camtasia Studio for editing. My needs are pretty simple, so I haven't found that I really wanted something that wasn't there (other than "automatically fix the audio" which I'll talk about below).
Here's a screenshot (which looks better on a big monitor with lots of stuff showing/zoomed in):
I won't go over all the features of Camtasia; you can learn about that from the website. But there are a few things that I really like and I use frequently.
As mentioned in the "Screen Capture" section, Camtasia Recorder captures the mouse as a separate object. This means that we can remove it or add effects during the editing process:
I don't generally do too much with the cursor. Primarily, I uncheck the "Mouse cursor visible" option during the slide portions of my videos. I really like that I don't have to think about this during recording, and I can easily fix it during editing.
You can also add "click" effects so that there is a visual change when the mouse is clicked. I don't generally use these, but I see how it could be very useful in a product tutorial where you're showing someone how to use an application.
I also spend quite a bit of time with the Audio tab:
I don't use the processing options (like the volume leveling or noise removal); I don't really have the problems that these options solve. But I do use the "Silence" button quite a bit as well as the "Volume down" button. I'll talk more about these when I go through the production process.
Finally, I really like the Callouts that are available:
I frequently use the "Spotlight" option (which is shown here). And I also use the arrows quite a bit. These are really useful for drawing the viewer's eye to just the right thing on the screen. I've also done a bit of animation with the arrows (to move an arrow around the screen to point to different things). But I usually don't get too complex with my video effects.
And there are lots of other features available as well, including zooming, scene transitions, and narration.
Video Rendering
Camtasia also offers quite a few options when rendering video. You can render to various formats, different bitrates and resolutions, and you can also add a watermark to the video during the rendering process (this is how I get the Pluralsight logo in the bottom right corner of those videos).
Camtasia will also let you do batch rendering of multiple projects. This is really good for something like Pluralsight where I usually have 7 to 10 separate clips that make up a module.
Hardware
Here's a picture of my "studio":
As you can see, things are pretty simple. The piece of hardware that makes the biggest difference to the quality of the screencast is the microphone.
Microphone
I use the RØde Podcaster microphone along with a swing arm and shock mount (Amazon link). This is a USB mic that plugs straight into my laptop, and I'll have to say that this is the best purchase that I've ever made. To demonstrate the difference that good audio makes, compare the following videos:
Laptop mic: C# Properties
Podcaster: Anatomy of a Lambda Expression
Now you might say that I should have used a headset mic instead of the built-in laptop mic, but it turned out that the mic built in to my MacBook Air was actually better quality that the Logitech headset that I had. Either way, the Podcaster is way ahead of the other options.
I also know a lot of screencasters who use the Blue Yeti (Amazon link) and are happy with the results. Again, you'll probably also want a swing arm (for convenience) and a shock mount to isolate the microphone from vibrations from the keyboard. You may also want a pop filter (the Podcaster has an integrated pop filter which works pretty well).
Swing Arm / Work Table
The microphone is attached to a table that is 40 inches tall. This is a good height for me. And because the microphone is mounted on a swing arm, I have the option of recording standing up or sitting down (on a barstool). Being able to change positions is especially welcome when doing all-day recording sessions.
The swing arm is also good because I can simply move the microphone out of the way when it comes time to edit. With my setup, I can move the microphone so that it sits behind the monitor.
Shuttle Control
My newest piece of hardware is a shuttle control. I have the ShuttlePRO V.2 from Contour Design (Amazon link). I've only used this on one video so far (C# Generics Part 1), but this has made a *huge* difference. One of my biggest frustrations in editing was getting exactly the right frame (especially when trying to adjust audio or edit portions of video). The mouse is too fiddly, and the keyboard behaves inconsistently depending on what has focus. The shuttle control eliminates that frustration.
[Update: I've used this shuttle control to edit about a dozen videos now. It's a lifesaver for navigating through the timeline.]
I picked this one based on a blog entry from Scott Hanselman (Human Computer Interaction Diversity - 7 Devices that AREN'T a mouse and keyboard). He points out that Tod created a Camtasia profile for this device (Camtasia Studio and the ShuttlePro V2). I'm sure that I'll be customizing the buttons for the things that work for me, but I've found these settings to be a really good starting point.
[Update 03/03/2014: It turns out that Tod is Tod Gentille, another Pluralsight author, and I got to meet him at the Pluralsight Author's Summit this weekend. He's a great guy, and I was happy to be able to thank him in person for creating these settings.]
Laptop
This stuff plugs into a 13" MacBook Air that's a couple years old (i7-2667M and 4GB RAM). I haven't had any problems with video capture, audio capture, or performance -- no stuttering or other craziness. One big advantage of this laptop is the SSD; I've been told that having an SSD (opposed to a spinning hard drive) makes a huge difference in overall editing and rendering experience.
Rendering is acceptable. I generally record and output at 1024x768, and rendering times are about 1 to 1 (meaning 1 minute of video takes about 1 minute to render). The CPUs max out during rendering, so make sure your computer is well-ventilated during this process.
Process
Finally, I'll talk about my process. This is what works for me. I have published about 20 hours of finished video over the last 6 months, and I've done a bit of tweaking of my process along the way.
Preparation
Preparing for a video takes a bit of time. This is the "creative" part of the process. And this is where I plan, create slides and put together the demo code.
I often get asked about whether I write a script. The answer is, I have a partial script. For the "slide" portion of the videos, I put together a script. I'm pretty good with live presentations, but for some reason, I trip over myself when I'm trying to talk slides during a screencast. So for this portion, I script out what I'm going to say.
For the "demo" portion of the videos, I have an outline of the points I want to hit along with any code snippets that I'm going to type. I'm really good at talking coherently while I'm coding (for some strange reason), so I don't need a full script. In addition, a full script would probably be distracting since I couldn't read from the script and type code at the same time.
Recording
It took me a while to "dial in" the settings for my microphone. I spent a bit of time playing with the recording volume settings and varying my distance from the mic. If it was too high, then I got a lot of background noise. If recording volume was too low, then I had to be really close to the mic to be picked up. I ended up with a setting of 90 and a distance of 2 inches from the microphone.
I got a really good tip from another screencaster. To help you keep a consistent distance from the microphone, attach a gauge of some sort. I have a chopstick rubber-banded to the side of the microphone that sticks out 2 inches. This helps me keep a consistent distance during the recording process (with the Podcaster, you speak into the end of the mic). Here's a picture of that:
As far as the recording process itself, I will do separate recordings for "slides" and "demos". And for Pluralsight where I may have multiple "slide" clips in a row, I record each clip separately.
When I'm recording, I don't stop recording if I make a mistake. I simply take a breath and try the line again. For slides, this isn't too difficult. Sometimes I need to backup a slide or hit the back arrow to reset an animation, but it's pretty easy to keep going.
For demos, it gets a little trickier. Since I started with live presentations before I did screencasts, I had a tendency to point to things with the mouse. But with screencasting, I "point" to things during the editing process by adding arrows and callouts. So, I generally try to stay "hands off the mouse" as much as possible during recording. This also makes it easier to do clean edits when I'm fixing mistakes.
Now, I'm pretty fortunate: I can talk and type at the same time. So, I record the audio and demo all at once. I do make mistakes (a lot). I'm often backing up to a previous point in the code to try a line again. If you don't let the mistakes frustrate you, it's pretty easy to keep going. That's what editing is for; it doesn't have to be a perfect recording.
If you can't talk and type at the same time, you can always add narration later on during the editing process. I know of a couple of screencasters who do this.
Editing
I spend the bulk of my time in editing. In order to try to streamline the process, I've adapted to making 3 editing passes.
Pass #1: Cut To Length
The first pass is cutting out all of the mistakes. This step takes a long time as I go through and cut out the "dead air" when I pause to get the right words, and also when I need to cut out the mistakes. I find that I cut out about 30% of the raw footage during this pass.
For example, the C# Generics Part 1 video was a little over 25 minutes of raw footage that I cut down to 14 minutes of completed video. For my Dependency Injection On-Ramp course for Pluralsight, I had 3 hours 48 minutes of raw video that was cut down to 2 hours and 37 minutes of final footage. I've found that this is pretty consistent, and I can estimate the final video length based on the raw footage. (This is all based on my recording quirks and mistakes, of course.)
Pass #2: Audio Adjustment
I would love to have a "fix audio" button. Like I said, I spent a bit of time dialing in the settings for my microphone to get the cleanest recording that I can, but I still need to do some audio editing.
I don't get any background noise in my setup, but I do have smacking noises that come from my lips and gums. This is generally when I open my mouth to start a phrase. I still haven't figured out what causes this, and I've tried a few different things (wet mouth, dry mouth), and a friend of mine even sent me an interesting link about it: http://filmsound.org/QA/mouthclick.htm.
I've learned to see these extraneous noises in the audio waveform, and I can select and silence them pretty easily. I'll also remove breathing sounds, but I don't need to do this very often. After enough experience recording, I find myself turning my head away from the mic if I need to take a big breath.
I don't have much of a problem with "pops" (hard "P" or "H" sounds). As mentioned above, the Podcaster has a built-in pop filter, and that works pretty well. I get an occasional "pop", and these are easy to fix by locating the sound (usually just 1 frame) and then clicking the "Volume down" button about 5 times. These are fairly rare in my recordings, though.
I have tried the noise removal tools in Camtasia, but they don't seem to pick up the noises that I'm concerned about. And if I change the settings so that they do get picked up, then it alters the rest of the audio as well. So for the time being, I'm sticking with the manual process even though it's a bit more work.
The reason the "Audio Adjustment" pass is separate from the "Cut To Length" pass is that I found I was spending time fixing audio on segments that I later cut out. That was a waste of time. And I also found that it was easier for me to concentrate on one thing at a time.
Pass #3: Video Effects
The last step of the process is adding the video effects. This includes simple things like transitions between "slides" and "demo" and also adding the callouts and animation. This is usually the easiest and fastest editing pass because I generally don't add effects to "slide" portions, and I limit my effects during "demo" portions to highlighting important elements.
Rendering
Rendering is pretty straight-forward. As mentioned above, my laptop renders at about a 1 to 1 ratio. And I have the common rendering settings saved in Camtasia. So, this step is usually just hitting the "Produce and share" button, picking the saved setting, and choosing an output location.
Rendering with Camtasia has been rock-solid. But I will give you a warning if you output to WMV files (which I do): Windows Media Player has a problem playing WMVs (yikes!). So, when you're watching the output in Media Player, you may see "smearing" of the cursor or multiple pointers on the screen at the same time. But don't worry, if you view the video with another player (like VLC) or upload it to YouTube, it will look just fine.
Wrap Up
I've actually been wanting to do screencasts for quite a while. I'm glad that I finally started in earnest last year. I'm hoping to fill up my YouTube channel with lots of good stuff this year (and I'll have some new Pluralsight courses as well).
So, now you know how I produce a screencast. These are the things that work for me. Hopefully, you'll be able to use some of these tips in your own productions.
Happy Coding!
Friday, February 14, 2014
LINQ and the Functional Approach
I've been diving more and more into functional programming. And the more I learn about it, the more interesting it becomes to me.
What I've recently discovered is that I've been promoting the functional approach for 3-1/2 years. I've been a big fan of LINQ (Language INtegrated Query) in C# since it first released back in C# 3.0. The constructs seemed very natural to me -- and I'm talking about the method syntax here; I keep query syntax in my SQL code.
One of my earliest presentations is Learn to Love Lambas. I first presented this in October 2010, and the original samples used Silverlight. I'm still giving this presentation (but the samples have been revised to use WPF). So, let's see how I've been pushing functional programming without being conscious of it.
The Imperative Method
We start with an imperative method. This is because the primary focus of the presentation is on lambda expressions, so I start out with familiar constructs. Here's that method:
The idea behind this is to show the "captured variables" functionality that we get with lambda expressions (and anonymous methods in general). If you want full details, you can check out the presentation materials: http://www.jeremybytes.com/Demos.aspx#LLL.
Here's the functionality. We have a list box in our UI which is called PersonListBox. We want to refresh the data but maintain the selected item in the list if there is one. So before we refresh the data, we save off the currently selected item with the "selectedPerson" variable. Then we reload the list box in the line where we set the "ItemsSource" property. Then after the new data is loaded, we set the selected item of the list box based on our saved variable.
This code is very imperative. Let's take a closer look at the "foreach" loop where we actually set the selected item:
This code is not all that difficult to parse. We have the "foreach" loop to iterate through all of the items in our list box. Now, the "Items" collection is not strongly-typed. What this means is that when we pull items out, they are of type "object". In our "foreach" statement, we specify that we want a "Person" object instead. Internally, this will result in cast of the "object" to a "Person".
Inside the loop, we use an "if" statement to see if the current item matches our saved "selectedPerson". There isn't a good primary key on this data, so we compare the first name and last name properties. If we do find a match, then we assign this item as the "SelectedItem" of our list box.
The Problems of This Approach
This code operates exactly the way we want it to. The problem that I have is that this is not very approachable to a developer looking at this for the first time -- that's why I just explained it step-by-step. (And step-by-step instructions exactly describes imperative programming.) So, let's ask a couple of questions.
What are we trying to do? Well, our ultimate goal is to set the "SelectedItem" property of our list box. But that code is buried two levels down:
I'd prefer to make what we're really trying to do more obvious. But my questions don't stop there.
What happens if there is more than one match? What happens if there is no match found? We can answer these questions, but we do need to stop and think about things a bit. I'd rather not have to spend those cycles figuring out all the possibilities.
Let's look at an alternate approach.
The Functional Method
Instead of telling the computer what to do (i.e., loop through this collection, look for a match, if you find one then set this value), we'll tell the computer what we want. This is where LINQ comes in.
Here's the code that will replace the "foreach" loop that we have above:
This actually looks like multiple statements because it's broken up into several lines, but this is actually a single statement (notice the one semi-colon at the very end). This chains together a couple of methods, and each method gives us a small piece of functionality that we need. Let's break this down a bit.
As mentioned above, "PersonListBox.Items" is not strongly typed. When we call "OfType<Person>()" on that collection, it returns an IEnumerable<Person> object -- something that is strongly-typed that we can iterate through. This is the equivalent of the cast that we had in the "foreach" loop.
Once we have an IEnumerable<Person>, we can use the LINQ extension methods to do all sorts of cool stuff. I mean a whole lot of cool stuff. If you're not aware of them, spend a little time looking through this list: http://msdn.microsoft.com/en-us/library/system.linq.enumerable_methods(v=vs.110).aspx.
The method that I've picked out is "FirstOrDefault". This will return us an item based on the predicate that we have as the parameter, which is a lambda expression. (Again, for more details on lambdas, check out the presentation materials.)
The Benefits of This Approach
Let's go back to our questions from above and see how the functional approach helps us understand this code more easily (arguably, it's "more easily" after we're familiar with LINQ and lambdas, but I encourage every C# developer to get comfortable with those).
What are we trying to do? What we're trying to do is really obvious in this case; we're setting the "SelectedItem" property of our list box. It's right there at the beginning of the statement.
What happens if there's more than one match? The "FirstOrDefault" method will return the first matching item that it finds. If there is more than one match, the others are ignored.
What happens if there is no match found? The "FirstOrDefault" method will give us a default value if it cannot find a match. In this case, since "Person" is a value type, it will return "null" if there is no match.
Now, obviously, we need to know about the behavior of "FirstOrDefault" to understand this. And this is why we should really learn the LINQ methods. There are a lot of options.
For example, if we wanted to throw an exception if no match was found, then we could use the "First" method which will do just that. If we don't want to allow for more than one match, we can use the "Single" method. "Single" will throw an exception if there is more than one match found or if there is no match found. (There's also a "SingleOrDefault" method that will return "null" if no match is found.)
Hopefully, this will encourage you to go learn the LINQ methods. Here's that link again in case you missed it before: http://msdn.microsoft.com/en-us/library/system.linq.enumerable_methods(v=vs.110).aspx.
How is This More "Functional"?
Why would we consider this to be more "functional" than our original foreach loop? Well, with the "foreach" loop, we are telling the computer what to do. (1) Loop through the collection, (2) cast the items to "Person", (3) check if the current item is a match, (4) if so, assign it to our property.
With the new approach, we are telling the computer what we want. We want an item from the list box items that matches our criteria (which happens to be another method in the form of a lambda expression). Once we have that result, we assign it to the "SelectedItems" property of the list box.
Wrap Up
So, it turns out that I've been promoting functional style since 2010 without even realizing it. This is probably one of the reasons why I'm interested in functional programming -- it's something that I picked up on unconsciously, and my brain really wants to work this way.
I'm going to keep plugging away at this. I'm currently reading Tomas Petricek's book Real-World Functional Programming: With Examples in F# and C# (Amazon link), and I'm looking forward to applying more of these techniques in my everyday programming.
Happy Coding!
What I've recently discovered is that I've been promoting the functional approach for 3-1/2 years. I've been a big fan of LINQ (Language INtegrated Query) in C# since it first released back in C# 3.0. The constructs seemed very natural to me -- and I'm talking about the method syntax here; I keep query syntax in my SQL code.
One of my earliest presentations is Learn to Love Lambas. I first presented this in October 2010, and the original samples used Silverlight. I'm still giving this presentation (but the samples have been revised to use WPF). So, let's see how I've been pushing functional programming without being conscious of it.
The Imperative Method
We start with an imperative method. This is because the primary focus of the presentation is on lambda expressions, so I start out with familiar constructs. Here's that method:
The idea behind this is to show the "captured variables" functionality that we get with lambda expressions (and anonymous methods in general). If you want full details, you can check out the presentation materials: http://www.jeremybytes.com/Demos.aspx#LLL.
Here's the functionality. We have a list box in our UI which is called PersonListBox. We want to refresh the data but maintain the selected item in the list if there is one. So before we refresh the data, we save off the currently selected item with the "selectedPerson" variable. Then we reload the list box in the line where we set the "ItemsSource" property. Then after the new data is loaded, we set the selected item of the list box based on our saved variable.
This code is very imperative. Let's take a closer look at the "foreach" loop where we actually set the selected item:
This code is not all that difficult to parse. We have the "foreach" loop to iterate through all of the items in our list box. Now, the "Items" collection is not strongly-typed. What this means is that when we pull items out, they are of type "object". In our "foreach" statement, we specify that we want a "Person" object instead. Internally, this will result in cast of the "object" to a "Person".
Inside the loop, we use an "if" statement to see if the current item matches our saved "selectedPerson". There isn't a good primary key on this data, so we compare the first name and last name properties. If we do find a match, then we assign this item as the "SelectedItem" of our list box.
The Problems of This Approach
This code operates exactly the way we want it to. The problem that I have is that this is not very approachable to a developer looking at this for the first time -- that's why I just explained it step-by-step. (And step-by-step instructions exactly describes imperative programming.) So, let's ask a couple of questions.
What are we trying to do? Well, our ultimate goal is to set the "SelectedItem" property of our list box. But that code is buried two levels down:
What happens if there is more than one match? What happens if there is no match found? We can answer these questions, but we do need to stop and think about things a bit. I'd rather not have to spend those cycles figuring out all the possibilities.
Let's look at an alternate approach.
The Functional Method
Instead of telling the computer what to do (i.e., loop through this collection, look for a match, if you find one then set this value), we'll tell the computer what we want. This is where LINQ comes in.
Here's the code that will replace the "foreach" loop that we have above:
As mentioned above, "PersonListBox.Items" is not strongly typed. When we call "OfType<Person>()" on that collection, it returns an IEnumerable<Person> object -- something that is strongly-typed that we can iterate through. This is the equivalent of the cast that we had in the "foreach" loop.
Once we have an IEnumerable<Person>, we can use the LINQ extension methods to do all sorts of cool stuff. I mean a whole lot of cool stuff. If you're not aware of them, spend a little time looking through this list: http://msdn.microsoft.com/en-us/library/system.linq.enumerable_methods(v=vs.110).aspx.
The method that I've picked out is "FirstOrDefault". This will return us an item based on the predicate that we have as the parameter, which is a lambda expression. (Again, for more details on lambdas, check out the presentation materials.)
The Benefits of This Approach
Let's go back to our questions from above and see how the functional approach helps us understand this code more easily (arguably, it's "more easily" after we're familiar with LINQ and lambdas, but I encourage every C# developer to get comfortable with those).
What are we trying to do? What we're trying to do is really obvious in this case; we're setting the "SelectedItem" property of our list box. It's right there at the beginning of the statement.
What happens if there's more than one match? The "FirstOrDefault" method will return the first matching item that it finds. If there is more than one match, the others are ignored.
What happens if there is no match found? The "FirstOrDefault" method will give us a default value if it cannot find a match. In this case, since "Person" is a value type, it will return "null" if there is no match.
Now, obviously, we need to know about the behavior of "FirstOrDefault" to understand this. And this is why we should really learn the LINQ methods. There are a lot of options.
For example, if we wanted to throw an exception if no match was found, then we could use the "First" method which will do just that. If we don't want to allow for more than one match, we can use the "Single" method. "Single" will throw an exception if there is more than one match found or if there is no match found. (There's also a "SingleOrDefault" method that will return "null" if no match is found.)
Hopefully, this will encourage you to go learn the LINQ methods. Here's that link again in case you missed it before: http://msdn.microsoft.com/en-us/library/system.linq.enumerable_methods(v=vs.110).aspx.
How is This More "Functional"?
Why would we consider this to be more "functional" than our original foreach loop? Well, with the "foreach" loop, we are telling the computer what to do. (1) Loop through the collection, (2) cast the items to "Person", (3) check if the current item is a match, (4) if so, assign it to our property.
With the new approach, we are telling the computer what we want. We want an item from the list box items that matches our criteria (which happens to be another method in the form of a lambda expression). Once we have that result, we assign it to the "SelectedItems" property of the list box.
Wrap Up
So, it turns out that I've been promoting functional style since 2010 without even realizing it. This is probably one of the reasons why I'm interested in functional programming -- it's something that I picked up on unconsciously, and my brain really wants to work this way.
I'm going to keep plugging away at this. I'm currently reading Tomas Petricek's book Real-World Functional Programming: With Examples in F# and C# (Amazon link), and I'm looking forward to applying more of these techniques in my everyday programming.
Happy Coding!
Thursday, February 13, 2014
Using Build Events in Visual Studio to Make Life Easier
The Build Events options in Visual Studio can make your life much easier. I use them primarily to copy files around after building projects, but we can use them to run any commands that we want. Let's take a look at the options, and then I'll show how I use them.
Update May 2020: With .NET Core, I've stopped using the '$(SolutionDir)' macro (and other macros relating to the solution). Read why here: Post-Build Events and .NET Core. But don't stop reading this article; it has lots of useful information.
Build Events
Build Events are part of the Project options, and they are often overlooked. If we open up the "Properties" for a project, we get the following screen:
This lets us run command-line commands either before the project is built (pre-build events) or after (post-build events). Although, we do get a drop-down to select if the post-build event runs or not. The default value "On successful build" is usually where I keep this value. This way, the post-build event will *not* run if there is a build failure. Other options include "Always" and "When the build updates the project output."
You may be curious about this last option. Visual Studio is lazy about compiling things, so it will only build if the source files have changed. If we select "When the build updates the project output", then the post-build event will only run if the compiler updates the output files on this project.
We can basically put anything in here that we could run from the command line. And Visual Studio provides macros to make it easier. So, even though we can type straight into these text boxes, it's much better to click the "Edit Pre-build..." or "Edit Post-build..." buttons to bring up the dialog.
The Build Event Dialog
The build event dialog is where things start to get interesting. Well, they aren't actually very interesting at first (this dialog is already populated with a command which we'll talk about in just a bit):
But, when we click the "Macros", we can see how Visual Studio can really help us out:
This shows us a number of macros that we can use in our commands. We can add the macros by simply double-clicking on the item. This will add a corresponding placeholder that has the format $(xxx). These macros will be replaced with actual values when the event runs.
Here's a complete list of the available macros (this is from Visual Studio 2013, but this dialog has not changed for quite a while):
Let's take a look at the "TargetDir" macro. In the value column, we see this expanded out to the full path of the output folder.
But here's why using macros is compelling: If I move this project to a different folder, this TargetDir macro will expand to the new location. If I change the configuration from "Release" to "Debug", the macro will expand to the corresponding output folder (bin\Debug\ vs. bin\Release\).
There are a number of other interesting items. And we can get various levels of detail, depending on our needs. For example, look at the various "Project" macros in the list above.
ProjectName is just the name of the project itself
ProjectFileName includes the project name, plus the file extension
ProjectPath includes the fully-qualified path to the project, and includes the project file name/extension
ProjectDir is the fully-qualified path to the project, but does *not* include the project file name
And we can see similar naming schemes for "Target" and "Solution". Lots of cool stuff.
How I Use Post-Build Events
I primarily use post-build events to copy files around. For example, a number of my sample applications use late-binding. This means that certain dlls are not present at compile time; instead, they are referenced dynamically at runtime.
The problem with this is that I would normally need to copy the late-bound dlls into the project output folder (the same folder as the executable) in order for the application to work. Instead of manually copying the files, I use a post-build event to automatically copy them over.
So, when my examples use dynamically-loaded repositories, I can simply copy over the already-compiled assemblies from a solution folder. Here's the "LateBindingRepositories" folder from one of my solutions:
In addition to the assemblies, this also has some additional files. The "People.txt" file is the data file used by the CSV repository (PersonRepository.CSV.dll). The SQL repository (PersonRepository.SQL.dll) uses Entity Framework and also pulls its data from a local database file (People.sdf).
I need to get these files into the output folder for my main WPF application. So, I have a post-build step in that project:
So, let's parse this command.
We're using "xcopy" to copy files from one folder to another. This is how we had to copy files back before Windows (now I feel old). This has 2 main parameters: the source and the destination.
For the source, we are asking for the solution folder with the $(SolutionDir) macro. If we look at the macro list above, we can see how this expands out, and notice that it includes the trailing path delimiter. Then we go to the "LateBindingRepositories" folder and ask for all files (the *.*). This will copy the assemblies (dlls), plus the other files in that folder.
For the destination, we use the output directory with $(TargetDir). This will be the same folder as our executable (in bin\Debug or bin\Release depending on the Solution Configuration).
The "/Y" switch tells xcopy to overwrite any existing files. This is here because xcopy will normally give you an overwrite prompt if it finds an existing file. We can't get that prompt from a build event.
Important Note About Paths and File Names
One important thing to notice about the source and destination paths: they are wrapped in quotes. If there are any spaces in the path or file name, then we need quotes around it so that xcopy knows that this is a single argument (a space delimits arguments on the command line).
I've run into this issue because I don't normally have spaces in my project paths. But then someone copied the project to their machine (which did have spaces in the path) and the post-build step stopped working.
Other Uses
Another way that I've used post-build events is for dynamically loaded modules. Prism (from the Microsoft Patterns & Practices team) makes it really easy to load up modules from a particular folder. This way, we don't need them to all be present at compile time.
So, in each of my modules, I have a post-build step to copy the output to $(SolutionDir)Modules. And then in my shell application (the one that loads the modules), I have a post-build that copies the modules from $(SolutionDir)Modules into the appropriate folder that the application is looking for.
And we can get more creative as well. I'm using post-build events to copy files around. But we can also use them to run applications such as testing tools, linting tools, validation tools, IL re-writers, and obfuscators. This can help us automate these processes on our developer machines (although we would want to put these into a formal process on our build server).
Wrap Up
The Build Events can be a big time saver, especially if you find yourself copying files from one output folder to another. And the macros that are provided by Visual Studio make it really easy to use paths and file names that easily adapt when we move our projects or build with different configurations.
Happy Coding!
Build Events
Build Events are part of the Project options, and they are often overlooked. If we open up the "Properties" for a project, we get the following screen:
This lets us run command-line commands either before the project is built (pre-build events) or after (post-build events). Although, we do get a drop-down to select if the post-build event runs or not. The default value "On successful build" is usually where I keep this value. This way, the post-build event will *not* run if there is a build failure. Other options include "Always" and "When the build updates the project output."
You may be curious about this last option. Visual Studio is lazy about compiling things, so it will only build if the source files have changed. If we select "When the build updates the project output", then the post-build event will only run if the compiler updates the output files on this project.
We can basically put anything in here that we could run from the command line. And Visual Studio provides macros to make it easier. So, even though we can type straight into these text boxes, it's much better to click the "Edit Pre-build..." or "Edit Post-build..." buttons to bring up the dialog.
The Build Event Dialog
The build event dialog is where things start to get interesting. Well, they aren't actually very interesting at first (this dialog is already populated with a command which we'll talk about in just a bit):
But, when we click the "Macros", we can see how Visual Studio can really help us out:
This shows us a number of macros that we can use in our commands. We can add the macros by simply double-clicking on the item. This will add a corresponding placeholder that has the format $(xxx). These macros will be replaced with actual values when the event runs.
Here's a complete list of the available macros (this is from Visual Studio 2013, but this dialog has not changed for quite a while):
Let's take a look at the "TargetDir" macro. In the value column, we see this expanded out to the full path of the output folder.
But here's why using macros is compelling: If I move this project to a different folder, this TargetDir macro will expand to the new location. If I change the configuration from "Release" to "Debug", the macro will expand to the corresponding output folder (bin\Debug\ vs. bin\Release\).
There are a number of other interesting items. And we can get various levels of detail, depending on our needs. For example, look at the various "Project" macros in the list above.
ProjectName is just the name of the project itself
ProjectFileName includes the project name, plus the file extension
ProjectPath includes the fully-qualified path to the project, and includes the project file name/extension
ProjectDir is the fully-qualified path to the project, but does *not* include the project file name
And we can see similar naming schemes for "Target" and "Solution". Lots of cool stuff.
How I Use Post-Build Events
I primarily use post-build events to copy files around. For example, a number of my sample applications use late-binding. This means that certain dlls are not present at compile time; instead, they are referenced dynamically at runtime.
The problem with this is that I would normally need to copy the late-bound dlls into the project output folder (the same folder as the executable) in order for the application to work. Instead of manually copying the files, I use a post-build event to automatically copy them over.
So, when my examples use dynamically-loaded repositories, I can simply copy over the already-compiled assemblies from a solution folder. Here's the "LateBindingRepositories" folder from one of my solutions:
In addition to the assemblies, this also has some additional files. The "People.txt" file is the data file used by the CSV repository (PersonRepository.CSV.dll). The SQL repository (PersonRepository.SQL.dll) uses Entity Framework and also pulls its data from a local database file (People.sdf).
I need to get these files into the output folder for my main WPF application. So, I have a post-build step in that project:
So, let's parse this command.
We're using "xcopy" to copy files from one folder to another. This is how we had to copy files back before Windows (now I feel old). This has 2 main parameters: the source and the destination.
For the source, we are asking for the solution folder with the $(SolutionDir) macro. If we look at the macro list above, we can see how this expands out, and notice that it includes the trailing path delimiter. Then we go to the "LateBindingRepositories" folder and ask for all files (the *.*). This will copy the assemblies (dlls), plus the other files in that folder.
For the destination, we use the output directory with $(TargetDir). This will be the same folder as our executable (in bin\Debug or bin\Release depending on the Solution Configuration).
The "/Y" switch tells xcopy to overwrite any existing files. This is here because xcopy will normally give you an overwrite prompt if it finds an existing file. We can't get that prompt from a build event.
Important Note About Paths and File Names
One important thing to notice about the source and destination paths: they are wrapped in quotes. If there are any spaces in the path or file name, then we need quotes around it so that xcopy knows that this is a single argument (a space delimits arguments on the command line).
I've run into this issue because I don't normally have spaces in my project paths. But then someone copied the project to their machine (which did have spaces in the path) and the post-build step stopped working.
Other Uses
Another way that I've used post-build events is for dynamically loaded modules. Prism (from the Microsoft Patterns & Practices team) makes it really easy to load up modules from a particular folder. This way, we don't need them to all be present at compile time.
So, in each of my modules, I have a post-build step to copy the output to $(SolutionDir)Modules. And then in my shell application (the one that loads the modules), I have a post-build that copies the modules from $(SolutionDir)Modules into the appropriate folder that the application is looking for.
And we can get more creative as well. I'm using post-build events to copy files around. But we can also use them to run applications such as testing tools, linting tools, validation tools, IL re-writers, and obfuscators. This can help us automate these processes on our developer machines (although we would want to put these into a formal process on our build server).
Wrap Up
The Build Events can be a big time saver, especially if you find yourself copying files from one output folder to another. And the macros that are provided by Visual Studio make it really easy to use paths and file names that easily adapt when we move our projects or build with different configurations.
Happy Coding!
Tuesday, February 11, 2014
New Video: Generic vs. Non-Generic Collections
I've started a series of videos based on my presentation "T, Earl Grey, Hot: Generics in .NET". The first installments is now available on my YouTube channel: C# Generics - Part 1: Collections.
Generics are easy to take for granted, especially since they've been around since .NET 2.0. In this short clip (14 minutes), we compare a non-generic collection (ArrayList) to a generic collection (List<T>). And we see how the generics give us type safety and make our coding very easy.
Future videos will explore using generics in our own interfaces and methods. We'll see how generics can give us flexibility and reuse that would otherwise require us to write multiple classes and methods. Lots more to come.
For more information on the topic (and to download the code samples), visit my website: T, Earl Grey, Hot: Generics in .NET.
Happy Coding!
Generics are easy to take for granted, especially since they've been around since .NET 2.0. In this short clip (14 minutes), we compare a non-generic collection (ArrayList) to a generic collection (List<T>). And we see how the generics give us type safety and make our coding very easy.
Future videos will explore using generics in our own interfaces and methods. We'll see how generics can give us flexibility and reuse that would otherwise require us to write multiple classes and methods. Lots more to come.
For more information on the topic (and to download the code samples), visit my website: T, Earl Grey, Hot: Generics in .NET.
Happy Coding!
Thursday, February 6, 2014
Book Review: Parallel Programming with Microsoft .NET
I recently finished reading Parallel Programming with Microsoft .NET by Colin Campbell, Ralph Johnson, Ade Miller, and Stephen Toub (Amazon link or read for free online). This book has a rather intimidating subtitle: "Design Patterns for Decomposition and Coordination on Multicore Architectures", but the content is actually very approachable.
I picked up this book to get better familiarity with the Task Parallel Library (TPL), and it gave me the information I was looking for (plus, quite a bit of information that I didn't know I needed).
This is a fairly short book -- just 130 pages, plus the appendices; it is approachable; and it clearly describes the problems that can be solved with parallel programming.
What's Covered
Parallel Programming shows how to implement various scenarios using the Task Parallel Library (TPL) and PLINQ (Parallel Language INtegrated Query). This book is from 2010, so it is from the .NET 4.0 era (after the TPL was added, but before async/await).
What I really like about this book is that it focuses on problems and solutions. It shows several different patterns and ways to implement parallelism in our applications, but it has a primary concern of making sure that we select the solution that matches our problem and environment.
We can get an idea of that by looking at the chapter titles:
The introduction also gives a quick overview of what will be covered in the other chapters -- with a focus on making sure that we select the right parallel pattern for our problem.
Chapter 2 covers parallel loops. This includes Parallel.For, Parallel.ForEach and also AsParallel in PLINQ. It gives the basics of the syntax, but more importantly, it talks about situations were we *can* use these constructs.
We cannot simply change a "for" loop into "Parallel.For" without considering the work that is actually being done inside the loop. This could result in corrupted data (if multiple threads update the same values without proper synchronization) or deadlocks (if multiple threads try to update values with improper synchronization). And it's very likely that even if our code works, we may actually see slower performance than if the code ran sequentially (due to the added overhead of the additional threads).
So, I really appreciate that quite a bit of time is spent in determining whether our code is suitable for a parallel loop. If not, there may be some changes we can make to our code to facilitate it, or there may even be a better pattern for us to look at.
And we don't simply want to add locks to our code to handle synchronization. This can lead to slower performance and deadlocks. It's much better if we can architect our objects and methods so that they do not require synchronization. Instead of using a shared state that needs synchronization, we should look at ways to accomplish the same task without needing that shared state.
Other chapters cover the topics in a similar manner, and there are quite a few references from one chapter to another to explore different options. Just to give you an idea of what's covered in the chapters, here's a screenshot from the introduction (it's in the middle of this page: http://msdn.microsoft.com/en-us/library/ff963542.aspx)
Again, this is a great approach to understand the types of problems that can be addressed with parallel programming and how to make sure we use the right implementation for our problem.
Another thing that I really like: there are clear recommendations and examples on how to handle things like cancellation and exceptions. These things are quite a bit more complicated when we may have things running in parallel.
Parallel Programming vs. Async
Parallel programming and asynchronous programming are 2 different things. But they are related. We most often think of parallel programming as running the same function across multiple cores so that we can enhance performance. (And from the chart above, we see that there are several variations that let us take advantage of parallelism if we have some slightly different requirements.)
Asynchronous programming is generally more about "go do this, and let me know when you're done" without interrupting the flow of the rest of the program. So, we may kick off a long-running calculation on a separate thread so that our UI stays responsive.
But in both of these cases, the result is that we end up using multiple threads (most of the time). And so we see both parallel programming and asynchronous programming implemented with Tasks and the Task Parallel Library.
Last year, I reviewed Async in C# 5.0 (it looks like almost exactly a year ago, actually). And one of the caveats in my review is that you need a good understanding of Tasks before reading it. It turns out that Parallel Programming with Microsoft .NET give you (almost) all the information you need to understand how Tasks are used with async.
Is This Still Relevant?
This book is from 2010, and we've had several important updates and additions to the .NET framework. So, is this book still relevant?
The answer is "Yes". The Task Parallel Library has had a few additions (including the IProgress<T> interface), but the core functionality is still the same, including using a CancellationTokenSource, spinning up new Tasks, and dealing with AggregateExceptions.
I found the Appendix on Adapting Object-Oriented Patterns to be very interesting. I'm a big fan of design patterns (as long as they are used properly). The appendix takes several OO patterns and shows how we may need to modify them to make sure they are thread-safe. For example, the "standard" implementation of the Singleton pattern (which includes a class with a static instance field) can cause problems in a multi-threaded environment. The book provides several ways to make a thread-safe Singleton, including adding synchronization or by using Lazy<T> for the instance. Very cool stuff.
Wrap Up
I'm not actually expecting to dive into parallel programming too deeply. Historically, I haven't had too many business problems that really warranted it. But I do have a much better understanding of the Task Parallel Library and PLINQ. I have been using the TPL more and more in my code (primarily for the asynchronous bits), and this book has given me some good ideas on how I can start expanding into the parallel world.
Parallel Programming with Microsoft .NET is an approachable resource for anyone who is looking to better understand the problems of parallel development and also for developers who simply want a better understanding of the Task Parallel Library for use with async method calls. Personally, I expect to use this information as I dive deeper into functional programming in .NET.
Happy Coding!
I picked up this book to get better familiarity with the Task Parallel Library (TPL), and it gave me the information I was looking for (plus, quite a bit of information that I didn't know I needed).
This is a fairly short book -- just 130 pages, plus the appendices; it is approachable; and it clearly describes the problems that can be solved with parallel programming.
What's Covered
Parallel Programming shows how to implement various scenarios using the Task Parallel Library (TPL) and PLINQ (Parallel Language INtegrated Query). This book is from 2010, so it is from the .NET 4.0 era (after the TPL was added, but before async/await).
What I really like about this book is that it focuses on problems and solutions. It shows several different patterns and ways to implement parallelism in our applications, but it has a primary concern of making sure that we select the solution that matches our problem and environment.
We can get an idea of that by looking at the chapter titles:
- Introduction
- Parallel Loops
- Parallel Tasks
- Parallel Aggregation
- Futures
- Dynamic Task Parallelism
- Pipelines
- Appendix A - Adapting Object-Oriented Patterns
- Appendix B - Debugging and Profiling Parallel Applications
- Appendix C - Technology Overview
The introduction also gives a quick overview of what will be covered in the other chapters -- with a focus on making sure that we select the right parallel pattern for our problem.
Chapter 2 covers parallel loops. This includes Parallel.For, Parallel.ForEach and also AsParallel in PLINQ. It gives the basics of the syntax, but more importantly, it talks about situations were we *can* use these constructs.
We cannot simply change a "for" loop into "Parallel.For" without considering the work that is actually being done inside the loop. This could result in corrupted data (if multiple threads update the same values without proper synchronization) or deadlocks (if multiple threads try to update values with improper synchronization). And it's very likely that even if our code works, we may actually see slower performance than if the code ran sequentially (due to the added overhead of the additional threads).
So, I really appreciate that quite a bit of time is spent in determining whether our code is suitable for a parallel loop. If not, there may be some changes we can make to our code to facilitate it, or there may even be a better pattern for us to look at.
And we don't simply want to add locks to our code to handle synchronization. This can lead to slower performance and deadlocks. It's much better if we can architect our objects and methods so that they do not require synchronization. Instead of using a shared state that needs synchronization, we should look at ways to accomplish the same task without needing that shared state.
Other chapters cover the topics in a similar manner, and there are quite a few references from one chapter to another to explore different options. Just to give you an idea of what's covered in the chapters, here's a screenshot from the introduction (it's in the middle of this page: http://msdn.microsoft.com/en-us/library/ff963542.aspx)
Again, this is a great approach to understand the types of problems that can be addressed with parallel programming and how to make sure we use the right implementation for our problem.
Another thing that I really like: there are clear recommendations and examples on how to handle things like cancellation and exceptions. These things are quite a bit more complicated when we may have things running in parallel.
Parallel Programming vs. Async
Parallel programming and asynchronous programming are 2 different things. But they are related. We most often think of parallel programming as running the same function across multiple cores so that we can enhance performance. (And from the chart above, we see that there are several variations that let us take advantage of parallelism if we have some slightly different requirements.)
Asynchronous programming is generally more about "go do this, and let me know when you're done" without interrupting the flow of the rest of the program. So, we may kick off a long-running calculation on a separate thread so that our UI stays responsive.
But in both of these cases, the result is that we end up using multiple threads (most of the time). And so we see both parallel programming and asynchronous programming implemented with Tasks and the Task Parallel Library.
Last year, I reviewed Async in C# 5.0 (it looks like almost exactly a year ago, actually). And one of the caveats in my review is that you need a good understanding of Tasks before reading it. It turns out that Parallel Programming with Microsoft .NET give you (almost) all the information you need to understand how Tasks are used with async.
Is This Still Relevant?
This book is from 2010, and we've had several important updates and additions to the .NET framework. So, is this book still relevant?
The answer is "Yes". The Task Parallel Library has had a few additions (including the IProgress<T> interface), but the core functionality is still the same, including using a CancellationTokenSource, spinning up new Tasks, and dealing with AggregateExceptions.
I found the Appendix on Adapting Object-Oriented Patterns to be very interesting. I'm a big fan of design patterns (as long as they are used properly). The appendix takes several OO patterns and shows how we may need to modify them to make sure they are thread-safe. For example, the "standard" implementation of the Singleton pattern (which includes a class with a static instance field) can cause problems in a multi-threaded environment. The book provides several ways to make a thread-safe Singleton, including adding synchronization or by using Lazy<T> for the instance. Very cool stuff.
Wrap Up
I'm not actually expecting to dive into parallel programming too deeply. Historically, I haven't had too many business problems that really warranted it. But I do have a much better understanding of the Task Parallel Library and PLINQ. I have been using the TPL more and more in my code (primarily for the asynchronous bits), and this book has given me some good ideas on how I can start expanding into the parallel world.
Parallel Programming with Microsoft .NET is an approachable resource for anyone who is looking to better understand the problems of parallel development and also for developers who simply want a better understanding of the Task Parallel Library for use with async method calls. Personally, I expect to use this information as I dive deeper into functional programming in .NET.
Happy Coding!