MLP: FIM is the best show you’re probably not watching

In case you don’t know, it’s My Little Pony: Friendship Is Magic

It’s a show created by Lauren Faust (fyre-flye on deviantart)

https://i1.wp.com/www.watchcartoononline.com/thumbs/My-Little-Pony-Friendship-Is-Magic-Episode-22.jpg

Random: Last year Bill Clinton aced a quiz about the show.

I’ve started this blog post in my head a few times:

  • I first started it with something like “geek dad with daughter(s)? check out this show!”
  • then I realized it’s not really a ‘girly’ show and the gender discrimination was a bad idea anyway, so it became “geek dad?  check out this show!”
  • however, that was still gender-specific, so it became “geek parent?  check out this show!”
  • but I realize that I like the show and while I probably wouldn’t have known about it without the kids, I still like it 🙂

The name of the show is IMHO the part working the most against it, at least in terms of venturing out into the ‘greater geekdom’ – I’m sure it’s a double-edged sword, since the “My Little Pony” part is what’s going to get a set of people to initially look at it for nostalgia-related reasons.

Much like many animated productions these days, it’s got lots of references to jokes that the parents will get and laugh at that the kids won’t.  Those are as important on this show as they are to Pixar et al.

The 2-part show that I watched that ‘sealed the deal’ for me in terms of really liking this show was the ones to start this season (Season 2) of the show, named “The Return of Harmony”

Why?  Because I’m a fan of ST:TNG and one of my favorite characters there is Q (seriously, how can you not love this character? all-powerful, mischievous, intelligent – great mix!).  You might remember that Q was there right off the bat, as the first 2 episodes of the show were “Encounter at Farpoint” where he’s being, well, Q.  Whether you liked those particular episodes or not (WIl gave both part 1 and part 2 a C-, for instance), I think it’s easy to like the Q character, or at least the concept.

Q portrait.jpg

That’s very much what happens in this 2-part episode.  Here it’s a character named Discord that has broken free, but it’s very much the same as he toys with the group of friends, taking away their powers and putting them into a labyrinth separated from each other.

The MLP:FIM team even got the same actor who played Q (John de Lancie) to play this character, and he nails it just as well as you would expect – the tone, the mischief, the disdain, the chaos.  It was awesome to watch what felt like a crossover as Q was manifested again in this universe and still up to his ‘tricksy’ ways. 🙂 

Of course, I’m certainly not the only one to notice this 🙂

So why are you still reading this post?  Go watch the show!  Oh, and if you have kids, make sure they watch with you. 🙂

Advertisements

great write-up by a fellow introvert

The whole thing is a great read to help give insight, but this particular bit is what I find so hard to get across to others.  Trying to explain why I’m exhausted after things most people wouldn’t think twice about has been difficult.

http://blog.edwardamsden.com/2012/03/plight-of-introvert-rant.html

Any introvert in a group larger than 10 people is exerting tremendous energy to avoid intruding on the enjoyment of those who really do like such groups.

Windows Alt-PrintScreen includes anything on top of the app

A couple of the built-in ‘screen shot’ methods in Windows are:

  • hitting Ctrl + PrtSc (Control key + ‘Print Screen’ key) will take a screen shot of the entire screen (don’t have a multimon in front of me to test whether it’s just current screen or all of them)
  • hitting Alt + PrtSc will take a screen shot of just the active application (or the screen if no active app)

The latter is the one I typically use if I don’t have some other app (OneNote, EverNote, whatever) installed/running for screen clipping, or if I happen to want to take a screen shot of the entire app and not just some part of it.

So starting an instance of notepad and hitting Alt+PrtSc shows its typical result:

image

One behavior I already knew is that if the window is partially off the screen, it’ll only include the visible part – for instance:

image

One thing I didn’t realize before is that it apparently works by basically just doing a copy of rendered video memory (or similar) for the (visible) rectangle defined by the app.

The way I first hit this was having it include the task bar in a screen shot I made, similar to this, where the notepad is much taller, but it includes the higher z-order taskbar which is displayed on top:

image

It’s not just the taskbar – any other app that’s displayed on top (for instance, using User32.dll’s SetWindowPos or the higher-level versions like TopMost in WInForms) will similarly be included:

image

Anyway, I found it interesting. Smile

Async lesson 2 of N: async void probably isn’t what you want

In the previous post you might have noticed that we had a Console.ReadLine near the end of Main – while it’s relatively common for demo code to do so, in this case it was actually necessary for the program to work.

Why’s that?  Well, let’s remove it and see what we get for output:

And here’s what we get when we run it:

…\bin\Debug » .\ItsBigItsHeavyItsWood.exe
Starting to download url
http://www.google.com/
Starting to download url http://www.microsoft.com/
Starting to download url http://www.apple.com/
…\bin\Debug »

Wait, what happened?  We started the 3 downloads just fine, but then nothing else happened?  Why’s that?

SImply put ‘async void’ means ‘fire and forget’, and that means *really* forget.  You don’t know when (or even if) the item completes.

In this case, when the last DownloadUrl call happens, it hits the await, returns to Main, and Main has nothing left to do, so it returns.  Since that’s the entry point of the app, the process exits.

The fact that 3 tasks are still running doesn’t make any difference, Main returned.

If we want to care about whether something completes (and more specifically, knowing when it completes), then we have to tell the compiler that by changing the return type to ‘async Task’.

This is probably the ‘weirdest’ form of async method, since you end with a method that has no ‘return’ in it (not even a ‘yield return’) yet the method itself actually returns something (the Task).

Since we also need to do something to make sure these 3 tasks complete before we return from Main, we need to do something else.

A first reaction might be to ‘await DownloadUrl(url)’, but while that would both throw away our parallelism (since it would block the start of a task until the previous one completed), it also isn’t possible here since it would mean the containing method (Main) would need to be marked async, and you can’t do that with entry point methods, as the compiler lets you know:

error CS4009: ‘Foo.Program.Main()’: an entry point cannot be marked with the ‘async’ modifier

This means we’re back to using ‘normal’ TPL methods, which in this case means waiting on the tasks to complete. Since we want them all to complete, we need to Task.WaitAll which takes an array of tasks.  We could create our array of tasks to wait on in a variety of manners (foreach loop, etc.).

Now that we’re waiting on the tasks to complete, we get what we expected – they all complete and we get the expected output:

…\bin\Debug » .\ItsBigItsHeavyItsWood.exe
Starting to download url
http://www.google.com/
Starting to download url http://www.microsoft.com/
Starting to download url http://www.apple.com/
Downloaded 16694 bytes
Downloaded 34329 bytes
Downloaded 1020 bytes
…\bin\Debug »

Async lesson 1 of N: keep your async methods granular for better concurrency

This is the first in a series of posts trying to give bite-sized tidbits around async for people not familiar with Task or async but who want to take advantage of it without wanting to think too much about it. Smile

For this first post, I wanted to talk about async method granularity and how it affects concurrency.

Just as an example, let’s say you have some existing code that downloads data from 3 different URL’s.  In regular/synchronous code, it’s kind of six/half-dozen whether you have methods handle a collection of items or just one item at a time.  Either way, you’re going to foreach over it and handle each item:

If you try to switch this kind of code over to async by making the ‘simple’ changes (WebClient –> HttpClient, then await its download method), then it makes a big difference which one you end up with.

Changing the ‘takes a collection’ version would end up as:

(Yes, you would probably want to change the return type to Task, but I’m trying to minimize the diff for this example)

With this kind of change, you might think that you’d starting getting parallel downloads, but you’d be wrong. Smile

Fiddler shows up what’s going on, as the requests are still serialized.

image

Trying to explain ‘why’ is tricky without delving too much into Task or await, but I think (at least at the moment) the minimal conceptual bit is looking at the calling method, specifically that ‘await means returning back to the caller’.

(Yes, there’s lots more to it than that, but trying to simplify as much as I can without losing too much fidelity)

So with this kind of method, when we hit the await the caller can continue to do work, but there’s nothing for it (in this example) to really do.

More importantly, the start of the later web requests don’t happen until later iterations of the foreach loop, since the ‘await’ means that method is ‘stuck’ waiting for the web request until it can continue.

The ‘fix’ for this (assuming your intent is concurrency/parallelization of the http requests) is to move the boundary such that when the ‘await’ in the method does the ‘return to caller’, there’s actually more work (specifically, queuing more http requests) that the caller can do.

With the method like this, we’re doing the foreach at the caller instead (Main in this case).  This means that once we hit that ‘await’ keyword and the caller gets to continue, we actually have something meaningful we can do (calling DownloadUrl again and starting another request!)

This simple change in where we do the foreach results in the actual intended concurrency happening, as Fiddler shows us:

image

The lesson?  When making async methods, try to accept parameters that are at the granularity of your intended parallelization.  If you want different URL’s to get processed in parallel, then make sure your async method accepts just a url and not a collection of them.

NOTE: Yes, I’m glossing over lots of details and other options, but I’m trying not to complicate an already-too-long post. Smile

Should the BCL add an IEnumerableWithCount<T>?

NB: I’ve written and thrown away this blog post twice before in the past, as I would convince myself it was in the category of micro-optimization, but I’m finally going to post it this time, if only to get feedback to confirm that it’s a bad idea. Smile

There’s been a few times when it would be nice to have something be IEnumerable<T> but keep the data on how many items will be in the ‘stream’.  Certainly there are many cases where that’s not known ahead of time, and for those our existing IEnumerable<T> works perfectly. Smile

One thing in particular about IEnumerableWithCount<T> is that it could still be covariant like IEnumerable<T> is since we would only need to add a property with a getter.  That is a big difference from ICollection<T>, which is the current ‘lowest-level’ option for a collection interface that includes count.  ICollection<T> means you have to support lots of ‘write’ operations (Add/Clear/Remove) so it’s not what we want if we just want to have a ‘stream’ that has the additional ‘metadata’ of the count of items.

Even more more importantly, IEnumerableWithCount<T> would still support deferred-execution since it inherits from IEnumerable<T>

One key bit is that this interface will be able to ‘go between’ IEnumerable<T> and ICollection<T> – ICollection<T> (and therefore IList<T> and so forth) could also implement IEnumerableWithCount<T> (would just need to declare it, since it already has a Count property that matches).  This means most (if not all) of the collections we’re used to will get this interface ‘for free’.

Somewhat unfortunately, the canonical example of where an IEnumerableWithCount would have benefit is in the Select method of LINQ.

Much of LINQ-to-Objects includes interface checks to optimize performance if the input object supports a higher-level interface than the method requires.

For instance, let’s look at ToArray:

The real work is done in the ctor of Buffer, which is where we can see the first place that would use our new IEnumerableWithCount<T>:

This is where we see one of the places that could change if we added the new interface.

Currently the perf optimization is based on checking for ICollection<TElement> – if that interface is implemented by the source, then we get to skip all the reallocations (since we don’t know the final size) and create the final array directly.

The code also uses ICollection<T>’s CopyTo method to copy the actual elements, and since that seems out of scope for an IEnumerableWithCount, we’d probably just have 2 paths and IEnumerableWithCount would foreach, so that situation would be a little less CPU-efficient (but still better than today’s existing IEnumerable path in the code, since , but saving the reallocations (and generated garbage) is still a big potential perf win.

For the LINQ benefit in particular, we’d likely want a static System.Linq.EnumerableWithCount (similar to Enumerable and Queryable) that implemented the particular methods that can be done ‘better’ with count information.e

What methods would our EnumerableWithCount static class support?  These are the methods being used with LINQ-to-Objects that would benefit from having the compiler using an implementation by our EnumerableWithCount class instead of the current implementation in Enumerable.  If it’s not in this list, then falling back to the Enumerable version is fine (much like some methods aren’t implemented by Queryable), which is what the compiler will do for us.

  • Any method that doesn’t affect the number of items when it acts on the enumerable, of course
    • Select is the method that sticks out, certainly
    • OrderBy / OrderByDescending
    • ThenBy / ThenByDescending
    • Reverse
    • Cast
  • Concat can also benefit, as we’d be able to know the target item count in such a situation without enumerating (so concat’ing multiple inputs and then ToArray would also save all those reallocations)
  • Similarly, Skip and Take would benefit, since we’d know the resulting count due to both – in paging scenarios, for instance, this is very common
  • Count/LongCount is similar in its benefit, as it would be able to just return the Count property without actually enumerating.
  • DefaultIfEmpty would know if it’s already empty or not
  • Zip would know the final count (since it’s Min of the input counts)
  • SequenceEqual could check count mismatch first
  • Various methods that could either throw or return default(T) based on the count instead of having to iterate first, like:
    • ElementAt/ElementAtOrDefault
    • First/FirstOrDefault
    • Last/LastOrDefault
    • Single/SingleOrDefault (which would be able to throw for count > 1 without needing to enumerate twice)
  • ElementAt could throw on “index out of range” immediately without iterating
  • ElementAtOrDefault could similarly know whether to return default immediately
  • First/FirstOrDefault could similarly throw or return default without iterating
  • ToDictionary would know the final count and could preallocate buckets better
  • Range/Repeat/Empty would all be able to create this interface since the final count is known
  • Average would know the count to divide by without incrementing a counter 🙂

Some remaining questions:

  • If the compiler supported IEnumerableWithCount<T> (using the same ‘yield return’ syntax), how would it work?  Would the method declaration need an out param for count?
  • Should we create a matching IEnumeratorWithCount<T> as well?  I don’t think it’s necessary, but not really sure at this point.

Can I run Hyper-V in the consumer Win8 preview?

If you’re thinking of using the Win8 consumer preview and running virtual machines on it (especially to get the dynamic memory allocation feature that’s included), the thing to check for is whether your CPU supports Second Level Address Translation (SLAT).

NB: this is the first time the client OS has come with Hyper-V, so you don’t have to use the Win 8 Server to run Hyper-V.

http://technet.microsoft.com/en-us/library/hh857623.aspx

Hyper-V requires a 64-bit system that has Second Level Address Translation (SLAT).

Thankfully Mark Russinovich put up a new Sysinternals tool that helps with exactly this question: coreinfo.exe

image

What you want is for coreinfo –v to include a * (which means present) instead of a – (which means missing) for the feature that’s got “(SLAT)” in the line:

image

The above means the machine I ran it on has SLAT support and therefore can run the Win8 client Hyper-V fine.

C# 5 async/await: the oncoming flood of confusion

Caveat #1: First and foremost, let me state that I love, Love, LOVE the feature.  It’s a massive improvement in the language and is the key to enabling a change in the app landscape towards async-by-default, which in turn will mean massive gains in UI responsiveness and server scalability.  It’s HUGE.

Caveat #2: this kind of issue has arguably already happened in C# with iterator blocks/IEnumerable<T>/yield (for instance, how deferred execution bites people), and I’m really just trying to point out that the same is going to happen here, but to a larger extent.

With that said, there’s basically 2 buckets of people that I think about as using this feature to consume async API’s in the future.

Bucket #1 – People that are coming from the world of using async API’s already, but were doing so without the benefit of async/await

In this bucket you get lots of people that:

  • are likely to know what APM, EAP , and TAP refer to
  • may or may not have created their own async API’s, but are familiar with consuming them
  • Due to the growth curve, they’re most likely to be familiar with the EAP pattern of FooAsync() and a FooCompleted method
  • when writing EAP-type code, are probably as likely to write FooCompleted as a separate method as doing it inline as anonymous or a lambda
  • are likely already familiar with issues with thread affinity (typically via updating the UI on the ‘UI thread’)

These people (myself included) are typically happy with async/await because of what it enables for code readability and cleanliness.  Now you can have async code that you can read and (more importantly) reason about without getting lost in the details/ceremony of the pattern(s) used to implement it.  It also brings back the ability to use things like try/catch/finally and using blocks the way you did in synchronous code, so you can get rid of all the headaches you have without it in making sure to dispose of things in both success and failure cases (effectively doing ‘finally’ manually).  Yes, there are patterns/techniques/libraries for improving this story, but none ‘out of the box’ and none (IMHO) to this extent.  The TPL itself does a good job of improving the code for such things, but there’s only so much it can do as a library.

NOTE: this isn’t necessarily saying these are ‘better’ developers, as in many cases some of them were just forced into this bucket (for instance, Silverlight forcing async API’s), just that they’re at least somewhat familiar with the terrain of the world that async/await lives in.

Bucket #2 – People that aren’t familiar with using async API’s, and are either ‘forced’ (Silverlight/WP7/Metro) into async or see async/await as the way to get rid of their UI blocking or concurrency issues ‘for free’

For people that might normally avoid async API’s (when synchronous versions are available), looking at async/await code is surprisingly familiar.  Gone are the things like lambdas and event hookups and the like.  Gone even are the usages of Task or Task<T> and associated things like TaskFactory, AggregateException, etc., which are great if you’re coming from APM/EAP, but still more complicated and less obvious than ‘regular’ synchronous code.

One big ‘gotcha’ here is that ‘await’ doesn’t do what you might think of it as doing based on the name.  When you read some code that does var foo = await something.GetFooAsync(); without any additional context, the tendency will be to think “oh, cool, it calls this async methods, but then waits for it to complete, assigns the result to foo, and we can go back to our normal/regular/synchronous code”.  IOW, ‘await’ can be interpreted as something that just bridges the gap between async and sync API’s – using await just lets you call async API’s but still keep your original code.  Yay!  Now I can keep writing code the way I always have and not have to deal with all the ugly code!  When the compiler tells them that using ‘await’ means they have to mark the containing method as ‘async’, they (or a ‘quick fix’ / ReSharper / CodeRush / whatever) will do so and they’ll move on.

Of course, those familiar with the feature know that’s not what happens at all, and unfortunately it’s harder to explain (and even harder for people to know what’s going on to search for it in the first place) due to fact that the above (flawed) logic is actually correct within the scope of that method.  When looking at and reasoning about that method, you can think of it that way (that’s what this enables, after all).  The problem, of course, is that the place where the runtime behavior has changed in terms of how things behave is in the caller.  Because of this, when looking at demos and snippets of code, it’s much harder to reason about runtime behavior if you’re not familiar with at least the concept that the rest of the method is made a continuation on a task and the method returns.

Worst of all, if you thought deferred execution was hard to track down when it bit you and once you realized what was happening, you needed to ToArray/ToList/whatever, you’re in for a whole new world of difficulty here.

Much like the iterator block support (and IEnumerable in general), the deferred execution is great (memory savings in particular), but if you’re either not aware or not thinking about it at the right time, it’ll bite you.

What to do?

In an ideal world, everyone using async/await will have a chance to learn how it works and get familiar enough with it so that when they use it, they’re aware ahead of time of the potential gotchas.  In particular, Jon Skeet’s EduAsync series is a great place to start.

In the real world, lots of people will be using it as the ‘least code’ and ‘simplest’ approach to using async API’s that they’re forced to use (as Metro shows us, forcing async in your API surface area is a trend that’s going to continue).

So, as I help people (friends, co-workers, StackOverflow users, whatever) with their pain points around async/await due to how it works, especially issues that seem to crop up often, I’ll try to make (short, hopefully!) posts about the issue with hopefully helpful guidance on what you can do to try to minimize the chances of being bitten by it.  Something akin to ‘async/await for the rest of us’ or ‘async/await cliff notes’ I guess.

I have no idea how many or how often they’ll happen, but hopefully at least one will help someone at some point. Smile