Mental Pandiculation

Yesterday, I discovered the dark, evil, brooding side to using lambdas to hook up events, even though they look so cool. Here’s an example:

public void GetCustomerOrders(string storeId)
{
    Proxy.DocumentDownloadCompleted += (data) => this.HandleCustomerOrders(data);
    Proxy.GetXML("customerorderservice.svc/customerorders/" + storeId);
}

All well and good, that works like a champ. Usually. As long as you don’t want to ever unhook the event.

But let’s say you want to do something like this when the response comes back.

private void HandleCustomerOrders(Stream data)
{
    base.Proxy.DocumentDownloadCompleted -= this.HandleCustomerOrders;

    // Do something interesting here.
}

Maybe you want to do this because you have multiple handlers hooked up to the same event and in reality, you only want to handle one at a time. The problem here is that the unhooking does nothing here because that HandleCustomerOrders isn’t actually the function handling the event. When the compiler finds a lambda, it does one of two things. If there are variables that the lambda closes around, the compiler creates an anonymous private class on the enclosing class. If there aren’t any variables to close around, the compiler creates a new anonymous function on the current class. In either case, the event you think you are hooking up to is actually being hooked up to the created anonymous function. When you try to unhook, nothing happens because it was never hooked up to begin with.

In cases where you want to have the ability to unhook from events, you can’t use lambdas to hook up your events. You have to be a bit more old school, if less hip and do this.

public void GetCustomerOrders(string storeId)
{
    base.Proxy.DocumentDownloadCompleted += this.HandleCustomerOrders;
    base.Proxy.GetXML("customerorderservice.svc/customerorders/1");
}

By doing that, you will be able to unhook the events when the response happens. This public service announcement brought to you by the lost 2 hours of my dumbfounded life yesterday afternoon.

This is a public service announcement.

If you’ve got a class like this:

[DataContract(Namespace = "")]
    [XmlRoot(Namespace = "")]
    [Table(Name = "dbo.PurchaseOrder")]
    public class PurchaseOrder : DomainBase
    {
        [DataMember]
        [Column(IsPrimaryKey = true, IsDbGenerated = false, CanBeNull = false)]
        public Guid Id { get; set; }

        [DataMember]
        [Association(Name = "FK_PurchaseOrder_PurchaseOrderDetails", OtherKey = "PurchaseOrderId", Storage = "details")]
        public IList Details
        {
            get
            {
                return this.details;
            }
            set
            {
                SetOrAssign(ref this.details, value);
            }
        }

        private EntitySet details;
    }

and a class like this:

    [DataContract(Namespace = "")]
    [XmlRoot(Namespace = "")]
    [Table(Name = "dbo.PurchaseOrderDetail")]
    public class PurchaseOrderDetail
    {
        [DataMember]
        [Column(IsPrimaryKey = true, IsDbGenerated = false, CanBeNull = false)]
        public Guid Id { get; set; }

        [DataMember, Column]
        public Guid PurchaseOrderId { get; set; }

        [DataMember, Column]
        public Guid ItemId { get; set; }

        [DataMember]
        [Association(Name = "FK_PurchaseOrderDetail_PurchaseOrder", Storage = "purchaseOrder", ThisKey = "PurchaseOrderId",
            IsForeignKey = true)]
        public PurchaseOrder PurchaseOrder
        {
            get
            {
                return this.purchaseOrder.Entity;
            }
            set
            {
                this.purchaseOrder.Entity = value;
            }
        }

        private EntityRef purchaseOrder;
    }

and you’re getting highly bizarre results when you try to call your web service like mysterious second calls even though you know you’re only calling it once, try to remember that Linq-To-Sql still doesn’t support back references like other ORMs you’ve loved before. Delete the reference to the parent object from the child object (in this case, that sneaky little purchase order on the purchase order detail) and rejoice in the fact that all is right with the world again. Go drink beer, eat sausage and terrorize puppies (or whatever it is you do for relaxation).

I don’t know why I can’t get this through my thick skull. Just because I expect something to act a logical certain way doesn’t mean it will.

From the public service announcement department, this is an issue I recently struggled with and since I didn’t find much help on the web, I thought I’d contribute. If you’re trying to hit WCF services (ours are RESTful but I’m pretty sure this happens on regular services as well), you may get weird security exceptions like this:

at System.Net.BrowserHttpWebRequest.InternalEndGetResponse(IAsyncResult asyncResult)
   at System.Net.BrowserHttpWebRequest.<>c__DisplayClass5.b__4(Object sendState)
   at System.Net.AsyncHelper.<>c__DisplayClass2.b__0(Object sendState) 

This can happen even if you’re using a clientaccesspolicy.xml file which is supposed to be one of the supported ways to get around cross domain issues in Silverlight according to Microsoft. Problem is, that doesn’t seem to be true. I examined the requests on the wire using Wireshark and what was actually happening was that my Silverlight application was asking for the clientaccesspolicy.xml and the server was responding with a 304 Unchanged status. Silverlight didn’t seem to know what to do with this so it would continue to throw the SecurityException above.

As it turns out, the crossdomain.xml file is the way to go if you want to have your Silverlight application talk to WCF. Put this at the web root and suddenly, everything works like a champ. Funny how the crossdomain.xml file is from Flash, the very application Silverlight is trying to take market share from.

Irony. It’s what’s for breakfast.

When Continuous Integration just isn’t hardcore enough. What an amazing and fascinating place that must be to work, an environment where discipline to their process enables them to deploy code to production up to 50 times a day.

The scripts that monitor statistics and perform analysis on the result of the partial rollout are ingenious. But all of this is made possible by human discipline to a process that make the impossible easy. I’d guess it’s probably a joy to work in that kind of environment, one where each team member has responsibility to all other team members to make sure the environment stays clean.

On my current project, we’re using WCF and RESTful services to provide data to a Silverlight application. We have our own IOC/DI framework but we’re also toying with Ninject in that capacity and it was my job this week to hook it up to the service layer. I came across Heinrich’s description of the process but it needs a little tweak to work with RESTful services. Instead of having your NinjectServiceHost class derive from ServiceHost, use WebServiceHost instead. Once that’s done, Ninject works like a champ.

If you’re using NInject on a Framework 3.5 project and you start to get fun System.Runtime.CompilerServices.ExtensionAttribute compiler errors, you can do what I did and recompile the NInject source to create happy libraries for 3.5. You may have to alter the nant build script for NInject to not treat warnings as errors to get a successful build but that should do the trick. More info here on this bug in the released NInject libraries.

Steve Yegge has written a fascinating, thought-provoking article on embedded systems. In it, he discusses embedded systems ala games ala Mario Kart and muses on the invisible boundary around all embedded systems, how information gets into and out of embedded system and the possible ramifications when we think about all things as embedded systems.

A key point:

In our discussion so far, I have intentionally blurred the distinction between the host system (such as a fish tank or a game console device) and the host system’s host system (such as your bedroom or living room). But you’ve probably noticed by now that all host systems are embedded in some larger system. This is just the way things work. The fish tank is in your bedroom, which is a system embedded in a house, which is a system embedded in a neighborhood, embedded in a county, a nation, a continent, a planet, a solar system, a galaxy, a universe.

It’s perhaps not as clear in the case of fish tanks, but host systems often overlap and even cooperate. A city is composed of many interleaved subsystems. So is your body. It’s not always a simple containment relationship. Systems are made of, and communicate with, other systems.

But one way or another, all systems are embedded systems.

If all systems are embedded systems, isn’t it possible that our little corner of the galaxy is an embedded system that is controlled in some way by information that enters our system, unknown to us, through the holes that Steve talks about? And couldn’t our universe be another embedded system within some host system that we can’t even begin to comprehend but that is controlling our universe in invisible ways? Isn’t this what God (or Buddha or The Pink Unicorn or your deity of choice) does? He works in mysterious ways, right? But if he does, and if all systems are in some way embedded systems, what is God’s (or Buddha’s or The Pink Unicorn’s) host system? Whoops, my mind just exploded.

It’s an approachable article even if you aren’t a programmer and I highly recommend it. It IS long so print it out and read it in the bathroom or on the train but do read it. It’s not so much about programming (though I think he’ll get there eventually) as it is about metaphysical questions about our existence. At least to me it is.

Here’s some fun code:

private void AddButton_Click(object sender, RoutedEventArgs e)
{
  if (user.IsStupid)
    {
      MessageBox.Show("Hey Stupid, don't be so stupid in the future", "Information", MessageBoxButton.OK, MessageBoxImage.Error);
      return;
    }
    // TODO Treat smart users well here
}

Of all the truly evil things in the world, Information only Message Boxes have to rate in the top 3 right below Pol Pot and above the Dallas Cowboy’s performance this season. They take focus away from the UI, they supposedly require input except that there’s only one way to input information and they generally serve no purpose. I can’t think of a single place where an Informational Message Box makes sense. Stop doing it. Please.

This rant brought to you by my impending vacation sponsored by Citibank. I apparently need one.

One of the tenets that agile proponents often tout as the best of the best is the team room. A team room is a centralized location where the entire team works. Typically, there are tons of whiteboards around, a big space and the concept of personal space is thrown out the window if there happens to be one nearby. The supposed benefits come from the high bandwidth communication that a situation like this fosters. No more does a team member have to wait to have his question answered via email or IM. He can just tap anyone on the shoulder (or just stand up and announce his problem) and presto, problem solved by whoever knows the answer.

There are about 40 things wrong with this scenario that seem to be typically glossed over by the proponents of team rooms. The first, but certainly not the worst, is that it seems to be a myth that this is an agile tenet. It’s not found in Scrum. It’s not on the XP Rules page. I don’t see it in the Agile Manifesto. Pretty sure it’s not in Lean anywhere. This site says powerful communication is one of the seven core principles of agile management and makes a huge generalization that “The single most effective means to communicate powerfully, is to put all the team in a room together where they can do their work, every day for the majority of the work time” but provides no links or studies to back up this rather extreme assertion. Overall, having the team in a team room is at best, according to the links to the variants of agile above, a minor bonus, not even important enough to write into the rules.

The second way that a team room is just wrong is the effect that noise and interruption have on task completion, specifically cognitive tasks like programming. Google “Effects of noise on task completion” or “Effects of interruptions on task completion” and take your pick of psychological and sociological studies of this on human performance. They are so prevalent that you can even find free ones which is saying something in the field of psychological research. Even when you find one that supports crazy ideas like “radical colocation” which I wrote about here, you find that the study involved quiet places where programmers could go to actually do, you know, REAL work. There just isn’t any evidence out there that team rooms actually improve the code. There is anecdotal evidence that they improve the process that leads to the code but there is a MOUNTAIN of evidence that when the code has to be written, noise and interruptions lead to longer task completion, greater numbers of errors/bugs and higher frustration/annoyance by the subjects.

The third and final reason why team rooms suck is the fact that 50% of the time teams spend in the room involve non-work related issues ranging from discussions of the hot chick in HR with the short skirt, discussions about guns/football/basketball/your mother and who to blame for the awful stench after the team went to Abuelo’s for lunch. Put 5 people in a room together and you don’t get 8 hours of work out of them. You get about 4 if you’re lucky because the other four are spent doing non-work related things. It’s just human nature. It’s also the nature of the beast because when it is easy to interrupt someone, it is difficult to not be interrupted.

All of this is brought on by this post (I’m sorry to keep picking on Ken but he and I seem to have VASTLY different ideas of how agile works. He’s far more experienced than I am so I’m probably wrong on every count but until I get some hard evidence of it, I’m going to keep spouting off). In it, he examines the noise effect on teams and what that means for task completion. His numbers are

Normal Project
• 90 hours spent in lag time (based on an unlinked study)
• 10 hours spent on real work

Project with a team room
• 15 hours spent on real work (the real work takes 50% more time in a noisy environment)
• 20 hours of lag time
• 65 hours talking about porn (ok I made that up but where else are the 65 hours going to go?)

One thing this analysis leaves aside is the fact that it took you 50% more time to do work that now contains a probable factor of 5 more bugs. Which means someone has to find the bugs and fix the bugs. You took more time to write crappier code which surely can’t be looked favorably upon by agile proponents. On top of that, there is some evidence that you spend more time modifying existing code than you do on new code by a factor of 10. Even if you believe that you wrote the same quality of code in the 15 hours interrupted that you did in the 10 uninterrupted, you’re going to spend 50 more hours modifying it and debugging it. Like everything in life, you can’t just look at the pro side of things.

I don’t doubt that team rooms can work. The problem is, they fail far more often than they succeed for the same reason teams doing scrum fail far more than they succeed. If you don’t implement a team room or scrum perfectly, they fail to deliver on the goods. If they aren’t implemented with a chance for quiet concentration separate from the team, if they don’t involve meta-XP rules like code standards, pair programming, or continuous integration, they just aren’t going to do what you think they do. Just throwing a bunch of people in the same room together doesn’t do what you think it does. Like putting lipstick on a pig, it’s a not a prettier picture and you only stand to annoy the pig.

Further reading (as if you made it this far):
Holding A Program In One’s Head
Attention and Sex
Human Task Switches Considered Harmful

Ken argues that Agile (big A or little, your choice) is Gestalt. From this conclusion, he says that it’s a mistake to dogmatically follow a given process or proscribe particular tools when we’re trying to implement Agile and that instead, we should “. . .help remove organizational and sociological blocks that prevent teams from employing them.” While I think he may be right that Agile done correctly is Gestalt, I don’t think that his conclusion naturally follows.

For those of us who haven’t been in cognitive psych too recently, the Gestalt basically boils down to “the whole is greater than the sum of its parts” and is holistic in nature. Max Wetheimer, commonly considered one of the three founders of the Gestalt school, said this about Gestalt:

I stand at the window and see a house, trees, sky. Now on theoretical grounds I could try to count and say: “here they are. . .327 brightnesses and hues.” Do I have “327″? No, I see sky, house, trees; and no one can really have these “327″ as such. Furthermore, if in this strange calculation the house should have, say 120 and the trees 90 and the sky 117, I have in any event this combination, this segregation, and not, say 127 and 100 and 100; or 150 and 177. I see it in this particular combination, this particular segregation; and the sort of combination or segregation in which I see it is not simply up to my choice: it is almost impossible for me to see it in any desired combination that I may happen to choose. When I succeed in seeing some unusual combination, what a strange process it is. What surprise results, when, after looking at it a long time, after many attempts, I discover-under the influence of a very unrealistic set-that over there parts of the window frame make an N with a smooth branch. . .

(Emphasis mine)

So is Agile greater than the sum of its parts? I agree that it probably is. But here’s the kick in the pants: so is waterfall or RUP or whatever methodology of the week that you follow to write good software. The key word there is “parts” I think. As in, without ALL the parts, you don’t get the emergent factor of Gestalt. Waterfall can be Gestalt, just ask the people who write the software that runs the space shuttle. It becomes a matter of discipline in following a process precisely.

In my limited experience, what I find happens if you don’t prescribe a practice in a shop that’s trying to implement agile is that shop picks and chooses the pieces that it likes (typically short sprints and planning at the beginning of each sprint) and leaves aside all the pieces that they don’t like which are typically the parts more likely to engender success with Agile like producing deployable code after each sprint or having access to real users, not managerial standins. When this process fails to produce the expected results, it’s the methodologie’s fault even though in reality, the methodology was hardly implemented at all. There’s a reason why we call them methodologies and not suggestologies. There is a method involved and when you short cut it, you get short cut results.

I don’t think Ken Schwaber would ever suggest that leaving out pieces of Scrum would result in a better process. In fact, his book is full of case studies where teams were trying to implement Scrum by skipping important parts of the process.

Eric Gunnerson wrote about what he called “scrumbut”, that is, the practice of saying you are doing Scrum but you’re doing it differently. I’ve written in the past how Scrum is analogous with a jazz musician. When you are a beginner, you don’t have any business going off on your own and riffing some jazz chords because you don’t have the fundamentals necessary to do that. Scrum and Agile are the same in that as a beginner, you can’t decide what pieces of the methodology are good and bad for your team because you don’t have the fundamentals necessary to make that decision without having implemented the process precisely. I don’t think a mentor should ever advocate or facilitate such behavior either. Instead, he should explain why all of the pieces of a particular methodology are necessary.

A process without ALL of its most important parts becomes anti-Gestalt because then the process not only doesn’t produce the emergent behavior expected, it can also be blamed for the failures that inevitably follow. Comparing the picture in Ken’s post with the picture above, it becomes obvious that without the critical parts, the cross in the middle never emerges.

Agile has the potential to make writing software more successful. But that doesn’t mean that it will ever be easy or that you can skip parts you don’t like. Implementing agile correctly involves prescribing and then following an oftentimes difficult process. Only by doing that can agile become Gestalt.