My application is going to be an ESPN Headline browser essentially using the ESPN Developer API. This API really isn’t very interesting in that it gives about zero useful information to non-paying developers but there isn’t much I can do about that. If you want to follow along, you’ll need to request an API key from ESPN. This first part doesn’t involve any actual calls to ESPN so you can skip that step for now. All the code for this app is up on Github.

I’m using the Rails 3.1.3 and the lastest Jasmine code. The first thing you need to do is install the jasmine gem by adding it to your Gemfile

group :test do
  gem 'jasmine'
end

Once that’s done, run “bundle install”.

There is some jasmine initialization that needs to happen once the gem has been installed so run “bundle exec jasmine init”. This will create configuration files for jasmine as well as some sample tests and javascript that you can safely delete. At the time of publication of this post, Jasmine seems to still be expecting an earlier version of Rails because the generated code ends up in the “public” folder and the configuration files all reference that generated code. If you’re using Rails 3.1.3 and the new asset folders, you just need to update the jasmine.yml file in spec/javascripts/support to reference those folders. More on that in a minute.

Once we have the gem installed, we can write our first test. I’m following the convention of this guy and arranging my test code into spec/javascripts/** and my javascript into app/assets/javascripts/** where ** will match models, collections, views, etc. Fire up the Jasmine test runner by executing:

rake jasmine

and then hit localhost:8888 in your browser. You should see something like this:

Those five tests are the sample ones that Jasmine created. A conscientious developer would delete those but I’m lazy. Plus, it makes me feel better that five tests already pass. Now, we’re ready to write our first test. I created Headline.spec.js in spec/javascripts/models to contain the test code for my Headline Backbone model.

describe("Headline model", function () {
  beforeEach(function() {
    this.headline = new Headline({
      headline: "My Headline",
      description: "a description",
      source: "ESPN News",
      byline: "Brett Bim"
    });
  });

  describe("when instantiated", function() {
    it("exhibits the attributes", function() {
      expect(this.headline.get("headline")).toEqual("My Headline");
    });
  });

});

This is a really basic test that in actuality is testing Backbone functionality more than code but it’s a good first starting point to make sure everything is set up correctly. Refresh localhost:8888 and you’ll find out it’s not.

As you can see, Headline is not defined which is expected given that we haven’t defined it yet. Let’s do that now. Create Headline.js in app/assets/javascripts/models and add this code to it:

var Headline = Backbone.Model.extend();

This creates our model. Save the file and refresh localhost:8888. Surprisingly, you get the same failed test. This is because the Jasmine gem does not automatically reflect the asset structure from Rails 3.1.3. You need to explicitly specify where your files are. You can do this by going to spec/javascripts/support and opening the jasmine.yml file. There is a section for source files. You need to tell it where your source files are and add references to backbone.js and underscore.js as well.

src_files:
  - public/javascripts/prototype.js
  - public/javascripts/effects.js
  - public/javascripts/controls.js
  - public/javascripts/dragdrop.js
  - app/assets/javascripts/underscore.js
  - app/assets/javascripts/backbone.js
  - public/javascripts/application.js
  - public/javascripts/**/*.js
  - app/assets/javascripts/**/*.js

I’ve added the three lines pointing to app/assets which will inform Jasmine where to find those files. Once that’s done and the file is saved, we can refresh localhost:8888 again.

And our first test passes. We should now have Jasmine up and running for testing all our Backbone code. Stay tuned for another episode of “One Post Every 8 Months About Random Crap” where we discuss writing tank driving software using Lisp.

Anyone who says that a system must display perfect separation of concerns has either never written any large-scale software systems or is certifiable and can safely be ignored. The goal of writing software isn’t to produce perfect separation of concerns. That is only one of many possible means to get to the end result of long term maintainable software that produces some quantity of value. Writing any meaningful software without an eye on SoC results in a jumbled morass of unmaintainable and untestable crap. Does this mean that your HTML views should be completely dumb, devoid of logic? Of course not and no one with any real world experience would argue that. But if you don’t constantly strive to keep them as dumb as possible, you will end up with junk. It must be a constant focus, one that is in the forefront of every decision because if it is not, it is always easier to do what is wrong than what is right.

When I first read Atlas Shrugged, I was enthralled with the concept and philosophy behind the novel, that selfishness was inherently good and that every action should be judged based on one’s own self interest. Of course, I was 16 and frankly, the philosophy fit nicely into a 16 year old’s limited world view. As I got older, I began to realize that while acting in self interest as a general rule is good, taking it to the extremes that Objectivist philosophy did will result in a terribly narcissistic life.

The same thing happened when when I first discovered design patterns. Everything was a Factory. I loved Factories. Concrete Factories, Abstract Factories, Factory Factories, by God they were all fantastic. But slowly over time, I learned not every thing was a Factory and not every thing fit neatly into design patterns. But as a guiding rule, design patterns are good but it’s just as important to know when to not use one as it is to use one.

Separation of Concerns is the same. As an overall guiding principle, it is critically important. There are going to be times when ignoring it is in fact the right decision but those times are very few and far between and you better have a damn good reason for doing it. But not doing it because it’s never going to be possible to have a perfect separation isn’t one of those reasons.

Take a quantum leap forward into 2011. Over the last 3.5 days, I built a website to serve as my central home on the web, a place where I can aggregate my work, writing and other activities easily. I built the site in Rails, used git for source control and when it came time to deploy the site, Heroku for the hosting. To deploy my site, I had to do to things:

heroku create

and

git push heroku master

When that was done, I had a live, working test site on the internet that I could test in multiple browsers, get feedback on, etc. When I was ready to redirect my current home on the web to the new one, all I had to do was add a custom domain and Zerigo DNS add-on at Heroku, update my nameservers and presto chango, the production site was up and running.

On top of that, because branches are so painless and easy in git, I can create a new branch, work on it, merge it with the master and update the site all from the command line of my laptop. The idea of having to log into a hosting provider to upload files suddenly feels like waterboarding. If you have a significant test suite, it’s a matter of having your tests pass and then just updating your production site. Obviously, as a site grows, it won’t be quite that easy but it’s not going to get that much harder either.

The smart folks at Heroku have removed some of the major obstacles to web development so we can focus more on writing the code now and less on the administrative agony. On top of all that, for small sites like mine, the cost is negligible or non-existent. Lots of things aren’t better in 2011 than they were in 2002 (the world’s going to end next year) but web deployment, if you choose the right tools, is infinitely better in a way that’s almost inexpressible.

For those of you who aren’t country music fans, Johnny Cash sings about working in the the GM plant and building a Caddy one piece at a time by bringing the pieces home in his lunch pail over a number of years. In the end, he puts the car together and while he has a functioning, hot-rod Cadillac, it’s a bit, how shall we say, bolted together with duct tape and bailing wire. I can only imagine what it would be like to perform repairs on a car that’s built from a variety of parts over 10 or 15 years. It seems like a far fetched system but in the software world, it’s exactly what you get in a system that ignores technical debt over time to focus on delivery of features.

There’s been some writing lately on metaphors for technical debt which has caused me to think about what it means to have bad, messy, untested code in a system of any size. I think the idea of Johnny’s Caddy is an excellent metaphor for explaining both what technical debt is and how it affects the functionality and maintainability of a system. When you have three headlights and no tail fins, the car still runs but it’s a little odd looking. But when you try to dig into the guts and replace a ’52 carburetor bolted onto a a ’62 intake manifold, you’re going to have to have extremely specialized system knowledge about the car you are working on. If the original mechanic gets hit by a bus, the new guy can’t go to AutoZone and pick up a ’49, ’50, ’51, ’52, ’53, ’54, ’55, ’56, ’57, ’59 Caddy manual.

The same thing holds true for a system that’s been continually extended over a number of years to do more and more things for its users. If the code is never revisited to make it more maintainable or flexible and if it was originally written without consideration for future changes, the system becomes brittle and difficult to understand. Without a manual (either the domain knowledge of the original system builders or tests to protect developers from unintended consequences), modifying the system can become rather hazardous. Over time, if no attempts are made to bring the code into some semblance of normalcy, the system can even get to the point where it can’t be changed for fear of the consequences.

One thing to keep in mind, technical debt is a sign of success. Without the continual driver of success, systems don’t evolve to the point of having technical debt. But it’s important to always be aware of the debt, of its force and effect on the flexibility and maintainability of the system. Without constant attention, eventually the system becomes fragile and unstable, breaking unexpectedly when changes are made to seemingly unrelated sections of the app. Without some sort of design, either architectural through an initial design phase or protective through the use of solid tests, the system development is likely to eventually come to a slow halt because of the interrelatedness of the pieces.

And really, the only criterion of coolness worth checking out is “Are they named David?”

Now let me clarify: people named David are nifty. They pay their bills, don’t beat their wives, drink something other than Mickey’s Malt Liquor and generally are normal human beings. But you see, being named David is a choice and whenever anyone voluntarily keeps David as their name, I have to ask “Why?”

Don’t get me wrong, being named David isn’t an instant showstopper for being cool. It’s just that lots of people I’ve known that were named David just weren’t cool which is best explained by a really bad simile: Being named David is like being David Hasselhoff.

Being David Hasselhoff is awesome if you want to be a world famous cult figure who goes to awesome parties, sings great songs on YouTube and was once the star of a kick-ass show called Knight Rider with a bad ass car that talked to him and saved him from bad guys. But if you want to be in independent films with Natalie Portman, you can’t because Natalie Portman doesn’t hang out in Cannes with David Hasselhoff. If you want to win Academy Awards, you can’t because David Hasselhoff doesn’t win Academy Awards. In short, if you want to do anything that doesn’t involve being David Hasselhoff, you can’t because you ARE David Hasselhoff.

See, by being David Hasselhoff, he can’t be in independent films. Well he could be but that doesn’t really fit in my story line. Therefore, because David Hasselhoff can’t be in independent films with Natalie Portman (even though he can be, it’s not important, try to forget I pointed that flaw in my storyline out, OH LOOK A CHICKEN!), then it must be true that people named David aren’t cool.

Instead I look for people who aren’t named David because it’s more likely that they fit closely into my weird and oddly supported worldview that people named David can’t be cool. I don’t want someone named David, I want someone named not David.

Just by being named David, your chances of being cool are practically zero and thus, I’d rather have hung out with you after you took a long nap.

So what’s the moral of this whole ridiculously overgeneralized and poorly thought out story? Two things:

1. If you want to hang out with me, avoid being named David. Unless you’re David Hasselhoff, it does you no favors.
2. If you are someone else and you meet someone named David, take notice and immediately become suspicious. They probably aren’t David Hasselhoff and thus, not cool.

It might sound harsh, and it is (with bad punctuation usage to boot). But frankly, life is too short to live without strict black and white rules that involve sweeping over-generalizations and easy ways to get lots of people to link to your site.

Yes, there really is a point to all this.

Today, I needed to increase the innodb_buffer_pool_size value from the default 8MB to something useful for my purposes like 2 GB. You can’t do that from SqlYog or from the command line as it happens to be a readonly variable. So you need to create a configuration file. That file lives in root directory at /etc/my.cnf. I first tried to create this file using vi, got it all typed up and then when I saved it, was told that wasn’t going to fly, you don’t have the requisite permissions. Stupid *nix operating system. Not that I’m complaining but after the day I’ve had, I would have liked to have just created the file.

So back to the command prompt and try sudo vi my.cnf. Lo and behold that works like a champ. The file looked like this when I was done:
[mysqld]
innodb_buffer_pool_size=2G

Saved that, restarted the MySQL server and it had updated correctly as seen using SHOW VARIABLES;

Probably all very elementary stuff but for a guy who prefers not to get his hands dirty with database stuff, good to know for the future. Also learned about profiling which you can enable in a script in SqlYog with a SET profiling=1; at the beginning of your script and a SHOW profiles; at the end.

Quite awhile back, I dove into learning Python and the Pylons web framework and for the most part, I was happy with it. I never deployed an app using Pylons but my Python skills got to be where I could at least make things work. I enjoy the Python language but I always felt like testing was a second class citizen to a large degree. Maybe I just never read the right tutorial but tests and deployment seemed to be an afterthought. In Python’s defense, it never seemed to have the “cool kids” momentum like Ruby/Rails. And Pylons certainly has different goals than Rails given the fact that Pylons gives you some defaults but let’s you do what you want whereas Rails has the Rails Way and it’s clear you stray from it at your own risk.

But learning Pylons especially always felt like I was stabbing at zombies in the dark. I didn’t have a good mentor other than the web and while I certainly could have read code trying to figure out best practices, I’m lazy and would at least like to have some reasonable semblance of a roadmap to guide me. With Rails in general and the Rails Tutorial linked above specifically, the road map is more like a TomTom GPS system, giving you the steps you need to create and deploy applications in Rails right from the start. The emphasis on testing, source control and deployment is refreshing since I imagine that being a major headache of writing a decent sized web application in a new framework.

The use of convention over configuration, long a selling point for Rails, is wonderful. There are so many places over the last few days where I have compared my experience in the ASP.Net world to what I was learning in Rails and found the former terribly lacking. As an example, in ASP.Net MVC, named routes have to be created explicitly but in Rails, many useful named routes are created for you based on convention.

Another place Rails seems to have the advantage is database updates as you develop an application. I just went through a situation where I had to manually define how a couple of people would deal with updates to a database while working concurrently on a project. With Rails, it’s built in through the concept of migrations. So many pieces of the plumbing that you have to deal with on normal projects is just handled. I’m sure there are gotchas waiting to jump up and bite me but for someone first learning a framework and comparing it to other experiences in the past, it’s wonderfully refreshing to find so many pieces of the puzzle are taken care of for you.

On top of that, as someone who has struggled with how to coordinate programming styles and conventions on projects in the past, imagine my excitement when I learned about the Rails Way and convention over configuration. If you just do things the Rails Way, not only are certain things taken care of for you, but you can count on Rails code being comparable across projects. Imagine reading the open source code for a couple of ASP.net projects. While ASP.Net MVC has certainly moved the slider bar over towards convention, chances are the two projects would have drastically different flavors. Yet it seems to me that Rails projects would likely be reasonably similar.

Overall, I’m thrilled (obviously since I just ran through 700 words detailing my excitement) with Rails and the potential it has in my development toolbox. While it’s certainly early and I’ve no doubt been excited about things in the past, it’s encouraging to feel comfortable in a framework to some degree right from the start. Of course, major props has to go to Michael Hartl who wrote the tutorial I’m learning from. He’s done a fantastic job of laying out a good course through the material with reasons for doing things in a particular way. I intend to buy the screencasts once I work my way through the online book because I imagine they are filled with exactly the same kind of good teaching that is available in the book.

My experience was surprisingly similar to the post linked above, e.g. I went down the hardest path trying to solve the wrong test. I’ve since gone back and deleted the code but the pseudocode went something like this:

Break text into a list using a list comprehension based on the length of the column
Start looking for words with spaces
Try to repiece things together based on the column length and the last space
Cry

Clearly, this wasn’t going to work. About that time, we ran out of time and went to the Flying Saucer to drink beer and pretend like we weren’t geeks. But I decided to finish the kata this week during a slow period at work. Since I didn’t have Jerry there to talk me through how dumb I was, I started reading the post up until the point where it describes writing the wrong test and spending too much time trying to solve it. So I backed up, deleted everything and started down the easier path of solving all the non-space related issues first. Once I did that, and once I figured out that recursion was going to be a big help, the project really got easier.

The final solution is below:

def wrap(text, num):

  if len(text) <= num:
    return text
  elif text[num-1] == ' ':
    return text[:num].rstrip() + '\r\n' + wrap(text[num:], num)
  elif text[:num].find(' ') > -1:
    rightSpaceIdx = text.rfind(' ')
    return text[:rightSpaceIdx] + '\r\n' + wrap(text[rightSpaceIdx:].lstrip(' '), num)
  else:
    return text[:num] + '\r\n' + wrap(text[num:].lstrip(), num)

It’s fascinating how quickly the algorithm starts to come together when you write the correct test. The problem is, I’m terrible at all this so it’s going to take some time getting enough experience to correctly select what test to write next. I was writing a ton of code trying to solve a problem that was too hard and the solution I eventually would have come up with would have been effective but brittle.

The more I do TDD/BDD, the more I realize that it is *THE* way to develop software, especially if you’re working in a dynamic language. I’m currently working through the tutorials at Ruby Tutorials and it’s great to see that TDD is a fundamental part of the process. As I learned Python and Pylons, it was up to me to figure out best practices it seemed like and that’s workable but frustrating in the long run. I’m planning on doing more katas in an effort to improve my craft.

If you start getting an Import error that says “No module named content found”, you’ve run into it. According to the Pylons book, “Routes has a surprising legacy feature that means that if you don’t specify a controller and an action for a particular route, the implicit defaults of controller=’content’ and action=’index’ will be used for you”. This is certainly surprising to me and as always, things that are implicit tend to annoy me. Luckily, you can change this behavior. In your config/routing.py file, set map.explicit = True and then you’ll need to alter the route that is giving you problems to make the controller and action explicit. For example:

Before

def make_map(config):
    """Create, configure and return the routes Mapper"""
    map = Mapper(directory=config['pylons.paths']['controllers'],
                 always_scan=config['debug'])
    map.minimization = False
    map.explicit = False 

    # The ErrorController route (handles 404/500 error pages); it should
    # likely stay at the top, ensuring it can always be resolved
    map.connect('/error/{action}', controller='error')
    map.connect('/error/{action}/{id}', controller='error')

    # CUSTOM ROUTES HERE
    map.connect('schedule_entry', 'schedule/index/{scheduleDay}')
    map.connect('/{controller}/{action}')
    map.connect('/{controller}/{action}/{id}')

    return map

As you can see, the default behavior is for map.explicit to be set to False. The first route under CUSTOM ROUTES HERE is a named route that to my eye should match schedule up with a controller and index up with the action. Unfortunately, instead it tries to find the default implicit controller “content” which it can’t find and that throws the ImportError listed above. To fix it do this:

After

def make_map(config):
    """Create, configure and return the routes Mapper"""
    map = Mapper(directory=config['pylons.paths']['controllers'],
                 always_scan=config['debug'])
    map.minimization = False
    map.explicit = True 

    # The ErrorController route (handles 404/500 error pages); it should
    # likely stay at the top, ensuring it can always be resolved
    map.connect('/error/{action}', controller='error')
    map.connect('/error/{action}/{id}', controller='error')

    # CUSTOM ROUTES HERE
    map.connect('/schedule/index/{scheduleDay}', controller='schedule', action='index')
    map.connect('/{controller}/{action}')
    map.connect('/{controller}/{action}/{id}')

    return map

Now the map is set to explicit and the controller and action are explicitly specified which works just fine. This all may be an artifact my novice understanding of Routes but since the book documents it this way, I guess this is the way I’m going to do it.

From the Pylons docs on testing:

Pylons will provide several additional attributes for the paste.fixture response object that let you access various objects that were created during the web request:

1. session — Session object
2. req — Request object
3. c — Object containing variables passed to templates
4. g — Globals object

To use them, merely access the attributes of the response after you’ve used a get/post command:

response = app.get('/some/url')
assert response.session['var'] == 4
assert 'REQUEST_METHOD' in response.req.environ

As it turns out, that’s sort of true. I have a test that looks like this:

from nbapowerrank.tests import *

class TestGamedetailsController(TestController):

    def test_index(self):
        response = self.app.get(url(controller='admin/gamedetails', action='index'))
        self.assertEqual(len(response.c.games) > 0, True)

and a controller that looks like this:

from nbapowerrank.lib.base import BaseController, render
from nbapowerrank.model import meta
from nbapowerrank.model.game import Game

log = logging.getLogger(__name__)

class GamedetailsController(BaseController):
  def __before__(self):
    self.game_q = meta.Session.query(Game)

  def index(self):
    yesterday = datetime.today() - timedelta(1)
    c.games = self.game_q.filter_by(gamedate=yesterday).all()

    return render('/gamedetails.mako')

Unfortunately, contra the documentation that says the “c” alias will be available on the test response object, that test always fails with an AttributeError stating that in fact, the c attribute does not exist. It frustrated me even more because all the other attributes that are supposed to be on the response like session and g were in fact there. After doing some random digging, I came across this in the Pylons 1.0 roadmap: “Deprecate pylons.c, pylons.g, and pylons.buffet. These have been disrecommended since 0.9.7.”

Apparently, they are/have deprecated using the c alias for tmpl_context even though when you create a controller under Pylons 1.0, it still aliases the context as c. Sigh. So in order to test data that you have added to your context for templating, your test should use the explicit tmpl_context instead of the c like this:

from nbapowerrank.tests import *

class TestGamedetailsController(TestController):

    def test_index(self):
        response = self.app.get(url(controller='admin/gamedetails', action='index'))
        self.assertEqual(len(response.tmpl_context.games) > 0, True)

And then all will be well with your world. I do have to say that compared to both ASP.Net MVC and Rails, the testing support in Python/pylons seems to be a second class citizen. I’m not sure why that is because as a dynamic language, it seems to benefit greatly from a TDD approach. Maybe it’s just me getting back into a framework after a year’s worth of changes.

Anyway, that may help some people out there trying to search in the googleplex about TDD and Pylons.