Emulation March

An Emu, get it?

Programming communities are full of inspirational people. People who do great things. Not only great things, but specific things that inspire us. Whether it is blogging, working on open source projects or mentoring, I often look out and say to myself, "wow, I should do more of X." But, it can seem a bit overwhelming. After all, I don't have unlimited time. Add to that the seeming enormous task to start doing all those things.

Change is hard

Change starts with small steps, though. It is rare to be successful by making a huge shift in your life, not to mention developing it all into a habit. So, rather than looking at all the inspiring actions around our community and wishing that I was doing them more, I decided that I would pick just one. Thus was born "Emulation March."

The goal is simple: find someone that inspires you through a concrete action; make a vow to do that action with some regularity during the month of March; change your twitter bio (or other indication somewhere) to their name and the hashtag #emumarch. Simple as pie (note: simplicity also available as cake in certain areas).

You don't even have to be public about it. If you don't want to change your title or bio or whatever, that's cool. The month is about choosing an activity you want to pick up and doing it in a reasonable fashion. I find that being public often is a motivating factor to keep going. You might not. That's okay. But, if you want to be public, then blog about it, tweet about it (hashtag #emumarch), facebookize about it, google+-ercize about it, shout from the building tops.

Not Aaron, but WHAT HAIR!

My Emulation March Person

I admire people who contribute a lot to open source projects. I have contributed here and there, mostly through pairing with people, but I'd like to do it more deliberately.


A person I admire for contributing to open source (and lots of other things) is Aaron Patterson, our precious tenderlove of the ruby community. Not only does Aaron contribute to open source which makes our lives more productive, but he also brings us wonderful projects like Enterprise and Phuby that really enrich our lives. I think this is important, reminding us that coding should be like a full-head mullet: Party AND Business ALL AROUND!

So, I've chosen Aaron Patterson as my emulation march title.

Fierce Red Carpet Action

My Emulation March Activity

I vow to contribute to an open source project at least twice a week through the month of March.

I hope to build up a habit that will last longer than that, but Emulation March isn't about setting up expectations. I've even chosen my first open source contribution, which I'll push up in a couple days.

Who's your #emumarch inspiration?

[Update: My first contribution came in early by a day. Here is my gem based on Aaron's minitest-emoji. Mine adds RSpec support: Emoji-Test-Love)

I'm sorry

Yesterday I made a mistake. Without thinking, I put up a mean tweet about some code that Heather Arthur wrote. And I want to apologize.

I value being supportive. And I value being nice. I have thoughts around code, but they are all trumped by my strong belief that we should be working to bring each other up. Yesterday I went against all of those things, and I am ashamed. There are constructive ways to talk about code, but I did not use them. I will be thinking more strongly about these ways and how to use them in the future.

It is easy to forget that people write code. But it is important that we don't forget that. Writing code is easy, putting it out in front of the world isn't.

So, I'm sorry, Heather. There is no excuse for what I wrote.

(note: I won't be publishing comments on this post)

Why I Don't Use ActiveSupport::Concern

or, why I actively replace ActiveSupport::Concern with Ruby in codebases I work on

tl;dr

I don't use ActiveSupport::Concern.

• Unlike other Rails-based bastardization of concepts and names, ActiveSupport::Concern has very little to do with common connotations of "concern."
• It tightly couples a useful design concept to a single, possibly-least-optimal implementation of the idea (mixins).
• Its implementation is actual hiding a dependency management system. This hides design issues that are highlighted when you are more explicit.

However, if you want to use it, the world isn't going to end.

Background on why I noticed this

This summer, I spent some time at a local startup here in Chicago doing some work. I was brought in to build some features, look at their architecture, work on improving their test suite, etc. The system was young-ish (between 1- and 2-years old) and wasn't in bad shape. Features were still being added fairly readily (although starting to slow), and it wasn't TOO hard to find where things were. But the original developers had made some poor decisions, and you could see that if the same idioms were followed much more, the system would start to follow what is a common Rails progression, calcifying as heavy coupling starts to make it more and more difficult to make small changes without negatively influencing other parts.

Also, as their system grew, the lack of a good testing practice and structure slowed down their test time, both for the suite and (more importantly) the unit level. Heavy use of FactoryGirl-style integrated tests were making their test startup time less and less useful for the minute-to-minute development process. When running a single tests takes a long time, it is less likely that you'll run them frequently, so you lose one of the major benefits that a good test-driven process provides: design feedback as you flesh out your components, behaviors and interactions.

Again, their system wasn't in a horrible shape. I wasn't brought in to fix an already horrible situation. I was actually there to help keep them from going over the "super fast at first, then slows down tremendously" development cliff that haunts many of the Rails developers that I've talked to and spent time coding with.

They also had at least one missing domain concept in their system, as the original developers had waited a bit too long to focus on extracting the necessary design elements. This is a common problem I've seen: people (rightfully) wait to extract domain concepts until they present themselves, then miss the point in time when they can be easily extracted. So, it was a little bit past where reifying this fundamental concept was going to be a bit more work than they were comfortable taking. Since then, we've worked through a baby-step process to slowly bring the concept to life as a central figure in the design.

They were following a fairly idiomatic Rails design, focusing on putting business logic into their models. They tried to keep their controllers focused on the interaction between the user and the models. Of course, as many people who have worked with Rails know, this often leads to very large models. It also has a tendency to create complex interactions between your models. This results in large dependency graphs between objects, instead of a more streamlined tree-looking dependency diagram (also can be considered an acyclic graph). So, you have Model A depending on services from Model B, which depend on both Model A and Model C, which can lead to very complicated relationships and paths through your codebase.

The codebase had a combination of very large ActiveRecord model class files, as well as a hefty dose of extracted module code that was then added to the model via an include. The modules all used ActiveSupport::Concern to manage their inclusion, which is where this blog post comes from.

What is a concern?

One of the first steps I do when trying to pull apart large AR models is to group the methods into actual conceptual "concerns."  By moving these groups into modules and mixing them into the AR class, I can also get a good sense of the dependency graph between them. In general, I like these concerns to be orthogonal to each other. If I find that they are dependent on each other, it is usually an indication that there is a helper object of some sort waiting to be extracted.

Orthogonality of concerns

Although there are times when it can be the simplest, or at least an interim, solution, allowing a module to depend on another module should be treated as a design smell. This smell often flags a missing, but needed, service object. Unfortunately, idiomatic Rails design seems to shy away from more service-focused objects, relying on the "everything is in the AR model" design.

A good way to make sure that these concern modules are truly orthogonal is to mix them into your AR module without loading the other modules. This allows you to test just their code. If you find that you can't load them independently, then they have dependencies on each other and are perhaps hiding another concern. Loading modules in isolation is fairly easy to do, just create an empty version of the undesired modules before you require your AR module. Doing this allows you to run isolated tests of specific concern modules when doing development, but also allows you to have the same tests run in the context of loading all concern modules when running your whole suite. It works great, and I use this technique extensively.

NOTE: this requires an isolation-focus for testing your AR models. That is, you shouldn't need to load up your entire application just to test your AR models. You can see the way I do it here.

Isn't ActiveSupport::Concern just an alternate syntax for include/extend?

However, when I tried doing this on this specific codebase, the full suite wasn't working. The real versions of some of the modules weren't loading. Something was changing the way that Ruby's include was working. After some investigation, I narrowed it down to the modules that were relying on ActiveSupport::Concern, rather than just using Ruby's built-in include functionality. This shocked me, as I assumed that ActiveSupport::Concern was just an alternate syntax for Ruby's include/extend functionality. I was working with a pre-existing codebase, so I was just going by what I had read in blog posts, which emphasized the syntax. Now, personally, I dislike the syntax and find it to be one of the more unnecessary "syntactic sugar" pieces in an otherwise useful set of helpers provided with Rails.

Whoa! Dependency Management?

So, I went looking at the code for ActiveSupport::Concern. Go look at it, yourself, as well. ActiveSupport::Concern is focused on mucking around with the loading of modules in the case that you've allowed one module to depend on another. Having modules depending on other modules is a design smell, but, as with all design discussions, there are always some edge cases where this is needed and potentially the simplest solution. What caught me off guard, though, was that a component that was focused on managing a specific design was wrapped in a name that has nothing to do with that, namely "Concern."

Rails history of naming problems

Giles Bowkett has been writing some great stuff lately about Rails, OO and design. His book, Rails As She Is Spoke (yes, you should get it), has a good section on the problem with ActiveRecord as a name for the implementation in Rails. I won't go into too much detail, but it revolves around the misrepresentation of ActiveRecord, the ORM, and active record, the design pattern. Along with MVC, Controllers, Models, and pretty much the entirety of the "testing" names, Rails has had a history of bastardizing and confusing the meaning of already established terms.

It is common for Rails to be the first point of contact for developers with these terms. Many of the people who come to Rails either are beginners or come from programming cultures with less emphasis on software design and the history of where these terms come from. The result is that they now have a skewed view of these terms from the larger software industry as a whole. As an example, talk to a testing professional and use the terms that Rails uses for the different types of tests. You'll get some interesting looks.

Let's do the time-warp again

And now we have the term "concern" being bandied about. DHH recently wrote a reasonable blog post about separating out the different concerns of your AR models into modules. Unfortunately, he consistently confuses ActiveSupport::Concern (dependency management for module inclusion) with concern (orthogonal, possibly reusable modules containing conceptually related groups of functionality). By doing this, Rails once again is co-opting a term that is commonly used and has a well-established mechanism in Ruby (include/extend) and replacing it with a capitalized extra dependency. By presenting ActiveSupport::Concern as the same as "concern" the naming is again confused. And, unlike the other parts of Rails, this one has no added benefit related to even the most vague definition of concerns. In fact, it adds complexity to your system that wasn't there before when relying on include/extend.

It also tightly ties the idea of a "concern" to a concrete implementation in ActiveSupport. I use the idea of concerns a lot to guide me as I build the design for my systems. Sometimes the idea leads me to a module mixin, sometimes to a wrapper-type class, and sometimes to a helper class that I delegate requests to.

Do I need this dependency management functionality?

With the complexity (both inherent in the problem and caused by the choice of design) in Rails, I can see a need for a component that does what ActiveSupport::Concern does, managing module-to-module dependencies. But, why not have this as a separate, well-named component? On the surface, Rails purports to have a focus on usability, and it succeeds in a lot of areas, but historically it has messed up when it comes to names. While the other names at least have a passing similarity to their common connotation, though, ActiveSupport::Concern doesn't seem to have any actual link to anything but the vaguest of definitions of concern.

However, being explicit about this allows us to get necessary feedback from our design when we start creating strange dependency chains.

Adding complexity to Ruby's mixins

Because of the built-in dependency management overriding that this contains, you now have to jump through hoops to test ActiveSupport::Concern mixins in isolation of other mix-ins.

As an example, suppose you have the idea of authorization for certain behaviors. It could be argued (I've heard it, in fact) that authorization is a cross-cutting concern across models. So, why not implement this as a Concern? This is a reasonable idea, because you can then test that the authorization code is working, then declaratively lay it down on the rest of your system. However, this causes another small issue. In order to test the behavior of your component, you have to provide some form of setup around the authorization stuff. That adds unrelated complexity to your testing, masking the reason behind the test. Do this with enough "concerns" and you'll have a tremendous amount of cruft in your tests. Now, change one of them and all your tests break. Sound familiar? This is a common situation where the fragility of tests is a direct pointer to an implementation problem in your underlying design.

The current implementation of ActiveSupport::Concerns doesn't allow you to simply swap out the Authorization concern with a dummy implementation, due to its real nature as a dependency-management component.

And this is in place of Ruby's built-in mixin support with include/extend. Josh Cheek has a good blog post where he talks about include vs extend and some ways to use them.  Yes, you should go read it.

What it actually does

So, ActiveSupport::Concern appears to have 2 functions: an alternate syntax ("sugar" is a matter of opinion, I personally think it is less useful than Ruby's) for specifying how a module gets included into a class; and, a complex dependency management system (which has absolutely nothing to do with the concept of concerns).

But I like concerns

I love the idea of concerns. I use the concept all the time to guide my design decisions. In fact, I agree with DHH on a high level. If you replaced all references to ActiveSupport::Concern with just simple concern/include/extend in DHH's post about them, I think that his points are valid. In Rails, this is the place where you should be focused on separation of concerns. However, based on my experiences, mixins shouldn't be the primary mechanism for doing this in your system (I think wrappers or helpers are better way of eliminating duplication of knowledge/behavior), but it does fit with idiomatic "everything is in your model" Rails designs.

What should it be called?

ActiveSupport has a whole slew of useful additions to built in Ruby classes. Why not just have this be an augmentation of Module, focused on the dependency management?

Ultimate Question: should you use it?

I don't use them. I find that they add complexity to my system without providing any value. Adding dependencies and hiding complexity is a choice that you should make according to your desires and experience.

But, this blog post isn't about convincing you not to use them; it is about explaining the reasons why I don't use them. I prefer using Ruby's built-in, more flexible include/extend syntax. Your mileage may vary depending on your choice.

And, most importantly, the world won't end if you do. I would just ask yourself if it actually provides value. Rails has some awesome parts that make it very useful, but you should think before blindly using it for more than what it really is: a set of reusable helpers to route from a url to a method and a passably useful ORM. That part in-between those: that's your application, it is worth putting thought and care there.

Update: Other Options

People mentioned two alternatives to include/extend that don't have the naming- and dependency-management-craziness in them. Plus, the code is a bit more clear IMNSHO.
Shared by Ara Howard - The more complex of the two, supporting deferred execution of the code. Very interesting.
Augmentations by Henrik Nyh - Incredibly simple and focused.
Check them out, and especially check out the code! I'm not saying you should use them, but I definitely have a better feel about them than ActiveSupport::Concern.

And finally, here's a picture of Zak the Cat 

You just read a long blog post. You deserve to smile.

Thanks to Sarah Gray, Giles Bowkett and Thorsten Böttger for reviewing this.

Roman Numerals Kata with Commentary

I recently participated in an awesome Kata Battle at 8th Light against the worthy adversary, Craig Demyanovitch. The kata was the wonderful Roman Numerals Kata, converting arabic numbers to their roman numeral equivalent.

While practicing the kata, I noticed a lot of interesting things around the decisions I made in the solution. So, I thought I would record a katacast and lay over some voice commentary around it.

I'm putting two videos here, with and without commentary.

Here is the kata WITH commentary. It runs about 16 minutes and 40 seconds, so it isn't too bad to watch. I'd love to hear any constructive feedback in the comments. Version with no commentary is at the bottom of this post. I'd highly recommend watching it full-screen, so you can actually see it.

Resources:

• Transformation Priority Premise
• 4 Rules of Simple Design
• Roman Numeral Kata

Here it is with no commentary, if you just want to see me run through it at a reasonable speed.

Being honest vs Making excuses

The other day, I was involved in a conversation with a group of developers. Several of them work at a large company whose product is built on Ruby on Rails. We were talking about the development environment there, specifically the test suite. The codebase is quite a few years old and has a very slow test suite in part due to some design issues in the codebase. I asked what their test suite was currently running at, and I don't remember the exact figure, but it definitely had a lower bound at 30 minutes (I think it might have been an hour). I then asked about just the unit test suite, the one you should be running frequently while developing. The speed for that was on the order of many minutes. Someone else then asked how it was to run a single set of examples, the type that's focused on whatever part of the codebase you're actively working on, the type that should be run almost constantly while writing code. This was 'better,' being on the order of 30-45 seconds. I remarked that this was still horrible, especially when working on a piece of the system that didn't need specific features of Rails*. One person remarked that this was inevitable with a large codebase. I disagreed. That is, I disagreed with the statement that the size of the codebase was the cause. Slow unit test suites and the inability to quickly run focused subsets are an indication of a design problem with the codebase, not the size. As we talked more, I started to notice something that I've seen before but had a hard time placing: extreme rationalization of bad situations.

This got me thinking. Often times, we choose to be in certain situations and then, rather than admitting that the situation is bad and dysfunctional, then convince ourselves -- and justify to others -- that this is the only way it can be. While we are free to choose trade-offs for whatever situation we place ourselves in, it is important to be honest with ourselves about the causes of the situation. For example, the above company has interesting computer-science-types of problems which makes up for the bad situation regarding the actual development process.

If we are not honest about the causes of a bad situation, we pose a significant danger to those who are either less-experienced or uncertain about their own situation. Take a slow test suite, for example. Having a test suite that actively hinders a developer from running portions of it while developing is a deterent even to run the suite at all. As the run time climbs into minutes, the test suite becomes an antagonist, rather than the helper and guide that it should be. Instead of rationalizing, say you have some serious design flaws in your system. Or, say you have a large codebase and a huge team consisting of people with varying desires to write automated tests. When we mask the real causes of a problem, it has the potential to confuse less-experienced developers who look to us for guidance. For example, thinking a slow test suite is inevitable could stop someone from asking for help optimizing their own codebase while there is still time. If you are talking to a less-experienced developer and you mask the fundamental problems, what message are you conveying to them?

The point of this post is not to point out how people are wrong or that you have to fix, or are able to fix, a bad situation. I just want to raise a reminder that there is a difference between rationalization and being honest with the reasons for something. By rationalizing, we are fooling ourselves and those who learn from us into thinking that you can't do better and it isn't worth trying. By being honest with the reasons, we have the opportunity to both learn from our mistakes, and teach others what pitfalls might await given certain decisions.

So, the next time you find yourself talking with someone and describing a less-than-optimal situation, ask yourself whether you are being honest about the causes. It doesn't mean you have the power or inclination to fix the problem, but talking about the causes can lead to valuable conversations about how we can do things better in the future.

also... can we stop using the terms 'ivory tower' or 'real world' when rationalizing our situations? As DHH said, "The real world isn’t a place, it’s an excuse. It’s a justification for not trying." (other inspirational quotes)

*In fact, I will put forward you can't do an effective test-driven development cycle with a feedback loop that long.

As always, thoughtful comments are happily accepted.

My talk from SCNA2010

I was honored to be asked to give the closing talk at the Software Craftsmanship North America conference (SCNA) last year. It was a fantastic event filled will great thought, exciting conversations and wonderful people. This was the talk I gave introducing the idea of positivember.

Dates have been announced for this year's version of SCNA, November 18-19, 2011, so make sure to go check it out.

The video from my talk has been put up, so I wanted to share it. You can watch it on this page, full screen or click over to vimeo to watch it.

SCNA 2010 / Corey Haines from Brian Pratt on Vimeo.

Turbulence, measuring the turbulent nature of your code

Recently, Michael Feathers has been investigating the idea of mining all the data in our source code repositories to start finding information about our codebase and system design. He wrote an article about possibly using a churn v complexity chart to look for areas that could use some refactoring love called "Getting Empirical about Refactoring".
Since he has joined Obtiva as Chief Scientist, he is here in Chicago fairly frequently. I had the pleasure of spending a morning with him, and we naturally talked about his ideas. I was inspired to build a short bash script that generated a churn graph for my own codebase on MercuryApp. Chad Fowler took my short script and merged it with a script he wrote to run Flog over the codebase. While Michael, Chad and I were in Boulder, CO, for a coderetreat this past weekend, we put together a project called Turbulence.

Turbulence is a gem that you install and run in the directory of your code. It does a churn report, combines it with Flog data, and generates a nice scatter plot view. You can find instructions on the Turbulence project page. Although it currently only support Ruby code, we have plans for expanding the project to support other languages.

One goal is to have people take a screenshot of their graph and post it to twitter with the hashtag #codeturb. This will allow us to view the graphs on hashalbum and see what we can see.

Another goal is to have people send us their data, so we can do some analysis on different contexts. If you'd like to take part in this project, please email me. We are working on finishing up an anonymizer function that will mask the filenames, in case you might worry about directory structures/file names giving away your codebase's dirty secrets.

We are also proposing a series of talks at conferences outlining both results, recommendations and tools for helping analyzing your design based on the shape of your metrics. Keep watch for those.

I've uploaded my current graph for MercuryApp, you can see it in all its glory by clicking on the picture below.