A language that doesn’t affect the way you think about programming, is not worth knowing.—Alan Perlis

Alan Perlis, the first head of the Carnegie Mellon University Computer Science Department and the first recipient of the Turing Award, recorded some of his accumlated knowledge about programming in a series of one sentence statements. You can read more about his Epigrams here.

I especially like number 19, about learning programming languages (quoted above). That’s the big reason the Pragmatic Programmers recommend learning a new language every year. To expand the way you think.

Thinking Different

Here’s an example of one way that Ruby affects the way you can approach a problem. I was preparing for my Dependency Injection talk (to be given as OSCON) and was working out the details of an example program I was using to demonstrate some DI features. I chose the “Mark IV Coffee Maker” problem from “Robert Martin:http://objectmentor.com (used with permission). I love the coffee maker problem and have used in C++ and Java courses over the years. It is simple enough to grasp in a teaching scenario, but rich enough in objects to be interesting. And because it is a physical control system, a lot of the objects are simple analogs to physical devices and therefore easy for beginning OO designers to work with.

I’ve written code for this example, oh, probably half a dozen times over the years; sometimes in C++, sometimes in Java (and once in Eiffel if I recall correctly). The solution changes with each iteration, but there are some common themes that keep appearing.

Java Coffee

One small part of the problem deals with a Brewer object that needs to control a Heater object (which implements an On/Off device interface) and a relief valve (which conforms to an open/close device interface). This is an excellent place to introduce two design patterns to the students. The first is the composite pattern where we create a Boiler object that is a composite of both the Heater and Relief valve. The Boiler becomes a single object to the brewer which just issues on/off commands to the boiler. The boiler is responsible for making sure all its subcomponents (the heater and the relief valve) are properly operated.

The composite pattern works by allowing the composite to forward the on/off commands to each of its subcomponents, but the valve wants to recieve open/close commands, not on/off commands. Here is the second pattern that emerges from this portion of the design: the Adapter Pattern. By wrapping the relief valve in an on/off adapter, the boiler composite sees a unified on/off interface that is translated for the valve to an open/close interface.

For the Dependency Injection talk, I was developing both a Java verison and a Ruby version in parallel. The Java version was straight forward and I implemented it like I normally do. The only big design choice is how to map on/off commands to open/close. For the coffee maker, on needs to map to close and “off” needs to map to “open” (when the boiler is off, the relief valve needs to be open to relieve the steam pressure).

The opposite mapping might make sense in other problems, and you alway think about making classes like the OnOffAdapter a bit more flexible so that it can be used in the other situations. I considered adding a flag to the adapter constructor to allow the user to select the desired mapping, but decided against it. I didn’t need it for this problem and felt the additional logic would just get in the way.

The Ruby Adapter

So now it was time to write the Ruby version. The straight forward implementation of of the Ruby adapter is simply:

  class OnOffAdapter
    def initialize(device)
      @device = device
    end
    def on
      @device.close
    end
    def off
      @device.open
    end
  end

Once written, I again considered adding a flag, but suddenly realized there was a better solution:

  class OnOffAdapter
    def initialize(device, on_command, off_command)
      @device = device
      @on_command = on_command
      @off_command = off_command
    end
    def on
      @device.__send__(@on_command)
    end
    def off
      @device.__send__(@off_command)
    end
  end

Instead of a flag that would only switch between two different on/off and open/close mappings, by adding the commands themselves to the interface I get an adapter that will adapter any device to an on/off interface (well, almost any device … keep reading).

To use the device in my coffee maker example, all I need is:

   valve_adapter = OnOffAdapter.new(relief_valve, :close, :open)

Once I took the step of making the open/close commands generic, I realized that I could make the on/off generic as well.

  class Adapter
    def initialize(device, mapping={})
      @device = device
      @mapping = mapping
    end
    def method_missing(sym, *args, &block)
      command = @mapping[sym]
      if command.nil?
        super
      else
        @device.__send__(command, *args, &block)
      end
    end
  end

Usage is now:

  valve_adapter = Adapter.new(relief_valve,
    :on  => :close,
    :off => :open)

I now have a generic adapter that will do a one-to-one mapping from one set of methods to another set of methods.

Continued Refinement

But is doesn’t have to stop there. As an exercise for the student, consider what it would take to add the following refinments:

• Allow mappings that translated on and off commands to something like brightness_level(10) and brightness_level(3). (Hint: think about procs).

• Allowed methods that were not mapped to pass through unchanged.

• The current version creates an object that does the adaptation. If you need a bunch of similar adapters, would it be better to return a class whose instances are adapters implementing the defined mapping? How would one implement that.

Beyond Java

The point of this posting is not to bash on Java. Certainly you can create generic adapters in Java that can do much the same mappings as the Ruby version. But getting from the specific to the generic adapter is a big step, big enough that my YAGNI filtered kicked in and I didn’t go down that path in Java.

In Ruby, the step was small enough that refining the adapter was very easy and natural. And the end result yielded some worthwhile insites into the adapter pattern as well.

Someone reported an interesting problem in Rake, and I thought you might enjoy the problem and its resolution.

The Problem

Consider the following Rakefile (I’ve left out some of the uninteresting parts):

task :run

BUILD_DIR = 'build'
TARGET_DIR = 'build/copies'

FileList['src/*'].each do |src|
  directory TARGET_DIR
  target = File.join TARGET_DIR, File.basename(src)
  file target => [src, TARGET_DIR] do
    cp src, target
    sleep 3
  end
  task :run => target
end

Assume the src directory has a lot of files, and that it takes a while to copy them to the build directory (I artificially slowed down the copy by including a sleep command). Also assume for this first run, the build directory has not been created yet.

The first time you run “rake run”, you will see …

  cp src/foo1 build/copies/foo1
  cp src/foo2 build/copies/foo2
  cp src/foo3 build/copies/foo3
  ...

and so one for each of the copies.

Now run “rake run” again. You shouldn’t see any copies because the source files have all been copied at this point and there is no more work to do … but instead you will see (if you are running rake 0.5.3 or earlier) a number of duplicate copies being performed.

Analysis

Why are those extra copies performed? Each target file build/copies/foo^{n^ is dependent upon the source file src/foo}n^ (so it gets updated when the source changes) and the target directory (so the directory is created by the time the file is copied).

The first time task run is invoked, it populates the target directory with each copy, and in doing so updates the time stamp of the target directory. The next time run is invoked, the earliest target files are out of date with respect to the timestamp on the directory. Rake thinks it needs to update the targets, hence the extra copies.

A Solution

Rake already supports two kinds of tasks. Task objects always run when invoked and are useful for defining simple jobs that need to be performed whenever invoked. FileTasks are different in the they are only invoked if (1) the file they are associated with is does not exist, or (2) the time stamp of any prerequisites are newer than the target file.

What we need for directories is a task that runs when a file needs created but (1) doesn’t trigger on timestamps and (2) returns a timestamp that is earlier than any time stamp of files that depend upon it.

It turns out this is fairly easy in Rake. Tasks define two methods, needed? and timestamp. The first is easy … only return true if the file doesn’t exist.

  def needed?
    ! File.exist?(name)
  end

Handling timestamp was a bit more interesting. What is the earliest possible time stamp? I played around with Time.mktime to find the earliest possible time stamp it could encode, but gave up after a bit. Even if I found it, it would be an implementation dependent issue. I wanted an object that would report it is less than any timestamp.

Class EarlyTime

The first pass at an EarlyTime object was simple:

  class EarlyTime
    include Comparable
    include Singleton

    def <=>(other)
      -1
    end
  end

We made the comparison operator (<=>) always return -1. This means that an object of EarlyTime will claim to be smaller than any other object. The Comparable inclusion makes sure all the comparison operators are properly defined (based on <=>). The Singleton inclusion make sure the is only one copy of the early time (we only need one … really, one of few times I’ve used Singleton).

This works for early_time < time, but how do we handle time < early_time? The Time class doesn’t know about EarlyTime, so it won’t return the right result (in fact it will choke on the value).

We just need to teach Time about the new class:

  class Time
    alias pre_early_time_compare :<=>
    def <=>(other)
      if Rake::EarlyTime === other
        - other.<=>(self)
      else
        pre_early_time_compare(other)
      end
    end     
  end

We create an alias the existing comparison operator <=>. Then we redefine <=> to check for an EarlyTime value. If other is an early time, we redispatch the comparison to the early time value and reverse the sign of the result. If other is not an EarlyTime, then we invoke the old behavior through the alias we created.

Beta Rake

If you want to try the new version of Rake, I’ve uploaded a beta version to my betagems site. You can get it via:

gem install rake --source http://onestepback.org/betagems

Version 0.5.4.3 is the latest beta version. Once I get a little time on it, I’ll make a 0.5.5 release.

Thanks

Thanks to Martin Fowler for pointing out this problem and correctly deducing the reason behind the problem.

Preparations have been made, presentations have been written, and tickets have been ordered.

Talks for OSCON 2005

As mentioned here earlier, I’m doing a talk on Dependency Injection at OSCON this year. I’ve given the talk twice now for practice (thanks to both the Cincinnati XP Users Group and the Cincinnati Java Users Group for being my guinea pigs). Both practice runs went fairly well. I rewrote about half the talk between the first and second trial runs and am much happier with the result. The hardest part is knowing what not to say, otherwise I could talk for hours (OSCON presentations are 45 minutes long).

What I haven’t mentioned is that I’ll be giving a second talk there as well. It seems there was cancellation in the Ruby track and they had another opening, so the “10 Things Every Java Programmer Should Know About Ruby” is going to OSCON as well. This should be good, the “10 Things” talk is a fun one to give.

However, I did a practice run on the “10 Things” talk at the Cincinnati Programmers Guild last week and the timing was way off. I ended up talking for nearly an hour and 45 minutes. I’m going to have to to some serious cutting and trimming on that talk to get it into the 45 minute window, actually make that 35 minutes because I would like to leave 10 minutes for questions.

If you are in the Columbus Ohio area and would like to hear yet another practice run, I’ll be at the Columbus Ruby Brigade on July 19th. It will probably be the dependency injection talk. Time and location hasn’t been set quite yet, but I’ll announce more when I know more.

Cincinnatians at OSCON

John Wilger has announced he will be at OSCON, and Chris Nelson (Mr. Trails) will be there speaking about his Trails project (see this java.net article for a preview to Chris’s talk). Both John and Chris live within a few miles of my house, so I officially declare Finneytown to be the OSCON-off-cite center of the conference. From elsewhere in Cincinnati, a friend from the Cincinnati Linux Users Group has mentioned to me that he is also going. Anyone else planning on going?

All I can say is Wow!

Rails Day, the Aftermath

(If you don’t know what rails day is, look here: Rails Day Blog. It is a 24 hour contest to implement a web application in Rails.)

Obviously I can say more than wow. I don’t have a lot of time, so just a few quick points here.

• Don’t try to setup your brand new Mac-mini for Rails Day if you’ve never used it for developement and are not that familiar with Macs in the first place. Not that Macs are a poor development machine, far from it! It’s just a poor decision to “try out a Mac” the morning of Rails Day. I wasted way too much time trying to get it configured. I finally switched back to my trusty laptop.

• Favorite quote of the day: I heard this from the other Rails Day team in the room with us:

Person 1: I think we should use transactional testing.

Person 2: I agree. Just a second … I want to watch you set it up. (He then turns back to his machine to finish a commit. He returns in a few seconds). Ok, I’m ready.

Person 1: Oh, I’m done already.

Person 2: Damn! That’s what I hate about Rails. You blink and you miss something.

• Ha! We way overplanned our application. John started at midnight and implemented the DB schema we talked though in our planning meeting. When we started connecting things up, we discovered that is was way more complicated than what we could accomplish in one day, so we ripped out the extra stuff and concentrated on the features that we were working on at the moment. Eventually we might need the complexity, but it was only getting on our way. YAGNI!

• Thanks to Fusion Alliance for procuring the Room. It was perfect for our needs (well, except for a distinct lack of power outlets ... we had to bring in a few extra power strips).

• Thanks to Lisa Kaminski (SARK) for the fruit basket and the Pizza.

• Although we didn’t get everything done, we did get to put in some AJAX like features. That’s the first time I have played with AJAX and Rails does a really nice job of making it easy.

• Here’s our stats:

+----------------------+-------+-------+---------+---------+-----+-------+
| Name                 | Lines |   LOC | Classes | Methods | M/C | LOC/M |
+----------------------+-------+-------+---------+---------+-----+-------+
| Helpers              |     9 |     8 |       0 |       0 |   0 |     0 |
| Controllers          |   148 |   126 |       4 |      21 |   5 |     4 |
| APIs                 |     0 |     0 |       0 |       0 |   0 |     0 |
| Components           |     0 |     0 |       0 |       0 |   0 |     0 |
|   Functionals        |   189 |   148 |       6 |      26 |   4 |     3 |
| Models               |   170 |   133 |       4 |      26 |   6 |     3 |
|   Units              |   175 |   142 |       3 |      28 |   9 |     3 |
+----------------------+-------+-------+---------+---------+-----+-------+
| Total                |   691 |   557 |      17 |     101 |   5 |     3 |
+----------------------+-------+-------+---------+---------+-----+-------+
  Code LOC: 267     Test LOC: 290     Code to Test Ratio: 1:1.1

• Note to self: Next year have practice run to smooth out the kinks.

• Teams in Cincinati:
  • Team 1 (JEWEL … personal food log)
    • Jim Weirich
    • John Wilger
    • Rob Biedenharn
  • Team 2 (Photo album/organizer)
    • Scott Barron
    • Doug Alcorn
    • Mark Windholtz

• Mark brought in a couple bottles of Ruby Sake, which we used to celebrate the end of the run.

Had a great time … I suspect the other teams did too.

It looks like I am going to O’Reilly’s Open Source Conference this year.

Dependency Injection at OSCON 2005

My proposal for a talk at OSCON 2005 on Dependency Injection and Dynamic Languages has been accepted. The acceptance notification came quite a bit later than I expected so when it finally came I was pleasantly surprised. Speaker registration opened last week and I’m all signed up and ready to go.

Well … almost ready. There’s the small detail of finishing the presentation and getting the materials to O’Reilly in a timely fashion. (details, details …)

The talk is going to be an extension of the blog entry I made last fall on Dependency Injection In Ruby, but will go further and deeper into the subject. I think it will be a lot of fun.

If you can make it to the conference, come on by and introduce yourself. I’d love to meet you.

Yes, I saw Star Wars, Episode III. But that’s not the war I’m talking about.

Episode III

My daughter and I went to the midnight showing of “Revenge of the Sith”. (Quick review: Yes, worth seeing; no, Lucas still can’t write a love scene.) But this blog is not about the movie, it’s about the event.

Imagine this picture… You are in a room full of Star Wars geeks, many of whom are dressed in elaborate costumes with expensive (as in over $200) light sabers. You have been waiting for hours for the show to start, and have suffered through the trailers. The obligatory “Please turn off all cell phones” and “Remember our concession stand” message are now playing and the movie event of the year is just seconds away from starting.

Pause now with this picture in your mind and think about how the Star Wars movies always start. I know you’ve seen them. The big STAR WARS logo flashes on screen, the brass fanfare sounds and the words “In a galaxy far, far away” start scrolling on the screen.

Also remember that if you are at the midnight showing, then you are there not only for the movie (which you will probably see several times over the next few weeks), but for the experience, the event! The final movie is showing and you just want to drink in the experience and feel the mood.

And then seconds before the movie starts… the projector skips a beat and the frame gets out of synchronization with the projector. The top half of the screen is now showing at the bottom, and the bottom of the screen is at the top. The logo was split across the screen so that it read:

   WARS
  -----
  STARS

The room was in an uproar! Chants of “FIX IT” and “RESTART” echoed throughout the theater. The crowd reacted as if the projector malfunction was ruining the whole experience. Someone in the back of the room stood up and blocked the projection beam so that the picture could not be seen … and the crowd cheered because they did not want to see a flawed first performance (why shutting one’s eyes was not sufficient was never clear to me).

The theater staff was not exactly helpful. At first someone came out and said that they would get it fixed shortly and would restart the film. And they did get it fixed within a few minutes, but didn’t restart it. It didn’t matter, the crowd was so upset that they kept chanting “RESTART” after the picture was fixed. Finally the staff came out and said that weren’t able to restart it (never sure just why), and that the problem was in all the theaters (Really? In all 20 theaters in the movieplex? In what way are the theaters linked so that the same problem happens in all them at the same time?)

Finally it was obvious that they weren’t restarting it, and the crowd settled down and we could begin to watch the show. At that point, we were in the middle of a space battle (I’m not quite sure who was fighting who), and there were little drilling droids on Obi wan’s fighter. Oh well, it was Star Wars and we settled in to enjoy it.

An aside: You got to believe that a significant portion of that midnight showing crowd was planning on multiple viewings of the movie. Lets say two-thirds were planning on seeing it more than once. With 20 sold out theaters with 200 to 300 people per theater makes (around) 5000 people there that night. Now if two-thirds of 5000 were going to pay to see the movie twice, but now have a free pass, that right around $30,000 that was lost that night. Wow. That’s an expensive mistake.

Oh, and I have one other piece of advice to those going to midnight Star Wars showings. If you are going to go in costume, go all out and do it right. If you show up with a Darth Vadar helmet that looks like it came from a K-Mart Halloween costume rack, and the Darth Vadar standing next to you looks as if he just stepped off the movie set ... well, you look pretty silly.

Just go see the movie and enjoy it.

Mark Probert on the Ruby-Talk mailing lists asks: “I am not sure how to create a set of Rake rules to do the following. Can anyone prove assistance?”

I had planned for the next Rake tutorial to go into using prepackaged task libraries, but Mark’s question highlights an interesting problem (and resolution) with rules. I posted an answer to Mark on the list, but why waste a perfectly good explaination when I can recycle it here.

The Problem

Mark has two separate source directories (he calls them src_a and src_b, but I suspect they are more creatively named in real life. Both directories contain .c files. However, the kicker is that all the object files are to be placed in a single directory (named obj) no matter which source directory contained the original C code.

The rule we introduced in the last tutorial isn’t powerful enough to move the .o file into the obj directory. We need to tweek it just a bit.

The .o File Rule

Here is the rule in question.

rule '.o' => '.c' do |t|
  sh "cc -c -o #{t.name} #{t.source}" 
end

To recap, the rule specifies how to create a .o file from a similarly named .c file. But as noted above, the .c are in a different location. We can fix this by giving the rule a general purpose function that transforms the object file name into the correct source file name.

Finding the Source File.

But how do we find the source file? Assuming we have a constant named SRC that contains a list of all our source files, this simple find command will do the trick (assume objfile is the name of the object file):

SRC.find { |s| File.basename(s, '.c') == File.basename(objfile, '.o') }

Wrapping this in a method (named find_source) gives us a nice way to find the source file.

Tweeking the Rule

We can now write the rule like this…

rule '.o' => lambda { |objfile| find_source(objfile) } do |t|
  sh "cc -c -o #{t.name} #{t.source}" 
end

The Whole Rakefile

Just a couple notes about the Rakefile

1. Note that we invoke the OBJDIR task directly in the rule. Because it is a rule, there is no opportunity to list OBJDIR as an explicit dependency. By invoking it directly inside the rule, we will build that directory if it is needed (but only if it is needed).
2. If searching the SRC list has performance problems (because SRC is very long), then an alternative is to create a mapping of object names to source names at the top of the file. Then finding the source name is a simple hash lookup.

Rakefile

require 'rake/clean'

PROG = "foo" 
LIBNAME = PROG
LIBFILE = "lib#{LIBNAME}.a" 

SRC = FileList['**/*.c']

OBJDIR = 'obj'
OBJ = SRC.collect { |fn| File.join(OBJDIR, File.basename(fn).ext('o')) }

CLEAN.include(OBJ, OBJDIR, LIBFILE)
CLOBBER.include(PROG)

task :default => [:build, :run]

task :build => [PROG]

task :run => [PROG] do
  sh "./#{PROG}" 
end

file PROG => [LIBFILE] do
  sh "cc -o #{PROG} -L . -l#{LIBNAME}" 
end

file LIBFILE => OBJ do
  sh "ar cr #{LIBFILE} #{OBJ}" 
  sh "ranlib #{LIBFILE}" 
end

directory OBJDIR

rule '.o' => lambda{ |objfile| find_source(objfile) } do |t|
  Task[OBJDIR].invoke
  sh "cc -c -o #{t.name} #{t.source}" 
end

def find_source(objfile)
  base = File.basename(objfile, '.o')
  SRC.find { |s| File.basename(s, '.c') == base }
end

Alternatives

On possible alternative is to replace the rule with a loop that explicitly creates tasks to compile each .c file. It might look something like this:

SRC.each do |srcfile|
  objfile = File.join(OBJDIR, File.basename(srcfile).ext('o'))
  file objfile => [srcfile, OBJDIR] do
    sh "cc -c -o #{objfile} #{srcfile}" 
  end
end

What I like about this solution is the ability to put the OBJDIR dependency directly in the task definition.

Ruby and celestial mechanics… two great tastes that taste great together.

I saw this mentioned on the Ruby-Talk mailing list. The Kali Code for Dense Stellar Systems by Piet Hut and Jun Makino is a conversation between two astrophysists as they build a computer simulation of the N-Body problem. And they are using Ruby to write the simulation. It is a great introduction to the Ruby language in addition to being a fun problem to solve.

My first job out of college was at Cape Canaveral working on deep space trajectory programs. It has been many years since I flexed my math muscles in celestial mechanices, so it was kinda fun to see that stuff again. I’ve only read through the second chapter, but I’m eagerly looking forward to the rest.

If you are looking to learn Ruby and aren’t frightened by a bit of math (well, perhaps more than a bit), you might enjoy this.

Rake is a tool for controlling builds. In this part of the Rake tutorial, we see how to organize the Rake actions to apply to many similar tasks.

In the RakeTutorialIntroduction, we talked about the basics of specifying dependencies and associating actions to build the files. We ended up with a nice Rakefile that built our simple C program, but with some duplication in the build rules.

But First, Some Extra Rake Targets

But before we get into all that, lets add some convience targets to our Rakefile. First of all, it would be nice to have a default target that is invoked when we don’t give any explicit task names to rake. The default target looks like this:

   task :default => ["hello"]

Until now, the only kind of task we have seen in Rake are file tasks. File tasks are knowledgable about time stamps on files. A file task will not execute its action unless the file it represents doesn’t exist, or is older than any of its prerequisites.

A non-file task (or just plain “task”) does not represent the creation of a file. Since there is no timestamp for comparison, non-file tasks always execute their actions (if they have any). Since the default task does not represent a file named “default”, we use a regular non-file task for this purpose. Non-file tasks just use the task keyword (instead of the file keyword).

Here are a couple of other really useful tasks that I almost always include in a Rakefile.

Remove temporary files created during the build process.

Remove all files generated during the build process.

clean tidies up the directories and removes any files that generated as part of the build process, but are not the final goal of the build process. For example, the .o files used to link up the final executable hello program would fall in this category. After the executable program is built, the .o files are no longer needed and will be removed by saying “rake clean”.

clobber is like clean, but even more aggressive. “rake clobber” will remove all files that are not part of the original package. It should return a project to the “just checked out of CVS” state. So it removes the final executable program as well as the files removed by clean.

In fact, these tasks are so common, Rake comes with a predefined library that implements clean and clobber.

But every project is different, how do we specify which files are to be cleaned and clobbered on a per project basis?

The answer is File lists.

File Lists to the Rescue

A file list is simply a list of file names. Since a lot of what Rake does involves files and lists of those files, a file list has some special features to make manipulating file names rather easy.

Suppose you want a list of all the C files in your project. You could add this to your rake file:

  SRC = FileList['*.c']

This will collect all the files ending in ”.c” in the top level directory of your project. File lists understand glob patterns (i.e. things like "*.c") and will find all the matching files.

By the way, no matter where you invoke it, rake always executes in the directory where the Rakefile is found. This keeps your path names consistent without depending on the current directory the user interactive shell.

The clean and clobber tasks use file lists to manage the files to remove. So if we want to clean up all the .o files in a project we could try …

  CLEAN = FileList['*.o']

(CLEAN is the file list associated with the clean task. I’ll let you guess the name of the file list associated with clobber).

The Rakefile So Far …

With the addtion of a few extra tasks, our Rakefile now looks like this. Notice the require ‘rake/clean’ line used to enable the clean and clobber tasks.

  require 'rake/clean'

  CLEAN.include('*.o')
  CLOBBER.include('hello')

  task :default => ["hello"]

  file 'main.o' => ["main.c", "greet.h"] do
    sh "cc -c -o main.o main.c" 
  end

  file 'greet.o' => ['greet.c'] do
    sh "cc -c -o greet.o greet.c" 
  end

  file "hello" => ["main.o", "greet.o"] do
    sh "cc -o hello main.o greet.o" 
  end

Ok, now its time to address the redundant compile commands.

Dynamically Building Tasks

The command to compile the main.c and greet.c files is identical, except for the name of the files involved. The simpliest and most direct way to address the problem is to define the compile task in a loop. Perhaps something like this …

  SRC = FileList['*.c']
  SRC.each do |fn|
    obj = fn.sub(/\.[^.]*$/, '.o')
    file obj  do
      sh "cc -c -o #{obj} #{fn}" 
    end
  end

Just a couple things to note about the above code.

• The dependencies are not specified. This is a common where we specify the dependents at one place and the actions in another. Rake is smart enough to combine the dependencies with the actions.

• Although the task was named after the .o (which is, after all, what we want to generate), the file list is defined in terms of the .c files. Why?

The simple reason is that file lists search for file names that exist in the file system. We have no guarantee that the .o files even exist at this point (indeed, the will not after invoking the clean task). The .c are source and will always be there.

Rake Can Automatically Generate Tasks

Defining tasks in a loop is pretty cool, but is really not needed in a number of simple cases. Rake can automatically generate file based tasks according to some simple pattern matching rules.

For example, we can capture the above logic in a single rule … no need to find all the source files and iterate through them.

  rule '.o' => '.c' do |t|
    sh "cc -c -o #{t.name} #{t.source}" 
  end

The above rule says that if you want to generate a file ending in .o, then you if you have a file with the same base name, but ending in .c, then you can generate the .o from the .c.

t.name is the name of the task, and in file based tasks will be the name of the file we are trying to generate. t.source is the name of the source file, i.e. the one that matches the second have of the rule pattern. t.source is only valid in the body of a rule.

Rules are actually much more flexible than you are led to believe here. But that’s an advanced topic that we will save for another day.

Final Rakefile

Here is our final resule. Notice how we use the SRC and OBJ file lists to manage our lists of scource files and object files.

  require 'rake/clean'

  CLEAN.include('*.o')
  CLOBBER.include('hello')

  task :default => ["hello"]

  SRC = FileList['*.c']
  OBJ = SRC.ext('o')

  rule '.o' => '.c' do |t|
    sh "cc -c -o #{t.name} #{t.source}" 
  end

  file "hello" => OBJ do
    sh "cc -o hello #{OBJ}" 
  end

  # File dependencies go here ...
  file 'main.o' => ['main.c', 'greet.h']
  file 'greet.o' => ['greet.c']

Up Next

In our next tutorial, we will look at using Rake to handle some tasks other than compiling C code.

Received via EMail:

I have just started using the excellent Rake tool (thanks, Jim!) and I am at a bit of a loss on how to proceed. I am attempting to create unit test for some C++ code I am creating, […]

To adderess this lack, I’m going to post several Rake tutorial articles that will take you through some of the basics. Eventually, I’ll organize the articles into a document somewhere.

Here’s the first one!

The Problem

We will start with a very simple build problem, the type of problem that make (and now rake) were desiged to deal with.

Suppose I am a C programmer and I have a simple C program consisting of the following files.

main.c

  #include "greet.h"
  int main() {
      greet ("World");
      return 0;
  }

greet.h

  extern void greet(const char * who);

greet.c

  #include <stdio.h>
  void greet (const char * who) {
      printf ("Hello, %s\n", who);
  }

(Yes, it really is the old standard "Hello, World" program. I did say we were starting with the basics!)

To compile and run this collection of files, a simple shell script like the following is adequate.

build.sh

  cc -c -o main.o main.c
  cc -c -o greet.o greet.c
  cc -o hello main.o greet.o

For those not familiar with compiling C code, the cc command is the C compiler. It generates an output file (specified by the -o flag) from the source files listed on the command line.

Running it gives us the following results …

  $ build.sh
  $ ./hello
  Hello, World

Building C Programs

Compiling C programs is really a two step process. First you compile all the source code file into object files. Then you take all the object files and link them together to make the executable.

The following figure illustrates the progression from source files to object files to executable program.

Our program is so small that there is little benefit in doing more than the three line build script above. However, as projects grow, there are more and more source files and object files to manage. Recompiling everything for a simple one line change in a single source file gets old quickly. It is much more efficient to just recompile the few files that change and then relink.

But how do we know what to recompile? Keeping track of that would be quite error prone if we tried to do that by hand. Here is where Rake become useful.

File Dependencies

First, lets take a look at when files need to be recompiled. Consider the main.o. Obviously if the main.c file changes, then we need to rebuild main.o. But are the other files that can trigger a recompile of main.o?

Actually, yes. Looking at the source of main.c, we see that it includes the header file greet.h. That means any changes in greet.h could possibly effect the main.o file as well.

We say that main.o has a dependency on the files main.c and greet.h. We can capture this dependency in Rake with the following line:

  file "main.o" => ["main.c", "greet.h"]

The rake dependency declaration is just regular Ruby code. We take advantage of the fact that we can construct hash arguments on the fly, and that Ruby doesn’t require parenthesis around the method arguement to create a file task declaration that reads very naturally to the humans reading the rake file. But its still just Ruby code.

Likewise, we can declare the dependencies for creating the "greet.o" file as well.

  file "greet.o" => ["greet.c"]

greet.c does include stdio.h, but since that is a system header file and not subject to change (often), we can leave it out of the dependency list.

Finally we can declare the dependencies for the executable program hello. It just depends on the two object files.

  file "hello" => ["main.o", "greet.o"]

Notice that we only have to declare the direct dependencies of hello. Yes, hello depends on main.o which in turn depends on main.c. But the .c files are not directly used in building hello, so they can safely be omitted from the list.

Building the Files

We have carefully specified how the files are related. Now we need to say what Rake would have to do to build the files when needed.

This part is pretty simple. The three line build script that we started with contains all the commands needed to build the program. We just need to put those actions with the right set of dependencies. Use a Ruby do / end block to capture actions …

The result looks like this:

Rakefile

  file 'main.o' => ["main.c", "greet.h"] do
    sh "cc -c -o main.o main.c"
  end

  file 'greet.o' => ['greet.c'] do
    sh "cc -c -o greet.o greet.c"
  end

  file "hello" => ["main.o", "greet.o"] do
    sh "cc -o hello main.o greet.o"
  end

Trying it out

So, let’s see if it works!

  $ rake hello
  (in /home/jim/pgm/rake/intro)
  cc -c -o main.o main.c
  cc -c -o greet.o greet.c
  cc -o hello main.o greet.o

The command line rake hello instructs rake to look through its list of tasks and find one called "hello". It then checks hello’s dependencies and builds them if required. Finally, when everything is ready it builds hello by executing the C compiler command.

Rake dutifully reports what it is doing as it goes along. We can see that each compiler invocation is done in the correct order, building the main program at the end. So, does the program work? Let’s find out.

  $ ./hello
  Hello, World

Success!

But what happens when we change a file. Lets change the greet function in greet.c to print "Hi" instead of hello.

  $ xemacs greet.c
  $ rake hello
  (in /home/jim/pgm/rake/intro)
  cc -c -o greet.o greet.c
  cc -o hello main.o greet.o
  $
  $ ./hello
  Hi, World

Notice that it recompiles greet.c making a new greet.o. And then it needs to relink hello with the new greet.o. Then it is done. There is no need to recompile main.c since it never changed.

What do you think will happend if we run Rake again?

  $ rake hello
  (in /home/jim/pgm/rake/intro)
  $

That’s right … nothing. Everything is up to date with its dependencies, so there is no work for Rake to do.

Ok, sure. Rake is a bit of overkill for only two source files and a header. But imagine a large project with hundreds of files and dependencies. All of a sudden, a tool like Rake becomes very attractive.

Summary

What have we learned? Building a Rakefile involves identifying dependencies and the actions required to create the target files. Then declaring the dependencies and actions are as simple as writing them down in standard Ruby code. Rake then handles the details of building

What’s Up Next

We notice that even our small example has a bit of duplication in it. We have specify how to compile both C file separately, even though the only difference is the files that are used. The next installment will look at fixing that problem as well as introduce non-file based tasks, rules and file lists.

Until then … Code Red, Code Ruby.