Building Cassandra Thrift Bindings on OS X

A few weeks ago, I came to Rackspace to work full-time on Cassandra in their cloud division.  So far, I'm having fun and learning new things.  Apart from Cassandra, one of the projects I get to figure out is Thrift.  Thrift is a tool that allows you to define a service interface and then generate stubbed service bindings in different programming languages.  A programmer then takes the generated code and makes it do the things it is supposed to.  In an ideal world, this simplifies the process of, say, stubbing in a PHP client that can speak to a server stubbed in Java.

Right now, I'm the lone wolf in the office doing development on OS X.  The glitches so far have been minor, but I was forewarned that I might want to reconsider [using linux] when it came time to work with Thrift.  Well, that time started today.  I've been a faithful linux user for about 10 years, but I've been a faithful Mac user even longer.  I'm not ready to make the switch to full-time linux yet; I like my Mac.

Fortunately, Google was my friend when it came to figuring out the secrets of building Thrift on OS X.  Credit goes to Nathan Ostgard and his blog post for getting me going in the right direction.

1.  You definitely want to install macports.
2.  Install boost and log4j
sudo port install boost
sudo port install jakarta-log4j

3.  Download and install thrift
curl -o thrift.tgz "http://gitweb.thrift-rpc.org/?p=thrift.git;a=snapshot;h=HEAD;sf=tgz"
tar -xvf thrift.tgz
cd thrift
echo "thrift.extra.cpath = /opt/local/share/java/jakarta-log4j.jar" > ~/.thrift-build.properties
./bootstrap.sh
./configure --prefix=/opt/local

4.  You're going to get an error during configure:
./configure: line 16440: syntax error near unexpected token `MONO,'
./configure: line 16440: `  PKG_CHECK_MODULES(MONO, mono >= 2.0.0, net_3_5=yes, net_3_5=no)'

I couldn't figure out how to tell configure "no csharp, please" through the command line, so I just commented out lines 16439-16442 and ran configure again:

./configure --prefix=/opt/local

5.  You know the drill:
make
sudo make install

That's it for Thrift.  The next step is to generate the Cassandra client.  The Cassandra wiki has steps to generate a python client.  This works fine except that the thrift python module was installed to a place where the OS X python can't see it. You'll get the following error if you try to run Cassandra-remote:

Traceback (most recent call last):
  File "./Cassandra-remote", line 11, in
    from thrift.transport import TTransport
ImportError: No module named thrift.transport

There is probably a right way to fix this problem, a way that is right for OS X, but I had no patience.  I added the the following line to my ~/.profile:

export PYTHONPATH=/usr/lib/python2.6/site-packages

Restart terminal, navigate back to the directory where the python client was generated and try again.  Cassandra-remote should spew out a verbose usage directive.

That's it; you're done.

If you found this useful, or have feedback, please let me know.  I use gmail (gdusbabek).  I also emit the occasional tweet; just follow gdusbabek. 

If you're interested in learning more about Cassandra, there is an active and helpful IRC channel (#cassandra) on freenode and mailing lists as well.  The wiki also contains useful information for beginners.

The Saddest Story Ever Told

If you depend on a computer like I do, this one will make your innards shrivel up inside you and convince you to become a hermit...

The display unit in my company-owned MacBookPro when belly up on Monday night (thank you, Nvidia), one day before the one-year warranty would expire. (This was actually fortuitous, as my employer did spring for Applecare, but it was lost and never registered to my Macbook. Another story.) I quickly made plans to take the ailing machine to the Apple Store in Salt Lake City on Tuesday. Then I set about deciding on how I would work until I got it back. I am a remote employee; no IT staff to give me a loaner when equipment goes bad.

Here is the lay of the computers in my house:
1. Nicole has a 2.5 year old white Macbook (single core Intel 1.2 GHz?)
2. The kids have a brand new dual core Intel Atom 1.6GHz machine. It has a 1.5TB drive in it and serves mainly as a network storage device.
3. MythTv runs on a very old AMD Athlon 1.3 GHz machine.

Using the Myth server was out of the question. Not only is it too old to be useful in my work, it has reached mission-critical status in our household. We need our TV shows.

Nicole offered, and I breifly considered using her computer. I decided not to because I thought the kids computer might be a bit more powerful (though not a mac) and I didn't want to bother setting up a my dev environment there. You see, I had a plan.

The Plan involved using the kids computer. It was new, though underpowered, but I figured I could make it work fine. Before taking my macbook in I made a copy of the windows virtual machine I sometimes work in. I figured I could install VMWare Player on the kids computer, turn on the VM and I'd be in good shape. Setting up a non-development machine to do development work usually consumes about half a day for me, so I was keen on avoiding this.

But it wasn't meant to be.

The only file system on their computer that had space for the VM was formatted XFS, and I had been having little problems with it that I had mostly chosen to ignore. VMWare started complaining about 30 seconds after bringing the VM up. Then the OS dropped the volume completely. Ouch.

I ran xfs_repair and re-mounted the volume to try again. I'm a hopeful person and besides that, I needed to get some work done. Same exact problem. Ugh.

At this point my thoughts were focused on two things: First, I had a few work tasks that must get accomplished, and I still didn't have a dev environment to do those tasks. Second, I needed to get rid of XFS on that volume. It would not be viable in the short- or long-term.

First things first... I still haven't put the new 250GB hard drive in the myth server to replace the dead one. I decided to stick a USB enclosure on it, format it as ext3 and move the windows VM to it. At least then I could get into the VM without it crashing. The VM ended up performing quite poorly (due to running over USB), but would be ok for an afternoon of work.

Next I had to figure out how to get rid of the crappy XFS partition that was giving me grief. The only problem was that it had about 750GB worth of data that I couldn't just delete. It contained my music backups (expendable), DVD rips (not very expendable, but I could live without them), myth backups (needed for a full recovery if I ever got around to replacing the dead drive in the myth machine) and other assorted backups.

Luckly, I happened to have an extra 1.5TB drive laying around. I plan to put together a new Myth server in a few weeks and have started assembling the new hardware. I briefly tried installing windows XP on that drive (wasn't looking forward to working in a VM), but the aged XP installer didn't like the humongous drove. Oh well. You know the drill by now: USB enclosure, format ext3, and move data off the XFS volume. Ubuntu estimated it would take about 8 hours for the transfer, so I went to bed.

In the morning, the copy looked good, so I removed the USB drive and set it aside. Next I unmounted the XFS partition and reformatted it as ext3. Then I moved my 30 GB VM onto it and fired it up.

It works a lot better. Not only is it stable, but it runs a lot faster (this was expected). I suppose I can work like this for a week until my Macbook comes back.

The next step will be to restore the backup data off the USB drive back onto the backup machine. I'm losing the stomach for hardware maintenance, so I'm going to put it off for a day or two.

NOTE: I've used every model of 15-inch MacBookPro/PowerBook since the Aluminum G4. Each one has needed repairs within the first year (one even shipped with a defective trackpad). My take: always spring for the extended warranty.

iTMS link generator API

I set about looking for an API that utilizes the iTunes Music Store link maker, thinking they would be all over the place (deep links that is, and includes referral program ids). Five minutes of half-hearted googling didn't get me anywhere I needed to be, so I decided to take matters into my own hands. (Note: either I am a crappy googler, or the Apple legal team keeps these things off the air. Seriously, this code must have already been written 10 or 12 times now.)

A bit of poking around revealed that Apple (edgesuite, linkshare, linksynergy, whoever!) uses GET for their search forms. This makes writing a scraper API as complicated as parsing HTML, which is pretty easy in this day and age. I decided to use python, because I'm learning it by forcing myself to use it for all my extra-curricular projects.

I've released the source code under the code section at my website. The license is, uh... liberal. So don't be afraid to use it if you find it useful. It depends on the excellent BeautifulSoup library to do the heavy HTML lifting.

Here is some iTunes deep-linky goodness for you. These are the top 5 most played songs in my library. Buy them:
1. New Slang, by The Shins
2. Valley Winter Song, by Fountains of Wayne
3. Dreams Anymore, by The Magnetic Fields
4. Trust, by Gravenhurst
5. Saint Simon, by The Shins

BTW, I've resurrected Tagfriendly as the mp3 blog aggregator I've been working on, soon to have iTMS referral links. It isn't much, but I'm adding features all the time.

(Tagfriendly was, at one time, an automatic ID3 editing tool I created that mostly worked.)

Adding New Views to iPhone an Application

It is becoming clear that iPhone apps are in my future. So I've been taking time lately to familiarize myself with the development environment. For me, this means getting to know Xcode and Objective-C.

One possible way to code an iPhone app involves using views. In fact, there is a wizard that lets you create a simple iPhone app that includes a view. It makes things handy.

But what happens when you want to include another view? There is no wizard to make the process easy. It took me about 30 minutes to figure out how to do this, so I thought I'd create a post out of it in the hope that it might make things a bit easier for someone in the future (myself included).

You've just created a simple Xcode iPhone project that uses a view. Xcode leaves you with a controller (which is the view) and an app delegate. For the purpose of this demonstration, I called my project "SimpleView". The files of interest are SimpleViewAppDelegate (.m and .h).

We now wish to create a second view and hook it up to the delegate.

First thing is to create the controller class.

File > New File > Cocoa Touch Classes > UIViewController subclass.

I called my class SecondView.m. Press the finish button.

Next, we create the xib. Xibs are the artifacts produced by Interface Builder and make up the GUI of most applications.

File > New File > User Interfaces > View XIB.

Unless you have very good reasons otherwise, I think it makes good sense to name the xib the same as the controller. I named by xib SecondView.xib.

After that we need to connect the xib to the controller and vice versa. Most of this is done in Interface Builder, but we need to do a few things in Xcode so that information is available to Interface Builder.

First open SimpleViewAppDelegate.h. Note the @class and then create an IBOutlet in the @interface section for your new view. This exposes it to InterfaceBuilder. Make sure to create a @property too.

Then open SimpleViewAppDelegate.m. Import SecondView.h and synthesize the new controller and tell the window to use it instead of the default view.

That is all that needs to be done in the code. Everything else will happen from Interface Builder. Open up SecondView.xib into Interface Builder. You can do this by double-clicking on SecondView.xib from Xcode.

Notice how "File's Owner" is a "NSObject". We need to set that as "SecondView."


Select the "Object Identity" tab in the property exporer. Under "Class Identity" change the class from "NSObject" to "SecondView."

Next, click on the "Connections" tab (second one, with the blue arrow). We need to link the view in SecondView.m with the actual view in the xib. Do this by dragging the little grey circle next to "view" in the Inspector to the "View" element in the main xib window.

The UI in the Inspector then changes to show the linkage.

After that, open up MainWindow.xib. We need to tell it about this shiny new view we've created. This is done by dragging a "View Controller" object from the library into the main xib window.


The controller is now there, but we need to make sure to set it to the right class.


Do this by setting the class name in the identify tab of the inspector to "SecondView," much the same as you did when you were setting the "File's Owner" earlier.

Next, in the main xib window select "SimpleView App Delegate" and select the "connections" tab in the Inspector. Notice how the Inspector indicates "secondView" has no connections. Rectify this by dragging its grey circle and dropping it on the "Second View" controller you have just added to the main xib window.

Then click on the "Attributes" tab of the Inspector. "NIB Name" should be empty. You want to set that to the name of the xib you created for the view. In our case this is "SecondView."


That's it, you've just create a new view. You can force your application to use it by modifying your controller such that it loads "secondView.view" instead of viewController.view inside of "applicationDidFinishLaunching."

Now on to the gripes...

It took me at least 30 minutes to figure this out the first time I did it. Immediately after that I decided to try it again. My thinking was along the lines of "ok, this should only take five minutes." But no, it took me about 15 minutes instead. Now it could be that I am just incredibly dense (the accusation has been leveled at me more than once), or it could be that the process is full of minute details that are easy to be forgotten.

It seems to me that this process just begs to be put into a wizard. To create another view, the exact same steps will have to be followed--you gain nothing by having just already done it once. This is the kind of thing that the Microsoft dev tools really shine at, and something all IDEs should aspire to: take the drudgery out of development.

Update: Added a title, 24 Aug.