Book Review: Javascript The Definitive Guide 6th Ed.

TL;DR: This book is an excellent reference.

I decided to pick this book up a few years ago when I found out I'd be writing significant pieces of software using Node.js.  I had used Javascript in the past for client-side work in the browser, so I was familiar with it, but didn't have the deep understanding I figured I'd need to be productive on a large project.

I have the PDF version of the book.

Part I contains chapters that go into a lot of detail about how Javascript works.  This is where you can learn about types, Javascript's functional nature and how to build classes.  This is excellent material if your goal is to understand language internals.

There is a chapter that isn't there though: "Crap to Watch Out For."  This would include information about IEEE754 floating points, null vs undefined, truthy/falsy and so on.  Instead, that information is sprinkled around the book.  (More often than not, you figure it out on your own through sad experience.)

The chapter on server side Javascript was more or less pointless.  It was neither a good primer for Rhino or Node.js.

Part III contains an exhaustive reference of core Javascript APIs.  I found this section indispensable.  Every method of every key class type is documented.  Some include examples.  I found the cross-references (linked in the PDF) extremely helpful.

Parts II and IVwere not useful to me since I was not doing client side programming.

Overall, I'd rate this book positively.  It helped me figure things out and continues to serve as a reference.

Full Disclosure: This book was given to me by O'Reilly with the hope that I'd publish a review on it.  Other than the book, I have received no other consideration from O'Reilly.

Calculating Long Running Averages

Something I've been working on has me calculating averages over time on data arriving into a system.  I don't know beforehand how many pieces of data there will be, which makes the calculation difficult.  This operation needs to be done for both floating point and integers.

The floating point solution turned out to be fairly simple:

double average = 0d;
int count = 0;

// when a number comes in, do this:
average += (new_number - average) / ++count;

Here's how it works:  Realize that when a new number comes in, it is going to pull the average slightly higher or lower.  That difference is this calculation.  Since count is always growing this means that numbers arriving later have far less influence on the average unless they are fairly large (as it should be).

The integer solution looks different even though it is essentially the same thing.  What complicates the problem is that I want to avoid using floating point math if at all possible.  There are several different computations that would work, but here is the one that seemed to follow the true average as much as possible:

long average = 0, remainder = 0;
int count = 0;

// when a number comes in, do this:
count++;
average += (new_number + remainder) / count;
remainder = (new_number + remainder) % count;

Here is how it works:  since I cannot add incremental pieces of a whole number as they arrive (as I did in the floating point example), those incremental chunks are tracked in remainder.  Eventually, if the inputs are fairly uniform, enough delta builds up in remainder to add or subtract a number from the average, or it happens right away if the new number is significantly higher or lower than the average.

I wanted to verify correctness of the algorithm, so I wrote a simulator.  I found early on, the averages vary from the true average a bit, but this variance goes away as count increases.  To compensate for this, I keep track of the sum of all new_numbers and calculate average the normal way until either sum or count overcome predetermined thresholds.

Thoughts on Rdio

Clearly, each of the services get some things right.  I could go into detail about why I like Rdio and chose it over its competitors (Last.fm, Pandora and Slacker), but this post is mainly about why Rdio is right for me and a what would make it better.

The Good
Playlists
I like Rdio mainly because it gives me flexible listening options:

• It lets me have playlists with cherry-picked songs.
• It can generate playlists based on an artist.
• I can queue up entire albums.
• I can listen to songs offline using my android phone.

API
Although they didn't have it when I started my subscription, Rdio have added a public API, which is cool.  This means that developers will create interesting applications that use Rdio (unlike say, Pandora). I've played around with it a bit and it is pretty natural.

Discovery
I absolutely *love* that I can listen to entire albums at once.  Remember that band you heard that one song from once and you meant to go back and check them out?  Only, you had a terrible time finding a decent way to sample all their music and you weren't ready to make the commitment of actually paying for it.  (We have all made that mistake before.)  Rdio solves that problem: queue up their catalog and listen to everything for a day or two.  Then go buy, or not.

Room For Improvement
Offline Content

• I wish the desktop application had offline mode like the mobile app.  
• Managing offline media from the mobile application is cumbersome, especially if you have synced a lot of songs.  
• Managing media from the desktop application is even more difficult.
• It would be awesome if I could create a station based on an artist and then sync those songs (in much the same way I can create a playlist and sync those songs with one action).

Most of my grief with offline songs would be eliminated if there were a way to expire the offline content so I didn't have to manage it myself (like a DVR).  E.g.: keep this song for a [week, month, etc.], delete it after I play it [once, twice, three times, etc.]

Playlists
Rdio does not have "genius mode."  I would like to generate an awesome playlist based on a single song of my choosing (like iTunes).  They have artist-based playlists, but it isn't the same.  Theoretically, an enterprising programmer should be able to create software that exports an iTunes genius playlist (or any iTunes playlist) to an Rdio playlist.  Sounds like a fun weekend project.

I wish I could create multiple queues (e.g.: a "work" queue, a "home" queue, and a "quiet sunday" queue).  I could use playlists for this but I cannot add entire albums to playlist unless I add the songs one at a time.

The Rdio machine learning algorithms could take a lesson from Pandora or Last.fm.  They are not that good.

If you use queues and then go listen to another song, your currently playing queue item (an album) is lost.  This used to bite me a lot and annoyed me until I eventually changed my habits.  This could be resolved if Rdio just remember where I was in the queue when I play something adhoc.

Streaming
It takes too long to go from pushing the play button on my mobile player to actually hearing something.  If this is a problem of pushing bits, maybe Rdio could push some low-quality data (doesn't require as much bandwidth) at first to get some data on the users player quickly, then come in later with the full-quality audio.  If this is a latency problem, my bad.

Rating, Scrobbling and Data
It would be cool to scrobble love/hate from Rdio.  Currently I have to go to the Last.fm website to do it.  Also, it would be nice to rate songs (like iTunes) or tag them as I listen to them.  This is helpful for music discovery at times when I do not really want to be distracted by the process of copying down a song or artist name.

Something to Show For it All
This isn't a deal-breaker (none of it is, after all--I am paying for the service).  It saddens me that if I give Rdio $10 a month for the next 10 years and then stop, I will have nothing to show for it but the memories.  It would be great if I could keep some of the music permanently.

I realize that it is the same value proposition as cable TV and that I would have my cake and eat it too.  But folks--this is music.  I am used to keeping what I pay for.

Summary
I like Rdio.  Out of all the players so far, it works best for me.  It as the most flexible listening options of any streaming music service, but I think there is room for improvement.  I hope they keep up the good work.

Introducing Casserole

I've you've done much work with Cassandra clusters, you've probably gotten very familiar with bin/nodetool, which is the command line utility for poking Cassandra nodes.  If you are like most people, you have probably developed a love/hate relationship with it (your -h and -p fingers get sore quickly).

Well, here is something else you can love and hate.  Casserole is a gui tool that encapsulates some of the functionality of nodetool.  Right now, it primarily monitors clusters, but the groundwork is in for performing operations as well.

As the readme says: this tool currently sucks.  It hasn't been tested much and I've only worked on it at odd times over the last few weeks.  If you find bugs, please report them!  Even better, fix them and send me pull requests.

I've got branches that target 0.7-beta3 and 0.7-rc1 (yes, things are still changing too much IMO).  `ant run` should get you off the ground quickly.  Sorry: no 0.6 support at the moment.  My plan is to maintain Casserole as long as there is interest, or place it in cassandra/contrib if there is interest for that.

RESTful Cassandra

A lot of people, when first learning about Cassandra, wonder why there isn't any easier (say, RESTful) way to perform operations.  I did.  It didn't take someone very long to point out that it mainly has to do with performance.  Cassandra spends a significant amount of resources marshaling data and Thrift currently does that very efficiently.

So I put away my RESTlessness.

I've heard more people lately clamoring for the feature, so I gave some thought about how I'd go about it.  One approach would be to wrap Thrift.  That would be nice from a coupling standpoint, but I think performance would be pretty crappy.  After all, it is just adding another layer of marshaling that needs to be done; nobody needs that.

I eventually arrived at the decision that an HTTP Cassandra Daemon/Server pair similar to the existing Avro and Thrift versions would do the trick.  It would basically be a straight port, with a few minor caveats.  One big thing is that HTTP uses a new connection for each request, so storing Cassandra session information in threadlocals is gone out the window.  This means that authentication needs to be abandoned, done with every request, or we need to use HTTP sessions.  Punt.

Today, while half-listening to some lectures at ICOODB I decided to see how hard it would be to throw something together.  I ended up with two classes containing stripped down implementations of get and set.  I pushed the whole thing to github if anybody is interested.  I used the built-in Sun HTTP service because I didn't want any extra dependencies and building services on top of it is pretty straightforward.  Results are returned in JSON format that should match what you would see if you used sstable2json to export data.

This is clearly a proof of concept, but I think it demonstrates that the idea is sound and could be implemented fairly quickly.  Maintenance would be another story.  One problem with maintaining the Cassandra Avro bindings is that they regularly get out of sync with what is capable using Thrift.  An HTTP Cassandra wrapper would suffer the same fate without an active champion.  I'm interested, but I'm not *that* interested.

Anyway, have fun.

-- DETAILS --
The following URI formats are expected:
/get/keyspace/column_family/row_id/super_column/column_start/column_end/consistency_level.  If you don't want to pass it in, leave it blank.  Empty strings are interpreted as null when appropriate.

Here is an example from my tests:  http://127.0.0.1:9160/get/Keyspace1/Standard1/1//10/11/ONE

The main thing to deduce here is that the super column is empty (see the double slash?).  If you haven't realized by now, I've gone ahead with the assumption that your keys and column names are strings.  This isn't good enough.  All the details we need to become type-aware are available in the comparator for the column family.  As a shortcut for now, you can append "?asString" to the end of the URI to have all byte[] values converted to strings.  Without it they are displayed as hex.

Updating works the same way: /set/keyspace/coumn_family/row_key/super_column/column/value/consistency_level

e.g.: http://127.0.0.1:9160/set/Keyspace1/Standard1/1//11/aaa/ONE


UPDATE: I went ahead and created two additional implementations that use jetty (bare and with servlets).  This generated a bit more code, but opens up the way to getting sophisticated with sessions.

Running Multiple Cassandra Nodes on a Single Host

One of the first Cassandra tickets I worked on had me reviewing some code that visualized the node ring.  Properly testing the code required that I run a cluster. 

But I didn't have access to a cluster. Neither did I feel like creating a virtual cluster by building a VM and cloning it several times.  What I wanted was to run several instances of Cassandra on a single machine with multiple interfaces, all pointed at the same compiled code (without multiple svn checkouts).

The Cassandra wiki explains how to tweak Cassandra settings by editing cassandra.in.sh, but doesn't explain what needs to be done to run concurrent instances.

It turned out not to be too difficult.  I figured it might be daunting enough to Cassandra noobs (of whom we're seeing more of lately due to some great exposure), that a blog post might be helpful. 

This tutorial assumes that you'll want to run multiple instances of Cassandra on code built by ant and not a standalone jar.  I am also assuming that you are a) just playing around, or b) intend to do some development.  This is not a tutorial explaining how Cassandra should be run in production.

Note: I apologize for the way this looks.  Blogger is not a friend of ordered lists.

1. Make sure you've got aliases to localhost (e.g.: 127.0.0.2, 127.0.0.3, etc.).  Mac OS X doesn't have this enabled by default, so you'll have to manually create aliases:
   
   

   sudo ifconfig lo0 alias 127.0.0.2 up

   sudo ifconfig lo0 alias 127.0.0.3 up

2. Decide where you're going to keep things.  You can keep them with your code, but that just isn't neat.  Pick a directory somewhere, call it $cass_stuff.
3. Then, for each node in your little cluster, do this:
   
   
   1. From your svn checkout, copy the conf directory into $cass_stuff.  You can rename it to something like conf0 (or conf1, etc.).  I'll assume $conf from here on out.
   2. Copy bin/cassandra.in.sh to $cass_stuff.  Give it a name that helps you associate it with the conf directory you just created (node0.in.sh or whatever).
   3. Open node0.in.sh in an editor and make the following changes:
      
      
      1. Hardcode cassandra_home to the location of your trunk.  This will give you the flexibility to run Cassandra from anywhere.
      2. Set CASSANDRA_CONF to the conf directory you just created.
      3. In the JVM_OPTS change the jdwp address= setting.  The default is 8888, but you should include the unique IP you chose for this node along with the port, e.g.: 127.0.0.2:8888.  Not specifying a host causes the debugger to bind to 0.0.0.0:8888 and you'll have port binding problems when you bring up more than one node.
      4. pick a unique port for com.sun.management.jmxremote.port, but make sure you have at least one node listening on 8080 since all the Cassandra tools assume JMX is listening there.  Unfortunately, you can't pick the JMX host, 0.0.0.0 is assumed.  I was under the impression this could be changed by specifying java.rmi.server.hostname, but had no luck going down that road.  (Please leave a comment if you figure out a way for this to work, but I think it might be hopeless.)
   4. Open $cass_stuff/$conf/storage-conf.xml in an editor and make the following changes:
      
      
      1. specify unique locations for CommitLogDirectory and DataFileDirectory.  Don't bother with CalloutLocation or StagingFileDirectory.
      2. replace ListenAddress with the IP of your host.
      3. replace RPCAddress with the IP of your host.

To run you may wish to use another script for each node:

#!/bin/sh

CASSANDRA_INCLUDE=$cass_stuff/
export CASSANDRA_INCLUDE
cd
bin/cassandra -f

One downside to this approach is that if you're tracking trunk, it is your responsibility to make sure you notice changes to the default storage-conf.xml and cassandra.in.sh and apply them to your environments.

Cassandra is supported by an active and welcoming community.  If you'd like to learn more about the project, check out our wiki, mailing list or hop on #cassandra on freenode.

Dear Entrepreneurs, this is something I would pay for...

Dear Entrepreneurs,

This is something I would gladly pay $20 a month for...

A device that, according to my tastes, downloads new music from the Internet whenever it connects.  I would be able to listen to music without restriction while I am disconnected from the network.  I wouldn't own the music, except for roughly 20 tracks a month that I select which would then become mine as MP3s (for FLAC or whatever DRM-less technology makes sense).  I could then load them into iTunes, give them to my brother, or (if I'm feeling sinister) make them available on a P2P network.

The music could come from anywhere: iTunes, Amazon, The Labels, or artists themselves.

The content sources exist.  The recommendation engines exist.  Devices exist. 

I suspect the audience/market exists.  (At least, I hope so.  If not, and nobody is willing to pay for music, we're going to need to find another model.  And it will still necessarily involve a money exchange between producers and cosumers and/or advertisers.)

Is there such a system already?

Christmas Mix 2009

I've been making  Christmas mixes for my family the last couple years.  It's not your typical Bing Crosby stuff, and requires some digging on my part.  I finally started blogging about it last year and think I'm going to make it a tradition.  So here goes... Christmas with an indie slant.  And I did a better job checking on the lyrics this year for family appropriateness.

The links this year are coming at you from Lala by way of Google.  Message me if things stop working.  (This blog post has turned out much like my Christmas shopping: it gets sloppy towards the end.)

1.  "Holiday Road" by Matt Pond PA.  This is the only repeat from last years list.  I love this song because the vacation movies still connect with me at a level I am entirely uncomfortable with. 

2.  "Blue Christmas" by Dread Zeppelin.  Believe it or not, there is a nice smattering of Christmas to choose from with these guys. Where else can you get Elvis, Led Zeppelin, Reggae and Christmas in one track?

3.  "Christmas is Going to the Dogs" by Eels.  Hard choice between this and "Everything's Gonna Be Cool This Christmas".

4.  "I Wish It Was Christmas Today"  by Julian Casablancas.  This one is for the kids.  I wish that I could still feel the way I did when I was a young boy after Thanksgiving.  Christmas, although only four weeks away, seemed like it sat on the other side of eternity.  As an adult, it comes and goes so fast I barely have time to enjoy it.  Message to kids: enjoy it while you can.  Responsibility steals the fun from Christmas!

5.  "Christmas Time is Here Again (Bring Out the Joy!)" by My Morning Jacket.  Peaceful.  I'll let you google for this one.  It's a live take from a radio broadcast.

6.  "Listening to Otis Redding at Home During Christmas" by Okkervil River.  Not a traditional Christmas tune, but a good one to follow MMJ, if only for the indie vibe.  This song reminds me of "New Slang" by The Shins, but with less jade and desperation.  Slightly more hopeful. :)

7.  "X-Mas Card" by MU330.  Not my normal thing, but the instrumental intro with the horns is fun.

8.  "Yule Shoot Your Eye Out" by Fall Out Boy.  If you haven't checked out "Can You See Santa From the Southside," now is the time to skedaddle over to Amazon and do so.

9.  "Baby, It's Cold Outside (Mulato Beat Remix)" by Louis Armstrong and Velma Middleton.  Shopko gave away a Christmas sampler in 2004 and this was on it.  This is, by far, the Christmas album that gets play in our house (not the one this song links to).  It comes on while we're preparing meals and we find ourselves breaking frequently to get our grooves on.  No kidding.  Six people from 2 to 35 shaking a leg in the kitchen.

10. "O Come All Ye Faithful" by Weezer.  Traditional Christmas tune done right by a modern band.

And some bonus songs from last years mix:

Bonus 1:  "Fairytale of New York" by the Pogues.  This one is definitely not for the kids and is a guilty pleasure of mine.  Who can resist: "You're a bum, you're a punk / You're an old slut on junk."  Ahh, the holidays.

Bonus 2:  "Frosty the Snowman" by the Cocteau Twins.  Year after year, my favorite Frosty rendition.

Building Cassandra Thrift Bindings on OS X

A few weeks ago, I came to Rackspace to work full-time on Cassandra in their cloud division.  So far, I'm having fun and learning new things.  Apart from Cassandra, one of the projects I get to figure out is Thrift.  Thrift is a tool that allows you to define a service interface and then generate stubbed service bindings in different programming languages.  A programmer then takes the generated code and makes it do the things it is supposed to.  In an ideal world, this simplifies the process of, say, stubbing in a PHP client that can speak to a server stubbed in Java.

Right now, I'm the lone wolf in the office doing development on OS X.  The glitches so far have been minor, but I was forewarned that I might want to reconsider [using linux] when it came time to work with Thrift.  Well, that time started today.  I've been a faithful linux user for about 10 years, but I've been a faithful Mac user even longer.  I'm not ready to make the switch to full-time linux yet; I like my Mac.

Fortunately, Google was my friend when it came to figuring out the secrets of building Thrift on OS X.  Credit goes to Nathan Ostgard and his blog post for getting me going in the right direction.

1.  You definitely want to install macports.
2.  Install boost and log4j
sudo port install boost
sudo port install jakarta-log4j

3.  Download and install thrift
curl -o thrift.tgz "http://gitweb.thrift-rpc.org/?p=thrift.git;a=snapshot;h=HEAD;sf=tgz"
tar -xvf thrift.tgz
cd thrift
echo "thrift.extra.cpath = /opt/local/share/java/jakarta-log4j.jar" > ~/.thrift-build.properties
./bootstrap.sh
./configure --prefix=/opt/local

4.  You're going to get an error during configure:
./configure: line 16440: syntax error near unexpected token `MONO,'
./configure: line 16440: `  PKG_CHECK_MODULES(MONO, mono >= 2.0.0, net_3_5=yes, net_3_5=no)'

I couldn't figure out how to tell configure "no csharp, please" through the command line, so I just commented out lines 16439-16442 and ran configure again:

./configure --prefix=/opt/local

5.  You know the drill:
make
sudo make install

That's it for Thrift.  The next step is to generate the Cassandra client.  The Cassandra wiki has steps to generate a python client.  This works fine except that the thrift python module was installed to a place where the OS X python can't see it. You'll get the following error if you try to run Cassandra-remote:

Traceback (most recent call last):
  File "./Cassandra-remote", line 11, in
    from thrift.transport import TTransport
ImportError: No module named thrift.transport

There is probably a right way to fix this problem, a way that is right for OS X, but I had no patience.  I added the the following line to my ~/.profile:

export PYTHONPATH=/usr/lib/python2.6/site-packages

Restart terminal, navigate back to the directory where the python client was generated and try again.  Cassandra-remote should spew out a verbose usage directive.

That's it; you're done.

If you found this useful, or have feedback, please let me know.  I use gmail (gdusbabek).  I also emit the occasional tweet; just follow gdusbabek. 

If you're interested in learning more about Cassandra, there is an active and helpful IRC channel (#cassandra) on freenode and mailing lists as well.  The wiki also contains useful information for beginners.

id3 for Python

I've been meaning to package up some python code I wrote earlier this year and release it for free as open source software.  Several things held me back from doing this.  The biggest reason, by far, is that I'm still not proficient at python, and feel like I'm exposing myself by putting this code out for the world to see.

But then I remembered that my blog doesn't have a lot of readers anyway.  So no worries there.  And besides, maybe I can garner some constructive criticism to make my python code better.  :)


http://www.dusbabek.org/~garyd/id3_python/

This library is capable of reading most any correct Id3v2.3 or 2.4 tag, some incorrect ones, and then fails gracefully when things get hopeless.  (I run smoke tests on my mp3 collection, which has a lot of nasty debris from the Napster years.) 

It supports unicode, and does a good job handing PIC/APIC tags.  I should also mention it is in production at Tagfriendly.

For those that do come across this, I'm still trying to figure something out.  All the documentation I've come across says that I should structure my directories as such:

MyModule/
    id3/
        __init__.py
        stuff.py
    setup.py
    tests.py

I include id3.py inside the id3/ directory because that's where I think it should go.  But then when I build and test the module, the only way I can access the code is if I import id3.id3, but I wish I only had to import id3.  Clearly, I'm not doing something right.

My solution, and I know this isn't right, is to do away with the id3/ directory altogether and just have id3.py rubbing elbows with setup.py and test.py.  Anyone know what gives?

Well, enough of all that... id3 for Python is available at http://www.dusbabek.org/~garyd/id3_python/.

P.S.  Thanks to my friends on IRC for reminding me about this.

My New Workout

I have been attending the gym regularly, religiously even, for the last 10 years for early morning workouts.  What started out as short 20 minute workouts for an out-of-shape 25 year old have turned into 45 minute cardio sessions where I routinely burn 900 calories.  It's been a very good thing for me; I wouldn't trade it for the world.

When I started, I would bring headphones and watch the morning news.  Then began a succession of portable music players that started with a Diamond Rio 500, which I cherished, and culminated with the two iPods I currently use.  Over the last several years, gradually, the monotony of a daily workout began combining with my music against me, hampering my motivation.  Maybe I can chalk it up to age, or plateauing, but regardless of the cause: I needed to find something else.

I thought that an iPod video (30 GB) would help, but it turns out the screen is much to small to watch while the rest of my body is moving on a treadmill or elliptical machine.  (It is still great on an airplane though.)  I've been using an iPod shuffle for the last two years, alternating between general conference sessions and music.  I tried podcasts for a while, but the iPod has such a crappy podcast interface--you can't queue them up in a playlist, and I don't want to fiddle with buttons in the middle of my workout.

Total frustration.

Then something happened.  My gym began installing televisions on all the cardio equipment.  I thought my problem was solved--I'd be able to go back to just bringing a pair of headphones again.  That joy was short lived, as I realized that most of what's on television in the early morning is generally crap (informercials) and sometimes downright crude (infomercials for porn--no kidding!).  And the news is, well, frankly: not worth watching anymore.  Back to square one...

I don't know why it took so long for the idea to sink in, but I realized that I could connect my iPod video directly to the televisions on the cardio machines using a $5 composite cable I already owned (thank you, Sony), and watch my iPod videos on the TVs.  I finally gave it a try this morning.  I watched two TED talks, and an APM Marketplace podcast about high-frequency trading.  The best part is that when it was all over, my 45 minute workout felt like only 15 minutes had gone by! 

Undeniable WIN!

My next task is to find more podcasts that don't suck.  I've already found a decent language training podcast, but what would be really nice is to get a hold of some of the Google tech talks, as they are usually excellent and I don't mind watching them over (indeed, some of them ought to be watched multiple times to absorb the information).  I couldn't find them through the iTunes store, but maybe some kind soul has been kind enough to make them available on the outside.  Google, ironically, has not.  In fact, the Google tech talks are strewn across video.google.com and youtube.com now that Youtube is part of Google.  The older video.google videos are easily downloadable, but the Youtube ones are not.  At least, not without some real work.

Then again, there are enough [free] things at iTunes U to keep my mornings occupied for a long time.  They have a special section just for computer science educational content.  Double plus woot!

Anyway, all this is a long way of saying something short: iPod + cardio TV == excellent workout.

Adventures in Javascript

Note: Blogger doesn't give me a good way to preview the image before it goes live.  If it is too small, my apologies.  I'll try to get that fixed before your news reader pulls the feed.

The Question
Sometimes I think programmers tend to accept things the way they are without really questioning them, especially when it comes to language quirks.  This is something I remember coming across when I was first learning Javascript.  I was reminded of it recently while reading a Javascript book.

Javascript pros will probably quickly recognize what's going on here.

At first I thought the third definition of MyFunc would wipe out the others.  And just to verify that declaring a function in that manner normally makes it into the assumed contexts, I did it with YourFunc.

What gives?

Go ahead, take a guess... 

The Answer
The interpreter evaluates standalone functions before the other expressions.  So it is as if 'function MyFunc...' were written before everything else.  This can be verified by calling MyFunc() in the first line of the script.  The expressions (which include assignments) are then evaluated.  So what appears as the second assignment of MyFunc is really the third, and the third is really first.

My Take on Newspapers in America

I have read numerous articles and blog posts over the last 12 months about the decline of the newspaper industry.  Having spent a few years in the trenches myself, and having opinions on the matter, I thought I could add something to the discussion. 

A few things about me:  From 2000 to 2007 I worked as a programmer for Dgital Technology International (DTI), one of the leading newspaper software vendors (editorial and advertising) in North America and Europe.  I worked primarily on editorial products.  I have made roughly 25 visits to customer sites to perform troubleshooting or assist with software installations or upgrades.  So I have an insiders look about how newspapers operate, albeit with a technical slant.  I never learned much about how newspapers are run though.  That kind of knowledge would have made writing this easier.  If there are any inaccuracies (and there probably are), feel free to call me out on them, either in the comments or by way of email.

What is happening to newspapers?
To sum it up, the business is drying up.  Circulation numbers are in decline.  Whenever this happens, advertisers tend to pull back.  The result is a net loss of revenue. 

But why is this happening now?  Can it be blamed on the recession?  I was around during the last recession.  There was a lot of talk in 2001 and 2002 about newspapers going extinct then, since advertising rates had fallen and circulation was in decline, much like it is now.  In fact, the biggest difference between now and then is the severity of the problem.  For me, watching the Rocky Mountain News shut down operations really opened my eyes to the magnitude of the problem being faced by print journalists.

So if the slump can be blamed on the economy that would mean that circulation would probably not have declined during the good times.  Right?  Let's check the numbers.  I'll use the New York Times as a metric, even though it's not a good metric for the entire industry.  Here are the ups and downs of circulation starting in 1998.  I got these numbers from the NYTimes corporate site. 

1999: +2.2%,
2000: +1.3%,
2001: +0.1%,
2002: +3.8%,
2003: -5.3 %,
2004: +0.3%,
2005: +0.2%,
2006: +0.5%,
2007: -1.9 %,
2008: -3.9%,
2009: -3.5%

What I see here is one good year, a few years of stagnation, and a several years (the most recent ones) of significant decline.  I don't think the decline of readership can be based on economic factors; there must be something else.

Cultural shift?
Maybe we're changing as a people.  Unscientific guessing tells me that we're working more and doing more things (after all, we're multi-taskers).  We don't have time to read the paper in the morning or when we come home at night.  Further, when we do make it home, there are so many other things available to occupy us.  100+ channels on the tube, a backlog of events on the TiVo and this miraculous internet invention all await us.  There may be studies to support this, but I don't care.  This is my hypothesis and I'm running with it: less people read newspapers because they have too many other things to do, and those things are far more interactive.

What can newspapers do?
There are two obvious solutions.  The first is to cut costs.  The second is to get more readers.  Let's take a look at each one individually. 

First, cutting costs...  Running a newspaper isn't cheap.  There are reporters, editors, more editors, press operators, delivery people, circulation people and advertising people.  Some papers have gone web-only.  That eliminates the press operators and delivery people, and with the right software-a good chunk of the circulation department too.  Give a few good salespeople the right software and analytics and I think advertising and sales would be covered.  Could they skimp on reporters and editors--the real content producers?  I think so, to a degree.  See, if newspapers focused on local news and stopped trying to compete with CNN.com and USA Today for national and world coverage, I think they would be able to own that niche for some time.  Radio took a similar path on the advent of the television age when home listeners declined.  They found a niche to occupy (the car) and did well there for years.

Since I am in the software business, I'll go ahead and say that the software could be better.  I'm not referring to the quality of the software, but rather what it does and how it used.  For instance, at DTI we had release cycles of around 6 months.  Imagine waiting 6 months to see something like Twitter or Facebook integration, or integrating the latest Adobe CS suite.  Shortening that cycle would have been difficult from quality and cost perspectives.  By "better" I mean more nimble.  Smaller pieces working together instead of a monolithic suite.  It would be hard to do, but I think it could be done.

Next is readers: how do they get more readers?  My first suggestion is to reach out to them wherever they are--not just on their front lawns.  Smart phone adoption is on the rise.  I get a good chunk of news through my T-Mobile G1.  It is easy to scan my syndication feeds when I'm in line for something.  But I don't pay for it, which means nothing is going back to the paper, or whoever generated the content in the first place.  So increasing readership alone isn't going to keep the papers afloat.

I just don't see myself paying for news.  The Internet has made me used to having things free and my way, and I think a lot of people are with me on this one.

But this content has value, right?  Surely, someone is willing to pay for it.  I mean, if the switch were turned off today and all news went away, there would be a hole in my life.  Americans need and crave news coverage.

As in the past, I think the advertisers will step up.  But they're probably going to want to know a little bit about you, to, you know, make sure you get the right ad.  Consumers benefit from this.  Even if you routinely ignore all adversing, under this scenario, you're ignoring advertising targeted to you for stuff that might actually fit your interests.  I have found myself, on occasion, clicking on Google ads, even though it rarely results in me buying something.  The process works.

My predictions
In ten years, there will still be newspapers.  But they'll be owned by local television stations, who will have taken over as the main supplier of news content.  This suits me: their stories are shorter and I think that fits the lifestyles of more people nowadays.  These newspapers will be free and contain copy taken from the newscasts and modified slightly for print.  The ads in the papers will be the result of TV spot upsells.  There won't be classifieds since you can't execute a full-text search on a newspaper.

This model cannot currently exist in the United States because laws prevent interests that own one type of media outlet from controlling another type.  But that will change soon, if it hasn't already started.  (Tracking the status of this without becoming an expert is well nigh impossible.)

I don't know much about what kind of software TV stations use to handle their content (I know a bit about the advertising side), but I seriously doubt it would integrate well with print content at this point.  The disciplines haven't converged enough yet.

It will be fun to follow this story for a few years to see how things pan out.

Simplifying the Trip

Our vacation starts soon.  We're going some place far away by way of jet. 

We've had a Sony MiniDV camcorder for almost 10 years.  It still functions perfectly, but with four children in tow, it has started to feel more like an anchor than anything else.  The bug bit last summer and I bought a smallish camcorder that recorded directly to a hard drive.  Two days later with a bit of buyers remorse and the utter realization that the camera was completely incompatible with the MacOS (even though the box claimed otherwise) I returned it.

I decided to be a bit more pragmatic about it this year.  After some reasearch I realized that I didn't want to spend good money on a small SD camcorder knowing that I would probably want a HD recorder in the next 5 years.  But I'm not ready for an HD recorder yet either (I don't have the storage capacity).

So what do I do?

It didn't take long for me to come across the Sony Webbie and the Flip Ultra HD.  These are small devices that record directly to flash in HD mpeg-4 format, priced in the $150-$200 range.  Now don't be fooled by the "HD" in the product names.  While both devices do record HD video, they do so using a codec that his highly compressed and lossy (think youtube HD quality).  They don't do well in low-light situations either.

All this got me to wondering what the video quality on my digital camera, a two year-old Sony DSC-T20, was like.  Turns out, it records 640x480 mpeg-4 at 30fps.  This falls in the range of "good enough for me" and it's technically HD like the Sony anf Flip.  After all, I'm not going to use it to record piano recitals, awards ceremonies or new babies.  I just need something good enough to capture the cute moments that I want to remember when I'm on a trip.  And let's face it--Nicole and I only ever go back and watch these videos once or twice a year anyway.  (Maybe they will become more important as we age.)

So it's settled--I'm going to use my digital camera to record video on this trip.  I did decide to spring for a larger memory stick though--8GB where there were only 2GB before, and a second battery for hot swapping.

Side question: I wonder how long before these devices converge?  Tangential jab: And will it include a cell phone and app store as well?

Stupid CSS Tricks

I have recently made the committment to wrap my head around CSS.  In other words,  I'm tired of guessing.  Part of this experience will have me documenting my discoveries on this blog.

Here is what I learned today:

Pseudo-classes aren't just for anchors.

That's right.  You're supposed to be able to apply standard pseudo-classes to just about any selector you can come up with.  This means you don't to rely on a) jQuery or b) onmouseover/onmouseout to do the hover-effect work for you, so long as you can do it all in CSS.

Here is some sample code:

<html>
    <head>
        <style type="text/css">
  
        /* a class for an element */
        span.my_hover_class:hover {
            background-color:red;
            cursor:pointer;
        }

        /* attached to nested elements. */
        div > div > span > span:hover {
            background-color:green;
            cursor:pointer;
        }
 
        </style>
    </head>
    <body>
        <span class="my_hover_class">Should hover red (span.my_hover_class)</span>
  
        <div>
            Should not have hover effect.
            <div>
                <span><span>Should hover green (div>div>span>span)<span><span>
            </div>
        </div>
    </body>

</html>

I would like to show the effect here, but the Blogger editor insists on tidying up my pasted HTML.  I uploaded the file to my website.

This example was tested in Firefox and Safari on a Mac.

Tagfriendly Updates, 1 Jun 2009

I've been quietly hacking away, albeit slowly, on the next iteration of my Tagfriendly project.  The back end is in a state I am happy with, so this version is all about making the front end suck less.

You can sample the work as it progresses at http://www.tagfriendly.com/tf2. This is truly alpha-quality work, lacking much in the way of CSS, but I link here because I know a few people follow this blog and are interested in it.  That, and since I'm not actively working on the current version, I feel the need to show something for my efforts.  :)  I push updated code live about once or twice a week.

I use more AJAX this time around, including fancy transitions and assembling the page in chunks.  I'm not sure if this is the right approach yet or not; I'll see how far it gets me.

In a few days I'll start work on the new player, now that I've discovered there is a published API for the Yahoo! Media Player (that is very hard to find via google).

And Now, A Completely Unrelated Gripe:

I offcially hate eBay.  I sell one or two items on eBay per year, so things are usually in a different place every time I need to sell something.  I can handle that, but two things tonight really bugged me.

1.  I could not find a way to sell my stuff without signing up for and agreeing to automatic payment of seller fees.  This needless hoop was in my face and prevented me from doing what I needed to do.

2.  I have an unopened seasons 1-5 boxed set of 7th Heaven that I want to get rid of.**  Well, in its wisdom, eBay has decided to cap shipping on DVDs at $3.  In other words, I couldn't charge more than $3 if I wanted to.  But this boxed set weighs about 5 pounds and will easily cost me $6-7 ship.  I realize what they're trying to do, which is crack down on S+H abuse, but at what cost?  They know the exact item I'm selling because they had me enter the SKU.  They should know how much it weights and what it would cost to ship through various methods.  Total fail, eBay; it's craigslist from here on out.

**I really thought Nicole liked the show and it would make an excellent Christmas gift.  It turns out she only watches the show on Myth because she can skip the commercials and watch it at 150% speed.  Total fail, Gary.

I don't blog about music as much as I would like.  I used to do it a lot.

I listen to music all day, just about every day.  I subscribe to *a lot* of music blogs.  So much that I decided to create my own blog aggregator to help me keep track of things (Tagfriendly.com, for those of you who haven't been there yet.  You can create music-related RSS feeds based on your own search criteria.).

So the problem is not that I have little to say about new music, or the music scene in general, but that I have *so much* to say.  If I didn't have to make a living doing Other Things, I could write about music (not that anybody would care to read it) all day, every day.  Watching the indie scene blossom over the last five or six years with the help of the Internet has been a special thing for me.  I listen to more music now than I ever have.  And hey, record executives: I buy more music than I ever did too (my wife will attest).  Even so, I know that I am just moving slowly across one facet of a very large musical stone.  And that is a good thing--I won't be getting bored any time soon.

To that end, I have decided to carve out a few minutes each day and devote them to writing about music.  Some of it will end up on this here blog, hopefully as entries about new music, or music that is at least new to me.

Let's get started...

Some good music that you're probably not listening to

"Oh My God" by Ida Maria
I thought it was Clap Your Hands Say Yeah at first, but it wasn't.  This is an honest song sung urgently almost desperately, and right on the verge of losing it.  "oh you think I'm in control... oh you think it's all for fun".  Moving and tender ("find a cure for my life... put a price on my soul"), but dishing it out at the same time.  This is the kind of song that makes it hard to stay in my chair.  (wikipedia link)   

"In the Night" by Basia Bulat
That is the sound of an autoharp, if you're wondering.  Organic and edgy, pop-folk for the aughts (what will we call this decade?).  I've been saying for several years now that the best new music is coming out of Scotland and Canada.  Bulat represents the latter, hailing from Ontario.  I like this song because it makes me feel good.  It carries a message about rising up above struggles ("Storm and shadows fall to pieces / to my heart like a comet / carry so that I can / soar like an eagle").  And while it's still possible for struggles to keep us down ("sometimes it takes the night to fall"), it doesn't have to be that way all the time.  (wikipedia link)

"New Moon" by Sambassadeur
When it comes to twee, you can't beat Tweeden.  Er... Sweden.  Sambassadeur are utterly forgettable, but still very pleasant (think: "spring time").  Proof that good music doesn't have to captivate or mesmerize; it just has to not make me throw up in my mouth.  I really don't know much about this band.  I suppose they are few Swedish twenty-somethings who will be around for a few years before moving on to meatier projects.  (wikipedia link)

Yes Virginia, there is a garden...

So many domestic projects and assignments lately leaves little time for writing about other things.  Some of those projects have me out in the yard though, so things could be worse.

During the first week in April I started making trips to the garden to assess things.  To my surprise, I discovered that my artichokes from last year survived the winter.  (Look at the picture!)

I first planted artichokes three seasons ago.  Starting from seeds I ended up with two plants that year.  Neither of them flowered though.  I knew that it would take two seasons for the plants to mature, so I bedded them down well (I thought) and let them have the winter off.

They died.

So I started the process over last year.  For some reason I ended up with a lot more plants.  Seven of them.  Every seed I planted (from the same seed pouch as the year before) germinated.  The season ended, winter came and I bedded the plants down again.  I used newspaper and tree leaves, same as I did the year before.

They made it this time.  This means that I should have a really good crop of artichokes this year.  Nicole and I are both looking forward to them.

I planted corn this year, for the first time ever.  Normally I skip corn because it is so easy to buy and, to be honest, it would block out a lot of sun other plants in my garden could be using.  To keep me interested, I opted for an heirloom popcorn with a shorter habit.  The kids are really digging the fact that we are growing popcorn in our garden this year.  I noticed the first shoots (corn is a monocot!) poking up on Monday.

Beans are in the ground, a week late, but should be sprouting any time now.  Bush beans this year--I'm tired of managing the trellis.

Peas were planted at the end of April and are about a foot tall now.

I had to buy tomato starts for the first time in four or five years.  I started my seeds in peat pellets as I normally do.  Then I transformed them to styrofoam cups filled with a starter mix, again, as I normally do.  I couldn't find my usual starter mix, so I had to use an off brand.  That was a mistake.  The transplants struggled after that, and really struggled when I began to harden them off a few weeks later.  I've all but given up on them now.

We're doing several different kinds of winter squash this year.  Giant pumpkins too.  The aim this year is to keep it low maintenance, as we're taking a looooooong vacation in the middle of the summer.  Hopefully it will take care of itself, so long as the watering system holds out.

Tweets and Scrobbling

My hobby site, Tagfriendly, got with the times last weekend and started using Twitter.  It tweets new songs as they are found, at a rate of once every 5 minutes.

Kind of noisy I thought, until people started following.  TF followers are, for the most part, record labels.  But there are others who are just interested, and others who are just... well, I'm not sure.  I'm still trying to figure out how the underbelly of Twitter operates.

I also took time to get some deeper Last.fm integration put together.  If you allow it, Tagfriendly can scrobble the songs you love/ban while you are listening to them inside of the TF player.  My next step is to give the TF player some UI love so that it can pop out to be its own window and do a few other fancy things. 

I took a quick peek to see what it would take to scrobble all songs played, but it would require the user to give TF his password.  I'm not comfortable with that, so until there is another way, TF won't be scrobbling played tracks.

I'm going to focus a bit more on the player, as I think it can make the site more fun.

Hacking the Yahoo! Media Player

I've been using the Yahoo! Media Player to stream Mp3s at Tagfriendly.  Minor glitches aside*, the only real complaints I have is that it isn't skinnable and there is no public API.

About a month ago I started prodding it to see what would make it squeak.  This post documents some of what I've found.

Disclaimer: I skimmed over the YMP terms of service and don't believe I'm breaking any of the rules.  You should also know that YMP is a) beta software, and b) a hosted application.  This means that any code you write or use that relies on specific methods or objects will be brittle and prone to breaking when Yahoo! releases updates.  That said, you're on your own; I didn't make you do anything.

YMP is one of those nifty internet tools you can use by simply embedding a <script> element in your markup.  If the page you've embedded it in contains links to mp3s, or if it links to an XSPF playlist, YMP picks it up and makes those MP3s streamable.  This is powerful if you're a non-technical blogger and want to have an embedded player on your site.

My aim is to leverage all of that, but to take the Yahoo! face off and give it my own.  I use jQuery to manipulate the Tagfriendly DOM, but really, any decent JS toolkit should allow you to get the same results.

The first thing you need to do is know when YMP is finished loading so that you can tell the UI to go away.  Due to the fact that the Javascript you embed includes a bootstrap that downloads other things, you can't count on YMP to be ready when your document is.  This is easily accomplished with a JS timer that polls to check whether or not YAHOO.MediaPlayer.setPlayerViewState is defined.  When it is, call YAHOO.MediaPlayer.setPlayerViewState(YAHOO.mediaplayer.View.DisplayState.HIDDEN); to make the YMP user interface go away.

Now you are in the drivers set to start working with the MediaPlayer object.  Here are some useful methods:

YAHOO.MediaPlayer.getTrackPosition()
     Gets the position offset (in seconds) of the currently playing (or paused) track.

YAHOO.MediaPlayer.getTrackDuration()
     Gets the track duration (in seconds).  YMP appears to grab this from ID3 tags in the mp3, so you can't always count on this piece of data to be there.

YAHOO.MediaPlayer.play()
     Tells the player to play the currently queued song.

YAHOO.MediaPlayer.pause()
     Tells the player to pause the currently playing song.

YAHOO.MediaPlayer.previous()
     Go back to the previous song.

YAHOO.MediaPlayer.controller.EventManager.onNextRequest.fire()
     I found it odd that there was no YAHOO.MediaPlayer.next() method.  This method does what you expect the non-existent next() would.  There are corresponding fire() objects for play, pause and previous as well.

YAHOO.MediaPlayer.getMetaData()   
     Returns an object that describes the currently queued song.  Useful properties there, gathered from ID3 and the XSPF, include 'title' and 'artistName'.  There're more if you care to look.

That's basically it--all you need to subvert the YMP UI and handle things your own way.  This really is a guerrila API hack, as I don't think the YMP designers intended a public API.  You can see the results  on the front page of Tagfriendly.  I went as far as providing a progress bar that gets updated as the song plays.  The Javascript source is freely available too.

In the future I plan on displaying cover art and linking the songs to their Tagfriendly description pages as well as to music stores.



* I suppose not so minor because it really bothers me:  YMP breaks down when you try to use a locally hosted XSPF file or the machine it is hosted on is stuck behind NAT.  Firebug reports that YMP executes a GET with the URI to the XSPF.  The backend of that GET does some data-munging of the XSPF contents.  The results contain a basic JSONified playlist.  The only problem is that if the [development] server that hosts the XSPF is behind NAT, that backend can't fetch anything.