My Take on Newspapers in America

I have read numerous articles and blog posts over the last 12 months about the decline of the newspaper industry.  Having spent a few years in the trenches myself, and having opinions on the matter, I thought I could add something to the discussion. 

A few things about me:  From 2000 to 2007 I worked as a programmer for Dgital Technology International (DTI), one of the leading newspaper software vendors (editorial and advertising) in North America and Europe.  I worked primarily on editorial products.  I have made roughly 25 visits to customer sites to perform troubleshooting or assist with software installations or upgrades.  So I have an insiders look about how newspapers operate, albeit with a technical slant.  I never learned much about how newspapers are run though.  That kind of knowledge would have made writing this easier.  If there are any inaccuracies (and there probably are), feel free to call me out on them, either in the comments or by way of email.

What is happening to newspapers?
To sum it up, the business is drying up.  Circulation numbers are in decline.  Whenever this happens, advertisers tend to pull back.  The result is a net loss of revenue. 

But why is this happening now?  Can it be blamed on the recession?  I was around during the last recession.  There was a lot of talk in 2001 and 2002 about newspapers going extinct then, since advertising rates had fallen and circulation was in decline, much like it is now.  In fact, the biggest difference between now and then is the severity of the problem.  For me, watching the Rocky Mountain News shut down operations really opened my eyes to the magnitude of the problem being faced by print journalists.

So if the slump can be blamed on the economy that would mean that circulation would probably not have declined during the good times.  Right?  Let's check the numbers.  I'll use the New York Times as a metric, even though it's not a good metric for the entire industry.  Here are the ups and downs of circulation starting in 1998.  I got these numbers from the NYTimes corporate site. 

1999: +2.2%,
2000: +1.3%,
2001: +0.1%,
2002: +3.8%,
2003: -5.3 %,
2004: +0.3%,
2005: +0.2%,
2006: +0.5%,
2007: -1.9 %,
2008: -3.9%,
2009: -3.5%

What I see here is one good year, a few years of stagnation, and a several years (the most recent ones) of significant decline.  I don't think the decline of readership can be based on economic factors; there must be something else.

Cultural shift?
Maybe we're changing as a people.  Unscientific guessing tells me that we're working more and doing more things (after all, we're multi-taskers).  We don't have time to read the paper in the morning or when we come home at night.  Further, when we do make it home, there are so many other things available to occupy us.  100+ channels on the tube, a backlog of events on the TiVo and this miraculous internet invention all await us.  There may be studies to support this, but I don't care.  This is my hypothesis and I'm running with it: less people read newspapers because they have too many other things to do, and those things are far more interactive.

What can newspapers do?
There are two obvious solutions.  The first is to cut costs.  The second is to get more readers.  Let's take a look at each one individually. 

First, cutting costs...  Running a newspaper isn't cheap.  There are reporters, editors, more editors, press operators, delivery people, circulation people and advertising people.  Some papers have gone web-only.  That eliminates the press operators and delivery people, and with the right software-a good chunk of the circulation department too.  Give a few good salespeople the right software and analytics and I think advertising and sales would be covered.  Could they skimp on reporters and editors--the real content producers?  I think so, to a degree.  See, if newspapers focused on local news and stopped trying to compete with CNN.com and USA Today for national and world coverage, I think they would be able to own that niche for some time.  Radio took a similar path on the advent of the television age when home listeners declined.  They found a niche to occupy (the car) and did well there for years.

Since I am in the software business, I'll go ahead and say that the software could be better.  I'm not referring to the quality of the software, but rather what it does and how it used.  For instance, at DTI we had release cycles of around 6 months.  Imagine waiting 6 months to see something like Twitter or Facebook integration, or integrating the latest Adobe CS suite.  Shortening that cycle would have been difficult from quality and cost perspectives.  By "better" I mean more nimble.  Smaller pieces working together instead of a monolithic suite.  It would be hard to do, but I think it could be done.

Next is readers: how do they get more readers?  My first suggestion is to reach out to them wherever they are--not just on their front lawns.  Smart phone adoption is on the rise.  I get a good chunk of news through my T-Mobile G1.  It is easy to scan my syndication feeds when I'm in line for something.  But I don't pay for it, which means nothing is going back to the paper, or whoever generated the content in the first place.  So increasing readership alone isn't going to keep the papers afloat.

I just don't see myself paying for news.  The Internet has made me used to having things free and my way, and I think a lot of people are with me on this one.

But this content has value, right?  Surely, someone is willing to pay for it.  I mean, if the switch were turned off today and all news went away, there would be a hole in my life.  Americans need and crave news coverage.

As in the past, I think the advertisers will step up.  But they're probably going to want to know a little bit about you, to, you know, make sure you get the right ad.  Consumers benefit from this.  Even if you routinely ignore all adversing, under this scenario, you're ignoring advertising targeted to you for stuff that might actually fit your interests.  I have found myself, on occasion, clicking on Google ads, even though it rarely results in me buying something.  The process works.

My predictions
In ten years, there will still be newspapers.  But they'll be owned by local television stations, who will have taken over as the main supplier of news content.  This suits me: their stories are shorter and I think that fits the lifestyles of more people nowadays.  These newspapers will be free and contain copy taken from the newscasts and modified slightly for print.  The ads in the papers will be the result of TV spot upsells.  There won't be classifieds since you can't execute a full-text search on a newspaper.

This model cannot currently exist in the United States because laws prevent interests that own one type of media outlet from controlling another type.  But that will change soon, if it hasn't already started.  (Tracking the status of this without becoming an expert is well nigh impossible.)

I don't know much about what kind of software TV stations use to handle their content (I know a bit about the advertising side), but I seriously doubt it would integrate well with print content at this point.  The disciplines haven't converged enough yet.

It will be fun to follow this story for a few years to see how things pan out.

Mp3 Blog Aggregator: status and some code

My last post was a lamentation about how the current set of mp3 blog aggregators don't do it for me, and at the same time a declaration that I would do something about it.

I've spent my spare moments this week hacking at the problem and it's starting to bear fruit.

The first of it is a simple id3 reader implemented in python. It simply reaches out over the tubes and grabs the id3 information from an mp3 that is hosted on a server somewhere. Nothing too complicated, except that it can be configured to extract any images that might be embedded in the mp3.

Knowledgeable readers might be asking: "why didn't he use one of the three or four existing python id3 libraries?" The answer is this: I planned on creating a blog crawler (mentioned later) and a website for this idea, and would do it all in python. As a warm-up exercise, I figured it would be good to create a simple id3 reader. I had already done it in Java, so it mainly became an act of seeing how the Java idioms I am currently used to translate over into python. (Note: if you bother to download and read the code, please be gentle. It's the first real python I've written. Feedback is appreciated too.)

The crawler is mostly done. It came together more quickly than I thought, although it still has rough edges. It runs a few times a day, notes new blog posts and gathers what information it finds into a database (postgres).

The website is where the work needs to be done now. I have gotten no further than creating a few simple query+display pages that I've been using to view results from the crawler. I've experimented with different ways to present data (entry-centric vs mp3-centric) and still haven't come up with something I like. I've got time though. And the longer I wait, the more useful data I'll have from the crawler.

I'm still using pylons for the website, although I had second thoughts after spending too much time fighting mako and the way it manhandled my nice unicode mp3 tags.

I have yet to tackle the problem of dynamic RSS generation, but I have some good ideas in my head for that.

MP3 Blog Aggregators

I started this post as a "Dear Lazyweb" but decided against it. I'm actually going to do something about this particular problem.

I subscribe to quite a few MP3 blogs. As I told a friend recently, "there is no quenching the thirst for new music." It occupies a fair amount of my internet time, but has lead me to good tunes. And good tunes translates to a happy Gary, so it's time well spent.

One of the biggest problems with MP3 blogs as they exist today is that if you come across something good, there is no easy way to say "find me more like this" without doing all the legwork yourself. Sure, the poster might mention "this sounds like X, Y, or Z," but that is just one persons opinion. I've tried a few mp3 blog aggregators and found a decent one, but there is still a lot noise and I'm not very happy with it. It is just that much better than the competition, which is poor to start with.

Social networking to the rescue. Audioscrobbler has silently been rolling out more APIs over the last 12 months, mostly without anybody noticing. They haven't shut off any of the old services that I currently use (the music section on my website), but they are requiring an API key to use the new services. One of the old services that has been reincarnated in the new is the "find simliar" feature, where an artist or song is supplied and related matches are returned. To be fair, Pandora does a better job of this than Last.fm, but Pandora has no API that I can use.

So the project, and I've already started pounding out code, is to scan the music blogs, figure out how they link to MP3s, grab the ID3 tag and then store that information in a database along with a link back to the original post. Several interesting things could be done with that information:

1. Find me posts (and mp3s) of related artists or songs. That isn't terribly interesting, but takes some of the legwork out of doing it manually.

2. Zeitgeist tracking. The difference between the good music blogs and the less-good blogs is that the good ones go out on their own to find new music, artists and information, rather than recycle what is being hashed on other blogs.

3. Search-based aggregate syndication feeds. Imagine being able to create an RSS/ATOM feed based on aritst or artist-similarity. This feed would aggregate all the posts that you find interesting. For example, you could create a feed that would return all posts mentioning Belle & Sebastian, or posts that contain references to artists similar to Belle & Sebastian. Pre-filtered information like this is a great time saver.

That's about it for now. I don't know if this kind of tool would be very useful for many. But it is fun to hack at and I've had a hard time lately finding recreational programming tasks that engage my passions. Also, this one is my first real venture into python, which is turning out to be quite fun. The website is in pylons, the spider is plain python, and they both communicate with the database using SQLAlchemy.

Wood

It amazes me that in terms of durability, beauty and quality in general, that wood is still high up on the list of "stuff to make things out of."

So many things have come and gone over the millenia (we don't write on parchment anymore). It is nice to see that some things are still with us and have no signs of being replaced.

In the Year 2000...

Dear Lazyweb,

I have a lot of ripped CDs and DVDs; I use MythTv and I have a mac.

And I want to access them all using Frontrow *and* Mythfrontend.

Thanks, Gary

This is one of those problems where the pieces are all 99% there, but not enough people want the solution bad enough (myself included) to put them all together.

Firefly can take a collection of mp3s and serve them up to iTunes/Frontrow. It can even handle video to an extent (must be mov or m4v). That is 50% of the problem, leaving only a way to get at the Myth TV shows.

There are several ways this could probably be handled. To me, the path of least resistance would be this:
1. A vlc setup to transcode and stream the MPEG-2 Myth tv shows into something iTunes can swallow. This is already doable.
2. A plugin for Firefly that will a) query mythtv using either the database or UPnP and then b) stream the video from vlc. There will be a little bit of glue that is needed to map items exported by the database/UPnP to urls for vlc.

I have two weeks off at Christmas. Maybe it's time to get my hands dirty and give something back to the community I've enjoyed for so long.

MediaMVP + DD-WRT + MythTV

Two years ago this month I set up our MythTV system. At first, we kept in in our living room where the computer was. We watched tv on a 22-inch flatscreen monitor. Nothing fancy, but it served us just fine.

About a year ago, we did some renovations on our main level that turned our long living room into a smaller living room plus an office for Nicole. The computer had to go. We stuck it downstairs in the basement family room and stuck the monitor in Nicole's new office. No more myth, at least not connected directly to the server.

For the last year, we've been watching TV primarily on our computers--either a 15-inch MacBook Pro, or a 13-inch MacBook. Again, nothing fancy, and it would be fine for us, except that we don't trust our kids to fire up Mythfrontend on our laptops. We know what they're capable of.

Without a doubt--I had to make it so the kids could watch Mythtv on the TV in the basement, while minimizing the likelihood of them damaging any equipment.

My first attempt was to mod an first gen XBox and install Linux. I ended up ruining a mod chip and probably ruining the xbox too. I put it back together yet to see if it still works.

Last week I bought a Hauppauge MediaMVP for about $70. After a non-trivial amount a tweaking, I managed to make it talk to my Myth system (thanks to mvpmc). It handles SD TV just fine and has component and S-video out. Perfect for our needs. The kids are going to be quite happy.

Minor gripe: I need to figure out how to change the font used by the on screen display. Our TV is 27 inches and I have a hard time reading the OSD.

For the technically inclined, here is what had to happen to make it all work:

1. On the backend, enable tftpd:
sudo apt-get install xinetd tftpd tftp

2. Create /etc/xinit.d/tftp and then restart xinetd.

service tftp
{
protocol = udp
port = 69
socket_type = dgram
wait = yes
user = nobody
server = /usr/sbin/in.tftpd
server_args = /tftpboot
disable = no
}

3. Create /tftpboot and place dongle.bin.mvpmc and dongle.bin.mvpmc.config there.
dongle.bin.mvpmc.config is:
mvpmc -s 10.10.2.32 -u mythtv -p mythtv -y 10.10.2.32 \
-T mythconverg --startup mythtv &
(yes, you now know my mythtv host, user and password!)

4. Create a dhcp boot environment. I use DD-WRT. The solution is to services > services and add some DNSMasqu otpions:
dhcp-boot=dongle.bin.mvpmc,,10.10.2.32
The IP address is the IP of the tftp server.

That's it. When I get enough motivation, I'll write up a set of proper steps and add it to my MythTV instructions.

Sign extension in java

I was talking with a programmer recently about a bit-twiddling problem. The conversation brought to mind things I learned a long time ago when I was new to Java.

One of the very unfun things about java is that there are no unsigned types. This means that if the hi-bit of a byte is on it gets extended when promoting a byte to an int. This applies to 8 bit unsigned values in the range from 128-255 (0x80-0xff).

This leads some some puzzling problems when you're byte munging in Java for the first time. Consider this:


System.out.println(Integer.toBinaryString(128));
> 10000000

System.out.println(Integer.toBinaryString(-128));
> 11111111111111111111111110000000

byte b = 0x80;
System.out.println(Integer.toBinaryString(b));
> 11111111111111111111111110000000

System.out.println(Integer.toBinaryString(0x80));
> 10000000

System.out.println(0x80 == b);
> false

Huh?

At first it seems silly that 0x80 != b when b was explicitly set to 0x80. The problem here is that java promotes b to an integer so it can be compared with 0x80 (which is already an integer, even though your mind wants to treat it as 8 bits). The process of promoting b (which is a negative number as far as a signed byte goes) extends the 1 in the hi-bit.

Another way to explain this is to say that casting 0x80 to a byte converts it from a positive 4-byte integer to a negative 1-byte integer. Casting 0x80 to a byte solves the problem:

System.out.println((byte)0x80 == b);
> true

The java libraries get around this problem by treating all bytes as integers (look at InputStream and OutputStream to get a feeling for this). In that case, it is probably more correct to:

System.out.println(0x80 == (0x000000ff & b));
> true

One more thing to be aware of this that java has an unsigned right shift operator: >>> that always shifts in a zero regardless of sign:

System.out.println(Integer.toBinaryString(b>>1));
> 11111111111111111111111111000000

System.out.println(Integer.toBinaryString(b>>>1));
> 1111111111111111111111111000000

Be aware that this will byte you (har!) when you assume you're shifting an 8-bit value though.

P.S. By popular demand (my wife), my next post will be non-technical.

Bayes Filtering in Javascript

I use google reader for my rss feeds. Something I'm surprised they haven't added yet are "predictive labels" that attempt to classify your feeds depending on how you've trained it.

Arguably, this would only be useful if you have a lot of feeds and wish to weed out noisy posts (like when a technical blogger starts making political posts--ugh!).

Thinking it would be a fun hack I set out to build a naive Bayes classifier in Javascript. It turned out to be easier than I thought.

Some observations:
1. My math is rusty.
2. Fancy mathematical diagrams sometimes don't translate so easily to code.
3. Edge cases still suck.

The fruits of my efforts are included in an iframe below. If you prefer, you can visit the actual page.

The next step would be to inject this into google reader somehow to predictively classify a post. I'm not sure of the best way to do this. Greasemonkey could handle capturing when a post is tagged, but physical storage would be required to store the probability graphs. That puts me squarely in the realm of traditional Firefox plugins. I'll have to do some more investigating.

Fun learning project though.

BEHOLD, MY IFRAME:

Spare Hacking Cycles

Many programmers hack in their spare time. I am no exception. I tend toward music-related projects, but have fun doing almost anything that involves a keyboard and a little thinking.

I got around to looking at Snowl about a week ago. The concept is interesting by itself, but my main intent was to get familiar with Firefox plugin APIs. Having investigated things I bit, I think that Gecko as an application platform is an under-realized gem. The notion that entire rich, cross-platform desktop applications can be written mostly in Javascript is very appealing.

A few observations:
1. Plugin development is painful. Instead of waiting for changes to recompile, I find myself waiting as I restart Firefox. It is entirely frustrating because, having done a fair bit of Javascript/HTML programming, I am used to just refreshing a browser page. I'm curious what hardcore plugin programmers do to take the edge off developing.

2. For the uninitiated, the Mozilla API documentation is difficult to grasp quickly. Some of the Mozilla docs I ran into referred me to XULPlanet, the docs of which claim to be obsolete and refer me back to Mozilla. When I needed a DOM parser, I was able to quickly find that one was available (by examining Components.classes), but it took me a fair amount of time to find the right documentation that showed me how to use it. (It turns out that the DOM parser available to Firefox a plugin uses roughly the same API used by DOMParser in standard HTML/Javascript pages.)

3. The community is very helpful. This makes up for some of the shortcomings of the documentation. It also helps that there are many open-source Firefox plugins already available to learn from--a great feature of open-source software in general. This is a great ecosystem to be working in.

Snowl uses an SQLite database to keep track of messages and other data. One neat Firefox plugin that ended up helping me figure things out was SQLite Manager. Just point at a file and it does the rest.

I managed to figure out enough of the existing Snowl codebase that I got simple Audioscrobbler loved/banned track support implemented in about a day. More work needs to be done to make the scrobbler messages more interesting.

The changes I have made so far are available in my (newly created) public hg repo.

Technical Hubris

Programmers must often strike a balance between correct and pragmatic. This is often the difference between do it right and merely get it done. Rarely does the intersection of the two equal one and the other. But if you ask just about any programmer, he/she will almost always prefer a correct approach versus a more-pragmatic-but-less-correct approach when given the choice. Why? That's just the way most programmers are wired. That, and we're kind of arrogant.

The conflict is that software users almost always prefer that software be usable (pragmatic) even if it comes at the expense of being correct.

A case in point: one problem I worked on recently was to generate a unique set of keyboard shortcuts for a given set of buttons. The stipulation is that a shortcut (keystroke) could only be used once and must be included in the text of the button. I created an algorithm that was guaranteed to find a solution if one existed. It did this by recursively finding the characters (shortcut candidates) that were unique to each button over the set of all characters already mapped and all buttons that had not been assigned a shortcut. When the recursive algorithm bottomed out, all possibilities had been exhausted. If there were any buttons left over it was because assigning any shortuct to one of the remaining buttons would require a shortcut to be taken from another button. My algorithm was correct and I was pleased that I had thought of it to boot.

So given two buttons "My Button One" and "My Button Only", it would return the shortcuts as "e" and "l" respectively (first unique character in each sequence). It turns out this isn't as intuitive as returning, say "M" and "B". Ouch! It would have been difficult to make the correctness argument in view of the seemingly nonsensical shortcut characters that were returned.

The solution I settled on was to alter the algorithm to prefer the first character of the first word, and then the second word, etc. when selecting a shortcut. If no shortcut is found it then starts using other characters from the text. Also, I hard-wired a few words to shortcuts (exit -> x, quit -> q, etc.). This algorithm isn't guaranteed to find a solution if one exists (it isn't correct), but it produces shortcuts that make more sense to users.* It is practical instead.

* Geeky/gory details: here is where the new algorithm breaks down. Consider the following buttons "ab" "bc" "cd" "ac". The modified algorithm would produce shortcuts "a" "b" "c" and nothing for the fourth button. But the old algorithm would have come up with "b" "c" "d" and "a". Chances are slim that such a deviant case would ever be encountered in the wild though.

Adding New Views to iPhone an Application

It is becoming clear that iPhone apps are in my future. So I've been taking time lately to familiarize myself with the development environment. For me, this means getting to know Xcode and Objective-C.

One possible way to code an iPhone app involves using views. In fact, there is a wizard that lets you create a simple iPhone app that includes a view. It makes things handy.

But what happens when you want to include another view? There is no wizard to make the process easy. It took me about 30 minutes to figure out how to do this, so I thought I'd create a post out of it in the hope that it might make things a bit easier for someone in the future (myself included).

You've just created a simple Xcode iPhone project that uses a view. Xcode leaves you with a controller (which is the view) and an app delegate. For the purpose of this demonstration, I called my project "SimpleView". The files of interest are SimpleViewAppDelegate (.m and .h).

We now wish to create a second view and hook it up to the delegate.

First thing is to create the controller class.

File > New File > Cocoa Touch Classes > UIViewController subclass.

I called my class SecondView.m. Press the finish button.

Next, we create the xib. Xibs are the artifacts produced by Interface Builder and make up the GUI of most applications.

File > New File > User Interfaces > View XIB.

Unless you have very good reasons otherwise, I think it makes good sense to name the xib the same as the controller. I named by xib SecondView.xib.

After that we need to connect the xib to the controller and vice versa. Most of this is done in Interface Builder, but we need to do a few things in Xcode so that information is available to Interface Builder.

First open SimpleViewAppDelegate.h. Note the @class and then create an IBOutlet in the @interface section for your new view. This exposes it to InterfaceBuilder. Make sure to create a @property too.

Then open SimpleViewAppDelegate.m. Import SecondView.h and synthesize the new controller and tell the window to use it instead of the default view.

That is all that needs to be done in the code. Everything else will happen from Interface Builder. Open up SecondView.xib into Interface Builder. You can do this by double-clicking on SecondView.xib from Xcode.

Notice how "File's Owner" is a "NSObject". We need to set that as "SecondView."


Select the "Object Identity" tab in the property exporer. Under "Class Identity" change the class from "NSObject" to "SecondView."

Next, click on the "Connections" tab (second one, with the blue arrow). We need to link the view in SecondView.m with the actual view in the xib. Do this by dragging the little grey circle next to "view" in the Inspector to the "View" element in the main xib window.

The UI in the Inspector then changes to show the linkage.

After that, open up MainWindow.xib. We need to tell it about this shiny new view we've created. This is done by dragging a "View Controller" object from the library into the main xib window.


The controller is now there, but we need to make sure to set it to the right class.


Do this by setting the class name in the identify tab of the inspector to "SecondView," much the same as you did when you were setting the "File's Owner" earlier.

Next, in the main xib window select "SimpleView App Delegate" and select the "connections" tab in the Inspector. Notice how the Inspector indicates "secondView" has no connections. Rectify this by dragging its grey circle and dropping it on the "Second View" controller you have just added to the main xib window.

Then click on the "Attributes" tab of the Inspector. "NIB Name" should be empty. You want to set that to the name of the xib you created for the view. In our case this is "SecondView."


That's it, you've just create a new view. You can force your application to use it by modifying your controller such that it loads "secondView.view" instead of viewController.view inside of "applicationDidFinishLaunching."

Now on to the gripes...

It took me at least 30 minutes to figure this out the first time I did it. Immediately after that I decided to try it again. My thinking was along the lines of "ok, this should only take five minutes." But no, it took me about 15 minutes instead. Now it could be that I am just incredibly dense (the accusation has been leveled at me more than once), or it could be that the process is full of minute details that are easy to be forgotten.

It seems to me that this process just begs to be put into a wizard. To create another view, the exact same steps will have to be followed--you gain nothing by having just already done it once. This is the kind of thing that the Microsoft dev tools really shine at, and something all IDEs should aspire to: take the drudgery out of development.

Update: Added a title, 24 Aug.

AM Radio

Old passions have been stirring within me...

As a boy, I had a great interest in radio. It fascinated me that five dollars in parts allowed me to pick up signals originating half-way across the world. By the time I was 13 I had made quite a few receievers and even a few simple transmitters. For the longest time I ignored the AM and FM radios that were all around me, preferring to search out exotic shortwave broadcasts and far-slung CB and ham radio operators.

On a family road trip, my father turned the radio on to AM and tuned in station KOA which originates out of Denver, Colorado. It was late at night and we were in the western U.S., somewhere between central Texas and Montana--I don't remember where exactly. But the point is that were were a long ways away from Denver. I asked Dad how we could pick up such a far away station and he explained that a few stations had extra powerful signals that could be picked up from great distances. I thought that was cool.

Later on at home I tried to tune in KOA without success. At first I was puzzled. It took time and research to figure out how things worked. By rule, KOA and other stations like it must wait until sunset to turn up the juice to 50,000 watts. By that time the local and regional stations using a frequency (AM 850 in the case of KOA) are off the air. This rule was created by the FCC in the early days of radio before there were a lot of broadcasters. The argument was that rural listeners with no local radio station would benefit from the signal pumped out by these far-away big stations. These stations are known as clear channel stations (not to be confused with Clear Channel Communications).

Radio manufacturers took advantage of this. My father has an antique radio. Manufactured by Packard Bell (not the same company that made computers in the 1990's), I'd say it was made in the early 1950's. The neat thing about it is that the tuner is divided into two halves. One side, labeled "northwest" contains the call signs of the regional clear channel stations stacked above the numeric frequencies. The other side, labled "southwest" is marked the same, except with different stations. Is suspect there is a corresponding layout marked for "northeast" and "southeast" that was marketed in those regions as well.

If you're bored one night, and you have an AM radio nearby, I suggest you give it a try. It is fun to see which is the furthest station you can tune in. (Note: you'll get a better signal if you tune in close to an equinox.) The Internet is full of sites that will give you helpful pointers.

I wonder how much tubes are going for on eBay these days...

Captain Stupid

Captain Stupid strikes again...

I bought a used Xbox (version 1.0) a few weeks ago, intending to convert it into a cheap (heh) frontend machine for my MythTv system. I knew the project would require some soldering, but I was not worried. I've been using a soldering iron since I was 12. Years of experience would make this a simple project.

Three--no four--things doomed me:
1. My soldering iron was too hot for pcb work.
2. I had the wrong kind of solder (electrical, but not the pcb stuff).
3. I didn't have a braid to wick up extra solder.
4. I stubbornly insisted that 1, 2 and 3 (above) didn't matter--I knew what I was doing.

I just ruined a $30 Xbox mod chip and will be lucky if I haven't ruined the Xbox motherboard too. When the stinging pain (mental, not physical--I managed to not burn myself) dulls down, I'll get another mod chip. Then I'll find someone with the right kind of soldering iron and do this right.

Is there a lesson here? Yes.

Have I learned it? Yes.

Will Captain Stupid continue his fearless quest to bungle simple projects and waste good money? Quite possible.

And an entire level of my house stinks of solder.

Moving finished

I've finished moving everything over. My blog, my website, my email, etc. I didn't realize how many services I had running on that 300 Mhz PIII in my basement until I decided I wanted to shut the the machine off.

It looks like the only thing that didn't make it was pictures from the old blog. I thought that blogger was importing them automatically, but it appeared to be hot-linking them instead.

I don't know if my motivation goes that far. I don't remember which of all posts have images in the first place, so I would a) have to figure that out and b) figure out which image was linked. All images are in the photo archive anyway, so it isn't that big of a deal to me.

In some ways, this is the end of an era. Years ago, I always thought it would be cool to host my website out of the house. Broadband internet was new then and the nerd factor of having it all down in my basement was high. A few hardware failures later, I find I don't feel quite the same way anymore. I've always been good about keeping backups of all my data, but the level of butt-pain from knowing the website was down after a PSU failed was quite high. I won't miss that.

OSX: Java to Ruby

I now completely understand why Apple has dropped support for the Cocoa-Java bindings (that were quite nifty)--they've moved on to Ruby. I stumbled across the Ruby developer samples in the dev kit the other night and was amazed at the depth of integration with the language.

First, the UI can be defined using nibs (Cocoa-Java never had that). Next, Ruby is more closely linked to Objective-C than Java ever was. I didn't get too deep into it, but the application I looked at started life in a .m file and quickly transferred all control to Ruby. I don't know how easily Ruby could cede back into Objective-C (I assume Ruby has some kind of native API), but it all looks possible.

Ruby, it seems, is the new Java. The writing is clear and unmistakable--Apple is dumping Java. I am a little miffed because Java is my first language. Don't worry--I'll get over it. But I would like to understand better what made this happen. Keep in mind that I am and outsider looking in, but here are a few ideas:

1. The relationship with Sun and Apple has soured.

Sun used to have at least two Java engineers working on the Apple campus. That went away years ago though (when Sun first started having problems). However, Sun and Apple were pretty clearly working together and must have had a somewhat healthy relationship. I'm not sure how much the difficulty of licensing Java3D soured this relationship, but gosh, it must have had some effects. Currently, as a developer, I feel as if I'm sitting in the middle of an adversarial relationship (Java 6, where are you?). Either Apple is unwilling or unable to get help from Sun.

2. Apple realized that developers were moving on.

Java isn't the fresh new face it once was. Developers are slowly realizing that some of its warts will not be going away anytime soon (Sun moves so slowly in this regard), and have begun experimenting. Groovy, JRuby and now Scala are obvious examples that Java developers yearn for the expressiveness of dynamic languages. And dynamic languages (mainly Ruby and Python) move ahead adding new features at a rate that makes Java seem antiquated. It appears that Python will even drop features (break old code!) in future releases. Amen!

3. Apple was frustrated.

At Sun. At itself. At developers. The Cocoa-Java bridge never did develop much of a following. The only product that I [knowingly] used that was Cocoa-Java based was Lux, a [very fun] Risk clone.

I don't know what the take is. C++ programmers were probably having similar thoughts in 1996. (...What!? Product X is adding support for that new toy language?) Life marches on, and so does technology. At some point Apple will move on from Ruby, or Erlang or Python or whatever the cool new cat of programming languages is. As evidenced by the fact that there is still a market for Cobol programmers, we need not worry about becoming antiquated too soon. :)

Updates

1. dd-wrt has (since November) released firmware updates for my WHR-G125. I have just installed it and bandwidth monitoring is indeed working. The latest firmware seems to be located here.

2. So much for my sons having good taste in music. A neighbor gave them a CD full of Pokémon songs and that is all they listen to now.

Tagfriendly showing signs of life

I have found that the time I have allotted for recreational programming has decreased significantly in the last two years. Part of that reason was that I was bringing quite a bit of work home. I have nipped that for the most part. Another reason is that we've had new children come into our family--one each in 2005 and 2007. Things are finally starting to simmer down on a personal level, so I thought I'd pick back up again.

About a month ago I started hacking away on Tagfriendly. Again. And of course, being the programmer I am, I started from scratch.

It's amazing what 18 months away from a project can do to ones perspective. I spent so much effort placing the application within a framework that it took away from the real efforts--manhandling Freedb and Lucene. The never version is quite a bit different, better I think. More nimble for sure.

Here are some of the changes:
Old: Much effort was focused on having the work done in producer/consumer queues. My justification was that I didn't want to waste CPU cycles doing nothing.
New: Broke the project into smaller tasks: replicate freedb data in database, convert to relational, build index.

Old: Did everything in Java.
New: Expanding my horizons by giving Ruby a try. It also helped that Lucene now has Solr, which saves me a lot of work. For starters, I don't have to bother with putting a smart (as in cacheing and paging) HTTP framework in front of Lucene anymore. Second, Solr makes it easy to interface with other platforms (like rails). Java is the hammer I've preferred for years, but having tried a few other tools, I'm ready to view it as that--just another tool. LoC economy comes into play at some point and Java just doesn't win there.

Old: Used PostgreSQL
New: Using Caché from Intersystems. This stems from my experience working at DTI . However, I remembered how much I disliked fighting Postgres with regard to false integrity constraint violations stemming from the "many updates" phase where, literally, TONS of inserts were taking place. Caché handles this just fine--and the single user license is free.

There are still lots of rough edges. The difference is that the last time I did this, I tended exert a lot of energy polishing the rough edges at each step rather than get real work done. Now I let the edges stay rough and focus on the core tasks--getting that index searchable. The polish will come later.

Other things that will change... There will not be a java thick client. It was fun for play, but this app, if it is every deemed useful, will be in the form of a plugin to an other application--written in whatever language/platform that makes the most sense. This is done partly because I would like to be exposed to other programming platforms/environments. Also, I'm trying to just have fun.

Upgraded Myth

I just finished upgrading my MythTV system. Given the pain it took me to get it running smoothly in the first place, I was worried about the upgrade.

Things went smoothly though. The only problem I ran into was my own fault. I forgot to move the mythweb php files over into my htdocs area and set permissions appropriately. Five minutes later I was in business.

I changed the data grabber to Schedules Direct using mythtv-setup and ran mythfilldatabase. I went from having 5 days of data to 14. It feels good to have this out of the way--like a yearly physical or something.

Quickies

Here is a Magnetic Fields show recorded in Toronto, 2004. Old, but if you're a MF fan, it's good stuff. I don't know how I've never come across it before. The sound quality is excellent, by the way.

I'm looking forward to the new record, due out in, umm... 2008.

I ought to post a few updates since this will probably be the only post this month or next.

1. Sadie (I call here "Sadie-my-lady") is doing well. She gets up usually once, sometimes twice a night. It could be worse.
2. Things are going well at the new job. I am uncomfortable with a few things though. If I don't go into details in the future, the big deal is this: At the last job I was one of the sharper pencils in the can. But here, at the new place, I am the dullest. In the long run, if I can hack it, this is probably a good thing for me. I'll come away from it a better person. But in the mean time, it's killing me just to keep up.
3. Garden--dropped off the radar when Sadie was born. I've managed to go out and pick a few tomatoes here and there though. And we've also got a honkin' big pumpkin waiting for Hallow'een.
4. My TV listing data runs out on Saturday unless I find time to upgrade my MythTV system. I've already shelled out for a Schedules Direct account, but haven't managed to put everything together yet.
5. School started and the kids are liking it. I can't wait for soccer season to finish though.

Stuff I Did Not Know, Pt. I

I think I'm going to make this a regular feature of my blog. (Wait... nothing about my blog smacks of regular or consistent, unless you count the fact that I regularly and consistently am not regular and consistent about posting entries. Wait... let's not go there--I could be arguing with myself all day).

Stuff I Did Not Know:

The Javascript language continues to evolve.

Firefox 2 has been out for almost a year. Introduced with Firefox 2 is support for Javascript 1.7. This came as two announcements to me: 1) Wow! Firefox is upgrading its support for Javascript, and 2) Wow! There exists a new version of Javascript!

You can read all about at the Mozilla Dev Center. Enjoy.

What struck me is the way Javascript is growing up. You need a little historical context here... To me, Javascript has always been the little language that could. For years I have been struck that very few people give it the time of day. Sure, it is handy for manipulating your HTML DOMs and doing other funky HTML tricks, but it is also wonderfully expressive and actually VERY FUN to program in.

And keep in mind that it is very accessible [to programmers]. I mean, it is in EVERY Firefox (unlike Python and Ruby, mind you). But very few people give it credit for being what it actually is: a first-rate programming language.

I'm not going to get into the new features--go read up for yourself. But I am glad to see that Javascript continues to grow an mature. Maybe someday we'll have a real Javascript compiler and some Desktop-class apps that do a little more than Konfabulator and Apple Dashboard.