December 4, 2008

89.3 The Current and the Mysterious Non-Expanding Playlist

Let me say this much up front: I still love The Current. I still have the dial on my car tuned there permanently, I still listen to the podcasts when I get a chance, I'm still a member. I'm saying this because the rest of this post is going to sound like Current-bashing. I still think it's a wonderful station - I just don't like the direction it feels we're heading.

A short primer: A few years back, a magical public radio station was born. It billed itself as "the antiformat" station, gave the DJs a massive amount of freedom, and played music that was always fresh, varied, and exciting (and usually quite good besides). Then, somewhere along the line, someone decreed that certain songs needed to get certain amounts of airtime. DJs started being told what their playlists should contain. One DJ quit over the issue. And people like me started wondering why the same song was playing every day on my 20-minute commute. Not that it's a bad song, just... I don't need to hear it every single day. I don't need to hear any song every day.

But, rather than complain anecdotally, I decided to use the power of numbers. The Current makes a massive history of their playlist publicly available on their website, dating back to 2005. So I wrote a screen-scraper in Python to pull all the songs off the site and store them in a sqlite database, which I could then run queries on and make pretty spreadsheets and graphs.

On methods: I tried to normalize all the data before storage, such as stripping non-alphanumeric characters and converting to lowercase letters. This helps increase correct matches. I also ran queries against songs grouped by (artist, title) to avoid false matches on title alone. I don't think I screwed anything up, but I have no formal training in statistics, so no promises. All code used to collect and analyze the data, as well as the spreadsheets and graphs of the results, are available for download under the GPL here.

The question I wanted to answer was "is The Current's playlist shrinking, and how badly?" Generally speaking, a "good" playlist should play many different songs, and not play any particular songs too frequently. The challenge is to coax a subjective measurement like "good"-ness out of a massive pile of song listings.

The first measure I have is the "unique song ratio" - that is the number of distinct songs played in a period of time compared to the total number of songs played in that time. So it should be a fairly good measure of how much variety a playlist is offering. Higher is better - it means of the total playcount, there is a larger selection of songs played.
Unique song ratio
The numbers themselves are somewhat arbitrary, but there's a pretty clear and shocking trend visible here. Somewhere near the end of '07, things take a massive dive. The ratio over a week, which was hovering around 0.9, drops to nearly 0.6. It makes sense that the ratio over a month is lower all along - over the course of a month, it becomes much more likely that the song you're playing has already been aired. But when the giant dip in the graph levels out, the ratio over a week has leveled out right around where the ratio over a month used to be. That can't be good.

Similarly, we have average song plays, or the number of times a typical song will be played over a period of time.
Average song plays
That same programming shift is visible here, peaking at an average of 2.5 plays per song per month and leveling out over 2.

Of course, if The Current played every song exactly twice a month, I wouldn't have much room to complain (I might wonder if the director of programming had some nuerotic tendencies, but that's a separate issue). My concerns lie more in if certain songs are being overplayed. To further address that, let's measure the maximum playcount - the highest number of times any one song is played in a period of time.
Highest playcount for a single song
Again, the same trend is plainly visible. And this time, the numbers themselves are troubling. The recent end of the graph is somewhere between 60 and 70. That's enough to play the most popular song for a given month more than twice a day, every single day. The weekly count is up near 20, which is almost three times a day for that week.

So... ouch. This isn't just a minor tweak to programming. To me, this looks like a shift in the very identity of the station. And I don't think I like the new Current as much as the old one.

I don't want to get too hyperbolic. I'm sure these numbers would still look very good put up against a Clear Channel subsidiary, or really just about any commercial station. I would have loved to compile some numbers from one of those stations to have a good laugh, but sadly I couldn't find any that made old playlists available. If you know of one, I'd be interested to hear.

All the complaints flying around are not because we haven't counted our blessings - it's because we know just how lucky we are, and we're afraid we're slowly losing our treasured station to the mainstream. So no, it's not the end of the world, and I'm not convinced 89.3 has sold out to The Man just yet. But I used to describe The Current to my friends as "single-handedly saving radio." And I'm starting to wonder if I can still count on them for that. Maybe it's time to lay the responsibility in Triple J's hands.


I want to close with one more analysis. Curious if drive time or other factors would affect the playlist at all, I ran a set of queries for the same uniqueness ratio as above, but now broken up into two hour time slots throughout the week (and yes, I included the weekend, whether that's good or bad).
Unique song ratio by time block
The orange line along the bottom is the monthly value, included just for reference. As you can see, most of the time slots follow the general trend towards less variety very closely. There are three slots, however, that don't: those from 4AM through 10AM. The Morning Show runs from 5-9AM. Strangely, the 6-8AM slot actually takes an upturn as everything else heads down. Did they ramp up their eclectic selection in reaction to the station's overall homogenization? I don't know. At any rate, woo yay Morning Show! Too bad it's ending forever in a week.

So Sad...

November 5, 2008

Visualizing sorting algorithms







Enough exposition! Let's move on the the results.

Insertion Sort

Well, we had to start somewhere...

Insertion sort (with borders)

Insertion sort (no borders)

Merge Sort

Now we're talking. O(nlogn), wooooo!

Merge sort (with borders)

Here's an un-bordered merge sort:
Merge sort (no borders)



Quicksort (with borders)

One without borders:
Quicksort (no borders)

Bubble Sort

That's right! A very special treat for you all!

Bubble Sort


Other Stuff


Quicksort on a value-only gradient.
Quicksort (value only)

Hues quicksort

Hues merge sort


October 28, 2008

How Bulk Political Mailing Should Be Done

A few days ago, I came across a real gem of a political mailing from the Republican Party of Minnesota. To a casual observer, it might just look like a sad smear attempt by a party hijacked by reactionaries and lacking any real substance. But I saw beyond the partisan hackery, and realized that this mailing had far more potential. It could be something truly great. And behold, with some scissors and glue, I made it so.


For best results, read in your best Dan LaFontaine voice (and watch a preview or two in his memory).

August 27, 2008

Why I sign with PGP


Hash: SHA1
And at the bottom is something like this:

Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Promote trust on the internet - Use PGP!






Why You Need PGP










How PGP Works (the short version)




A Brief Interlude on Trust





The Future of Trust


One More Time, Why?


Final Thoughts


August 1, 2008

Hacking Grinnell Laundry

Disclaimer: I am giving you the knowledge, I am not dictating how it should be used. That's up to you to decide.

Easy steps to being a Grinnell outlaw:
  1. Put your laundry into a dryer. Pay with your P-card.
  2. Hit "Delicates" or the other option that isn't "Colors/Whites". Your dryer should fire up with something like 70 minutes on it.
  3. Open the dryer door. Close it again. The dryer will start blinking Select Cycle.
  4. Hit "Colors/Whites". The dryer display indicates that it is using the Colors setting (i.e. higher heat) but with all 70 minutes or whatever you got out of step 2.

For the record, I haven't actually checked to see if the dryer is doing what it claims to be doing. It could be so dumb that it is displaying one setting while giving you lower heat. I don't know. I do know that Grinnell laundry machines are crap.

The Laundry Day Boxer Problem

As a certified lazy person, I've been known to put off chores like, say, laundry for quite some time. And since I can keep wearing the same pair of pants until I drop food on them or something, it's easy to let things slide a little too long. To moderate my laziness in this regard, I use a tried and true lazy person technique: laundry day boxers.

This is the pair of boxers that stays in the drawer until all the other boxers are gone. This is the pair that is so garish that even in your sleep-addled 8am state, you can't help but notice what you're putting on, and realize that this means the grace period is up. You can't put it off any longer. Your laundry day boxers are a constant reminder that you need to do laundry now.

I know some of you own laundry day boxers. Don't be shy. Mine are bright orange, with little bats and Frankenstein monsters on them. My old roommate had a pair that were silk, with a screenprint of Elvis on them - brilliant.

Now here's where I encounter a problem (see, that title was relevant after all!): I need something to wear on laundry day. That, naturally, is my laundry day boxers. But I can't wash them because I'm wearing them. So come next laundry day, what will alert me that it is, in fact, laundry day? My laundry day boxers are now dirty and in the laundry, where they are useless to me. Useless!

Having ruminated long and hard on this problem, the only solution I can come up with is to obtain a second pair of laundry day boxers. At that point it's just a bootstrapping problem to arrange things so that one pair remains clean the first time I do laundry post-acquisition. Then, come that first laundry day, the dirty pair gets cleaned while I wear the clean pair, and I alternate from then on.

It doesn't seem quite right though. I really feel like each person should only own one pair of laundry day boxers. Call me old-fashioned, but when it comes to laundry day boxers, I believe in monogamy. I mean, when you're cavorting around in your purple sequined laundry day boxers, how do you think the ones with the Taco Bell chihuahua feel, down at the bottom of the hamper? Maybe I'm just off on this particular topic. Anyone out there own more than one pair of laundry day boxers? Maybe if I got two identical pairs, that might be a reasonable compromise - it's like I'm wearing the same pair, even if they're actually twins. And with that, I've strayed into borderline creepy analogy territory. Moving on.

Is this two-boxer solution the only way? Am I overlooking something painfully obvious? I feel like one of you people who likes game theory or something should be able to model this mathematically. This is serious business, folks.

April 22, 2008

Hello, World

This is a good blog. It is probably not the best blog.

If you're looking for the stories of my travels in the land down under, those are at