The Big Data Platform Landscape

The number of Big Data software platforms has exploded. To wrap my mind around the available databases, query and analysis systems, and the types of users that actually these sorts of technologies, I created a mind map in FreeMind.

This map is certainly not complete, but it provides a good background on what’s out there. I hope to continue growing this map as I learn about new technologies. I’ve also noted a few topics that look like strong potential research areas.

Check it out here: Scott Sawyer’s mind map of Big Data platforms (or download the FreeMind file).

Leave a reply or tweet at me (@10flow) if you’ve got a technology to share!

Migrating playlists from iTunes (Windows) to Rhythmbox (Ubuntu)

Despite my affinity to Linux and free software, I’m a sucker for Apple hardware. In particular, I have an iPhone and a few old iPods. To keep these devices reliably synchronized with my music collection, I’m stuck using iTunes in a Windows VM. I’ve built up some playlists I like, so I wanted to export these playlists from iTunes (version 10.6) and import them into Rhythmbox (2.96) on Ubuntu.

I put together a Python script that converts an exported iTunes playlist into Rhythmbox’s format. It works well if you move entire collection from Windows to Linux, preserving the directory structure. I just wanted to share it here with some brief instructions on how to use it.

Step 1: Export the iTunes playlist (before killing your Windows partition!). Simply right click the playlist and pick “Export…”. Then, save it to a text file. Move this file with your music collection to your Linux box.

Step 2: Download the script (importpl.py, 1.5 KB) and modify the “win_path” and “lin_path” variables (lines 5 and 6 below) to reflect the proper paths. The script will take care of changing Windows’ backslashes to forward slashes.

#!/usr/bin/python

############################################################
#USER CONFIGURATION: SET YOUR PATHS
win_path = 'C:\\Users\\Scott\\Music\\' #be sure to use double backslashes!
lin_path = '/home/scott/Music/'
############################################################

import codecs
import csv
import sys
import os

file_in = sys.argv[1]
file_out = file_in + ".pls"

filename = file_in.split('/')[-1]
parts = filename.split('.')
if len(parts) > 1:
    pl_name = ".".join(parts[:-1])
else:
    pl_name = filename

f1 = codecs.open(file_in,'r','utf-16')

lines = []
for line in f1:
    lines.append(line.split('\t'))
f1.close()

f2 = codecs.open(file_out,'w','utf-8')

f2.write('\n')
f2.write('X-GNOME-Title={0}\n'.format(pl_name))
f2.write('NumberOfEntries={0}\n'.format(len(lines)-1))

count = 0

for i,l in enumerate(lines[1:]):
    win_file = l[-1].strip()
    title = l[0].strip()

    lin_file = win_file.replace(win_path,lin_path)
    lin_file = lin_file.replace('\\','/')

    if os.path.exists(lin_file):
        count += 1
        try:
            lin_uri = 'file://{0}'.format(lin_file.replace(' ','%20'))
            #write it out
            f2.write('File{0}={1}\n'.format(count,lin_uri))
            f2.write('Title{0}={1}\n'.format(count,title))
        except:
            print "Couldn't write a file " + title

    else:
        print 'Could not find ' + lin_file
f2.close()

print "Wrote {0} songs".format(count)

Step 3: Run the script to convert the iTunes playlist to the Rhythmbox “.pls” format. Here’s an example usage of the Python script:

$ python importpl.py iTunesPlaylist.txt LinuxPlaylist.pls

Step 4: Import your playlist into Rhythmbox. Just go to Music > Playlist > Load From File…

That got my playlists migrated to Ubuntu without too much trouble. Your mileage may vary. Leave a reply if you came up a better technique!