Ever got curious on something and then you’re unable to let it go until you’ve tried it out? Thought so. For me it started with the extracting of the FreeDB archive. What a beast! You start with a small innocent tar.bz2 archive of about 440 Mb. Extract the bz2 archive and you get something quite big, about 3.2 Gb tar archive, and it is now that you start thinking about the nagging about reading the FAQ before extracting. What a newbie can’t extract a simple archive?! Then I ran out of inodes on my partition… Cute.
The fact that the database contains almost 2 million files (2,000,000) is really a problem. If you don’t run out of disc space due to the overhead you’ll run out of inodes like I did. I was frankly surprised when the disc was full, I had 25 Gb free when I started and I had plenty of space left when it stoped. This was a partition using FAT32. Now I had a peek at the FAQ…
Next try, extracting to a partition using ext2, and this time I didn’t ran out of something. Thus it took my old disc several hours to extract all files. Now then? Here I was with a directory wasting about 7.4 Gb of disc space. Why is it distributed in this way? I would prefer to have the data in a MySQL database. There are some parsers out there for those of you that want to take the easy way out, but what’s the fun with that? So I wrote my own in Java and it is now released under the GPL, read more about the Viba IT FreeDB to MySQL parser.
Now when the week-end is drawing near my tip for you is to try it out. It’s not every day you’ll have the possibility to play around with this amount of data. For your information, once parsed and imported into MySQL the disc usage for the data will only be about 1.5 Gb including indicies for all tables. Nice, huh?