Thursday, August 13, 2009

On sorting files... part 3, exchange format

In this post I want to talk about the content of the metadata itself and the format in which it is distributed...

Let's take a similar example of metadata exchange: del.icio.us (or delicious.com now). I have to admit that delicious is the initial idea that got me thinking about this problem of file exchange sorting. What I liked about the delicious example was the API. Like a lot of web application, the API is a simple webpage that you call using arguments in the url. (you can see it here)

I propose either XML or JSON as these format are well understood and easy to implement but really, any sort of inter-exchange format, it could be implemented.
That metadata need to include:
  • Filename
  • Md5 & Sha1 Hash (optionally Sha256 / Sha512)
  • Description
  • List of Tags
  • optional website
A bit like the data del.icio.us stores about each bookmark.

A method of distribution of all that information also needs to be created. Either something like a url on the distribution server (like http://server/file-metadata/name/file.exe or http://server/file-metadata/index.php?&filename=file.exe) that would produce a page with the XML or JSON. Something that looks like:

{
"Filename": "executable.exe",
"hashsum": {
"md5": "d41d8cd98f00b204e9800998ecf8427e",
"sha1": "da39a3ee5e6b4b0d3255bfef95601890afd80709",
"sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
}
"metadata": {
"Description": "Description",
"tags": "Info, executable",
"Website": "http://www.wwww.com"
}
}

Or like:

<?xml version="1.0" encoding='UTF-8' ?>
<File Filename="executable.exe" >
<hashsum>

<md5>d41d8cd98f00b204e9800998ecf8427e</md5>
<sha1>da39a3ee5e6b4b0d3255bfef95601890afd80709</sha1>
<sha256>e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855</sha256>

</hashsum>
<metadata>

<Description>Description</Description>
<tags>Info, executable</tags>
<Website>Http://www.www.com </Website>

</metadata>
</File>

Or course the devil is in the details and this should be considered a draft and not a definitive thing. The notion of web API has been done before and I really just wanted to give an idea of how I think things could be done.

Update: I was specifically talking about a RESTful API. I could not remember the name when I wrote this post. Here are a couple of links on the subject:

On sorting files... part 2, the metadata

Continued from my previous post on sorting file, here is my own finding on metadata required.

First of all, not all metadata is made equal. There is a concept know as "metacrap": the fact that metadata is sometimes less useful then no metadata at all. Metadata is a fragile concept: if that part of the system is badly implemented it will crash before it has even left the ground.

Thus the source of metadata is crucial, and it must be verifiable. But I want to kept that part of the system for a future post. I want first to describe the metadata itself.

Let's take an example of existing well know metadata: the ID3 tag of mp3. That containers works great for mp3 files, but it has several downsides. The one relevant to this discussion is the fact that the ID3 is part of the file. So if, for example, you decide to change the genre of a song from "country" to "western", the file itself becomes a different file. The cryptographic hash of the file completly change and a md5sum would be useless. This hash is important because it guaranties that the file you get is the file you want.

Mp3 files are not the only type of file that have inline (inside the file) metadata. The Microsoft win32 binary files also contain some metadata. That information can be seen using "properties" on any executable file. (It's in the "details" tab.) Or by using the GetFileVersionInfo function for any developers. That information, like the id3 tag, is contained inside the file. But that metadata is more buiness data then actual useful 'sorting' information. And Microsoft never designed this information to be user-editable. It's only accessible to the developer of the application. It was never designed as a generic container, It's used to identify binaries.

There is a way to store additional information inside a file without modifying the file itself with either ressources forks or extended attributes but it's not supported on all filesystems. Mac OS X support resource fork in the HFS+ filesystem, as does NTFS with Alternate Data Streams (ADS).Extended Attributes are supported on Mac OSX HFS+, Linux's ext3 and BSD's UFS.

The only problem with storing this informatin in either NTFS ADS, resources forks or Extended attributes is that this information will be deleted if the file is moved to a FAT32 drive (used on thumbdrives).Simply downloading the files might strip the files of ressources fork or extended attributes.

And thus, it need to be distributed alongside the file, as either a single file or bundle of information about several files.

Next post will be about the metadata exchange format.

Wednesday, August 05, 2009

On sorting files... part 1

For months now, I've been looking for a way to automaticaly sort my files. There is a lot of software for creating and maintaining some sort of media library for pictures, movies or music, but there is almost nothing else for "the rest". On my home computer, "the rest" is composed of 2600 files that occupies a grand total of 45G (although 23G of that is composed of iso files of Linux and BSD). Almost all of these files were downloaded from a website somewhere, either application from sourceforge.net, game add-ons from The Elder Scroll Nexus , Fallout 3 Nexus or Simtropolis or drivers from various hardware manifacturers.

Right now, the way I manage these files is mostly by hand. Everything goes into a "Download" folder (that I have since my BBS days), and it's then sorted by hand into several different folders... and that folder is in a perpetual "TO DO:not completely sorted" state... (since those BBS days...)

I wish I could just simply drop these files in a folder where they would automatically get sorted...

...And I don't think it's impossible to implement. Each of the files I have specifically mentioned comes from websites where the content is already classified. Sourceforge classifies projects among various categories and so does all the gaming communities, with Mods, maps, etc. The only thing required is a method of exchanging metadata.

Seems easy enough, but it's not. When you start to think about that problem you realize quickly that it's not really a technical problem. It's more a problem of getting everyone to work together with the same standard.

I have prepared a series of blog posts for each of part of the problem, a sort of brain dump of what I came up with in the past months. A single post would create the "Wall of Text" or TL;DR (too Long; Didn't Read). (And if I try to write something too big, as the Thinkgeek t-shirt says "I never finish anyth")

Friday, May 29, 2009

Choosing Software

Wikipedia has really changed the way I do things... And I'm not talking about articles, I'm talking about choosing software.

Let me explain with an example. I recently installed Windows 7 on my computer (see previous post) and I wanted to try a new Media player. I've always been running Winamp and wanted something different...

The Wikipedia article on Winamp points to both a List of Media players and a Comparison of Media players. Using these lists, I found out that the best choice for me was Songbird. I also took notes of aTunes (and of course Winamp). I decided to use Songbird but I might change that decision because Songbird some interesting bugs on Windows 7 x64 (unsupported platform).

The same exercise can be done with CD burning Software points to a List of optical disc authoring software, which, in turn points to both CDBurnerXP and ImgBurn, both applications I have used in the past. It also shows that there is a CD Burning Open Source application on Windows called InfraRecorder (which I had never heard about).

I find this little "technique" of using "list of" article in Wikipedia very convenient.

Saturday, May 23, 2009

Re-installing Windows

I have a book behind me called "Backup & Recovery" from O'Reilly Publishing. I have learn a very basic yet very important lesson from that book... And If my memory is right it goes something like:
"Your backups are as good as your last restoration attempt."

The whole concept being that even though you have backups, unless your actually tried to restore those backups, they don't mean anything.

Since:
  • Microsoft came out with an RC of Windows 7
  • I ran Windows XP which could only see 3.5G out of 8G of physical RAM
  • I'm been making backups of everything on my computer
I decided it was time to test that backup strategy with a fresh install of Windows. Of course, because Windows 7 RC will shutdown in 2010, I will have to re-do this process all over again.

Thus, I am writing here everything that I have to do in case I need to do it again. (This posting will be updated if anything changes during the year)

This Machine is use for: Gaming, Web, E-mail and the occasional programming in C++ / QT.

List of software I need to get online (in alphabetical order):
  • 7zip
  • Adobe Air
  • Allsnap
  • ClipX
  • Boinc
  • Dropbox
  • Firefox
  • Flash player
  • Foxit Reader
  • gnucash
  • Launchy (no 64bit version, but 32bit seems to work)
  • Nokia Qt SDK
  • OpenOffice.org
  • Pidgin
  • Picasa
  • Process Explorer
  • Putty
  • QDevelop
  • SMplayer
  • Steam
  • Thunderbird
  • VirtualBox
  • Vlc
  • WinDirStat
  • WinSCP
  • Xming
List of Backup procedures:
  • Backed-up "Documents and Settings" Folder from Windows XP
  • Backed-up "My Documents" Folder from Windows XP
  • Backup list of mods in Oblivion Mod Manager and Fallout 3 mod Manager
  • Imaged the C: Partition with CloneZilla (just in case)

List of Restore procedures:
  • Moved the "My Documents", "My Pictures", etc as sub-folders of "D:\My Documents"
  • Show File Extentions. Always. (Control Panel -> Folder options)
  • Add "path=C:\Qt\2009.02\mingw\bin;C:\Qt\2009.02\qt\bin" variable in Control Panel -> System -> Advance -> Environment Variables
  • Re-installed Firefox extentions (including: Xmarks, Noscript, Delicious bookmarks, IE tab ,etc)
  • Restore Thunderbird directories:
  1. Documents and Settings\Ve2dmn\Application Data\Thunderbird to C:\Users\Ve2dmn\AppData\Roaming\Thunderbird
  2. Documents and Settings\Ve2dmn\Local Settings\Application Data\Thunderbird to C:\Users\Ve2dmn\AppData\Local\Thunderbird
The rest is mostly tweaking.

I guess I have proven to myself I'm somewhat protected against data loss now... Eventually I will have something like Time Machine, TimeVault or Flyback for Windows, but for now, I guess this will have to do :)

Friday, April 24, 2009

Bad English to French Translation...

Living in Quebec we sometime get very bad English to French mistranslations... Things like:
"Lamp Oil" translated to "Lamping Petrol"
"Made in Turkey" translated with "Made with turkey meat"
But the best one yet is on the washing instruction for one of my shirt.

Original text:
"See reverse for Care.
-Machine Wash cold with like color only
-Non-Chlorine bleach when needed
-Hang or Tumble dry low, Remove Promply
-Cool Iron when needed
-May be dry cleaned
-Do not Iron Decoration"

Was Translated to(For the French impraired):
"See Contrary for love
-The machine washes the cold with the loved color only
-The decoration not-chlorine when necessary
-Hang or fall dry, the low level removed on time
-Cool down some iron metal when necessary
-Cleaned. The month of May is dry.
-The decoration is not made of Iron"
Posted by Picasa
 
I though I had seen it all, but this is just a bit too much in my opinion.
Posted by Picasa

Monday, April 06, 2009

Small live stream recorder with Qt

I decided to try to write a small App with Qt (no, quicktime, but the Nokia toolkit, Qt)

The result is the screenshot you see. It's an app called, the "Twit Live Recorder V0.1 (Proof of Concept)". It's a live stream preview and recorder, with the ability to "split" the current file (it create a new file and add -1 or -2, -3 and so on...). It is very rough code, but it works. (you could probably crash it if you try hard enough)

Note: I took Twit Live as an example because I know that Leo gaves his O.K. to have his stream recorded (ODTV is great! You guys rules!). This could probably apply to any live stream that can be recorded with 'wget' or 'curl'. I won't give the urls here because I have no idea if it violates the terms of service for Stickam, Ustream or Bitgravity (or even Justin.tv). As always, uses your jugement and don't do anything illegal.

Also note that simply copying and pasting the code will probably not work. I removed some characters that were interpreted as htnl code and replace them with "quotes". If someone wants the original files, I could post them somewhere.

Final Note: This was done for fun, as an exercise. I'm probably not going to maintaint this application, as I don't see the use for it. (The OdTV people are already working on automation...) I'm not a full-time C++ developper and probably lack the experience to answer. If you have C++ or Qt Question you are probably better off asking it on a forum dedicated to that like StackOverflow.

The Code Behind the App
I used QDevelop to create the App, but it could probably be built with QtCreator, Kdevelop, MS Visual Studio or the command line.

The entire app is basically 5 files: a .pro project ( Twit Live Recorder.pro), a main.cpp , the .ui file (twit_live_recorder.ui), and the .cpp and .h of the main object (twitliverecorderimpl.cpp and .h )

The .pro "project" file:
TEMPLATE = app
QT += core webkit network
FORMS += ui/twit_live_recorder.ui
SOURCES += src/twitliverecorderimpl.cpp src/main.cpp
HEADERS += src/twitliverecorderimpl.h
CONFIG += build_all debug_and_release
TARGET = Twit_Live_Recorder

The file is pretty straightforward. Build a Qt app, with modules "core", "webkit" and "network", with name "Twit_Live_Recorder" (You can also see that I like to build both debug and release.) The qmake from Nokia can turn this file in the coresponding Makefile for the correct architecture and OS.

The main.cpp is also pretty straightforward:


#include "QtGui/QApplication"
#include "twitliverecorderimpl.h"

int main(int argc, char *argv[])
{
QApplication a(argc, argv);
TwitLiveRecorderImpl w;
w.show();
return a.exec();
}

Include the "Twit Live Recorder Implementation" called twitliverecorderimpl, create a QApp (to handle most of the OS stuff) , create a TwitLiveRecorderImpl dialog and show it.

I won't copy the content of the .ui because the XML code is not meant to be read. I can tell you it's a QMainWindows with a QWebview as a central widget (to get a preview of the stream). Also includes a QLabel and QLineEdit at the bottom to list the current file. Finally, the main windows includes a MenuBar and Toolbar with the following Qaction: "Record", "Split", "Stop","Quit", "About", and "About Qt"

Now on the the meat of the app, the Dialog Implementation. First the .h:


#ifndef TWITLIVERECORDERIMPL_H
#define TWITLIVERECORDERIMPL_H

#include "QMainWindow"
#include "QFile"
#include "QHttp"
#include "ui_twit_live_recorder.h"

class TwitLiveRecorderImpl : public QMainWindow, public Ui::TwitLiveRecorder
{
Q_OBJECT

public:
TwitLiveRecorderImpl( QWidget * parent = 0, Qt::WFlags f = 0 );

public slots:
void Http_error(bool error);
void Http_stateChanged (int state);

private slots:
void on_action_Quit_activated();
void on_actionAbout_activated();
void on_actionRecord_activated();
void on_actionSplit_Current_Recording_activated();
void on_actionStop_Current_Recording_activated();

private:
QString Host_url;
QString Stream_url;
QString BaseFileName;
QString ActiveFileName;
int SplitNumber;
QFile *ActiveFile;
QHttp *CurrentConnection;

};
#endif

As you can see, this object includes both a QFile and a QHttp private objects. What we do to save the stream is simply open a Http connection to the server and pass the QFile as the destination IO device. The on_action_ slots are function the Qt precompiler will automagically connect with the button on the UI. The Http_error and Http_stateChanged are there mostly to help debugging the Http stream.

Finally, the .ccp file. Let's go trough it one function at a time. First the header and Object Constructor:


#include "twitliverecorderimpl.h"
#include "QMessageBox"
#include "QWebSettings"
#include "QFileDialog"
#include "QFileInfo"
#include "QDate"
#include "QDebug"

TwitLiveRecorderImpl::TwitLiveRecorderImpl( QWidget * parent, Qt::WFlags f)
: QMainWindow(parent, f)
{
setupUi(this);

//Set Urls
webView->setUrl(QUrl("http://live.twit.tv"));
Host_url = "Server.whatever.com";
Stream_url = "http://Server.whatever.com/stream-url";

//Create a New Connection (unused until the record button is pressed)
CurrentConnection = new QHttp();
CurrentConnection->setHost(Host_url);

//Set Filename. Default file name looks like: "Twit-2009-03-25.flv"
BaseFileName = "Twit-" +
QDate::currentDate().toString("yyyy-MM-dd");
ActiveFileName ="";

//Split Number to have Twit-2009-03-25-1.flv, etc.
SplitNumber = 0 ;

//Enable Plugins so we can have Flash working
QWebSettings::globalSettings()->setAttribute(QWebSettings::PluginsEnabled,true);

//Connect to "About QT" in the menu
connect(actionAbout_Qt,SIGNAL(activated())
,qApp,SLOT(aboutQt ()));

//Connect Signal from http for debugging
connect(CurrentConnection,SIGNAL(done(bool))
,this,SLOT(Http_error(bool)));
connect(CurrentConnection,SIGNAL(stateChanged(int))
,this,SLOT(Http_stateChanged(int)));
}


Set the inital Values of each Variables as well as connect every signal not automagically connected.

Functions for the "Quit" and "About" Menu Options:

void TwitLiveRecorderImpl::on_action_Quit_activated()
{
close();
}

void TwitLiveRecorderImpl::on_actionAbout_activated()
{
QMessageBox::about( this, tr("Twit Live Recorder"),
tr("Twit Live Recorder v0.1(Proof of Concept)"
"Using QT"));
}



Function to help with Debugging the http connection:

void TwitLiveRecorderImpl::Http_error(bool error)
{
if(error)
{
//Debug Code goes here
}
else
//Debug Code goes here
}
void TwitLiveRecorderImpl::Http_stateChanged ( int state ) { //Debug Code goes here
}


3 Function are left for "Record", "Stop" and "Split". First the "Record" function:


void TwitLiveRecorderImpl::on_actionRecord_activated()
{
ActiveFileName = QFileDialog::getSaveFileName(
this, tr("Please Select File Name"),
"/"+ BaseFileName + ".flv");
if(ActiveFileName == "")
return;

//Change the look of the Buttons:
actionRecord->setDisabled(true);
actionSplit_Current_Recording->setDisabled(false);
actionStop_Current_Recording->setDisabled(false);

ActiveFile = new QFile(ActiveFileName);

//Open Connection to server
CurrentConnection->get(Stream_url,ActiveFile);

//Set Visible "Currently Recording"
Line_CurrentFileName->setText(ActiveFile->fileName());

SplitNumber = 0 ;
}


The important part is the File dialog where the user choose the name and location of the file, that file is then passed-on as a destination for a new connection. The button are also activated or deactivated, depending on which one. The "Stop" function does things in reverse:


void TwitLiveRecorderImpl::on_actionStop_Current_Recording_activated()
{
//Abort Connection to Server
CurrentConnection->abort();

//Reset Displayed File Name
Line_CurrentFileName->setText("");

//Change the look of the Buttons:
actionRecord->setDisabled(false);
actionSplit_Current_Recording->setDisabled(true);
actionStop_Current_Recording->setDisabled(true);
}



Finally the "split" Function:

void TwitLiveRecorderImpl::on_actionSplit_Current_Recording_activated()
{
QFile *PreviousFile = ActiveFile;
SplitNumber++;

//Create new File name based on the Active File
QFileInfo OldFile = QFileInfo(ActiveFileName);
QString NewFileName = OldFile.absolutePath() + OldFile.baseName() + "-" + QString::number( SplitNumber, 16 ) + "." + OldFile.completeSuffix();

//Next File is the new Active File
ActiveFile = new QFile(NewFileName);
ActiveFile->open(QIODevice::WriteOnly);

//Close the Old Connection and open a new one
CurrentConnection->abort();
CurrentConnection->setHost(Host_url);
CurrentConnection->get(Stream_url,ActiveFile);

//Set Visible "Currently Recording"
Line_CurrentFileName->setText(ActiveFile->fileName());
PreviousFile->close();
}


This function opens a new file based on the previous name. Right now it is probably pretty buggy: It does not check if the file already exist nor give any sort of feedback, but the basic functionality is there.

It was a fun app to write and I was surprised at how little I had in the end to get a functioning application. Of course, this doesn't show the all the debugging time I spent on it (a ratio of around 1:5 to 1:6). And I should also give credit where credit is due, becasue this would not have been possible with work from Qt and Webkit. I am standing on the shoulder of giant with this sort of work.

Tuesday, March 17, 2009

International Radio Operator Alphabet

If you work on the phone and must spell out names all the time, use the NATO, FAA and ICAO approved phonetic alphabet:
  • Alpha
  • Bravo
  • Charlie
  • Delta
  • Echo
  • Foxtrot
  • Golf
  • Hotel
  • India
  • Juliet
  • Kilo
  • Lima
  • Mike
  • November
  • Oscar
  • Papa
  • Quebec
  • Romeo
  • Sierra
  • Tango
  • Uniform
  • Victor
  • Whiskey
  • Xray
  • Yankee
  • Zulu

As a bonus, it works in several languages and it saves the embarrassment of mistranslation when spelling out words: "A as in Ananas"
(Pineapple is a commonly mistranslated word. Aside from the Japanese パイナップル (Literally PA-I-NA-T-PU-RU), almost every other language uses a variation of Ananas, Nanas or sometimes Pina.)

Wednesday, February 18, 2009

We live in an amazing world, yet nobody is happy

This "viral video" is going around on the internet (It's been taken out of Youtube because of copyright violation from NBC, but can be found elsewhere):

http://vodpod.com/watch/1335606-louis-ck-%E2%80%9Ceverything%E2%80%99s-amazing-nobody%E2%80%99s-happy%E2%80%9D
http://barefootmeg.multiply.com/video/item/56

The stand up comedian has a point. Technology has come a LONG way in 100 years. Yet we have so high expectations that we are never happy with what we got...
Just 100 years ago, the average plane would not go very far... Barely enough for military reconnaissance.
Just 50 years ago, The average computer was AT LEAST several cabinet and had less than 10K of memory, operating well under 1 MIPS.
Just 25 years ago, The average personnal computer didn't have a hard drive in it!
Just 20 years ago, cell phone were all analog.
Just 15 years ago, the website for CBC was created out of HTML 1.0
Just 10 years ago, the first blackberry was released
Just 5 years ago, No intel mac, no Iphones, first color Ipod
Just 4 years ago, Xbox 360
Just 3 years ago, Intel Macs

And yet we complain when, on the passenger seat of a car, an iPhone has trouble finding the nearest gas station in a densely populated area. Yet the poor thing has to compensate for relativity to calculate GPS coordinates, compensate for Doppler shift while registering with 40 differents cell towers along the way and transferring huge among of data, and using algorithms developed over the years to estimate the shortest route, just so we can find cheaper gas.

A spoiled generation :)
Blogged with the Flock Browser

Monday, January 26, 2009

A political question unanwered

Here's is one project I plan not to continue. I simply lost the motivation to do it. The reason I blog about it is that I wish someone will read this and will finally make it work.

But first a disclaimer: This is a really good and bad idea at the same time. It create a political minefield and possibly something controversial.

It's about the electoral map.

It all started back in November. There was talks in the media that the Quebec electoral map wasn't 100% correct. Some district were too populated and the map had to be re-drawn... The process was started but was too slow for it to be done for the upcoming election. It got me intrigued. I'm pretty sure the way the electoral map is drawn influence a lot the outcome of the election. But how? I wanted to find out.

I wasn't the first time this sort of questioning crossed my mind. Back in 2005, while I was still a student in my final year of Computer Engineering degree, I had a course on algorithms and graph theory. The kind of course were you learn that some problems are unsolvable, unless you have infinite time to solve them. I remember asking the teacher this very question: "Can't we use graphs to map were the votes are and tell if the electoral map is unjust?". He answered that it was possible, but no one would be willing to do this sort of stuff because it's too political.

And so, when the medias started talking about the electoral map, I remembered that talk I had with the teacher. I had an idea at the time of how one would analyze the various information sources to determine how well the electoral map is divided. Here's how I wanted to do it:

You make point of interests, containing all the votes in a giving sector. It could be a municipality of any other sort of subdivision. The more there is, the better. For example, in Quebec city, one could create a Point-of-interest at for voting spot, containing all the various boxes that were in that spot. Each spot has a weight based on the number of registered voters and is linked to the rest of the electoral district to form a single district. Once you have this map, you create "potential links", links from each P-O-I to various neighbors in other district (for example, the 3 closest neighbors not in the district). The create the various potential maps, you turn on and off each link until you have a legal, but different map.

OF course, there is a couple of problems with this approach.First, to do all this you need:
  • Coordinates for each voting poll
  • Number of voters for each voting poll
  • Results for each poll
  • District for each voting poll

Almost all this information is public, except for one: the exact location of each voting poll.The only information giving for each box is the municipality where the voting poll was. So all the maping information for the voting poll in Montreal, only says : "Location: Montreal". Hard to place the voting voting polls with so little information.

The second problem is the complexity of the entire process. My initial list had 1848 potentials P-O-I on the map. If you limits the links to the 5 closest neighbors, you still get somewhere between 3000 and 9000 links... Which means calculating between 2^3000 and 2^9000 possibilities... Which could take a very, very long time. (2^3000 is over 10^600, or 1 with 600 zeros)

After a couple of days of trying to find ways to optimize the algorithm, I simply gave up. There is not enough information to create a meaningful map, since the location information in large urban area is not very detailed and leads only to a partial answer...

It was a great idea, but I can't go further.

I guess my time is better spent elsewhere...

Monday, January 19, 2009

Hellsing (anime) is a perfect match for E Nomine

The music by E Nomine seem to be a perfect match for any sort of Hellsing AMV.

I found several, but I think this one is far better than the rest (I HAD to share this):

Monday, January 12, 2009

The Cost of SMS in Canada

Below is a copy of an E-mail I sent to Mr Jesse Brown who host the CBC podcast Search Engine:
http://www.cbc.ca/searchengine/

The topic was SMS pricing. (Show for 2009-01-12) I checked to see how much we were really paying for SMS compared to talking... my math says 500times... at minimum..

Can you check and tell me if I'm right?


All of the Telephone based information is based on the Creative Common Licensed book "Asterisk, The future of telephony", available at http://oreilly.com/catalog/9780596510480/ , just click the Online book for the entire PDF


Traditional telephone is 8bit sampling 8000 times per seconds, making traditional telephone 64kilobits per seconds. (or 8 kilobytes per second)Codecs might bring this down to anything between 8kbps and 15.2kbps.

One minute of talk can be anything between 480000bytes (at 64kbps) and 60000bytes (at 8kbps). This is, of course, low quality voice talk because a comparable MP3 file of the same length would be around 1024kilobytes (1MB). Let's just use the lower end for comparison.

To get a rough idea of pricing, let's take a talk plan that gives a 400 minutes for 40$. At that price, the cost of talking for 1 minute of talk (if you use all your available minutes) is 40$ / 400 minutes = 0.1$/m

So with 0.1$/m and 60000bytes/m talk costs around:
0.1/60000 = 0.000001667$/byte
or
0.000001667$/byte * (1024*1024) = 1.747626667$ per Megabytes at the low end.

SMS is, by comparison,very small. 140 Characters in 8bit mode or 160 characters in 7 bits encoding, making the content of the message only 140bytes. The overhead of such message makes the entire message under 200bytes, but let's use 200 bytes for comparison.

At 0.15$ per SMS, it cost:
0.15/200 = 0.00075
or
0.00075$/byte * (1024*1024) = 786.43$ per megabytes...

so it's around 500 times more expensive to send SMS at the base price, then to talk on my phone.

An equivalent price for SMS to be billed like talk would be:
0.000001667$/byte * 200 bytes/SMS = 0.00033$/SMS
or
0.33$ per 1000 messages...

Quite a difference, don't you think?

Friday, January 09, 2009

The Last Question, a short story

This is an old but fascinating short story written by Isaac Asimov:

The Last Question