[The first part of "Comet Story" was the story of failure. This part is no different with a slight twist at the end.]
Forgetting Comet was not such a bad thing actually. In the aftermath, I ended up passing the Zend Certified Engineer exam :-) But after the exam, the itch was back; time and again, I'll start thinking about Comet and online whiteboards. Regarding Comet, I was probably thinking deep inside my mind that a magic solution will appear somehow.
Talking about online whiteboards, there are two practical ways to go - HTML Canvas or Flash (Sorry, I do not consider Java applets a very practical solution). When I was trialling Meteor, I was using a Canvas-based whiteboard. Canvas is natively supported in Firefox, Safari, and Opera. The excellent excanvas Javascript library adds support for Canvas in Internet Explorer. Regarding Flash, there is not much to say - it just works iff a Flash plugin in available for your platform and your browser. For whatever reasons, I decided to go after Flash. The problem with Flash is kind of ethical instead of technical. Because Flash is not part of the web standards and a whole bunch of people accuse the Flash plugin of being a bloatware, it was really tough for me. I consoled myself by choosing OpenLaszlo as my development platform. OpenLaszlo is an Open Source Rich Internet Application development platform that can produce output in several formats including Flash. If you are familiar with Adobe Flex, then you can imagine how OpenLaszlo is. Instead of ActionScript, OpenLaszlo relies on JavaScript. But the interface definition is in XML. I was really excited about working on a totally new platform. Before this, I had no Flash development experience. Out of excitement, I read a whole book on OpenLaszlo. I had lots of practice and was getting a good grip on the platform. The OpenLaszlo book had a chapter on audio-video streaming. This was huge fun. Streaming requires a media server. Usually people go for the Flash Media Server from Adobe. I, being the Open Source fan I am, decided to go for Red5. At that time, Red5 was a very promising media server for Flash. Today, I am happy to say that it has fulfilled its promise. This Red5 came with a few demo applications and one of these applications was for video streaming. Quite naturally a video streaming application will need some kind of sample video and in this case, one of the videos was the trailer of the upcoming movie Ironman. The trailer was awesome!
I used it for testing my own video player and played it again and again. I cannot remember enjoying any of my own softwares so much. But then, the fun was in the video content and not in the software. No wonder youtube is so successful with such a simple setup. After all these video powered fun, it was time to roll the sleeves and get down to the real job - the construction of the whiteboard. And here, lady luck disappeared again :-(
Putting up a very basic whiteboard in OpenLaszlo was a piece of cake. It even worked! But after using it for a few minutes, it almost froze! And worst, I could not find why. A long time after that first dash with the whiteboard, I discovered that I was not the only one facing the problem and there was no known solution. Maybe it was some kind of bug in part of OpenLaszlo, but I'll never know. So that was the end of my stint with OpenLaszlo. After reading a whole book, after watching one of the best movie trailers, after contemplating a career in Flash development, I was standing totally empty handed.
While I was doing all these, Jason Kester of Expatriate Software (you should really read their About Us page :) released Twiddla.com which is an online whiteboard based on HTML Canvas. Instead of Comet, it uses polling. Other than this, Twiddla's whiteboard is the nearest to my wish.
My failure with OpenLaszlo convinced me that I'll have to work with the real thing and no Open Source alternative - that is, I'll have to learn ActionScript. As a matter of coincidence, Adobe released their Flex 3 SDK as Open Source around this time (Feb 2008, according to Wikipedia). It took me a while to find this out. So after a hiatus of a few months, I finally gathered enough energy to learn ActionScript at the beginning of 2009. It turned out that ActionScript is well capable of providing a bug free whiteboard and it can also interface with Comet servers. Things started to look really good. But I was seriously short of time to cook up the final meal :-(
In the few months between my move from OpenLaszlo to ActionScript, I have noticed quite a few mention of XMPP in various people's blogs. Their message was that XMPP will soon go big. This is the kind of message you'll hear about all sorts of things in all sorts of blogs from all sorts of people (eating Liquorice allsorts :) . So I didn't pay much heed. But I did start to pay more respect to XMPP although I was never a serious fan of online chatting. Both gmail chat and meebo uses Comet powered chat clients and gmail even uses XMPP as its chat protocol. Maybe these factors played a part to relate XMPP and Comet in my mind.
Then two things happened in Feb, 2009. First a link of Marek Majkowski's Comet powered collaborative text editor popped up in progit (Reddit's programming channel at programming.reddit.com). This editor uses something called the evServer as a Comet server. It is built in Django and routes the updates with RabbitMQ.
While going through blog posts on XMPP, I became aware of this term called "messaging platform". These platforms are a kind of telephone exchange for inter application messaging. Majek's article made me aware of using a messaging platform for passing updates between browsers. The idea works like this:
Now there is an ongoing debate about whether XMPP servers qualify as a messaging platform. Some say XMPP is the future of messaging; others say it is AMQP and not XMPP. RabbitMQ is the most popular AMQP server and ejabberd is the most popular XMPP server. So it's almost a RabbitMQ vs. ejabberd type argument :-) But I knew so little about messaging that I was not bothered at all. My attitude was - I'll use whatever works.
So why did I care so much about the messaging platform? Because they support authentication. They will allow me to send upgrades to only those browsers who are supposed to receive the upgrade. This will prevent intrusion.
The second incident was like this - my (then) co-worker Mike dropped me a link over Skype for something from the MySQL planet. Mike is from Transylvania and claims that he is a descendent of count Dracula. He is an interesting fellow for various other reasons. You just cannot ignore a link that Mike has sent you :) So I opened it. While I was at the page, I noticed lots of other links to various other planets, e.g. Gnome, KDE, etc. But my eye stopped on the link for the XMPP planet. I suspect that my recent awareness of XMPP's potential has played a part here. Once I was at the XMPP planet, the teaser that caught my eyes was a blog post on a JavaScript library for consuming BOSH services. If you have never heard of "BOSH" then don't worry because neither have I. After some background study, it was clear that BOSH is a specification for implementing a Comet server within an XMPP server. This will allow XMPP traffic to pass over HTTP both to and from XMPP server and web browser. The end result is, any XMPP server that implements BOSH can provide both a Comet server and a messaging platform. Two for the price of one; not bad :-) But to consume BOSH, we need a BOSH client that will run inside the web browser. A BOSH client is a complicated piece of JavaScript, but the blog post was about a new BOSH client called Strophe. with Strophe now available, there is no barrier to BOSH's widespread adoption. But I was happy to know about BOSH and Strophe for a different reason - I can play with one more Comet server and gain some more fun (or frustration) in the process.
At almost the same time, one more link caught my eyes at progit. It was Mihai Sucan's tutorial about building an HTML Canvas based paint application. This tutorial also came with an easy to understand demo application. So I now have the code for a paint application and Strophe for writing Comet applications all backed by a messaging platform. Quite naturally, the next step was to mash these all and fulfil my wish for a Comet powered collaborative whiteboard. I decided to give it a go on the next weekend.
When the weekend at last came, I spent the first few hours following Mihai's tutorial. Nice piece of write up I will say. So the next task was to learn to use Strophe. ejabberd already implements BOSH, so I spent the next few hours fighting to configure it as a BOSH capable XMPP server. No complain here either. Next it was time to run a few test apps that come with Strophe. Quite predictably, none of those worked! Whatever I do, they would not budge :-(
While I was dabbling with Canvas, Strophe and ejabberd, something was pecking at the back of my mind. It was the use of ejabberd as the messaging platform. After all, ejabberd is a chat server and not a messaging server. But RabbitMQ is a dedicated messaging server. I have always believed in the Unix philosophy of doing one thing well. So why am I now using a chat server for messaging instead of a dedicated messaging server? As soon as Strophe's test applications failed, this feeling became so strong that I almost decided to ditch ejabberd in favour of RabbitMQ. But RabbitMQ's JavaScript client seem to be a complexware and I avoided it. I realized that if I want to use RabbitMQ, I'll have to keep it behind a dedicated Comet server which was okay with me. Server side code (e.g. PHP) will then send updates directly to RabbitMQ and RabbitMQ will pass it to the Comet server which will in turn pass it to the web browser. So it was time to learn how to write a client for RabbitMQ in PHP. But when even this thing failed to work, it was too easy for me to discard RabbitMQ once and for all.
So I was back to square zero again. Unable to use Strophe, I was thinking what to do. Once again, I am not sure why I did what I did next. I pulled a copy of Strophy from its git repo and suddenly the test application worked!!! Prior to this I was trying with the recent stable release of Strophe. That stable release has given me quite an unstabling time.
Now I have all the cards in my hand. I have gathered all I need to know to build the application that I have wished to build for so long. My application will be built with HTML canvas, Strophe, and ejabberd. My joy knew no bounds when the application finally worked after two more days of work. But I'll leave the details of those two days for the next and last part of this series. In the mean time, if you are restless to see what I have actually built, then head on to this place.