The Twitter Sentimeter, Part Drei

Network World - So, the Sentimeter, my almost real-time Twitter sentiment analysis tool. If you need to catch up on where we're at you can check out the Gearhead columns from last week and the week before.

I chose to build the Sentimeter using Xcelsius, a development system that lets you attach visual and functional components to Excel spreadsheets and then generate a Flash movie.

I ended last week with a rough description of my homemade ASP.NET-based proxy server, which I created because the Flash-based Xcelsius is constrained by the security sandbox model that the Flash player enforces. This model prevents access to domains other than the one you get a Flash movie from to prevent crossdomain scripting exploits. You can implement a crossdomain policy file on your Web site to allow other domains to be accessed but this didn't seem to work for me as no matter what my crossdomain policy declared, my Xcelsius movies kept producing "Error #2032".

As I couldn't get rid of the dreaded "Error #2032" with a crossdomain policy and I needed to call multiple APIs (Twitter's Search API and the OpenAmplify's API) after the Sentimeter tool was loaded from my server so using a proxy service seemed like a good idea. Another compelling reason for using a proxy is that Xcelsius lacks a whole passel of features that would make interacting with Web services easy (for example, it has no functions to URL encode data).

I hacked up some ASP.NET code using VB scripting and voila! I could make a simple request to my server and it created the correct URLs and dispatched the request and then passed the returned data back to the movie.  I then found that one of my first problems was caching because browsers cache, ASP.NET caches, and so does OpenAmplify.  My Flash movie would make its first request and then get back the same answer for subsequent requests for a period of what appeared to be about 10 minutes. Very annoying so all caching had to be eliminated.

Getting rid of OpenAmplify's caching was easy: All I had to do was add "cache=disable", which is one of the new parameters in the recently released 1.1 version of the API. Then I had to disable ASP.NET's caching, which took a little research and fooling around to get the ASP pages to compile cleanly (really Microsoft, do the ASP.NET error messages have to be quite so opaque and vague? Really?). Now I could get a fresh response for each request.  

But hold on! Every now and then the Sentimeter would still show "Error #2032"! As far as Flash was concerned the movie was loaded from and every request made to the same domain so obviously something else was going on.

And here's where I found a tool that you absolutely need to have permanently installed and know how to use it:  Microsoft's Fiddler, a free Web debugging proxy that works with all Web browsers and applications and is simply excellent (I give Fiddler a 5 out of 5 rating). Fiddler Fiddler can log all HTTP and HTTPS traffic so you can inspect it, set breakpoints and "fiddle" with incoming or outgoing data. It also includes "a powerful event-based scripting subsystem, and can be extended using any .NET language." In short, it is a "must have" for your IT toolbox.

Next week: What I found with Fiddler.