The A-Z of programming languages: Falcon

Computerworld Australia is undertaking a series of investigations into the most widely used programming languages. Previously we have spoken to Larry Wall, creator of the Perl programming language, Don Syme, senior researcher at Microsoft Research Cambridge, who developed F#, Simon Peyton-Jones on the development of Haskell, Alfred V. Aho of AWK fame, S. Tucker Taft on the Ada 1995 and 2005 revisions, Microsoft about its server-side script engine ASP, Chet Ramey about his experiences maintaining Bash, Bjarne Stroustrup of C++ fame, and Charles H. Moore about the design and development of Forth.

We've also had a chat with the irreverent Don Woods about the development and uses of Intercal, as well as Stephen C. Johnson on YACC, Luca Cardelli on Modula-3, Walter Bright on D, Brendan Eich on JavaScript, Anders Hejlsberg on C#, Guido van Rossum on Python and Prof. Roberto Ierusalimschy on Lua. We most recently spoke to Steve Bourne, creator of the Bourne shell.

In this interview, Falcon creator Giancarlo Niccolai took some time to tell us about the development of Falcon, the power and influence of C++, and how the new multithreading design in Falcon Version 0.9 will innovate the scripting language panorama.

If you wish to submit any suggestions for programming languages or language authors you would like to see covered, please e-mail

What prompted the creation of Falcon? Part of my daily job was taking care of the status of servers streaming real time data through financial networks. A scripting facility in the control application would have been a godsend, as the alert conditions were too complex to be determined, constantly shifting and requiring constant supervision. I was not new to the topic, as I previously worked on the XHarbour project (a modern porting of the xbase languages), and I also did some research in the field.

The workload was heavy; even if the logic to be applied on each message was simple, data passing through was in the order of thousands of messages per second, each requiring prompt action, and each composed of about one to four kilobytes of raw data already de-serialised into complex C++ class hierarchies.

In terms of raw calculation power, the existing engines were adequate, but they were greedy. They considered their task as the most important thing to carry on the whole application, and so they didn't care very much about the time needed to set up a script, to launch a callback, to wrap external data or to provide them with data coming from the application at a very high rate.

It was also quite hard to use them in a multithread context. The only VM designed to work in multithreading (that is, to be used concurrently by different threads -- we had many connections and data streams to take care of) was Python, but it worked with a very primitive concept of multithreading, forcing global locks at every allocation and during various steps of each script. My test showed that this caused rapid slowdown, even in parts of the application not directly related with the script (i.e., in the parts preparing the data for the scripts before launching Python VMs).

Of all the possible scripting engines, LUA was the most adequate, but using it from concurrent threads posed some problems (at the time, the memory allocator wasn't thread-safe and needed to be refitted with wide global locks). Also, having to deal with wide integer data (prices on financial markets are often distributed as int64 with decimal divisor), I was worried about the fact that LUA provided only one type of numbers -- 58-bit precision floating point. This also caused severe rounding/precision problems in the financial area and are thus generally avoided.

There was so much work to do on those engines to make them able to meet the requirements for task that the alternatives were either dropping the idea of scripting the control application, and then the servers themselves on a second stage, or writing something new.

I hate to give up, so I started to work at HASTE (Haste Advanced Simple Text Evaluator), a scripting engine meant to be just a scripting engine, and to drive massive throughput of data with the host application.

Was there a particular problem the language aimed to solve? The main idea behind the early development of HASTE was the "integratability" with existing complex multithreaded applications and the interaction with real-time, urgent and massive data flow.

When I had something working, I soon realized that the ability to deal with raw data and the way the VM cooperated with the host application was very precious in areas where other scripting languages didn't shine like binary file parsing, image manipulation, gaming (not game scripting), and wide/international string manipulation, etc.

At the same time, I found the constructs in other scripting languages limiting. I could live with them, as imperative and prototype-based programming are quite powerful, and the "total OOP" approach of Ruby is fascinating, but now that I had HASTE working and doing fine (the HASTE VM was simpler, and slightly faster, than LUA's), I started thinking beyond the pure needs of the engine.

As a professional, that exact task for which I built HASTE was just a small part of my daily activities. Similarly to the way Larry Wall built Perl out of his needs (to parse a massive amount of unstructured log data set), I started to feel the need for higher logic to carry on my tasks -- complex analysis on structured data, pattern finding and decision-making.

I used to work with many languages, including C, C++, Java, Assembly, Lisp, Prolog, Clipper/Xbase, Delphi, SQL and of course Python, Lua, Perl and PHP, and I learned through time to employ the best instruments to solve the problem at hand. I felt the need for a tool flexible enough that it could cover my daily needs and drive new ideas.

Pure ideas are useful for the machine. A pure logic language, as Prolog, can explode the rules into raw machine code, being as fast as possible in finding solutions. Pure functional languages, as Erlang, can parallelize massive calculation automatically and compile time. Pure OOP languages, such as Ruby, can treat any entity just the same, reducing the complexity of the code needed to implement them.

But purity is never a good idea for the mind. The mind works towards unification and analogy, and solutions in the real world are usually more effective when a wide set of resolutive techniques can be employed. This is true even in mathematics, where you need to apply different resolutive techniques (and often also a good deal of fantasy and experience) to solve seemingly "mechanical" problems as the reduction of a differential equation. HASTE was terribly simple, a purely procedural language with arrays and dictionaries, but it had an interesting feature -- functions were considered normal items themselves. This gave me the idea of working towards a general-purpose language (beyond the scripting engine of HASTE) whose "programming paradigm" was "all and none".

So Falcon was born with the idea of having pure OOP (so that raw, hard C structures could be mapped into it without the need for a dictionary-like structure to be filled), but not being OOP, with the idea of having pure procedural structure (driven by old, dear functions and call-return workflow), but not being procedural, and with the idea of having functional constructs, but without being functional.

It was also developed with the idea to add new ideas into it besides, and throughout, the existing ideas. A set of expanding concepts to solve problems with new conceptual tools, to reduce the strain needed by the professional developer to find the right way to match its idea with the problem and to create a living solution. Not needing anymore to learn how to think in a language to have it to work out the solution, but having a language moving towards the way the programmer's mind solves problems. I needed a tool through which I could shape easily and flexibly solutions to higher logic problems, in different and ever-shifting domains (one day parse gigabytes of binary data in search for behavior patterns, the other day organizing classroom "turnations" for courses in my company), and to do that fast.

If there is one thing I'm proud of in Falcon, it's that it wasn't born for the most exotic reasons, but to address the problem of integration and empowerment of massive applications on one side and the necessity do solve complex logic and highly mutable problems on the other. Or, in other words, it was born as a necessary tool.

Few languages were born to address real problems, like Perl, Clipper, possibly C++ and C, which actually evolved from research/didactic university projects.

Why did you choose C++ to base Falcon on, rather than a lower-level language? What are the similarities between the two languages? I like OOP and I like the power of C. When I decided to go C++ for Falcon, I was positive that I would have used C for the low level stuff and C++ classes to shape higher concepts. All the applications I had to script were C++, or could integrate with C++. Also, I was told by a friend of mine working in the gaming industry that Virtual Calls was actually more efficient than switches, and we have lot of them in a scripting language. I tested the thing out for myself, and it turned out that modern processors are performing really well in presence of Virtual Calls, so many of the common problems were usually resolved with complex ifs, and switches could be resolved with Virtual Calls instead.

At the beginning I used also STL, which is usually much more performing than any dynamic typing-based library (STL maps are at least 10% faster than Python string dictionaries), but that carried on a relevant problem of interoperability with other programs. Falcon was also meant to be a scripting engine, and applications often have different ideas on which version of the STL they would like to use. Moving STL across DLLs is quite hellish, and it's a death blow to the binary compatibility of C++ modules (already less stable than C module binary compatibility by nature). Also, STL caused the code to grow quite a lot, and a lot more than I wanted; so, I temporarily switched back to dynamic typed structures, which are slower, to be sure to clear the interface across modules from external dependencies.

Recently, having found that new compilers are extremely efficient on the fast path of exception raising (actually faster than a single IF on an error condition), I have introduced exceptions where I had cascades of controls for error being generated deep inside the VM.

In short, Falcon uses C++ where it can bring advantages in terms of speed and code readability and maintainability, while still being C-oriented on several low-level aspects.

This may seem like reducing the interoperability with C applications, but this isn't the case. One of our first works as an open-source project was the preparation of the FXchat scripting plug-in for the famous Xchat program; as many know, the Xchat plugin API is pure (and raw) C. Yet, the interface blends gracefully in the framework without any interoperability problems, and even without any particular inefficiency, as the Falcon scripting engine is confined in its own loadable module and the FXchat module acts as a bridge. The code is even simple and direct, and it is easy to compare it against other scripting language plug-ins written in C that soon get much more complex than ours.

The same can be said for modules binding external libraries. We bind gracefully with both the DCOP library (written in C++) and the SDL library set (written in C), with complete interoperability and no performance or compatibility problems at all.

How is Falcon currently being adopted by developers? Falcon is still little-known on the scene, and with monsters like Python and Perl around, being fed by big institutions like REBOL and Erlang, the diffidence towards new products is great. On the other hand, it must be said that many developers who have been exposed to Falcon have been impressed by it, so much so that they didn't want to be without it anymore! Sebastian Sauer of the Kross project worked hard to have Falcon in Kross and KDE; Dennis Clarke at BlastWave is redesigning the BlastWave open-source package repository Web interface with Falcon and is helping in porting all the Falcon code base to Sun platforms -- AuroraUX SunOS distro has decided to adopt it as the official scripting language (along with ADA as the preferred heavyweight development language). We receive many "congrats" messages daily, but as we practically started yesterday (we went open source and began getting distributed a few months ago), we have the feeling that there are many interested developers taking a peek and staring from behind the window to see if the project gets consistent enough to ensure a stable platform for future development.

On this topic, we're starting to receive interesting proposals from some formal institutions. At the moment, it's just a matter of interest and work exchange, but if things go on growing with the rhythm we've been observing recently, we'll soon need to fire up an economic entity to back the Falcon PL project.

1 2 3 Page 1
Page 1 of 3
Shop Tech Products at Amazon