The A-Z of programming languages: Falcon

1 2 3 Page 2
Page 2 of 3

How is Falcon currently being adopted by developers? Falcon is still little-known on the scene, and with monsters like Python and Perl around, being fed by big institutions like REBOL and Erlang, the diffidence towards new products is great. On the other hand, it must be said that many developers who have been exposed to Falcon have been impressed by it, so much so that they didn't want to be without it anymore! Sebastian Sauer of the Kross project worked hard to have Falcon in Kross and KDE; Dennis Clarke at BlastWave is redesigning the BlastWave open-source package repository Web interface with Falcon and is helping in porting all the Falcon code base to Sun platforms -- AuroraUX SunOS distro has decided to adopt it as the official scripting language (along with ADA as the preferred heavyweight development language). We receive many "congrats" messages daily, but as we practically started yesterday (we went open source and began getting distributed a few months ago), we have the feeling that there are many interested developers taking a peek and staring from behind the window to see if the project gets consistent enough to ensure a stable platform for future development.

On this topic, we're starting to receive interesting proposals from some formal institutions. At the moment, it's just a matter of interest and work exchange, but if things go on growing with the rhythm we've been observing recently, we'll soon need to fire up an economic entity to back the Falcon PL project.

What's the Falcon ecosystem like? In the beginning, and for a few years, it was "just" me. I say "just" in quotes because Falcon wasn't meant as a spare-time project, but rather a professional project aiming to interact with my daily job on high-profile computing. The fact that I have been able to sell my skills and my consulting time, rather than my software, has allowed me to separate some internal projects that cannot be disclosed, from generic parts that I have shared via open source (there are also some older libraries of mine on SourceForge, which I employed on various projects). Since early 2007, some contributors have checked out the code and occasionally provided patches, but the first contributions other than mine to the repository are from 2008.

I have a community of about 10 to 12 developers, contributors and packagers actively working on the project, either externally or providing code in the core, or subsidiary modules, or on related projects. Their contributions are still few in number and weight, but entering in a project as wide and complex as Falcon requires time. We're also preparing a sort of "mini-SourceForge" to host Falcon-based and Falcon-related projects.

If developers want to know a bit about our project style, we are as open as an open-source project can be. New ideas have been constantly integrated into our engine, thanks to the help and suggestions of people either being stable in the project or just passing by and dropping a line. Although it has been impossible up to date, I am willing to pass down my knowledge to anyone willing to listen and lend a hand to the Falcon PL project. So, if developers are searching for a way to make a difference, stick with us and we'll make your vote to count!

Does the language have its own repository system? No. It may seem strange being said by me, but I really don't like to reinvent the wheel. We're using SVN, but we may switch to GIT or something more Web-oriented if the need arises. In this moment, I don't consider the commit history to be very relevant (with an exception for the 0.8 and 0.9 head branches), so switching to a new repository wouldn't pose any problem.

Falcon seems to have an emphasis on speed. Is this important within programming languages? Speed is not important within programming languages -- it is important for some tasks that may be solved by some programming language. As I said, I wrote HASTE out of a need for speed that wasn't addressed by any other scripting language, but Falcon evolved from HASTE for other reasons. If speed was everything, scripting languages wouldn't exist. On basic operations, the best in our class can perform about 30 times slower than C doing the same things, and that's definitely slow. Versatility, adaptability, maintainability, adequacy, integratability, complexity and many other factors play a major role in deciding which language to adopt.

Speed is the most important factor in the sense that is the prerequisite of everything, but it's not language-specific. Speed is determined by the complete input-processing-output line, and what a scripting language does into that line is usually a small part. If your IPO line doesn't meet the requirement, nothing else is relevant; in the case of early Falcon, no other scripting engine was able to let my applications to use them in an IPO line efficient enough to do the job on time. Once your whole system meets the IPO time requirements, speed ceases to be in the equation and everything else but speed becomes relevant. It's a binary choice: Your whole IPO line is either fast enough or not.

When we say "fast", we mean that we concentrated our development towards helping the whole system around Falcon to use it as fast as possible. VM speed is also relevant, as there are some tasks in which you want to use heavily VM-based calculations, but it plays a secondary role, in our view, with respect to the "service" offered to the surrounding world (applications, modules, threads) to let them run faster. This is why we have been able to live seven years without a single optimization on the VM itself, and this is why we're starting to optimize it now, when we have completed our reflection model and serviced the surrounding world the best we can.

How do Falcon's compile-time regular expression compilations compare with the library routines of other languages? Actually, compile-time regular expression was scheduled to be in by now (April 2009), but we went a bit long on the 0.9 release and this moved compile-time Regex development a bit forward in time. However, the topic is interesting, because it allows me to show three mechanisms at binding level that may be useful to the users of the scripting engine.

The plan is as follows: Falcon modular system provides to C++ a mechanism called "service". A service is a virtual class publishing to C++ what the module would publish to the scripts loading it. Since 0.8.12, the Falcon compiler has had a metacompiler that fires a complete virtual machine on request. Once we accept the idea of metacompilation, the compiler may also use the environmental settings to load the Regex module and use its methods from the service interface; that's exactly like calling the C functions directly, with just a virtual call indirection layer (which is totally irrelevant in the context of compiling a regular expression).

Since 0.9, items themselves are in charge of resolving operators through a function vector called item_co (item common operations). We may either introduce a new item type for strings generated as compile time regular expressions, and provide them with a item_co table partially derived from the other string item type, or just create them a string with a special marker (we already have string subtypes) and act with branches on equality/relational operators. On modern systems, a branch may cost more than a simple call in terms of CPU/memory times, so I would probably go for adding an item type (that would be also useful at script level to detect those special strings and handle them differently).

The fact that we want to use the Regex module at compile time is another interesting point for embedders. If we included regular expressions in the main engine, we would grow it some more and we would prevent the embedders from the ability of disabling this feature.

One of the reasons I wanted Falcon was to allow foreign, less-trusted scripts to be compiled remotely and sent in precompiled format to a server for remote execution. The executing server may want to disable some features for security reasons (it may forbid to use file i/o), and that just on some unprivileged VM, while the administrative scripts run at full power. That was impossible to do with the other scripting engines unless there were deep rewrites. Falcon modular system allows the modules to be inspected and modified by the application prior to injection into the requesting VMs. So, a server or application with differently privileged script areas can preload and reconfigure the modules it wishes the script to be able to use, preventing the loading of other modules, while letting privileged scripts to run unhindered.

Regexes are heavy, and not all embedding applications may wish their scripts to use them. In example, a MMORPG application may decide that AI bots have no use for regular expressions, and avoid providing the Regex module. At this point, the compiler would simply raise an error if it finds a r"..." string in a source, and the VM would raise an error if it has to deal with a pre-compiled Regex in a module. At the same time, as the Regex module is mandatory on any complete command line Falcon installation, command line scripts can use Regexes at the sole extra cost of dynamic load of the Regex module, which is irrelevant on a complete Falcon application, and that would be cached on repeated usage patterns as with the Web server modules.

Do you plan to develop your own Regex library to drive your regular expressions? No, we're happy with PCRE, which is the best library around in our opinion, and even if it's relatively huge, having it in a separate module loaded on need seems the way to go. We keep updated as possible with its development, providing native binding on some systems where PCRE is available (many Linux distributions) and shipping it directly in the module code where it is not available.

Is the embeddable aspect of Falcon versatile? I talked diffusely about that in the Regex example above, but other than the reconfigurability and sharing of pre-loaded modules across application VM, we have more. The VM itself has many virtual methods that can be overloaded by the target application, and is light enough to allow a one-VM-per-script model. Heavy scripts can have their own VM in the target application, and can happily be run each in its own thread; yet VMs can be recycled by de-linking just run scripts and linking new ones, keeping the existing modules so that they're already served to scripts needing them.

The VM itself can interact with the embedding application through periodic callbacks and sleep requests. For example, a flag can be set so that every sleep request in which the VM cannot swap in a co-routine ready to run is passed to the calling application, that can decide do use the idle time as it thinks best. For instance, this allows spectacular quasi-parallel effects in the FXChat binding, where the sleep() function allows Xchat to proceed. This may seem a secondary aspect, but other engines are actually very closed on this; once you start a script or issue a callback, all that the application can do is to hope that it ends soon. With Falcon you can interrupt the target VM with simple requests that will be honoured as soon as possible, and eventually resume it from the point it was suspended and inspected.

Since 0.9 we have introduced even a personalized object model. Falcon instances need not be full blown Falcon objects; the application may provide its own mapping from data to items traveling through the Falcon VM. Compare this with the need of creating a dictionary at each new instance, and having to map each property to a function retrieving data from the host program or from the binded library.

Other classes which you can override are the module loader, which may provide Falcon modules from other type of sources, or from internal storage in embedding applications, and since 0.9 the URI providers. Modules and embedding applications can register their own URI providers, so that opening a module in the app:// space would turn into a request to get a module from an internally provided resource, or opening a stream from a script from app:// would make possible to communicate binary data via streams to other parts of the application.

Frankly, we did our best to make our engine the most versatile around. They say LUA is very versatile, as you can reprogram it as you wish. But then, that is true for any open source project.

How will the new multithreading design in Version 0.9 innovate the scripting language panorama? There are two good reasons why multithreading in scripting languages are delicate matters (that many didn't even want to face). The first is that multithreading can break things. In "good multithreading" (multithreading which is allowed to actually exploit parallel computational power of modern architectures without excessive overhead), there is no way to recover from an error in a thread. A failing thread is a failing application, and that is a bit problematic to be framed in the controlled execution concept behind scripting language virtual machines.

The second reason is, as LUA developers point out, that a language where a = 0 is not deterministic cannot be proficiently used in multithreading. Some scripting language make a = 0 be deterministic and visible across threads by locking every assignment instruction, and that is a performance killer under many aspects. It doesn't only deplete performance on the script itself, but in case of concurrent programming in an application, it may severely deplete the host application performance by forcing it to unneeded context switches.

We opted for a pure agent based threading model. Each thread runs a separate virtual machine, and communication across threads can happen only through specialized data structures. In this way, each virtual machine can run totally unhindered by global synchronization. It is possible to share raw memory areas via the MemBuf item type, or to send complete objects created upon separate elaboration via a interthread item queue.

1 2 3 Page 2
Page 2 of 3
Shop Tech Products at Amazon