Skip to main content.

Saturday, January 01, 2011

Finally, Falcon has been released. The changes in the language are relatively few (few enough that it's binary compatible and released under the same release line of "Chimera"), but the changes in the overall project architecture are massive. Now, all the approved, or we may say, committee sponsored projects and modules are kept under the same development tree. This is a change deeper than it may seem at first glance: Falcon modules, and falcon related projects as Faldoc or Nest are now integrated in an unique development body. This has deep consequences in the mentality ruling the projects development guidelines.

For example, Faldoc had to be a stand-alone application up to date. In the new project-wide view, the modules used by Faldoc to scan for documentation entries and to render documentation code have been moved in the global module area; they are now a common asset that may be used also in other contexts.

Moving C code around from ancillary projects, as WOPI, and putting it in common as public modules will take a bit more, but the road is open. As creating new modules requires now far less complex procedures (it doesn't require creating a separate entry in the development site and a separate repository), it's definitely a lighter choice. Also, dependencies between modules can be checked at build time from a single CMAKE build structure; as such moving vital functions away and into a common structure increase interoperability and doesn't expose any drawback. It's a win-win choice.

We're about 2 months late on my schedule. I would have liked to issue the 0.9.8 version by the end of October, while we release by the end of December. The changes in the project structure required a deeper and more invasive review than I forecast. However, this project structure is definitive, and will take Falcon to version 1.0 and beyond. Actually, I don't see any need to revise it 'till 2.0.

Now, as the second step of the architectural change that must bring Falcon to 1.0, we're directly moving to 0.9.8 development. It's a somewhat complete rewrite of the inner engine (VM, item system, module system and compiler), yet it seems way less complex than what we undertake with this "Grand Release". The code involved, once removed comments and code that will become useless thanks to the new synergies, it's something around 10,000 lines. My development rate is around 1,000 line per day so the rewrite alone should take about 10 days of live work. The things to do have been studied and projected for a long time now, so it's just a matter of write them down; there isn't any new projectual step involved. Interface towards the external world, especially towards native modules, won't change dramatically. Maybe, object instantation and some part of the class definition will require a Search & Replace based refactor, but I think that this will take about 2/3 hours for even complex and hand-crafted modules as the GTK binding. Embedding applications will see the module load and link process slightly changed, but this rarely hits more than one single spot in the whole embedding program.

Finally, while I talked about a "total rewrite", it's not really a rewrite from scratch. The base code for VM and items stands, and it's really good actually; same for the modules. Just, a code reshape is needed there. The compiler must be rewritten as it's based on a dialogue between Bison and the Falcon::Compiler class, with the intermission of the "hack" called Interactive Compiler, with a structure called SynTree to hold the program definition. However, the Lexer is already a class we have, and it's very solid and well-tested; on the other side, the code generator is good and won't be changed except for removal of useless opcodes and a few lines of glue code with the new module system. Writing a compiler having a working lexer and a code generator in place is somewhat a trivial operation (although it may be lengthy).

On this spot, it is also important to remember that the new compiler will be fully reflexive. In other words, it shall be possible to interact with the compilation process either at compile or runtime, and inspect or alter the syntactic tree before the code is generated and run. In other words, other than compiling code slices in the caller context, it will be possible to compile a slice and inspect or change its composition before it is sent to the code generator and then to the VM for excecution.

The dual step of compilation and link will stay. This means that Falcon code will always stay a bit less dynamic than the code in the other scripting languages, but we have other means to be dynamic. Also, while a linked module is now a relatively monolithic structure, under the new module systems linking and unlinking modules on the fly will be a light operation, so that creating new code, using it and then throwing it away will be nearly indistinguishable from compiling it on-the-fly. But the link process has the advantage that can resolve some dependencies before runtime, spotting for common coding errors and optimizing searches into dictionaries so that they are performed just once. Optimization of method calls will take a bit longer (want to make a try at them while stepping into 1.0), but they are simply impossible without a link step.

Talking about optimization, I don't know if we'll be able to stuff anything in this release (stack-tail, dead-code pruning, loops unrolling, multiple reference caching, branch logic inversion and those basic optimization techniques are now a pressing need), but the new compiler model will produce better and simpler syntactic trees, and adding them at a later stage will be way easier. I have already a plan for 0.9.10 that includes other optimizations (small block memory management, string recycle), so this is something that may find its space there.

Other than the inner engine rewrite, this code, the release shall include a standard library redesign. This is highly needed to make the development of scripts stable from now on to 2.0 release. Everything that is not language-bound (directory handling, file handling, system related functions etc.) should be moved away from the core module and into separate modules. Theoretically, adding the proper "load/include" directive and eventually s&r refactor should do. I'd like to keep TimeStamp and other quasi-language classes in the core module, but will decide as the problem is undertaken.

The most massive change to the language will be the complete rewrite of the multitreading infrastructure as exposed to the modules. The inner engine is ok, it's just the way parallel processing is made available to the modules that will have to be changed. This may take the largest time in this development stage, and may call for changes in ancillary parts as message processing.

I'll report about the progress as it proceeds.


No comments yet

Add Comment

This item is closed, it's not possible to add new comments to it or to vote on it