Skip to main content.

Thursday, July 28, 2011

After a couple of days of work, I have found that the only sensible way to allow dynamic loading of modules is that to create references for static data and let those to be separately managed by the garbage collector.

Consider strings, arrays of flat items or even user-defined classes that might go in the static data of a dynamically loaded module. If some item is take from by the user of the loaded module and sent around, it must stay alive as long as needed. If necessary, it must even keep alive the module that is hosting it, otherwise it should just stay around even after the module has died. I wouldn't love to keep a plugin around just because it has returned "OK" from inside a function it exposes, and that OK is kept somewhere for future records...

Adding the requirement that anything that goes or may ever go outside the dynamic module boundaries must reference the module back would be a nightmare. It may make sense for (some) classes and functions (and I think we can lower this requirement to the external/native code ones) but it's totally unreasonable for i.e. strings. Also the reverse requirement, that is, that a module must stays alive 'till all the data born from it are gone is pretty heavy.

So, the only sensible way to handle plugin-based dynamic (dischargeable) modules is that of making their static data references as soon as the module is linked dynamically in a virtual machine. When the module is gone, the reference items might still be around and keeping alive the related data.

For this reason, re-introducing the references was in order before completing the module loading system.

In the process, I also improved the usability of references that was somewhat limited in the old engine. Here follows a talk about it in our #falcon chan at

jonnymind this is crucial to module loading, as I have found that the only consistent way to ensdure static data stability in dynamic modules is to create a reference out of them.
jonnymind Notice th power of the new references:
jonnymind Prototype{val=Ref {"World"}}
jonnymind >>> x = 100
jonnymind 100
jonnymind >>> proto
jonnymind Prototype{val=Ref {100}}
jonnymind ATM I decided that reference value can be altered only writing directly to a symbol.
jonnymind for instance,
jonnymind >>> proto.val = 1
jonnymind 1
jonnymind >>> proto
jonnymind Prototype{val=1}
jonnymind The reference is now broken.
jonnymind BUT
jonnymind >>> proto.mody = {v => val = self.val; val = v }
jonnymind (Method .anonymous())
jonnymind >>> proto.val = x
jonnymind Ref {100}
jonnymind >>> proto.mody( "Hello again" )
jonnymind >>> proto
jonnymind Prototype{mody=(Method .anonymous()), val=Ref {"Hello again"}}
jonnymind >>> x
jonnymind Ref {"Hello again"}
jonnymind I think it's important to consider the references alterable only as values. In this way altering properties or vector elements doesn't bring into surprises,
jonnymind and you always have a way to address the reference, if you need to.
jonnymind Also, foreign class may have different ideas about storing items.
jonnymind suppose I intercept x.v = prop
jonnymind The fact that I am sending a reference there is not so transparent if you're saving i.e. integers and expect to receive integers.
jonnymind OTOH,
jonnymind we may decide that the behavior of OUR classes is that of respecting references.
jonnymind ---
jonnymind like this:
jonnymind >>> x = "Hello"
jonnymind "Hello"
jonnymind >>> proto = p{
jonnymind ... var = $x
jonnymind ... }
jonnymind Prototype{var=Ref {"Hello"}}
jonnymind >>> proto.var = "World"
jonnymind "World"
jonnymind >>> x
jonnymind Ref {"World"}
jonnymind But then we need a way to break the reference.
billykater when would you need to break a reference? ( I am currently thinking c++ references ^^)
jonnymind For instance, when var needs to be something that's not x anymore.
jonnymind or when X need to do other things.
jonnymind Suppose that I want to assing var to another reference.
jonnymind i.e. because var is SEMANTICALLY a pointer to a falcon object that I decide somewhere else.
jonnymind and I want to take another reference.
jonnymind doing a thing like
jonnymind proto.var = $v1 won't work, as would cause var to reference v
jonnymind This is giving me an idea.
jonnymind Handling references transparently is a *hard* thing.
jonnymind error-prone and heavy
billykater hm the problem will be that every program will have to explicitly decide what to do with references it gets handed. suppose you have a function with 3 parameters that get stored internally. each program writer now has to decide how references are handled
jonnymind If we just had references as a language structure,
jonnymind ...
jonnymind but then, it would make no difference from a simple
jonnymind class Ref(x)
jonnymind data = x
jonnymind end
jonnymind No, references must give you some automatism.
billykater currently thinking about a way to handle it to not let them be the class you wrote above
jonnymind In short, you shouldn't even know you have a reference.
billykater class MyClass
billykater function func(x)
billykater self.x = x
billykater end
billykater end
billykater for the first c = MyClass(); c.func(ref) it will initialize the reference and the second c.func(someValue) would then set the reference to point to this value. this will become a nightmare for most programs.
billykater References should really be used with care
billykater or adding the reference as key to a dictionary. there can be some really interesting situations from this event
billykater d = [ ref => "a" ]; ref = "b" ^^
lumo_e what is wrong with our current model? (I mean
jonnymind Found
jonnymind lumo_e: a couple of things,
jonnymind for instance, the fact that you couldn't post references into objects or vector.
jonnymind However, I found a solution.
jonnymind The new engine has a method called Item::assign()
jonnymind it keeps track of a few things. Mainly, of copy-on-use patterns.
jonnymind It MUST be used when storing a value into a visible variable.
jonnymind i.e. into an array element, a property or a symbol.
jonnymind By just adding the dereference logic there...
jonnymind :-) done
billykater so assigning a reference to a reference will not make one reference hold the other one
jonnymind Exactly.
jonnymind Theoretically, it could be possible (but you still must mask references to yourself, the last test I checked in the paste).
jonnymind But practically, it's a nightmare.
jonnymind You can do that when you have explicit ways to access the referenced items;
jonnymind for instance, in lisp is legal to reference symbols from symbols, and even reference your own symbol.
jonnymind but you EXPLICITLY ask for dereference (which matches with evalueation).
jonnymind like
jonnymind (defq x 'x)
jonnymind x now evaluates to ... x
jonnymind But you always evaluate a single reference, and you are always told by the user when this evaluation must take place.
jonnymind here it's different.
jonnymind the different scheme of infix expressions suggest that references must be deep 1 step at worst.
jonnymind i.e. references can't reference other references.
jonnymind That makes things a lot easier in an infix-expression based language.


No comments yet

Add Comment

This item is closed, it's not possible to add new comments to it or to vote on it