Improving reading self in the Debugger for Clean Blocks

In the last post, we looked at reading the PseudoVariable thisContext in the Debugger. A summary of that might be: inspecting a variable compiled a DoIt, that DoIt it executed parametrised with the context that you look at in the debugger. As it is, in the end, just another method executing, the byte-code pushThisContext pushes the context of the executing DoIT method, not the context that you look at in the debugger.

For “self”, this is of course the same. Take Pharo11, add and insect “thisContext method inspect. self”

41 <52> pushThisContext
42 <80> send: method
43 <81> send: inspect
44 <D8> pop
45 <58> returnSelf

Thus even “self” is not really concerned with the context you are looking at, it is the “self” of the DoIt. The nice thing is that this is always (in the debugger) the correct object: In the case of blocks, the only time you look at a block in a debugger to read self is if you are in the home method of the block (the method that contains the definition of the block). And as the “self” of a block closure is the “self” of the home context, you get the value you expect.

Self and Clean Blocks

It gets interesting if you put clean blocks into the picture. Clean Blocks are blocks that just use information that is available at compile time. The compiler thus can pre-allocate them. For this discussion, we need to know that clean blocks do not have an outer Context, and self of the block is always nil.

Clean block support is there, you can enable it and even re-compile the whole image, but there are still some details to be improved before we can enable it. One of that is to fix all the debugger infrastructure to deal with blocks that have no receiver and no home context, this post thus is one tiny step of that work.

For this post, we just need to compile one method with clean blocks enabled, we create a class and method to play with (e.g. class TT with method tt):

tt
    <compilerOptions: #(+ optionCleanBlockClosure)>
    [ 1halt ] value 

(or instead of the pragma, enable the setting globally with the Settings browser)

With just sending a message to 1, the block can be compiled as a clean block. The halt will mean that the debugger opens on the context of the clean block being executed.

Let’s do that, execute “TT new tt” and we get a debugger:

It is interesting to look at the context, if you inspect the context, you see that the receiver is nil. Looking at the block closure, we see that it is indeed a CleanBlockClosure, outerContext is nil (and thus it can not follow the outerContext till it finds the home context).

So if we now just write “self” in the block and expect it (we do not save, we just want to explore), it evaluates to “nil”:

This is of course correct: The debugger looks at the home context of the block, we ask context for “self” and self in a context evaluating a clean block is nil, as the block does not know self.

But I am sure you are not happy with that. It feels wrong. The reason is that if you would accept the method (recompile), restart, then self would be “correct” (the instance of TT). This is because the compiler now sees the self, realizes that this block requires a full block and compiles a full block.

This is similar to accessing temps in blocks that do not access them. Change the method to have a temp, but one that is not accessed in the block:

tt 
    
    | temp |
    temp := 1.
    [ 1halt ] value.
    ^temp.

If you write “temp” into the block and inspect or print it, you get the value of temp:

The context of the executing block does not know the variable “temp” at all, it’s value is as unknown to it as the self is to the clean block. (This is true both if compiled as a clean block or a full block).

So why can we nevertheless read it? The reason:

• Looking up the variable by name of course works (after all, the compiler can compile a block that does access it, the compiler uses the same data for variable lookup as the reflective subsystem does)
• The reflective read is carefully build to allow reading this variable even from the block context that does not know it (by leveraging the stack of contexts)
• We force the compiler to compile all temp variable accesses as reflective accesses in a DoIt

So if we can “fake” it for temps (both clean and non-clean blocks), can we do it for “self in a clean block”, too?

If you think back at the discussion of reading thisContext, we fixed that by forcing reflective read for thisContext, too. And of course the PR that implements it, was done already for all PseudoVariables.

This means, reading self should, in Pharo12, already end up executing SelfVariable>>#readInContext:, so let’s just put a halt there, trigger a read in the debugger and, indeed:

One thing we now see is that reflective read of self is implemented to do exactly the same as “pushSelf”, it returns the receiver of the context we look at in the debugger:

readInContext: aContext
    self halt.
    ^aContext receiver

But that is not really correct: in case of a block, it should be the receiver of the home context. For all practical purposes (when in the debugger) they are the same, and for Clean Blocks we even know that the home Context is unknown… So how does this help if we would implement it like that:

readInContext: aContext
    ^aContext home receiver

The trick is that we can find a home context for clean blocks in some specific cases. Even though we can not get the home of a clean block via it’s outer context (as is is nil), we can find the home method if it happens to be on the stack. And in the debugger, we are in exactly that case!

So let’s try to fix #home for clean blocks.

The concept of active Home

The Debugger already needs to know if the home is currently on the stack. For that, it used a method #activeHome, in Pharo10 this looked like that:

activeHome
    | methodReturnContext |
    self isBlockContext ifFalse: [^self].
    self sender ifNil: [^nil].
    methodReturnContext := self methodReturnContext.
    ^self sender findContextSuchThat: [:ctxt | ctxt = methodReturnContext]

#activeHome returns the #home if it is currently on the stack, the debugger uses that to check if a “save and proceed” is possible when editing code in a Block. Until Pharo11, it was implemented to search up the sender chain until it finds the #home.

But it can be rewritten to do the same, but checking for the #homeMethod and using #findMethodContextSuchThat:

activeHome
    | homeMethod |
    self isBlockContext ifFalse: [^self].
    homeMethod := self homeMethod.
    ^self findMethodContextSuchThat: [:ctxt | ctxt method == homeMethod]

With #homeMethod being implemented to delegate to the bock:

Context>>homeMethod “Answer the method in which the receiver was defined, i.e. the context from which an ^-return ] should return from. Note: implemented to not need #home”

^ closureOrNil ifNil: [ self method ] ifNotNil: [ :closure | closure homeMethod ]

Where, if no #home via the outerContext is available, it asks the CompiledBlock:

BlockClosure>>homeMethod
    "return the home method. If no #home is available due to no outerContext, use the compiledBlock"
    ^ (self home
           ifNotNil: [ :homeContext | homeContext ]
           ifNil: [ self compiledBlock ]) method

Which uses the static #outerCode chain (CompiledBlocks encode a back-pointer to the enclosing block or method), with #method following #outerCode until it reaches a CompiledMethod:

CompiledBlock>>method
    "answer the compiled method that I am installed in, or nil if none.”
    ^self outerCode method

This was already done in Pharo11, see PR 12063

But we can now continue and use it to improve #home of Context, you see that it already has a (bad) workaround when the outerContext is nil:

home
    "Answer the context in which the receiver was defined, i.e. the context from which an ^-return ] should return from."

    closureOrNil ifNil: [ ^ self ].
    "this happens for clean blocks. We should later check if it is not better to return nil"
    closureOrNil outerContext ifNil: [ ^ self ].
    ^ closureOrNil outerContext home

returning self when the outerContext is nil (when we are in a clean block) is only correct for the home context of a clean block. We can do better, and use #activeHome and search for it on the stack (with the added bonus to correctly return nil if we do not find it).

home
    "Answer the context in which the receiver was defined, i.e. the context from which an ^-return ] should return from."

    closureOrNil ifNil: [ ^ self ].
    "no outerContext for clean blocks, we try to find it on the stack"
    ^ closureOrNil outerContext 
        ifNil: [ self activeHome ]
        ifNotNil: [:outer | outer home ]

And it works!

Let’s add some tests for #readInContext:, first the simple case for non-clean blocks:

SelfVariableTest >> testReadInContext [

    | var |
    var := self class lookupVar: #self.
    self assert: (var readInContext: thisContext) identicalTo: self.
    "read from a block context"
    self assert: [(var readInContext: thisContext)] value identicalTo: self
 ]

For clean blocks, we again use to the #compilerOptions: pragma to enable them. Here we want to test both the case where we can read self and the one where we just do not have enough information:

SelfVariableTest >> testReadInContextClean [
    <compilerOptions: #( +optionCleanBlockClosure)>
    | var block |
    "if context is one stack we can read"
    block := [ (thisContext lookupVar: #self) readInContext: thisContext ].
    self assert: block value equals: self.
    "but no chance if not, then it is nil"
    var := self class lookupVar: #self.
    block :=  [ thisContext ].
    self assert: (var readInContext: block value ) equals: nil
 ]

The full PR is here, you see that we had to add a nil guard in readInContext: for the case that we can not find a home context.

I am not sure if falling back to #activeHome in #home is really the best (maybe explicitly handling the nil and do the fallback to activeHome at the side of the caller is less magic), but we can change that easily later if it turns out the be better.

Another thing to look at is the inspector of the debugger showing self as nil (first line, the implicit self), but his is for another time.

Posted on May 31, 2023   #Pharo     #Variables