Simplifying the encoding of source pointers in Pharo12

Pharo still follows the original ST80 idea of storing source code on disk as strings (“everything is an object”, no, not code…).

I have to admit I don’t like the mechanism too much, hopefully we can replace this with something better in the future. But that should not mean that we can not improve this now.

Let’s look at sourcePointers. To be able to store source on disk, every CompiledMethod stores an integer that encodes 1) the file?, .sources or .changes and 2) the position in the file.

So for two files we need one bit. So we could just shift the position by one and then use the last bit, if it is 1, we are .changes.

But if you look at the code… it looks strange. I think this is because without a bootstrap, changes like these are hard to do and every change done (to be able to have larger pointers), had to be done in a backward compatible way.

So this is the old code:

fileIndexFromSourcePointer: anInteger
    "Return the index of the source file which contains the source chunk addressed by anInteger"

    ^ (anInteger bitAnd: 16r1000000) ~= 0
        ifTrue: [ 1 "sources file" ]
        ifFalse: [ anInteger >= 16r1000000
                ifTrue: [ 2 "changes file" ]
                ifFalse: [ 0    "compatibility with StandardSourceFileArray" ] ]
                
                
filePositionFromSourcePointer: anInteger
    "Return the position of the source chunk addressed by anInteger"

    | hi lo |
    hi := anInteger // 33554432.
    lo := anInteger \\ 16777216.
    ^ ((anInteger bitAnd: 16777216) ~= 0 or: [ anInteger < 16777216 "compatibility with StandardSourceFileArray" ])
        ifTrue: [ hi * 16777216 + lo    "sources file" ]
        ifFalse: [ (hi - 1) * 16777216 + lo "changes file" ]
        


sourcePointerFromFileIndex: index andPosition: position
        "Return a sourcePointer encoding the given file index and position"

        | hi lo |
        (index = 1 or: [index = 2])
            ifFalse: [self error: 'invalid source file index'].
        position < 0 ifTrue: [self error: 'invalid source code pointer'].
        hi := position // 16r1000000 *2 + index.
        lo := position \\ 16r1000000.
        ^ hi * 16r1000000 + lo

While this is the “just shift by one and use the last bit” version:

fileIndexFromSourcePointer: anInteger
    "Return the index of the source file which contains the source chunk addressed by anInteger"
    "1 is sources file, 2 is changes file"

    ^ (anInteger bitAnd: 1) + 1
    
filePositionFromSourcePointer: anInteger
    "Return the position of the source chunk addressed by anInteger, just shift by one"

    ^ anInteger >> 1
    
    
sourcePointerFromFileIndex: index andPosition: position
    "Return a sourcePointer encoding the given file index and position"

    (index = 1 or: [index = 2])
        ifFalse: [self error: 'invalid source file index'].
    position < 0 ifTrue: [self error: 'invalid source code pointer'].
    
    "we shift the position by one and set the last bit for .changes"
    ^ (position << 1) bitOr: index - 1

The PR https://github.com/pharo-project/pharo/pull/13583 was merged in Pharo12 already.

Posted on June 1, 2023   #Pharo