another lua curiosity: multiple return values

The previous curiosity was an example of something that didn't work, but one might argue should. This one is something that does work, and at first might be surprising. It revolves around the way Lua handles return values....

Let's define a simple functions:

function Produces( a )
 return a, a*2, a*6 -- doesn't matter what, just a few values
end 

The first behavior won't be too surprising on the surface, but it does hint something strange is happening:

print ( Produces( 3 ) ) -- prints: 3 6 18

Our multiple return values aren't bundled as an array, or a tuple as in other languages, they're just sitting on the stack waiting to be consumed by the input to the next function. In Python, for instance, same code would print:

def Produces( a ):
 return a, a*2, a*6 # doesn't matter what, just a few values

print( Produces( 3 ) ) # prints: (3,6,18)

Those parenthesis () printed are key. They indicate the return values are a unit: a tuple that is, in fact, a single object. To break them apart, you have to manually unpack them in someway.

For Lua, the fact they aren't packed together has implications beyond print, because it's just a regular function. Instead of calling print directly, let's call our own function:

function Consumes( a, b, c, d, e )
 print( a,b,c,d,e )
end

Consumes( 1, Produces(3) ) -- prints: 1 3 6 18 nil

Those values returned from Produces are just individual values hanging out on the Lua stack. In effect, we've injected the return values in the midst of another call:

• push Consumes
• push 1
• push Produces 
• push 3
• call Produces
• pop 3
• push 3
• push 6
• push 18
• return
• call Consumes -- sees: 1,3,6,18,nil

I can imagine this behavior could help when making declarative mini-languages -- functions would provide in-place shortcuts for more complex declarations -- but, the full implication isn't clear to me.

One good question would be, how far can we push this behavior? What would happen if we followed a call to "Produces" with something else?

Consumes( 1, Produces(3),  "end of the line" )

-- prints: 1  3  "end of the line"  nil  nil

Interestingly, our injected parameters are suddenly chopped off.  We might expect from the earlier behavior we'd see "1, 3, 6, 18, "end of line".

Maybe, Lua has seen the new value, and in someway reset the stack. Okay, so let's try one more:

Consumes( 1, Produces(3),  Produces(2) )

-- prints: 1  3  2  4  12

And, we get the old injection behavior again, but only for the trailing function.

When we compile the first Produces, Consumes example in luac and look at the op codes, just like the surmised push, pop behavior we see the injection in action:

 1 [3] CLOSURE   0 0 ; 0x100101200
 2 [1] SETGLOBAL 0 -1 ; Produces
 3 [7] CLOSURE   0 1 ; 0x1001013a0
 4 [5] SETGLOBAL 0 -2 ; Consumes
 5 [9] GETGLOBAL 0 -2 ; Consumes
 6 [9] LOADK     1 -3 ; 1
 7 [9] GETGLOBAL 2 -1 ; Produces
 8 [9] LOADK     3 -4 ; 3
 9 [9] CALL      2 2 0
 10 [9] CALL      0 0 1
 11 [9] RETURN    0 1

When we add "end of line", however, we can see the compiler has indeed altered the call sequence. In line 9, Lua pops two of the return values before pushing our string.

 7 [9] GETGLOBAL 2 -1 ; Produces
 8 [9] LOADK     3 -4 ; 3
 9 [9] CALL      2 2 2 <-- was: 2,2,0; now: 2,2,2 
 10 [9] LOADK     3 -5 ; "end of line"
 11 [9] CALL      0 4 1
 12 [9] RETURN    0 1

Upon reflection, I think this is all a requirement of proper tail calls. Lua agrees to be agnostic about return values for as long as it can, allowing recursion without overwhelming the stack. It's an interesting, curious, behavior to keep in the back of your head as shortcut for calling functions, but also as an occasional gotcha when you get more ( or less ) sent to a function than you might at first expect.

It might be interesting to take a look at a language like Erlang  ( which also supports tail calls ) to see how it handles functions like the "producer", "consumer". Maybe a good topic for a follow up post at some point.