map() command mangles return values

Affected version: 5.1.3 and most likely all older versions as well

Problem

In short current map() command is broken for arrays of return values different with length != 1 and also for certain data types.

Example with three return values. This is wildly inconsistent even between consecutive executions on the same kresd -f4 instances:

> map('1, 2, 3')
[1] => 1
[2] => 3
[3] => 1, 2, 3
[4] => 3

> map('1, 2, 3')
[1] => 1
[2] => 3
[3] => 1, 2, 3
[4] => 3

> map('1, 2, 3')
[1] => 1
[2] => 3
[3] => 1, 2, 3
[4] => 3

> map('1, 2, 3')
[1] => 1
[2] => 3
[3] => 1, 2, 3
[4] => 3

> map('1, 2, 3')
[1] => 1
[2] => 3
[3] => 1, 2, 3
[4] => 3

> map('1, 2, 3')
[1] => 1
[2] => 3
[3] => 3
[4] => 3

> map('1, 2, 3')
[1] => 1
[2] => 3
[3] => 3
[4] => 3

> map('1, 2, 3')
[1] => 1
[2] => 3
[3] => 3
[4] => 3

> map('1, 2, 3')
[1] => 1
[2] => 3
[3] => 3
[4] => 3

Example with nil - it gets turned into empty tables but not for all four instances:

> map('nil')
[1] => {
}
[2] => {
}
[3] => {
}

Example with errors - errors are incorrectly turned into strings so the map() caller cannot distinguish them from real string return values:

> map('error("test")')
[1] => test
[2] => test
[3] => test
[4] => test

In short it is utterly broken.

Proposal

map() is mostly broken and also undocumented so we change it more or less freely. Here is how I would do it:

• Follower instances will use pcall to determine if a the single call inside map() succeeded or not and send results back to leader

• map() running on leader instance will receive full outputs from pcall on followers and check if all calls succeeded. If not map() will throw out an error.

• Leader will check if number of return values from each instance is exactly one (from each follower) - if not map() on leader will throw out an error.

• if no error occured map() will return two tables:

  • first table with results from each instance in format {return value from first instance}, return value from second instance, return value from third instance, ...} - order of return values is not defined and cannot be relied on
  • second table with corresponding names of control sockets (? maybe we can skip this and add it later?)

• Usage if map() caller wants to receive multiple return values: map('{expression to be evaluated}') will produce table of tables {{return values from first instance}, {return values from second instance}, ...}, {name of first instance, name of second instance, ...}

• Usage if map() caller wants to also receive errors:

  • Caller can either wrap intended expression in pcall: map('{pcall(expression to be evaluated)}')
  • or we can define a table format which will be used by errors thrown from map()

This approach also allows to handle variable number of return values and also nil values in a safe manner. Let's define two helper functions:

function tab_pack(...)
	local tab = {...}
	tab.n = select('#', ...)
	return tab
end

function tab_unpack(tab)
	return unpack(tab, 1, tab.n)
end

Now map() caller can do: map('tab_pack(nil,2,nil)') and receive results like:

> map('tab_pack(nil,2,nil)')
[1] => {
    [2] => 2
    [n] => 3
}
[2] => {
    [2] => 2
    [n] => 3
}

The auto-generated table field n tells the caller that original result contained 3 values so the caller can iterate over [1],[2],[3] and safely find that first and third return value were nil.