Lua Select API, Lua based KeySelect & Customisable Smart Select #3757

whossname · 2024-09-22T06:52:27Z

Work done

Created a Lua based replacement for the existing hotkey selection code and added functionality to SmartSelect to utilise the same API to modify which units are selected when using drag select.

There were two main motivations for this:

the existing C++ API can be inconsistent and buggy in places. As far as I can tell the NameContains and Category rules are broken, and the select count/one/part conclusions includes units that are already selected, which can be counter intuitive.
writing it in Lua allows the code to be updated, maintained and improved more easily. Far more people should be able to work on the Lua code, and it should be easier to apply the changes.

An additional driver was being able to use the select syntax for SmartSelect commands in uikeys.txt.

The following filter was not implemented, I have no idea what it is meant to do: RulesParamEquals_<string>_<integer>

I was advised that no one uses it anyway. Also an extra rule AntiAir was added for selecting Anti-air units.

The relevant files are:

select_api.lua the common code used in both SmartSelect and KeySelect to decode selection command strings and cache the resulting select function
unit_smart_select.lua this widget was modified to use the select api for selectbox commands in uikeys.txt
unit_key_select.lua intended as a replacement for the existing select commands. This widget is disabled by default as we want users to test it before it is fully rolled out as a replacement.
compare_to_spring.lua a test script written to identify differences between the existing C++ select implementation and the new Lua select implementation. It's not a proper unit/integration test as the results require a lot of interpretation.

See discord thread for more details: https://discord.com/channels/549281623154229250/1266331814440730677

Setup

The changes to SmartSelect are merged in directly. It will require modification to the uikeys.txt file to use. Something like this:

bind          Any+shift  selectbox Not_Aircraft
bind           Any+ctrl  selectbox Buildoptions
bind            Any+alt  selectbox Aircraft
bind          Any+space  selectbox Builder_Aircraft

To test the KeySelect open widgets and enable the KeySelect widget.

Test steps

SmartSelect

Copy the above code into uikeys.txt and run /keyreload
Create a variety of air/land units
The shift, ctrl, alt and space modifiers should change which units are selected
Check that the existing SmartSelect drag selection key binds still work as expected

KeySelect

Run /wigetselector and enable KeySelect
Check that all of your keybinds still work

Known issues

There seems to be an issue where the NameContain filter doesn't work as expected if you have ClearSelection in the conclusion. It will centre on the unit, but not actually select it. I suspect it also happens for the Category filter. No idea why. Seeing as the NameContain and Category filters don't seem to work correctly in the existing version anyway, I don't see it as a deal breaker.

includes a refactor of the rule api - now receives udef, udefid, uid instead of just udefid

instead of parsing the rules on each button press, now it does it once and stores the rules in a lookup table

whossname · 2024-09-22T07:00:38Z

The changes to luamocks/SpringLuaApi.lua are unintentional. It's mostly removing white-space at the end of lines. I must have accidentally run a formatter at some stage.

sprunk · 2024-09-22T16:10:58Z

luaui/Widgets/unit_key_select.lua

+		author = "woss (Tyson Buzza)",
+		date = "Aug 24, 2024",
+		license = "Public Domain",
+		layer = -999999,


why -999999?

No idea. I think I was copying the smart select widget. I don't know what that number does.

You can replace it with 1 or 0, the layer is the loading order of the widget in relation to other widgets. We don't expect any conflicts to exist unless a game or user already declares another select action in a widget.

I made the changes but I can't see them in the discussions. Sorry, I don't use pull requests at work. I've always worked at small startups where it's easier to just review the changes with a local git client and discuss the code verbally. I'm probably missing some basic feature of pull requests here.

badosu · 2024-10-11T18:28:24Z

Just for context, the key addition here is the select api and possibility of replacing engine-level select action with an own lua managed one.

Reason is so that select keys can be customized to extend the select api in the future so the game does not depend on engine for it. If, after inclusion and continued use, we verify that all current engine features are satisfied with expected performance profile - this will be eligible for inclusion as engine basecontent and a proposal for engine select removal in the future.

I don't have any particular opinion on the smart select changes, a very small set of players actually even use the current smart select capabilities anyway, I leave that up to the team members to consider.

I'd be very glad if we could include the select api engine and select widget, even if enabled false - to at some point and basic testing enabling to true, and at some point to actually prove this api is a worthy replacement of the engine select command in basecontent.

Thank you.

badosu · 2024-10-11T18:33:03Z

luamocks/SpringLuaApi.lua

@@ -70,6 +70,7 @@ Engine = {
 }

 ---@param path string
+--- @return any
 function VFS.Include(path)


Can you revert this file? I understand this file is autogenerated.

Reverting this line is easy. I don't know where the other changes even came from. It's a few dozen instances of removing the white space at the end of the line.

Out of curiosity, is there a way of setting sensible return types in that file? I think (because this was 2 months ago at this stage) the IDE was yelling at me a lot for mismatching type specs.

I reverted the white space changes as well.

FYI #3919 makes this change unnecessary.

Incidentally, I'd be interested to know how these files are autogenerated. I thought this particular file is hand-written but there is a subfolder of auto-generated files.

WatchTheFort · 2024-10-11T19:52:54Z

I need to take a deeper look at the code, but from a first glance it looks like it is just duplicating the engine functionality, i.e. implementing a keybind syntax then using string parsing to turn a text file into Lua select commands.

This seems like the wrong approach, as it is sacrificing the flexibility of Lua. Wouldn't this make more sense as a widget, with the caveat that we need better modding support for custom selection keybinds?

badosu · 2024-10-11T20:01:50Z

Wouldn't this make more sense as a widget...?

Yes, that's what this PR does.

This seems like the wrong approach, as it is sacrificing the flexibility of Lua.

That's the opposite, we're using the flexibility of Lua to have the ability to extend engine functionality. It is duplicate, but the plan is to nuke engine select command anyway, this should be a basecontent widget instead sufficed it works satisfiably.

If the argument is to then, "if we can do anything then we dont need the old engine select constraints", that's a decent argument. But keep in mind, by first supporting base engine select syntax we are able to have this a first step into what will eventually be included in engine basecontent (the testing is priceless for me as an engine developer) and have a painless transition.

If the game deems this not fruitful for game concerns then sure, that's another argument, I just ask to give this a chance so we can improve our engine using this testbed. Proposal is to include at least the select widget and the api, it can come in as enabled false, that's fine as long as I can ask keybind savvy users to try it out in a first iteration of this process.

The smartselect changes I leave it up for the game maintainers to deliberate about.

WatchTheFort · 2024-10-16T15:01:08Z

luaui/Widgets/unit_key_select.lua

What is the purpose of this file?

Separating config and the widget proper lets people change the former without overriding the latter (which would make them miss out on functional improvements)

Why is it disabled?

To let players do a wider live test before becoming the default

It separates the functionality from the interface, allowing other widgets to use the functionality. I mostly did it to share common code between smart_select and key_select, but there should be other applications as well.

luaui/Widgets/Include/select_api.lua

WatchTheFort · 2024-10-16T15:59:11Z

luaui/Widgets/Include/select_api.lua

error is being used when an error is detected. Does this really warrant throwing an exception? Better to log an error and remove the widget, or just skip the individual bad filter if possible.

I thought the first was what this did (based on how it behaved during debugging), but I can see the second is a better option. What is the error logging function for BAR?

Search for use of Spring.Log and LOG.ERROR

WatchTheFort · 2024-10-16T16:00:11Z

luaui/Widgets/Include/select_api.lua

+			local myTeamId = Spring.GetMyTeamID()
+			return Spring.GetVisibleUnits(myTeamId)
+		end
+	elseif sourceDef == "PrevSelection" then


Suggested change

elseif sourceDef == "PrevSelection" then

elseif sourceDef == "PreviousSelection" then

luaui/Widgets/Include/select_api.lua

WatchTheFort · 2024-10-16T16:01:34Z

luaui/Widgets/Include/select_api.lua

+				return {}
+			end
+		end)
+	elseif startsWith(sourceDef, "FromMouseC_") then


Suggested change

elseif startsWith(sourceDef, "FromMouseC_") then

elseif startsWith(sourceDef, "FromMouseCylinder_") then

WatchTheFort · 2024-10-16T16:02:42Z

luaui/Widgets/Include/select_api.lua

For numeric filters, do they handle non-integers and negative numbers?

They all use this pattern:

local minRange = tonumber(getNextToken()) if not minRange then break end

I'm pretty sure that means it can be a negative float, which doesn't seem like an issue to me. Negative numbers aren't useful, but they don't break the logic. Any non-numbers will just break the parsing.

But will negative numbers break game behaviour? Are they ever even meaningful?

From memory, they don't break behaviour, but they are also never meaningful. Negative numbers behave the same as 0. It either includes all or excludes all.

Now that I think about it, I'm more worried about how 0 behaves than negative numbers. It will probably be treated as if it's a null.

whossname · 2024-10-17T04:32:45Z

I'll have a look this weekend. Could someone please clarify the workflow here? I haven't used PRs much. Am I meant to make the change and then mark the comment as resolved, or just reply to the comment?

- use global `CMD` and `UnitDefNames` lookups - use `split` string function - use localized `spGetSelectedUnits` - handle abbreviations - remove dead code - `getAlreadySelectedSet` as a local function - removed unused parameter from `parseConclusion` - fix hard coded 50% in `SelectPart_` conclusion - fix `x` not validated in `FromMouse` conclusion

WatchTheFort · 2024-10-31T14:32:03Z

luaui/Widgets/Include/select_api.lua

It would still be pascal case in the code, but would accept any casing without error. This prevents bug reports when people use the incorrect case in their uikeys.txt file. Other languages have case insensitive string compare, but this is Lua, so...

local function tokenMatches(token, key) return lower(token) == lower(key) end if tokenMatches(token, "InPrevSel") or tokenMatches(token, "InPreviousSelection") then

WatchTheFort · 2024-10-31T14:32:29Z

luaui/Widgets/Include/select_api.lua

WatchTheFort · 2024-10-31T14:33:30Z

luaui/Widgets/Include/select_api.lua

Search for use of Spring.Log and LOG.ERROR

WatchTheFort · 2024-10-31T14:34:14Z

luaui/Widgets/Include/select_api.lua

But will negative numbers break game behaviour? Are they ever even meaningful?

WatchTheFort · 2024-10-31T14:37:03Z

luaui/Widgets/Include/select_api.lua

+			end)
+		elseif token == "Patrolling" then
+			filters.patrolling = invertCurry(invert, function(_udef, _udefid, uid)
+				for i = 1, 4, 1 do


Add a comment saying why, but also see if this is actually needed, nothing worse than cargo cult programming.

Suggested change

for i = 1, 4, 1 do

-- First four positions in queue checked because that is what the engine does

for i = 1, 4, 1 do

WatchTheFort · 2024-10-31T14:39:36Z

luaui/Widgets/Include/select_api.lua

 local function parseNumber(input, fn)
 	local numStr = input:match("_([^_]+)")
 	local distance = tonumber(numStr)

 	if not distance then
-		error("Invalid input: expected a number after the underscore.")
+		error("Invalid input, expected a number after the underscore: " .. input)


If input is nil, this itself will throw an error

WatchTheFort · 2024-10-31T14:46:05Z

So apparently in a review, when you reply to a comment chain at the file level (i.e. not on a line of code), Github places the reply in the chain, but also adds a duplicate comment not connected to anything.

WatchTheFort · 2024-11-14T20:17:22Z

@whossname Any status updates?

whossname · 2024-11-15T00:02:39Z

Sorry, I've been busy. I probably won't get to this in the next month.

whossname added 30 commits July 6, 2024 23:43

first pass - hard coded options for air/notair/builder

d54c60e

fix airFilter, custom placeholder

52c9621

wip - custom filter

941213f

working prototype

3efbc66

Merge branch 'more_selectbox_options'

18db210

rough attempt at the easier rules, remove unnecessary aircraft filter

7c05fb1

implement rules that don't have arguments

b490b6d

includes a refactor of the rule api - now receives udef, udefid, uid instead of just udefid

slight reordering/grouping of rules

776013b

implementing the last rules

22fbc06

bug fixes

5009fb0

fix cmd rules, comment out category

c0ea245

rename customFilter to customRulesFilter, add customRulesLookup

0ac8cc4

instead of parsing the rules on each button press, now it does it once and stores the rules in a lookup table

add AntiAir rule

009cd6e

Merge branch 'more_selectbox_options'

a3177d1

separate select api from unit_smart_select

b996f98

basic test, better select api function names

1799e86

better test error messages, debugging tests, fix Idel, improve Builder

dc1b364

refactor repeating code unitPassesFilterRules, comment how to run tests

80ded54

isBuilder includes carrier ships, refactor createUnits function in test

f32d0ea

implement category rule

7b54995

test unit groups properly, count units selected, reorder ruledefs

37f1820

implement InPrevSel. Needed to test spring after api for Not_InPrevSel

ad31efd

replace "rules" with "filter" to match api definitions

661e25e

prep for apiCommand - refactor out applyApiFn and compare in test

f4a44d5

basic idea for command and source

884f55f

basic command implementation. tests finding odd errors

c46bbb2

debugging

7747b25

fix source, ignore more weirdness in the test

ab0ac1a

move types to select_api, localise most used Spring functions

b27674b

first pass unit_key_select module and wip select conclusion

2a6202e

whossname added 7 commits August 25, 2024 20:50

tests and some fixes for full commands. seeing some odd results

560a3a1

remove command counts, use circular array index instead

9d94ccf

Merge branch 'split_select_engine'

0f6f9ea

override select instead of using keyselect

e5c7d69

hotfix - throw errors on invalid tokens

af86f62

disable KeySelect by default, NameContains uses translatedHumanName

0bb0342

Merge branch 'beyond-all-reason:master' into master

59989d1

sprunk reviewed Sep 22, 2024

View reviewed changes

WatchTheFort self-requested a review October 10, 2024 19:46

badosu reviewed Oct 11, 2024

View reviewed changes

WatchTheFort marked this pull request as draft October 11, 2024 19:46

whossname added 2 commits October 12, 2024 10:23

remove unwanted return type, set sensible layer

0651a57

revert white space changes

c665c1b

WatchTheFort requested changes Oct 16, 2024

View reviewed changes

whossname marked this pull request as ready for review October 19, 2024 02:50

WatchTheFort self-requested a review October 24, 2024 19:37

WatchTheFort reviewed Oct 31, 2024

View reviewed changes

	elseif sourceDef == "PrevSelection" then
	elseif sourceDef == "PreviousSelection" then

	elseif startsWith(sourceDef, "FromMouseC_") then
	elseif startsWith(sourceDef, "FromMouseCylinder_") then

	for i = 1, 4, 1 do
	-- First four positions in queue checked because that is what the engine does
	for i = 1, 4, 1 do

Lua Select API, Lua based KeySelect & Customisable Smart Select #3757

Are you sure you want to change the base?

Lua Select API, Lua based KeySelect & Customisable Smart Select #3757

Conversation

whossname commented Sep 22, 2024

Work done

Setup

Test steps

Known issues

whossname commented Sep 22, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

badosu Oct 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

badosu commented Oct 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WatchTheFort commented Oct 11, 2024

badosu commented Oct 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

whossname commented Oct 17, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WatchTheFort commented Oct 31, 2024

WatchTheFort commented Nov 14, 2024

whossname commented Nov 15, 2024

badosu Oct 11, 2024 •

edited

Loading

badosu commented Oct 11, 2024 •

edited

Loading

badosu commented Oct 11, 2024 •

edited

Loading