Python bind for osh

[KingMob]

No description provided.

[andychu]

I ran the spec tests locally and got this -

case    osh     bash
Ignoring osh-cpp failure: 0 test bind (w/out flags) for adding bindings to readline fns
0       FAIL    ok      test bind (w/out flags) for adding bindings to readline fns
        0/5 ok
Ignoring osh-cpp failure: 1 test bind -r for removing bindings
1       FAIL    ok      test bind -r for removing bindings
        0/5 ok
Ignoring osh-cpp failure: 2 test bind -x for setting bindings to custom shell functions
2       FAIL    ok      test bind -x for setting bindings to custom shell functions
        0/5 ok
3       ok      ok      test bind -u for unsetting all bindings to a fn
4       ok      ok      test bind -q for querying bindings to a fn
5       ok      ok      test bind -m for setting bindings in specific keymaps
6       ok      ok      test bind -f for setting bindings from an inputrc init file

Is that what is expected to work?

The code looks reasonable, and this seems like good progress!

---

If so, I can help fix the C++ issues

[KingMob]

Is that what is expected to work?

Yes, this is what I expect. The failing tests don't have code implemented for them yet.

[andychu]

I pushed a couple minor commits to pass type checking and translation

So make sure to PULL before editing any code

---

It looks like we can get past the rest of the C++ build errors without much work too -- I will look at that

This looks VERY promising, thank you!

[andychu]

OK I pushed some stubs for C++ -- wasn't that bad, although there was one mycpp workaround necessary

Let's see what CI results looks like!

[andychu]

Oh weird there are some line_input.c compile errors that don't happen on my machine either

http://op.oilshell.org/uuu/github-jobs/8226/interactive.wwz/_tmp/soil/logs/py-all-and-ninja.txt

Maybe this is due to older GNU readline versions?

Maybe bash added a new flag recently that depends on a new version or something?

[KingMob]

Hmm, I haven't seen that error either. The rl_function_of_keyseq_len fn exists in my bash. I see it in lib/readline/readline.h and lib/readline/bind.c.

Looks like that particular fn was added six years ago for bash 5.0. I'm guessing python's readline wrapper uses an older version as its basis? Bash 4.4 used rl_bind_keyseq to do removal by passing a null ptr for the fn to bind to.

I'll look into what precise version python's readline was based off of. Using that as a basis is probably easier; bash/readline is old, but it's still changing, and I don't think I want to bring in the latest readline just for this. (Honestly, I'm not sure you want to use readline over a more modern lib, even without the license issue.)

[andychu]

OK I made a whole bunch of minor changes, and I think this is ready to merge! Here are what the tests look like now

• https://op.oilshell.org/uuu/github-jobs/8282/cpp-spec.wwz/_tmp/spec/osh-cpp/builtin-bind.html - C++ isn't complete yet, I added "oils_cpp_allowed_failures" metadata

• https://op.oilshell.org/uuu/github-jobs/8282/interactive.wwz/_tmp/spec/stateful/index.html - we are passing 4 of 6 tests now in Python, but not yet in C++

But I think this is great progress! Let me know what you think

[andychu]

Here's a summary of what I did

• disable the problem GNU readline function, and then disable the one flag that uses it. We can fix this in another commit
• fix some mycpp translation errors
• fix C++ compile errors
• change C++ stubs to be empty no-ops, rather than assert(0), to avoid breaking the ble.sh tests
• added spec test cases
  • removed the spec testdata golden file, since it was too dependent on the GNU readline version
  • instead we just do some simple sanity checks
• reformat with devtools/format.sh yapf-changed
• lint fix with test/lint.sh soil-run
  • Some of this is at - https://github.com/oils-for-unix/oils/wiki/Oil-Dev-Cheat-Sheet
• fix some C warnings in line_input.c

Let me know if you have any questions ... in general we have a boatload of tools that make this very mechanical

And I think concentrating on just Python is absolutely right for contributors, especially with something as hairy as bind. This code is unusual because it interfaces with an external library

Thanks for working on this! It DEFINITELY would not get done without your efforts. From working with this PR, I now know a little more about bind :-)

[KingMob]

Hmmm, looks like the new bind -q spec doesn't pass on non-cpp osh. Apparently, if it's not bound, it should write that to stdout, instead of stderr.

[andychu]

Yeah I noticed that, though it's arguable how much we have to match bash EXACTLY

i.e. we try to avoid "implementing bash bugs", which may be changed in later versions

For stdout/stderr, the heuristic I use is --

• is anyone going to parse this? if so, it should probably be on stdout
• if it's NOT meant to be parsed, then put it on stderr

i.e. I think stderr can "break" / change wording, but stdout should be frozen

---

If it's consistent with the rest of OSH, then we could keep it on stderr

Not sure though

[KingMob]

Oh, well...I already changed it. In particular, it failed when bind -q is called on a valid function name, but one that's not currently bound.

Whether bash is mistaken in writing that to stdout instead, or in returning a non-zero status code...I dunno. It does seem like something that could be parsed, though.

[KingMob]

There's a lot of CI checks failing their publish-* tasks...

Anyway, I think my part on the current PR is done. I have nothing else I want to change.

[andychu]

To check the CI failures, I go here:

https://op.oilshell.org/uuu/github-jobs/

and then click on your job:

https://op.oilshell.org/uuu/github-jobs/8291/

and then if you click on the first failure, you'll see:

https://op.oilshell.org/uuu/github-jobs/8291/cpp-coverage.wwz/_tmp/soil/logs/compare-gcc-clang.txt

	mycpp pass: PROTOTYPES
builtin/readline_osh.py:172 Use explicit len(obj) or 'obj is not None' for mystr, mylist, mydict

This is a mycpp error, which says that you should not use implicit booleans, basically because those 2 things are different in C++ !

We do a literal translation, so you have to be specific about what you want

[andychu]

Merged, thank you! Please keep us updated on Zulip