Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: echo-wrap tests failing #249

Closed
balupton opened this issue Aug 27, 2024 · 8 comments · Fixed by #250
Closed

Bug: echo-wrap tests failing #249

balupton opened this issue Aug 27, 2024 · 8 comments · Fixed by #250
Labels
bug Broken or unexpected

Comments

@balupton
Copy link
Member

echo-wrap tests are failing on CI, this may be due to non-strict requirement of gawk, and falling back to awk. If so the tests should consider this, similar to how other multi implementation commands work where there is a --tool= option.

@balupton balupton added the bug Broken or unexpected label Aug 27, 2024
@balupton
Copy link
Member Author

There is some crossover with #239

Where once it can be installed independently, then we can call setup-util-gawk.

@balupton
Copy link
Member Author

No idea why. Could be different systems treat the null or tab character as different lengths.

@balupton
Copy link
Member Author

balupton commented Sep 16, 2024

Still absolutely no idea why this is happening. It only happens on the CI containers. Running natively on ubuntu or macos does not produce this issue. If anyone can look into reproducing this and attempting to fix it, that would be appreciated.

@balupton
Copy link
Member Author

GitHub Vscode Codespaces is also unaffected with the tests passing...

@balupton
Copy link
Member Author

Maybe git on those instances is mangling the tab character, so I'll try this:
https://github.com/bevry/base/blob/master/.gitattributes

@balupton
Copy link
Member Author

Maybe git on those instances is mangling the tab character, so I'll try this:
https://github.com/bevry/base/blob/master/.gitattributes

this didn't work either: https://github.com/bevry/dorothy/actions/runs/10880225265/job/30186548263#step:4:2791

@balupton
Copy link
Member Author

Considering gawk requires brew on macos, and the precompile versions don't work due to gnu's dynamic linking, gawk will need to be dropped as a required dependency, which only gawk required usage is echo-wrap in which a reimplementation away from gawk may fix this???

#239 (comment)

balupton added a commit that referenced this issue Sep 16, 2024
@balupton
Copy link
Member Author

balupton commented Sep 16, 2024

solved by adding awk compat: 7deab4e

issues seems to have been the use a UTF8/multibyte character as the internal deliminator

balupton added a commit that referenced this issue Sep 17, 2024
/close #249 #241 #239

changes:

- replace common gsed/sed/sd/ripgrep/rg/grep/teip usage with new `echo-regexp` that uses deno, affects `convert-helper`, `config-helper`, `echo-magnet-hash`, `fs-(bytes|kilobytes|megabytes)`, `fs-speed`, `get-devices`, `get-local-to-remote`, `get-opensuse-release`, `get-volumes`, `git-helper`, `is-bash-version-outdated`, `mount-helper`, `network-interface`, `setup-dns`, `setup-util`, `setup-util-1password-cli`, `setup-util-nerd-fonts`, `setup-util-node`, `unziptar`, `what-is-my-(dns|gateway|interface|ip|submit)`, `get-codec`, `is-prefix`, `is-suffix`, `itunes-owners`, `mac-address-helper`, `setup-server`
- `bash.bash`:
    - simplify eval statements and support spaces in paths
    - add explicit names for `__print*` helpers, and slightly change their functionality to be aligned with expectations
        - **please change `__print_line ...` to `__print_lines ...` as in the future `__print_line` may not support arguments**
    - **removed deprecated non `__` prefixed aliases, e.g. `require_globstar` is now only `__require_globstar`**
- `ci/dorothy-workflow`: don't test PRs until review requested
- `ci/dorothy`: don't eslint as no Node.js code
- `ci/vscode`: add an awk syntax highlighting extension, and enable Deno
- `command-working`: give details of failures, add `dash` to exceptions
- `dorothy-config`/`config-helper`: significant work to better handle multiple instances (result is no longer garbled), output and process is now clearer, includes multiple strategies for multiple instances, uses new `echo-regexp` to replace `sd/rg/teip/grep` combo
- `dorothy` minimum deps now only `jq` and `deno`s,
- `down`: add `--url=...` support
- `echo-gnu-command`: fix it not outputting the command after installation
- `echo-if-empty`/`get-devices`/`get-local-to-remote`: fix sporadic success by adding missing `--stdin`
- `echo-lines`: now support `--[no-]quote` flags as alias for prior `--[no-]quoted` flag
- `echo-math`/`echo-(html|url)-(encode|decode)`: rewrote and added tests, now they work correctly, in the latter case `echo-(html|url)-coder` is now used
- `echo-wrap`: now awk compatible, removing dependency on `gawk`
- `eval-tester`: output escaped comparison if special characters were used
- `fs-(bytes|kilobytes|megabytes)`: use `dust` instead of `du` to achieve macos support, also fix crash in `fs-megabytes`
- `fs-rm`: fix trash error on alpine, as trash isn't available on alpine
- `get-codec`: removed `--trim` and made it default (prior verbosity available via `--verbose`), now sensible error if `ffprobe` is not available
- `get-codec`: rewrote and fixed
- `get-filesystem`: no longer fail silently when no args are provided
- `get-local-to-remote`: improve styling
- `get-opensuse-release`: simpler technique
- `get-url-(domain|protocol)` split into `get-url-(domain|protocol).ts` and add tests
- `is-match`: deprecated and moved in beta commands, as it is replaced by `echo-regexp -q` which it now uses internally
- `mac-address-helper`: fixed broken `new` functionality, and improved formatting of outputs
- `set-hostname`/`setup-hosts`/`setup-shell`: fix break when gsed isn't available, now uses the `echo-gnu-command` helper
- `setup-util-1password`: install stable instead of beta, as stable is now stable
- `setup-util-deno`: add apk, uninstall snap (as snap does not allow script execution), use official arm builds as that is now a thing, don't use official yet stupid `deno_install` script
- `setup-util-devel`: add `automake` for macos
- `setup-util-gawk`: add more package systems
- `setup-util-gsed`: add bottles for macos, and add uninstall build support
- `setup-util`: rename `DOWNLOAD_BUILD_EVAL` to `DOWNLOAD_BUILD_INSTALL` and add `DOWNLOAD_BUILD_UNINSTALL`, affected `setup-util-bash`, `setup-util-gawk`, `setup-util-gsed`
- `sparse-vault`: fix broken awk script, use `echo-regexp` instead
- `sparse-vault`: fix broken macos version fetching
- `styles.bash` make code styles more readable on light theme by swapping from dim to intense black, add code error, info, and notice styles
- `unziptar`: fix alpine compatibility
- `what-is-my-(dns|gateway|inteface|ip|subnet)`: rewrote and fixed
- add `.gitattributes` to prevent formatting issues in tests
- add `echo-escape-backslashes` and remove its functionality from `echo-escape-regexp-replacement` for compat with JavaScript Regular Expression
- add `fs-diff` to compute differences between files
- add `get-file` and `setup-util-file` to fix `unziptar` compat on systems without `file` preinstalled
- add `setup-util-frawk`
- add tests to `echo-math`, `echo-(html|url)-(coder|encode|decode)`, `echo-magnet-hash`, `fs-(bytes|kilobytes|megabytes)`, `get-url-(domain|protocol)`

needs more testing:

- `get-opensuse-release`
- `mount-helper`
- `setup-dns`
- `setup-server`
- `setup-util-gsed` on fresh macos
- `unziptar` on macos dmg files

squashes:

- be4b3ce
- ada62e0
- 6860035
- d86e060
- 2b24208
- 1fdae98
- 4506b7f
- 6618689
- 7022dba
- fe5050b
- 0e02dc9
- 7c7265d
- bb507bd
- 818d5c0
- b91656b
- 96d91bc
- 6838a37
- 7c7ed59
- a1f96ea
- 2e4d9da
- e2294c4
- 2a83b20
- da3cf12
- d334b8f
- 86de699
- 35593fe
- 62d42ca
- c723ea2
- e68e2e0
- 7deab4e
- ecee164
- 7cc4031
- e035a22
- e9a955a
balupton added a commit that referenced this issue Sep 17, 2024
/close #249 #241 #239

changes:

- replace common gsed/sed/sd/ripgrep/rg/grep/teip usage with new `echo-regexp` that uses deno, affects `convert-helper`, `config-helper`, `echo-magnet-hash`, `fs-(bytes|kilobytes|megabytes)`, `fs-speed`, `get-devices`, `get-local-to-remote`, `get-opensuse-release`, `get-volumes`, `git-helper`, `is-bash-version-outdated`, `mount-helper`, `network-interface`, `setup-dns`, `setup-util`, `setup-util-1password-cli`, `setup-util-nerd-fonts`, `setup-util-node`, `unziptar`, `what-is-my-(dns|gateway|interface|ip|submit)`, `get-codec`, `is-prefix`, `is-suffix`, `itunes-owners`, `mac-address-helper`, `setup-server`
- `bash.bash`:
    - simplify eval statements and support spaces in paths
    - add explicit names for `__print*` helpers, and slightly change their functionality to be aligned with expectations
        - **please change `__print_line ...` to `__print_lines ...` as in the future `__print_line` may not support arguments**
    - **removed deprecated non `__` prefixed aliases, e.g. `require_globstar` is now only `__require_globstar`**
- `ci/dorothy-workflow`: don't test PRs until review requested
- `ci/dorothy`: don't eslint as no Node.js code
- `ci/vscode`: add an awk syntax highlighting extension, and enable Deno
- `command-working`: give details of failures, add `dash` to exceptions
- `dorothy-config`/`config-helper`: significant work to better handle multiple instances (result is no longer garbled), output and process is now clearer, includes multiple strategies for multiple instances, uses new `echo-regexp` to replace `sd/rg/teip/grep` combo
- `dorothy` minimum deps now only `jq` and `deno`s,
- `down`: add `--url=...` support
- `echo-gnu-command`: fix it not outputting the command after installation
- `echo-if-empty`/`get-devices`/`get-local-to-remote`: fix sporadic success by adding missing `--stdin`
- `echo-lines`: now support `--[no-]quote` flags as alias for prior `--[no-]quoted` flag
- `echo-math`/`echo-(html|url)-(encode|decode)`: rewrote and added tests, now they work correctly, in the latter case `echo-(html|url)-coder` is now used
- `echo-wrap`: now awk compatible, removing dependency on `gawk`
- `eval-tester`: output escaped comparison if special characters were used
- `fs-(bytes|kilobytes|megabytes)`: use `dust` instead of `du` to achieve macos support, also fix crash in `fs-megabytes`
- `fs-rm`: fix trash error on alpine, as trash isn't available on alpine
- `get-codec`: removed `--trim` and made it default (prior verbosity available via `--verbose`), now sensible error if `ffprobe` is not available
- `get-codec`: rewrote and fixed
- `get-filesystem`: no longer fail silently when no args are provided
- `get-local-to-remote`: improve styling
- `get-opensuse-release`: simpler technique
- `get-url-(domain|protocol)` split into `get-url-(domain|protocol).ts` and add tests
- `is-match`: deprecated and moved in beta commands, as it is replaced by `echo-regexp -q` which it now uses internally
- `mac-address-helper`: fixed broken `new` functionality, and improved formatting of outputs
- `set-hostname`/`setup-hosts`/`setup-shell`: fix break when gsed isn't available, now uses the `echo-gnu-command` helper
- `setup-util-1password`: install stable instead of beta, as stable is now stable
- `setup-util-deno`: add apk, uninstall snap (as snap does not allow script execution), use official arm builds as that is now a thing, don't use official yet stupid `deno_install` script
- `setup-util-devel`: add `automake` for macos
- `setup-util-gawk`: add more package systems
- `setup-util-gsed`: add bottles for macos, and add uninstall build support
- `setup-util`: rename `DOWNLOAD_BUILD_EVAL` to `DOWNLOAD_BUILD_INSTALL` and add `DOWNLOAD_BUILD_UNINSTALL`, affected `setup-util-bash`, `setup-util-gawk`, `setup-util-gsed`
- `sparse-vault`: fix broken awk script, use `echo-regexp` instead
- `sparse-vault`: fix broken macos version fetching
- `styles.bash` make code styles more readable on light theme by swapping from dim to intense black, add code error, info, and notice styles
- `unziptar`: fix alpine compatibility
- `what-is-my-(dns|gateway|inteface|ip|subnet)`: rewrote and fixed
- add `.gitattributes` to prevent formatting issues in tests
- add `echo-escape-backslashes` and remove its functionality from `echo-escape-regexp-replacement` for compat with JavaScript Regular Expression
- add `fs-diff` to compute differences between files
- add `get-file` and `setup-util-file` to fix `unziptar` compat on systems without `file` preinstalled
- add `setup-util-frawk`
- add tests to `echo-math`, `echo-(html|url)-(coder|encode|decode)`, `echo-magnet-hash`, `fs-(bytes|kilobytes|megabytes)`, `get-url-(domain|protocol)`

needs more testing:

- `get-opensuse-release`
- `mount-helper`
- `setup-dns`
- `setup-server`
- `setup-util-gsed` on fresh macos
- `unziptar` on macos dmg files

squashes:

- be4b3ce
- ada62e0
- 6860035
- d86e060
- 2b24208
- 1fdae98
- 4506b7f
- 6618689
- 7022dba
- fe5050b
- 0e02dc9
- 7c7265d
- bb507bd
- 818d5c0
- b91656b
- 96d91bc
- 6838a37
- 7c7ed59
- a1f96ea
- 2e4d9da
- e2294c4
- 2a83b20
- da3cf12
- d334b8f
- 86de699
- 35593fe
- 62d42ca
- c723ea2
- e68e2e0
- 7deab4e
- ecee164
- 7cc4031
- e035a22
- e9a955a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Broken or unexpected
Development

Successfully merging a pull request may close this issue.

1 participant