support more file extensions for manifest

[ekoeryanto]

It is nice if we support more extension for manifest, like

1. app.yml
2. app.toml, etc.

[Ash258]

I am going to look for supporting yaml. Best option for this types files. Mainly for supporting block chomping and other advantages (#2609)

It should not be so hard.

1. It would only required extrernal module
2. Add wraper for parse_json like parse with arguments -Yaml -Json with default fallback
   1. Since ConvertFrom-Yaml work exactly like ConvertFrom-Json.
   2. It would require only 1 Global variable, which indicates extension
   3. Do same workflow with ConvertToPrettyYaml.
      1. Proper indentation
      2. Escape
   4. Binaries usually generate manifets, which require new function like writeManifest with extension parameter and then lines like: $json | ConvertToPrettyJson | WriteAllLines be extracted into that function
3. Of couse fix tests, all depends lines and keep backwards compatibility.
4. Bin for transfering json manifets to yaml manifests.

YAML Support Roadmap

• Blockers

  • Separate main bucket from scoop code? #1328
  • Stable and Bleeding Edge feature for Scoop #2939
  • [Discussion] State of repository and some thoughts on changes #2609
  • Extract supportings source code into custom repositories #3003

• Wave 1 - Wrapper function for loading manifest ([manifest] Present new Out-Manifest and Get-Manifest function #3332)

  • parse_json -> Get-Manifest { switch (extension) { ... default: _parseJson }
  • Replace all parse_json with wrapper

• Wave 2 ([manifest] Present new Out-Manifest and Get-Manifest function #3332)

  • Set-Manifest Path Content
  • replace all ::WriteAllLines

• Wave 3

  • Rename $json to $manifest

• Wave 4

  • .vscode settings
  • Download library into repository

• Wave 5

  • Extend validator to validat yml files

• Wave 6

  • Load module in main bin\scoop.ps1
  • Binaries
    • auto-pr
      • Inspect TODOs
    • checkver
      • ahoy
        • both architectures
      • bat
        • single architecture
      • ag
        • URL
    • autoupdate function call

• save_installed_manifest to save only json

• url_manifest function

• Commands

  • install.ps1
    • Copy-Item (manifest_path $app $bucket) "$dir\manifest.json" -> (parse_manifest (manifest_path $app $bucket))
    • Remote install yaml

• Tests

  • bin\*
    • Import Yaml module
  • Scoop-Format-Manifest.Tests.ps1 -> Scoop-Format-Manifest.json.Tests.ps1
  • Scoop-Format-Manifest.yml.Tests.ps1
    • Parse
    • Validate

• Tests

Create bucket YAMLs in $env:SCOOP\buckets. Keep only few manifests prefixed with yam (to make sure non of them exists in main):
(Wrap all these steps in some Scoop-AdoptTests.ps1)
Docker-compose service which will have command: powershell.exe -File C:/SCOOP/apps/scoop/current/Scoop-AdoptTests.ps1
docker/compose#1988 (comment)
https://docs.docker.com/compose/compose-file/#extension-fields

1. Make sure implementation work same for json and yaml.
2. Commands
   1. Installing
      1. Bucket folder
      2. Remote
      3. Local full path
   2. Uninstalling
      1. Remote
      2. Bucket folder
      3. Local relative file
   3. Info
   4. Reset
   5. Update
      1. Normal
      2. Force
      3. JSON converted to yaml
         1. Normal
         2. Force
3. Binaries
   1. checkver
   2. auto-pr
4. Syntax
   1. Normal validated and linted manifest
   2. Control characters / proper syntax mismatch
5. How to sanitize code to reflect new extension type
   1. Binaries

[shoogle]

It might be a good idea to replace JSON with a more readable format, but supporting more than one extension is not a good idea. It just means more documentation and unnecessary dependencies. Obviously you'd need a transition period where both JSON and the new format are supported but, keeping them both indefinitely is not a good idea.

[couleurm]

It might be a good idea to replace JSON with a more readable format, but supporting more than one extension is not a good idea. It just means more documentation and unnecessary dependencies. Obviously you'd need a transition period where both JSON and the new format are supported but, keeping them both indefinitely is not a good idea.

yaml is built on top of json, it wouldn't be an issue to mix them up

[niheaven]

@rashil2000 Why close such feature request issue?

[rashil2000]

Detailed discussion regarding manifests formats happened here #2609

[niheaven]

That one is closed, too...

Please keep feature request issues open as reminders, and wait for someone (maintainers or not) to realize them.

[goyalyashpal]

toml sucks,

but yeah, i myself came here this time to say that yaml (more precisely, strict-yaml) is much much better format for such use-cases as manifests.

• especially it's multiline support far far outshines the "single line string arrays" approach in json for storing the code for pre_install, pre_uninstall etc.
• and it's native support for comments mean that comments no longer require to be string-ed

i had thought that i would be ridiculed for bringing this up i.e. "support strict-yaml", so its sooo nice to see this long discussion here (which seems to be positive from quick glance)

[couleurm]

looking forward to see a strict yaml parser in pure powershell

[goyalyashpal]

strict yaml parser in pure powershell

powershell doesn't have FFI to libs having C-ABI?
'cz i have no plan of writing a library in powershell

it's kinda interesting to me that there is any 3P library for powershell lol:

$ scoop info powershell-yaml | yq '.Description' # -p props
PowerShell module for serializing and deserializing YAML

[couleurm]

doesnt this use YamlDotNet?

[goyalyashpal]

support comments (jsonc) in app manifests #5913

as comments help a lot in adding in manifest the *passive details like:

• alternate ways of accomplishing something, just for reference. like:
  • different ways for checkver: like different urls (e.g. for eclipse apps), different methods (github, sourceforge), etc.
• rapidly trying different things in manifest to get the details correct
  • different ways for extract_dir, pre_install, installer.script etc while trying to see which will work
• architectures u don't use (local manifests)
  as this will help reduce unnecessary package downloads for updating hashes of those in checkver.ps1 [-f|-up]
• blank keys while creating new manifests by hand
• any upcoming changes in the app manifest - while it is not landed in upstream stable yet
  like in smplayer Keep configurations in separate dedicated dir even for portable install smplayer-dev/smplayer#934
• anything detail worth paying attention to
• etc etc.

*passive i.e. not validated & errored-out-at, & only for human use.