Stubname field added to YML databases without any documentation

[CelesteBlue-dev]

Since commit 4e726a1 by @Princess-of-Sleeping, a stubname field was added to some (not all) libraries in the yml databases.

This leads to two issues:

1. What does this stubname mean? I could not find any documentation on this. This should be documented in a description of the YAML structure. Where is this stubname field used? I could not find any clear usage of this field. There could be a mistake with the term "library name".
2. I have a YAML parser in JAVA and this commit broke it with error com.esotericsoftware.yamlbeans.YamlReader$YamlReaderException: Line 7, column 17: Unable to find property 'stubname' on class. So I had to add this public String stubname; field to my Java class. This is not a major issue but this shows that YAML fields additions can break parsers. I would expect a YAML parser to just skip undeclared fields.

Please at least fix the first issue by answering my questions in some YAML documentation file on this repository. Regards

[isage]

It's target name of the generated stub static library. This is done because library name != stub name
For example, library name is SceAudioencUser, but stub file is SceAudioenc_stub.a

[CelesteBlue-dev]

I see. Thanks.

Where should we put a documentation about these YML databases? I advice in a new file docs/db_file_format.txt and in markdown.

[CelesteBlue-dev]

I have found another issue caused by the add of the 'stubname' YAML field. Indeed, running python build.py gives an error and stops on the first library having the 'stubname' field.

Before I updated vitasdk:

python build.py
warning: line: 0, column: 0, unknow tag 'version'.
warning: line: 0, column: 0, unknow tag 'version'.
warning: line: 0, column: 0, unknow tag 'version'.
warning: line: 0, column: 0, unknow tag 'version'.
warning: line: 0, column: 0, unknow tag 'version'.
warning: line: 0, column: 0, unknow tag 'version'.
warning: line: 0, column: 0, unknow tag 'version'.
warning: line: 0, column: 0, unknow tag 'version'.
error: line: 7, column: 18, unrecognised library key 'stubname'.

After I updated vitasdk:

python build.py
error: line: 7, column: 18, unrecognised library key 'stubname'.

These logs do not mention which file is faulty, line 7 of which file... build.py logging should be improved to help solving errors.

By deduction, it could be the fault of vita-libs-gen. There might be a link with vitasdk/vita-toolchain#246 because again it is @Princess-of-Sleeping's commit.

This could be probably fixed by updating vita-toolchain, but again, a YAML documentation is needed because it seems that vitasdk maintainers meet many issues related to this (see also issues in vita-toolchain).

[CelesteBlue-dev]

Another still open issue on this subject: vitasdk/vita-toolchain#95.

I hope that these issues will be fixed by documenting the YAML format then rewriting vita-libs-gen and other YAML parsers to skip unnecessary fields or by externalizing the optional fields to different files so that the mandatory YAML files can still be parsed without error.

[CelesteBlue-dev]

It seems that vdpm, which is what I used, does not install latest version of vita-headers and vita-toolchain.

Here is the proof that stubname was added to vita-libs-gen:
https://github.com/vitasdk/vita-toolchain/blob/08b913cfd1ce4659ae83386ab33f5349a1ff3242/src/vita-nid-db.h#L27

So I wonder why it does not work in vita-headers/build.py... Probably because vdpm did not install latest vita-toolchain... To confirm.

[zetanumbers]

Here's the stub lib name generation function that i've managed to isolate from the vita-libs-gen-2 code. Docs would have been usefull.

[CelesteBlue-dev]

Thank you @zetanumbers.
Is your rust code up-to-date with vita-libs-gen-2?
Are there any other difference between vita-libs-gen-2 and vita-libs-gen first series than "stub lib name generation function"?
On your linked repository I do not see any README. That pushes the issue further if people need to learn rust and read your code to find the usage and usefulness of vita-libs-gen-2...

[zetanumbers]

Is your rust code up-to-date with vita-libs-gen-2?

Yep. I'm tracking changes. I'm planning to add CI check for that.

Are there any other difference between vita-libs-gen-2 and vita-libs-gen first series than "stub lib name generation function"?

Not sure, code seemed pretty similair. Maybe it splits/joins some nids when compared. I've only had an issue with stub lib names.

On your linked repository I do not see any README. That pushes the issue further if people need to learn rust and read your code to find the usage and usefulness of vita-libs-gen-2...

This function is for internal use. It's not that hard to read. But if you need it: here's python version

[Princess-of-Sleeping]

vita-libs-gen was only able to process up to 1024 (?) entries from one yml at a time. vita-libs-gen-2 improved them to be able to handle multiple ymls at once and removed the maximum entry limit. Also added stubname for proper placement of db.

[Princess-of-Sleeping]

Document added to template.
https://github.com/vitasdk/vita-toolchain/blob/master/yml_template/nid_db_classic.yml