Lua program format

[oeed]

As discussed in #26 and #18, the Lua format is being split in to separate types.

There hasn't been much discussion of the extension for programs, so I've simply used .prg as it fits in with .tbl and .api. I've also said that .lua defaults to a Lua program and shouldn't be used which I'm sure people will be opinionated about.

Also see: #40 and #41.

[viluon]

What is the point in adding another extension? *.lua files cannot be loaded by os.loadAPI #40 (okay, they can, but who wants to access APIs like _G[ "foo.lua" ].bar()?), and *.prg files are essentially just Lua source code (*.tbl #41 files are not valid Lua source code. They miss a return statement).

Since switching to this program "format" offers no benefits compared to just sticking with *.lua and the *.prg extension is nowhere near as descriptive and can be confusing, I would argue against adopting this. Either just leave *.lua or create a new standard that actually contains features for stuff like metadata, resources, APIs, or fonts, as modern *.exe files do.

Also, programs can be represented in bytecode too. This standard however specifies *.prg files (essentially Lua programs) to be plain text. That is like saying that LuaLua (repo) portable programs (i.e. LuaLua source code compiled, i.e. regular Lua bytecode, which is portable) aren't programs, although they run under the very same VM (actually I'm not sure if LuaLua works this way, there aren't good docs on that, but you get what I mean).

• Resolve redundancy of a new extension (choose one)
  • a) Rewrite the program specification with new features
    • Add features that make programs distinguishable from source code
    • Create utilities for parsing the new file format
    • Allow bytecode
    • With different operating systems and environments, it is unlikely that a single program specification will exist (although I hope for these best-case scenarios). "Lua program" is quite an unclear name, as most of these future formats will probably use Lua source or byte code. I'd suggest a different, more descriptive and less official-sounding name
  • b) Change default extension to *.lua
    • Rename the proposed standard (the format is not a format for programs, but for source code)
    • Proposed standard should not use lua/program, but rather lua/source or similar
    • Create or link to utilities for parsing and loading Lua source code

[viluon]

Yes @demhydraz, ELF is a very good example of a powerful program file format, which can be used for different (4) purposes. Although unlike ELF, a Lua program should rather use a single extension (looking at you, none 😡 ).

[viluon]

No, you should set up aliases for that @demhydraz. OpenComputers, for example, appends a *.lua extension automagically when said file exists in the cwd. Extensions aren't an issue, they are a blessing!

[oeed]

@viluon did you not read the discussion in #18? I get what you mean about extensions being a pain (although to be honest, Dan, what the hell... add extension support for os.loadAPI already).

The whole point of this is to give them separate MIMEs and to do that properly we really need to give them separate extensions.

[lyqyd]

Thanks for your incredibly insightful contribution there.

I don't agree that separate MIMEs require separate extensions, or even any extensions at all. The MIME type of a file is unrelated to what it is named, and I don't believe I've seen a metadata suggestion yet that would inextricably link the two. Yes, looking at extensions to tell what something is can be convenient, but it's not the only way to go.

[oeed]

@lyqyd you're correct about no metadata standard yet, although the one I'm currently working on does do that. It clearly won't be popular though.

Sure, when you save the file under a metadata system you'll be able to choose. But when it's not, other than using extensions what are the other ways to go?

[oeed]

Also @viluon, rejected is if the proposal cannot be changed to fit other people's requirements. This has not been rejected yet.

[viluon]

@viluon did you not read the discussion in #18? I get what you mean about extensions being a pain (although to be honest, Dan, what the hell... add extension support for os.loadAPI already).

What are you referring to, exactly @oeed?

The whole point of this is to give them separate MIMEs and to do that properly we really need to give them separate extensions.

I don't see *.lua being used by any other format.

Also @viluon, rejected is if the proposal cannot be changed to fit other people's requirements. This has not been rejected yet.

Ah, sorry for that. My thinking was that since the current form is not acceptable, I should label it as rejected but work-in-progress at the same time.

meta However, it's quite unclear (to me) how should we treat PRs that have to change so that they can be merged. I think it'd be actually good if we close them and let the author rewrite the thing, then either open a new PR or reopen the old one. This would follow the idea of PRs being published when they are ready to be merged, not earlier. I messed up when I opened a PR for the tutorial branch, when it's clearly not ready to be merged yet, so sorry for that 😕 But what about future PRs? Closing them immediately is bad, because they can be reworked and they are easily visible when kept open. Keeping them open while they're WIP is bad, because PRs should be published when they're ready, and the PR might stay open for a very long time, with collaborators unsure if it's ready yet.

[oeed]

What are you referring to, exactly @oeed?

Sorry, I saw '18 days ago' and wrote 18 instead of #26. I'm referring to this:

There's lots of things in CC that a valid lua file could be:

lua/data
lua/program
lua/library

So why are we collapsing all of them to one mime type? We are missing an opportunity here.

Essentially we need to split Lua file formats up.

Rename the proposed standard (the format is not a format for programs, but for source code)

This is for programs, APIs can be regarded as source code too (and arguable tables too). Hence, this is not for source code; there needs to be a distinction. Essentially, an executable.

I don't see *.lua being used by any other format.

Well, it potentially could be used by an API, a program or data. I think I'll leave this whole extensions issue on hold until I finish my metadata proposal where we can sort out the relationship between extensions and MIMEs.

However, it's quite unclear (to me) how should we treat PRs...

I think this needs a discussion on Gitter.

[lyqyd]

I'd be fine with changing the default extension to .lua (or no extension), and the mime to lua/source. I don't think it would be useful to change this standard to anything other than the basic "lua program" that you'd see essentially everywhere in ComputerCraft. A separate standard could be created later on with other fancy features, which could use lua/program and the .prg extension.

[viluon]

Exactly what I thought the best solution would be @lyqyd, 👍 !

[oeed]

Okay, so are we saying have a Lua source file and a Lua table format; so no distinction between API and programs?

[viluon]

Pretty much, yeah. Standardizing APIs seems kind of pointless to me, and programs are really just source code. If they'd have more features than just storing plain text then why not, but since they don't, I'd just call them what they are. lua/source

[lyqyd]

I think there's still value in making a distinction there. APIs often won't do anything except put functions into their environment table. Yes, they're both valid lua, but they are different things.

On Feb 10, 2016, at 03:07, Oliver Cooper notifications@github.com wrote:

Okay, so are we saying have a Lua source file and a Lua table format; so no distinction between API and programs?

—
Reply to this email directly or view it on GitHub.

[oeed]

@lyqyd I do agree with you there, but your previous post seems to contradict this a little. Are you essentially saying there should be a lua/source MIME for non-API code (i.e. executable code) using the .lua extension?

[lyqyd]

Sorry for any confusion. For clarity, I would use lua/source for any valid lua file (.lua or no extension) that could be thrown through shell.run in ComputerCraft. I would also use lua/api (or lua/library, don't care which) for valid lua programs that are intended to be loaded via os.loadAPI. Most of those don't do anything except put stuff in their environment table when you run them, so it's a useful distinction. However, a program auto-detecting MIME by inspecting files could label API files as lua/source and not technically be incorrect, which I think is what @viluon is getting it.

I'm not sure that actually clarified anything, dammit. Let me know if I'm still being confusing. :)

[lyqyd]

These are standards for use within the context of ComputerCraft. That MIME string is unnecessarily verbose, and we don't need to make things more difficult just to try to "comply" with external standards when doing so provides zero benefit.

[oeed]

Okay, I think I understand what you're saying @lyqyd. I'll update the standards soon with those changes.

@demhydraz, as Lyqyd said, this isn't the 'real world'. Additionally, as we technically are the standards organisation, x- is incorrect.

[lyqyd]

I've assigned oeed 'cause he said he'd update the standards soon, but I don't see any new commits since then. :-)

[oeed]

Yeah sorry, I haven't really had tons of time to do much CC related stuff.