Enhancement: Prepare for a formal release #87

lcn2 · 2022-02-21T19:51:49Z

Before the mkiocccentry repo has a formal release, certain things should be performed. This request an action item list to serve as a reminder before this is done.

TODO:

Release 0.9.9 actions: (once all the above TODO actions are done)

Code complete actions: (once all above Release 0.9.9 actions done)

Final test: (once Code complete actions is done) - try on many reasonably up to date platforms:

git stash list
Verify nothing is stashed.
If any stashed files are found, remove stashes and restart this section from the beginning.
inspect make clobber ; git status --ignored
Verify that the correct files are being excluded.
If any unusual files are being ignored, resolve the issue and restart this section from the beginning.
git status
Verify that the directory tree is now clean.
If the directory tree is not clean, resolve the issue and restart this section from the beginning.
make release or make prep as needed
Verify that the make action is OK.
If any problems are found, fix them and restart this section from the beginning.
make all
Verify that the make all does nothing and prints nothing after doing make release.
If make all does something and/or prints something, fix the issue and restart this section from the beginning.
git status
Re-verify that the directory is now clean.
If the directory tree is not clean, resolve the issue and restart this section from the beginning.
inspect make clobber all ; make clobber ; git status
Verify the directory is now clean after make clobber.
If the directory tree is not clean, resolve the issue and restart this section from the beginning.
inspect git status --ignored
Verify that the correct files are being excluded.
If any unusual files are being ignored, resolve the issue and restart this section from the beginning.
inspect make clobber all test
Verify that the make action is OK.
If any problems are found, fix them and restart this section from the beginning.

Formal release 1.0.0 (once Final test is done)

The text was updated successfully, but these errors were encountered:

xexyl · 2022-02-21T19:58:55Z

Just to be sure: you have it in issues because you want feedback, right?

As for:

Remove any trailing blanks from lines
i.e., look for: /\s+$/

I often do this. I have it visible in git diff (I'm not actually sure if this is a default but I think it might be because of coloured diff and the fact that a single space can make a diff). I actually have commented out in my .vimrc a way to do this when saving files but I have it commented out exactly because of the git diff being annoying (and not wanting to make it much of the commit).

Anyway this is how you can do it in .vimrc:

        autocmd! BufWritePre *.c %s/\s\+$//e
        autocmd! BufWritePre *.h %s/\s\+$//e

That will automatically remove spaces at end of lines on save. I know I also have commented out (I'm unsure why in this case):

let c_space_errors = 1

Perhaps it highlights spaces? I'm not sure.

Are you thinking of changing the name of the repo btw? I wasn't sure from the first part of your post. And now I'll be leaving for the day.

EDIT 0, 12 January 2023

Typo fixes.

For now, added -Werror to make it easier to notice, stop and fix when there is a compile warning. As per Enhancement: Prepare for a formal release #87 this -Werror needs to be removed before a formal release of the mkiocccentry tool set. Once the 1st formal release of the mkiocccentry tool set has been made, the -Werror compile option should NOT be a default.

lcn2 · 2022-02-21T20:12:32Z

Just to be sure: you have it in issues because you want feedback, right?

Yes.

lcn2 · 2022-02-21T20:14:48Z

Are you thinking of changing the name of the repo btw? I wasn't sure from the first part of your post. And now I'll be leaving for the day.

Should the repo name change? If so, to what and why?

xexyl · 2022-02-21T20:15:48Z

Are you thinking of changing the name of the repo btw? I wasn't sure from the first part of your post. And now I'll be leaving for the day.

Should the repo name change? If so, to what and why?

I don't think so, no. I just got that impression that you might be thinking of it - from the first part of your post. It was a very quick glance.

xexyl · 2022-02-21T20:19:33Z

Just to be sure: you have it in issues because you want feedback, right?

As for:
Remove any trailing blanks from lines
i.e., look for: /\s+$/
I often do this. I have it visible in git diff (I'm not actually sure if this is a default but I think it might be because of coloured diff and the fact that a single space can make a diff). I actually have commented out in my .vimrc a way to do this when saving files but I have it commend out exactly because of the git diff being annoying (and not wanting to make it much of the commit).

Anyway this is how you can do it in .vimrc:
        autocmd! BufWritePre *.c %s/\s\+$//e
        autocmd! BufWritePre *.h %s/\s\+$//e
That will automatically remove spaces at end of lines on save. I know I also have commented out (I'm unsure why in this case):

Of course we could also use sed to do a single pass on this with the -i option if it just needs to be done at the end but maybe the vim config is useful too.

And now I really am going. Have a great afternoon! I'll read more in the other thread tomorrow (or maybe later today and react tomorrow).

Cheers.

lcn2 · 2022-02-21T20:26:19Z

Anyway this is how you can do it in .vimrc:
        autocmd! BufWritePre *.c %s/\s\+$//e
        autocmd! BufWritePre *.h %s/\s\+$//e
That will automatically remove spaces at end of lines on save. I know I also have commented out (I'm unsure why in this case):

Nice hack!

let c_space_errors = 1
Perhaps it highlights spaces? I'm not sure.

The vim help says:

*c_space_errors*                trailing white space and spaces before a <Tab>

But it does not seem to do anything.

xexyl · 2022-02-22T00:56:45Z

Anyway this is how you can do it in .vimrc:
        autocmd! BufWritePre *.c %s/\s\+$//e
        autocmd! BufWritePre *.h %s/\s\+$//e
That will automatically remove spaces at end of lines on save. I know I also have commented out (I'm unsure why in this case):
Nice hack!

Yes. And I remembered that although I have it disabled I also added to my .vimrc that it attempts to read in from the current directory the file .override.vim which means I could enable it for this repo. Want me to do that?

let c_space_errors = 1
Perhaps it highlights spaces? I'm not sure.
The vim help says:
*c_space_errors*                trailing white space and spaces before a <Tab>
But it does not seem to do anything.

Yes I looked at that too. I don’t know what it does and it’s possible that that’s why I have it disabled. But I have this vague memory that I disabled it due to not liking the effect it had. But what that effect is I have not the slightest clue.

I will reply to the other things you wrote in the morning. I hope you have a great night! If you have any thoughts on the other things I mentioned I would be very interested in them but it sounds like you have quite some changes you’re making so perhaps that’s why you haven’t got to it yet.

Well no rush on my behalf. I will add what I can do and the problems can be worked out later.

Good night!

EDIT on 13 April 2022: typo fix. Bloody phone typing.

xexyl · 2022-02-22T01:03:12Z

On one more thing real quick. I have a mate who has an IRIX box. He had to disable ssh but I might be able to help him download the repo to that computer and get it to work.

If there are any problems I could probably fix them. Actually years ago he had me port linux_logo to BSD. I was going to help him get some things working on this computer but it’s now rather impossible for me to do that since ssh is disabled. But I probably can get him to test this repo.

There are some other things in the list of things to do that I can help with. Documentation included. I should make man pages for the recent tools - the decode and encode tools. I just have been focused on the ones that haven’t been completed yet. But perhaps the next time I need a break (not from being too tired) I can do that. We shall see.

And with that I am going to put the phone down. Expect a pull request tomorrow!

lcn2 · 2022-02-22T02:41:02Z

On one more thing real quick. I have a mate who has an IRIX box. He had to disable ssh but I might be able to help him download the repo to that computer and get it to work.

IRIX as in SGI IRIX? If so, what model, if we may ask?

FYI: Landon worked at SGI for a number of years.

xexyl · 2022-02-22T10:49:08Z

On one more thing real quick. I have a mate who has an IRIX box. He had to disable ssh but I might be able to help him download the repo to that computer and get it to work.

IRIX as in SGI IRIX? If so, what model, if we may ask?

I think SGI yes but I'm not sure which model. I can ask him if you like.

FYI: Landon worked at SGI for a number of years.

Very cool! Your history is fascinating and very very cool - thank you so much for sharing all these details with me! You have a lot to be proud of and I'm also honoured to be working with you on this repo so thank you for that too!

xexyl · 2022-02-22T11:17:45Z

Anyway this is how you can do it in .vimrc:
        autocmd! BufWritePre *.c %s/\s\+$//e
        autocmd! BufWritePre *.h %s/\s\+$//e
That will automatically remove spaces at end of lines on save. I know I also have commented out (I'm unsure why in this case):
Nice hack!
Yes. And I remembered that although I have it disabled I also added to my .vimrc that it attempts to read in from the current directory the file .ovveride.vim which means I could enable it for this repo. Want me to do that?
let c_space_errors = 1
Perhaps it highlights spaces? I'm not sure.

I was right about what that option does - it highlights trailing spaces. See the screenshot

.

Edit: Which of course has a comment that's not true, perhaps from yank and paste. I'll fix it.

lcn2 · 2022-02-22T15:35:35Z

On one more thing real quick. I have a mate who has an IRIX box. He had to disable ssh but I might be able to help him download the repo to that computer and get it to work.

IRIX as in SGI IRIX? If so, what model, if we may ask?

I think SGI yes but I'm not sure which model. I can ask him if you like.

FYI: Landon worked at SGI for a number of years.

Very cool! Your history is fascinating and very very cool - thank you so much for sharing all these details with me! You have a lot to be proud of and I'm also honoured to be working with you on this repo so thank you for that too!

I have an SGI O2, but I don't power it up these days. Silicon Graphics was a fun company in its day.

xexyl · 2022-02-22T16:24:00Z

On one more thing real quick. I have a mate who has an IRIX box. He had to disable ssh but I might be able to help him download the repo to that computer and get it to work.

IRIX as in SGI IRIX? If so, what model, if we may ask?

I think SGI yes but I'm not sure which model. I can ask him if you like.

FYI: Landon worked at SGI for a number of years.

Very cool! Your history is fascinating and very very cool - thank you so much for sharing all these details with me! You have a lot to be proud of and I'm also honoured to be working with you on this repo so thank you for that too!

I have an SGI O2, but I don't power it up these days. Silicon Graphics was a fun company in its day.

I bet it was fun! My mum actually worked for the telco and she got to use the original AT&T Unix. I was always jealous of that. She also one time got to use a punchcard (which I was also jealous of) - but that was on strike duty.

I asked my friend what model - hopefully he'll get back to me sometime in the near future.

xexyl · 2022-02-22T16:57:08Z

On one more thing real quick. I have a mate who has an IRIX box. He had to disable ssh but I might be able to help him download the repo to that computer and get it to work.

IRIX as in SGI IRIX? If so, what model, if we may ask?

I think SGI yes but I'm not sure which model. I can ask him if you like.

FYI: Landon worked at SGI for a number of years.

Very cool! Your history is fascinating and very very cool - thank you so much for sharing all these details with me! You have a lot to be proud of and I'm also honoured to be working with you on this repo so thank you for that too!

I have an SGI O2, but I don't power it up these days. Silicon Graphics was a fun company in its day.

That's the same model he has.

xexyl · 2022-04-13T00:03:54Z

About to go to sleep but I just by chance saw this. I am too tired to even read it but it might be something we can use at some point:

https://developers.redhat.com/articles/2022/04/12/state-static-analysis-gcc-12-compiler

xexyl · 2022-04-13T15:30:41Z

I thought of two other things to do. One I'll do soon most likely; the other can probably come later.

The first one is change 1st, 2nd etc. to words. Similarly '2 args' should be 'two args' and so on. This should be easy enough to find most cases (the first part very easy but the second part might take a bit more time because [1-9]+ has some things that shouldn't be changed. Two examples:

In util.c:

util.c:     * must be a UUID.  The UUID has the 36 character format:
util.c:     *	    xxxxxxxx-xxxx-4xxx-axxx-xxxxxxxxxxxx
util.c:     * where 'x' is a hex character.  The 4 is the UUID version and the variant
util.c:     * 1.

The 36 is correct and the 4 and 1 should stay the same as well and there might be other places where this applies (and anything > 9 should be digits except maybe for one hundred and the like: I suppose that this is up to debate on the latter part though). Another example that can't be updated is the UTF-8 map.

Of course I could change the regex to be just [0-9] or maybe [1-9] but you get the idea: it's not always clear cut.

But actually all of it might be up to debate in some minds except I think the 1st -> first and similar should be done and for args two args etc. The rest can be decided but then it should also be consistent which I guess is the most important thing.

The second thing is we should give a bit more help when tools are missing that can be downloaded somewhere. In particular we give a download link to tar but only for linux. It might be an idea to explain how to do it for macOS for example. I guess it's more extreme to check which system is being used and print the correct links/help but it did cross my mind so figured I'd mention it anyway.

Something else just occurred to me: does anyone have any big endian machines to test everything under?

lcn2 · 2022-04-13T17:52:41Z

The first one is change 1st, 2nd etc. to words. Similarly '2 args' should be 'two args' and so on ...

This item has been added to the TODO list above.

xexyl · 2022-04-13T17:56:08Z

The first one is change 1st, 2nd etc. to words. Similarly '2 args' should be 'two args' and so on ...

This item has been added to the TODO list above.

I can do the that one fairly easily but I don't want to stomp on any of your pending changes so I'm waiting on that. Hoping the pull request that I did today does not but as I said I can redo the JSON renaming again.

lcn2 · 2022-04-13T17:59:12Z

The second thing is we should give a bit more help when tools are missing that can be downloaded somewhere. In particular we give a download link to tar but only for linux. It might be an idea to explain how to do it for macOS for example. I guess it's more extreme to check which system is being used and print the correct links/help but it did cross my mind so figured I'd mention it anyway.

That does seem a bit extreme. For the make all that people will need to use, the minimum is the C compiler and common command line tools. For those with systems that lack something like tar, we mention where to get it.

That is probably as far as we should go. Downloading software should be the responsibility of the user.
it may be too far to download and try to install something.

In the worst case, the user simply needs to find a system that as common command line tools.
This repo does NOT have to be compiled on the system for which the entry was built.
Users can copy the source for their entry to a system that as common command line tools
and compile / use this repo there.

xexyl · 2022-04-13T18:01:05Z

The second thing is we should give a bit more help when tools are missing that can be downloaded somewhere. In particular we give a download link to tar but only for linux. It might be an idea to explain how to do it for macOS for example. I guess it's more extreme to check which system is being used and print the correct links/help but it did cross my mind so figured I'd mention it anyway.

That does seem a bit extreme. For the make all that people will need to use, the minimum is the C compiler and common command line tools. For those with systems that lack something like tar, we mention where to get it.

That is probably as far as we should go. Downloading software should be the responsibility of the user. it may be too far to download and try to install something.

In the worst case, the user simply needs to find a system that as common command line tools. This repo does NOT have to be compiled on the system for which the entry was built. Users can copy the source for their entry to a system that as common command line tools and compile / use this repo there.

Well I'd agree that the last part is extreme. I am not sure if giving them a link for macOS is though. On the other hand macOS has tar built in as well so it's probably fine.

I wasn't suggesting downloading or installing though! That would be very bad! I meant simply giving them a link for macOS as well. But again that's probably not necessary.

lcn2 · 2022-04-13T18:03:10Z

I meant simply giving them a link for macOS as well.

The problem would be to detect macOS and then to determine what link:

source code somewhere
HomeBrew command
Macports command

This is probably too much to do on behalf of the user for a system that lacks common command line tools?

lcn2 · 2022-04-13T18:03:54Z

Something else just occurred to me: does anyone have any big endian machines to test everything under?

We don't. Do you think this is really needed?

xexyl · 2022-04-13T18:04:42Z

I meant simply giving them a link for macOS as well.

The problem would be to detect macOS and then to determine what link:

source code somewhere

HomeBrew command

Macports command

This is probably too much to do on behalf of the user for a system that lacks common command line tools?

True. I was thinking of the default location but as noted that's not even necessary. And in fact if they have the common command line tools they'll be able to compile the tools so it shouldn't be necessary anyway.

So forget that idea. Question is does the tar link for GNU need to be there?

lcn2 · 2022-04-13T18:07:10Z

Question is does the tar link for GNU need to be there?

We don't think ti could hurt: or that could be removed if there is a problem.

xexyl · 2022-04-13T18:08:21Z

Question is does the tar link for GNU need to be there?

We don't think ti could hurt: or that could be removed if there is a problem.

I agree. It doesn't hurt at all to be there. I leave that up to you.

xexyl · 2022-04-13T18:09:57Z

Something else just occurred to me: does anyone have any big endian machines to test everything under?

We don't. Do you think this is really needed?

I don't: I only mentioned it because you had said you wanted to test as many systems as possible so I figured maybe bring up endianness as well.

xexyl · 2023-02-04T19:39:22Z

af4229c looks very nice!

..however there is a problem. Now bug_report.sh fails. One moment... will see if I can find out what it is quickly.

Okay this is my fault. I forgot that the bad/ files do have to fail in some form always. For now since we're trying to get to a code freeze I'll just remove that one and rename the bad one related to semantics to be the typical one. The issue can be resolved another time.

Will do a pull request momentarily.

UPDATE 0

See 0f44c0e.

Sorry about that! I forgot about this and it didn't seem like I'd have to run a test script for that reason. Done with changes now.

lcn2 · 2023-02-04T20:02:52Z

af4229c looks very nice!

..however there is a problem. Now bug_report.sh fails. One moment... will see if I can find out what it is quickly.

Okay this is my fault. I forgot that the bad/ files do have to fail in some form always. For now since we're trying to get to a code freeze I'll just remove that one and rename the bad one related to semantics to be the typical one. The issue can be resolved another time.

Will do a pull request momentarily.

UPDATE 0

See 0f44c0e.

Sorry about that! I forgot about this and it didn't seem like I'd have to run a test script for that reason. Done with changes now.

All is well, thank you @xexyl

xexyl · 2023-02-04T20:04:00Z

af4229c looks very nice!

..however there is a problem. Now bug_report.sh fails. One moment... will see if I can find out what it is quickly.

Okay this is my fault. I forgot that the bad/ files do have to fail in some form always. For now since we're trying to get to a code freeze I'll just remove that one and rename the bad one related to semantics to be the typical one. The issue can be resolved another time.
Will do a pull request momentarily.

UPDATE 0

See 0f44c0e.
Sorry about that! I forgot about this and it didn't seem like I'd have to run a test script for that reason. Done with changes now.

All is well, thank you @xexyl

Appreciate that!

Well hope you have a great day and I look forward to see what comes next. Going back to reading again.

lcn2 · 2023-02-04T20:37:22Z

There is annoyance, that while not a fatal error, is something that will need to be addressed when a version that is "1.0 2023-02-04" is updated.

For example we now have multiple versions with the value of "1.0 2023-02-04", I.e.:

- CHKENTRY_VERSION
- FNAMCHK_VERSION
- MKIOCCCENTRY_VERSION
- TXZCHK_VERSION

The current vermod.sh cannot easily change just one of the strings "1.0 2023-02-04" for an individual application.

We also encoded this issue when we had to change IOCCC_YEAR from 2022 to 2023. In order to change all occurrences we had to do the following:

./soup/vermod.sh -Q -F '"IOCCC_year" : 2022' '"IOCCC_year" : 2023'
./soup/vermod.sh -Q -F '"IOCCC_year":2022' '"IOCCC_year":2023'

The vermod.sh will need to be improved to be able too modify an individual JSON value, and for whitespace or lack of whitespace around affected JSON tokens.

Again, this is not a bug .. more like a mis-feature :-)

We recommend that you consider, when the jparse repo is made, to add a "change a JSON member value" tool that would update the value preserve the existing whitespace.

xexyl · 2023-02-04T20:55:57Z

There is annoyance, that while not a fatal error, is something that will need to be addressed when a version that is "1.0 2023-02-04" is updated.

Good catch. I didn't think of that.

For example we now have multiple versions with the value of "1.0 2023-02-04", I.e.:
- CHKENTRY_VERSION
- FNAMCHK_VERSION
- MKIOCCCENTRY_VERSION
- TXZCHK_VERSION
The current vermod.sh cannot easily change just one of the strings "1.0 2023-02-04" for an individual application.

True.

Maybe it should use sed instead and take a name (field) it has to update?

We also encoded this issue when we had to change IOCCC_YEAR from 2022 to 2023. In order to change all occurrences we had to do the following:
./soup/vermod.sh -Q -F '"IOCCC_year" : 2022' '"IOCCC_year" : 2023'
./soup/vermod.sh -Q -F '"IOCCC_year":2022' '"IOCCC_year":2023'
The vermod.sh will need to be improved to be able too modify an individual JSON value, and for whitespace or lack of whitespace around affected JSON tokens.

Good point - the whitespace.

Again, this is not a bug .. more like a mis-feature :-)

We recommend that you consider, when the jparse repo is made, to add a "change a JSON member value" tool that would update the value preserve the existing whitespace.

Good idea. Let's hope I remember it. But if not I'm sure you can remind me. I could set a reminder but I don't really know when the repo will be crated so it might not be that helpful.

lcn2 · 2023-02-05T03:12:08Z

One problem test_ioccc/txzchk_test.sh is that the test breaks whenever soup/reset_tstamp.sh changes MIN_TIMESTAMP.

We will need to fix make release after commit a505b95:

TODO: test_ioccc/txzchk_test.sh fails non-zero exit code: 1, fix the txzchk tests.

UPDATE 0

commit 7dd7872 addressed the problem.

However test_ioccc/txzchk_test.sh needs to be improved to change txzchk_test.sh test err files accordingly.

lcn2 · 2023-02-05T05:17:32Z

Please perform all of the actions under the Final test: (once Code complete actions is done) on many reasonably up to date platforms.

The seemingly redundant actions under that test, when performed in order uncovered a number of minor Makefile issues that resulted in multiple commits to fix all discovered issues.

We can now pass without any issues, on all of the platforms (that were reasonably up to date) that we tested. As such s have checked off all buy the last issue for that section.

When you, @xexyl, let us know that you did not find any issues .. or you found them and resolved then with a new pull request, we will then check off all the items of that section and proceed to the Formal release 1.0.0 section.

xexyl · 2023-02-05T10:38:02Z

One problem test_ioccc/txzchk_test.sh is that the test breaks whenever soup/reset_tstamp.sh changes MIN_TIMESTAMP.

We will need to fix make release after commit a505b95:

TODO: test_ioccc/txzchk_test.sh fails non-zero exit code: 1, fix the txzchk tests.

UPDATE 0

commit 7dd7872 addressed the problem.

Thank you.

However test_ioccc/txzchk_test.sh needs to be improved to change txzchk_test.sh test err files accordingly.

I guess you don't mean the fact the contents of the err files have to be changed so much as the filenames? If it's the contents it's easy enough to address as I put instructions in the README.md under that directory:

--

Important note for adding files to the bad subdirectory

Whenever a new bad test file is added one must generate the proper err file. To
simplify it you can from the top level directory where the mkiocccentry.c source
file is located:

for i in ./test_ioccc/test_txzchk/bad/*.txt; do ./txzchk -q -v 0 -w -T -E txt -F ./test_ioccc/fnamchk "$i" 2>"$i.err" ; done

then do:

git add ./test_txzchk/bad/*.txt ./test_txzchk/bad/*.err

(assuming that there aren't any other files with those extensions that should
not be there). We could have the test_txzchk.sh script do this but the problem
is we need to manually inspect that the errors are correct.

--

But this can be done for new versions as well. However if you mean the filenames being invalid because the timestamp changes (and thus breaking fnamchk) then I can indeed see that as a problem. Do you have any suggestions on how to change it?

UPDATE 0

Okay I can see indeed that the file names changed as I feared. Any idea how to address this? One thought is kind of a workaround but it could work.

For the files under bad/ we don't have to worry about them so much. We simply update the files like the instructions say. Even if a bad filename is there it's okay because txzchk tests everything so updating the error files will address it (plus we don't necessarily know which files are meant to have invalid filenames).

As far as the good files go if we simply change it so that they all have the same timestamp we can do a rename of the files to be the new timestamp.

Would that suffice?

.. and I forgot to say: I hope you're having a nice sleep!

xexyl · 2023-02-05T10:43:16Z

Please perform all of the actions under the Final test: (once Code complete actions is done) on many reasonably up to date platforms.

The seemingly redundant actions under that test, when performed in order uncovered a number of minor Makefile issues that resulted in multiple commits to fix all discovered issues.

That's good they were resolved.

We can now pass without any issues, on all of the platforms (that were reasonably up to date) that we tested. As such s have checked off all buy the last issue for that section.

I currently have:

CentOS 7
Fedora 36
macOS (the most recent one but then you would have already tested it)

Now as far as Fedora I am aware that Fedora 37 is out and I can certainly upgrade and test it there though I imagine there's no problem.

I know that you have a debian box but I don't know what else you have. Well I know you have an SGI IRIX box but is that worth testing? If so can you boot it up to do so?

Can you think of other systems we might try in say a VirtualBox instance ?

When you, @xexyl, let us know that you did not find any issues .. or you found them and resolved then with a new pull request, we will then check off all the items of that section and proceed to the Formal release 1.0.0 section.

Well like I said I don't have many systems and the only linux systems I have are RedHat based but I'll do that now. When I'm more awake I can look at upgrading the fedora box (needs to be done anyway) and after that test the repo under it.

I had access to a rocky linux box but I currently do not. I will have to see if my good friend is able to reset my password. I somehow lost it and I did not put an ssh key in yet. He's been quite busy though. That might be a system we could try in a VirtualBox - as well as the other one that jumped in when Red Hat announced that they were axing CentOS.

UPDATE 0

make release passes on fedora 36.

make release will not pass under CentOS because it does not have a recent enough version of flex/bison. However make prep does work due to our backup files. Well that and make tags is non-fatal (see below). After installing ctags that test passes as well.

For make tags prior to installing ctags:

rm -f tags
cp -f -v .local.dir.tags tags
cp: cannot stat '.local.dir.tags': No such file or directory
make[2]: *** [all_tags] Error 1
make[1]: *** [all_tags] Error 2
make[1]: Leaving directory `/home/xexyl/ioccc/xexyl-mkiocccentry'
make: *** [tags] Error 2

...

One or more tests failed:

	make_action 28: make -f ./Makefile tags: non-zero exit code: 2

One or more tests failed.

See test_ioccc/test_ioccc.log for more details.

make prep: ERROR: ./test_ioccc/prep.sh exit code: 28

make prep: see build.log for details
make: *** [prep] Error 28

and build.log has as you would expect:

ctags -w -f .local.dir.tags dbg.c dbg_example.c dbg_test.c  dbg_test.c dbg.h
bash: ctags: command not found
make[2]: [local_dir_tags] Error 127 (ignored)

But now that the tool is installed all is okay in that way.

xexyl · 2023-02-05T10:57:30Z

As far as Fedora goes there's something to think about though it would require a lot of VirtualBox installs (or whatever emulator/whatever): fedora has a number of editions. See https://fedoramagazine.org/announcing-fedora-37/ for some of them.

We could install server edition for example. They now also support raspberry pi though I don't have one of those.

xexyl · 2023-02-05T11:11:00Z

Please perform all of the actions under the Final test: (once Code complete actions is done) on many reasonably up to date platforms.

The seemingly redundant actions under that test, when performed in order uncovered a number of minor Makefile issues that resulted in multiple commits to fix all discovered issues.

That's good they were resolved.

We can now pass without any issues, on all of the platforms (that were reasonably up to date) that we tested. As such s have checked off all buy the last issue for that section.

I currently have:

CentOS 7

Fedora 36

macOS (the most recent one but then you would have already tested it)

Making some changes to mkiocccentry_test.sh so I can test bsdtar ...

UPDATE 0

This I believe is done but now I have to test it under fedora and macOS and CentOS. If all is OK I will update the man pages (I also made the -T/-t options of txzchk_test.sh to consistent with other tools which had the options swapped) and make a pul request.

xexyl · 2023-02-05T11:42:24Z

Hmm ... this is curious: running make test and chkentry_test.sh works fine.

But running ./test_ioccc/chkentry_test.sh does not.

I'll fix this as well.

UPDATE 0

I think I know why but it might be difficult or at least annoying to fix. I'll update in a bit .. right now also working on the other changes I referred to above about the txzchk test scripts.

UPDATE 2

Yes 2 not 1: I had previously updated it but I figured out the problem and it's resolved. I had interrupted make test earlier and I forgot / did not think of it so the chkentry semantics checks failed because of that. The other problem still remains however. Will work on that soonish I hope.

xexyl · 2023-02-05T12:15:16Z

Please perform all of the actions under the Final test: (once Code complete actions is done) on many reasonably up to date platforms.

The seemingly redundant actions under that test, when performed in order uncovered a number of minor Makefile issues that resulted in multiple commits to fix all discovered issues.

That's good they were resolved.

We can now pass without any issues, on all of the platforms (that were reasonably up to date) that we tested. As such s have checked off all buy the last issue for that section.

I currently have:

CentOS 7

Fedora 36

macOS (the most recent one but then you would have already tested it)

Making some changes to mkiocccentry_test.sh so I can test bsdtar ...

UPDATE 0

This I believe is done but now I have to test it under fedora and macOS and CentOS. If all is OK I will update the man pages (I also made the -T/-t options of txzchk_test.sh to consistent with other tools which had the options swapped) and make a pul request.

Works under macOS and fedora. Have to test CentOS.

But: great news! The following tar works:

bsdtar 3.5.3 - libarchive 3.5.3 zlib/1.2.11 liblzma/5.2.5 bz2lib/1.0.8 liblz4/1.9.3 libzstd/1.5.2

Now I don't know if that's the version in say FreeBSD or any other BSD but at least that does work. Both mkiocccentry tests and also txzchk tests pass.

UPDATE 0

The improvements to the mkiocccentry test script also works under CentOS.

UPDATE 1

Under CentOS bsdtar also works!

$ bsdtar --version
bsdtar 3.1.2 - libarchive 3.1.2

Will see if I can find out if there are any real differences with it and those in say FreeBSD.

.. but if it doesn't they have GNU tar ... so probably okay anyway. Maybe we could update the FAQ on that? I'll look at that sometime soon.

UPDATE 2

bsdtar in macOS also works being:

$ bsdtar --version
bsdtar 3.5.3 - libarchive 3.5.3 zlib/1.2.11 liblzma/5.0.5 bz2lib/1.0.8

xexyl · 2023-02-05T13:03:59Z

See 781dbb9 for test script enhancements.

Now to resolve the issue of chkentry_test.sh not working when it's directly invoked.

A side note of the commit above: in txzchk_test.sh the -t / -T options had their meanings swapped to be more consistent with other tools.

UPDATE 0

Oh I need to add an FAQ entry about tar for BSD or at least FreeBSD .. rather update the entry about tar.

FAQ updated with commit edd1441.

UPDATE 1

As far as the chkentry_test.sh issue it was very simple to fix after all. It was just a path change in value that needed to be done and as of commit 683e074 it now works.

xexyl · 2023-02-05T13:21:24Z

QUESTION about txzchk and absolute paths (for abuse of tarball)

For txzchk should we use the -P option to check for absolute paths? Probably not necessary but I can imagine something like:

... /test-0 ...

which although would be okay when the / is stripped off technically it has an absolute path in it.

Seems FreeBSD tar supports this option as we'll.

Obviously when you extract entries you wouldn't use that option but it would enable the check I added to txzchk checks (or let it work). I think all I would have to do is add the option -P to the tar invocation (listing format of course).

What do you think?

xexyl · 2023-02-05T13:46:52Z

.. downloading Fedora 37 ... will take a while and then a while after that to install and test everything. I don't have to use the installer - just reboot into the installer and all should be good - but it'll take a while anyway.

I'll let you know when I'm able to test it. Hopefully that's today but we'll see.

What systems have you tested and what systems should we check further ?

xexyl · 2023-02-05T13:58:31Z

After making some fixes to the code I found a BSD tar implementation that does NOT work. The error of course has to do with the -J option:

test_ioccc/txzchk_test.sh -t ../bsdtar -T ./txzchk -F ./test_ioccc/fnamchk -d test_ioccc/test_txzchk -Z /Users/cody/code/xexyl-mkiocccentry
test_ioccc/txzchk_test.sh: ERROR: ../bsdtar --format=v7 -cJf .txzchk_test.XXXXXXXXXX.tarball.txz .txzchk_test.XXXXXXXXXX.test_file 2>.txzchk_test.XXXXXXXXXX.tar_err.out exit code: 1
test_ioccc/txzchk_test.sh: ERROR: did not find a non-empty tarball: .txzchk_test.XXXXXXXXXX.tarball.txz
test_ioccc/txzchk_test.sh: Notice: tar stderr follows:
Notice: ../bsdtar: illegal option -- -
Notice: Copyright 1990 Summer Institute of Linguistics
Notice: ../bsdtar: illegal option -- J
Notice: usage: bsdtar -t|-x [-s] [-v] [-f archive] [-n blocking] [filespec(s)]
Notice:     -t           tell the contents of the archive
Notice:     -x           extract files
Notice:     -s           swap adjacent bytes (for reading from 680x0 systems)
Notice:     -v           work verbosely rather than tersely
Notice:     -f archive   set the archive file (default = /dev/mt0)
Notice:     -n blocking  set the blocking factor (default = 20)
Notice: Either -t or -x is required.  The other switches are optional.
Notice: After the switches, files or directories may be listed.  If given, only
Notice: those items will be retrieved.  If no files or directories are specified,
Notice: then bsdtar retrieves the entire archive.
test_ioccc/txzchk_test.sh: Notice: end of tar stderr
./test_ioccc/ioccc_test.sh: ERROR: test_ioccc/txzchk_test.sh non-zero exit code: 1

The code in question is: https://www.cs.ait.ac.th/~on/O/oreilly/unix/upt/examples/bsdtar/bsdtar/bsdtar.c

which did indeed as noted need some fixes to be compiled. But then one could look at the getopt call to determine that anyway. Hoping to find the FreeBSD version. That btw is from 1990, the one I linked to, so it's probably not that surprising.

UPDATE 0

It seems that FreeBSD tar might support -J. From: https://github.com/freebsd/freebsd-src/blob/main/contrib/libarchive/tar/bsdtar.c

we see the option.

Not sure about v7 format yet.

xexyl · 2023-02-05T14:17:07Z

After making some fixes to the code I found a BSD tar implementation that does NOT work. The error of course has to do with the -J option:
test_ioccc/txzchk_test.sh -t ../bsdtar -T ./txzchk -F ./test_ioccc/fnamchk -d test_ioccc/test_txzchk -Z /Users/cody/code/xexyl-mkiocccentry
test_ioccc/txzchk_test.sh: ERROR: ../bsdtar --format=v7 -cJf .txzchk_test.XXXXXXXXXX.tarball.txz .txzchk_test.XXXXXXXXXX.test_file 2>.txzchk_test.XXXXXXXXXX.tar_err.out exit code: 1
test_ioccc/txzchk_test.sh: ERROR: did not find a non-empty tarball: .txzchk_test.XXXXXXXXXX.tarball.txz
test_ioccc/txzchk_test.sh: Notice: tar stderr follows:
Notice: ../bsdtar: illegal option -- -
Notice: Copyright 1990 Summer Institute of Linguistics
Notice: ../bsdtar: illegal option -- J
Notice: usage: bsdtar -t|-x [-s] [-v] [-f archive] [-n blocking] [filespec(s)]
Notice:     -t           tell the contents of the archive
Notice:     -x           extract files
Notice:     -s           swap adjacent bytes (for reading from 680x0 systems)
Notice:     -v           work verbosely rather than tersely
Notice:     -f archive   set the archive file (default = /dev/mt0)
Notice:     -n blocking  set the blocking factor (default = 20)
Notice: Either -t or -x is required.  The other switches are optional.
Notice: After the switches, files or directories may be listed.  If given, only
Notice: those items will be retrieved.  If no files or directories are specified,
Notice: then bsdtar retrieves the entire archive.
test_ioccc/txzchk_test.sh: Notice: end of tar stderr
./test_ioccc/ioccc_test.sh: ERROR: test_ioccc/txzchk_test.sh non-zero exit code: 1
The code in question is: https://www.cs.ait.ac.th/~on/O/oreilly/unix/upt/examples/bsdtar/bsdtar/bsdtar.c

which did indeed as noted need some fixes to be compiled. But then one could look at the getopt call to determine that anyway. Hoping to find the FreeBSD version. That btw is from 1990, the one I linked to, so it's probably not that surprising.

UPDATE 0

It seems that FreeBSD tar might support -J. From: https://github.com/freebsd/freebsd-src/blob/main/contrib/libarchive/tar/bsdtar.c

we see the option.

Not sure about v7 format yet.

BSD

As I reported before - at least according to man page of tar (not tried lib archive format) OpenBSD does not support either.. But then as you said if someone uses such an outdated system then it's their choice and they can always install GNU tar.

I think the same applies for FreeBSD so unless you can think of other systems that we should test maybe we're good to go ? I can't imagine any modern linux system cannot support this.

xexyl · 2023-02-05T14:32:48Z

I have to go afk a bit but when I'm back I will perform the tasks you asked me to do now that the above issues are resolved.

Stay tuned.

xexyl · 2023-02-05T16:17:19Z

I have to go afk a bit but when I'm back I will perform the tasks you asked me to do now that the above issues are resolved.

Stay tuned.

Done! With a caveat though: it's from a clone of my fork with the updates branch as it had some fixes.

I just triggered the fedora box to reboot into the upgrader. Later on I can test everything under fedora 37.

Anything else you can think of ?

I have to say I'm going to be sad that the 'Questions' issue will be closed though as I've really enjoyed it. Should I ask the timezone question now ? Another time ? (see what I did ? :-) )

xexyl · 2023-02-05T16:48:39Z

As far as tar goes. It seems that some tar implementations might include extra information in verbose output. FreeBSD is included. Again this might not be a problem if we warn the user but what do we warn them of? It appears to support -J and it possibly supports v7 as well. If so then this might be a problem.

It goes like:

/*
 * Display information about the current file.
 *
 * The format here roughly duplicates the output of 'ls -l'.
 * This is based on SUSv2, where 'tar tv' is documented as
 * listing additional information in an "unspecified format,"
 * and 'pax -l' is documented as using the same format as 'ls -l'.
 */

What do you think ?

UPDATE 0

This might be relevant:

	/* Extra information for links. */
	if (archive_entry_hardlink(entry)) /* Hard link */
		safe_fprintf(out, " link to %s",
		    archive_entry_hardlink(entry));
	else if (archive_entry_symlink(entry)) /* Symbolic link */
		safe_fprintf(out, " -> %s", archive_entry_symlink(entry));

xexyl · 2023-02-05T17:05:14Z

After updating the CHANGES.md I think everything is good unless you want txzchk to use -P to test for absolute paths - which might or might not be needed since even if it doesn't flag absolute paths you would notice it when extracting it.

Up to you but I think that all I would have to do is update the invocation to use the -P flag.

UPDATE 0

Yes it looks like I did indeed put a test in for absolute paths it's just not going to be triggered since tar strips absolute paths.

xexyl · 2023-02-05T17:07:28Z

BTW: with the test script improvements / fixes the versions of three were incremented:

Changed IOCCC_TEST_VERSION from "1.0 2023-02-04" to "1.0.1 2023-02-05".
Changed MKIOCCCENTRY_TEST_VERSION from "1.0 2023-02-04" to "1.0.1 2023-02-05".
Changed TXZCHK_TEST_VERSION from "1.0 2023-02-04" to "1.0.1 2023-02-05".

xexyl · 2023-02-05T18:41:38Z

Maybe this is nitpicky but: shouldn't the man pages be installed with 0644 permissions rather than 0444 ?

xexyl · 2023-02-05T18:44:37Z

Another thing: the licence file has the copyright for mkiocccentry as 2021. Should it be updated to say 2021-2023 or 2021, 2022, 2023 or just 2023 ?

xexyl · 2023-02-05T19:14:16Z

With commits 0da9df7 and 2a28d68 I've fixed some potential (though unlikely) formatting issues in the man pages, namely with \ and ' (the latter only in the case of commands and the former as in a literal \).

There are cases where we use a \(.. construct to use >= but that's not consistent. Perhaps it should have been but I think it's too late to do that now. I just wanted to get these in.

FYI I just by chance found it in the man page mandoc_char .. I wasn't looking for this. I just came across it in almost mindlessness of checking different man pages (something like that).

Do you want this documented in the CHANGES ?

Doing other things now anyway though I'll check if the fedora box is back up and if it is I'll do the tests you asked me to do on that one as well.

xexyl · 2023-02-05T19:19:43Z

Status update: Final test: (once Code complete actions is done)

I've done this successfully on:

fedora 36
fedora 37
CentOS 7
macOS Ventura (if that's the most recent one .. think so but too tired to really look)

Doing other things now but if you want me to make any last minute changes (like the idea with txzchk that I mentioned) please let me know.

Will be afk for a while though.

lcn2 · 2023-02-06T05:46:17Z

Version 1.0.0 has been published!

xexyl · 2023-02-06T11:16:02Z

Version 1.0.0 has been published!

Awesome! And a job well done! It was an honour and privilege to work with you! I thank you also for all the fun (and funny stories) and I'm glad you got a great deal of amusement out of my stories (and memes - plus memes of others)! The joy and privilege and pride cannot be overstated (nor understated)! The same with the friendship!

I look forward to working with you on the other projects now!

lcn2 added the enhancement New feature or request label Feb 21, 2022

lcn2 mentioned this issue Mar 17, 2022

Enhancement: create the jinfochk tool - check on the contents of an .info.json file #68

Closed

lcn2 mentioned this issue Apr 5, 2022

JSON (including parser/lexer) updates #144

Merged

lcn2 closed this as completed Feb 6, 2023

Enhancement: Prepare for a formal release #87

Enhancement: Prepare for a formal release #87

Comments

lcn2 commented Feb 21, 2022 • edited Loading

TODO:

Release 0.9.9 actions: (once all the above TODO actions are done)

Code complete actions: (once all above Release 0.9.9 actions done)

Final test: (once Code complete actions is done) - try on many reasonably up to date platforms:

Formal release 1.0.0 (once Final test is done)

xexyl commented Feb 21, 2022 • edited Loading

EDIT 0, 12 January 2023

lcn2 commented Feb 21, 2022

lcn2 commented Feb 21, 2022

xexyl commented Feb 21, 2022

xexyl commented Feb 21, 2022

lcn2 commented Feb 21, 2022

xexyl commented Feb 22, 2022 • edited Loading

xexyl commented Feb 22, 2022

lcn2 commented Feb 22, 2022

xexyl commented Feb 22, 2022

xexyl commented Feb 22, 2022 • edited Loading

lcn2 commented Feb 22, 2022

xexyl commented Feb 22, 2022

xexyl commented Feb 22, 2022

xexyl commented Apr 13, 2022

xexyl commented Apr 13, 2022 • edited Loading

lcn2 commented Apr 13, 2022

xexyl commented Apr 13, 2022

lcn2 commented Apr 13, 2022

xexyl commented Apr 13, 2022

lcn2 commented Apr 13, 2022

lcn2 commented Apr 13, 2022

xexyl commented Apr 13, 2022

lcn2 commented Apr 13, 2022

xexyl commented Apr 13, 2022

xexyl commented Apr 13, 2022

xexyl commented Feb 4, 2023 • edited Loading

UPDATE 0

lcn2 commented Feb 4, 2023

UPDATE 0

xexyl commented Feb 4, 2023

UPDATE 0

lcn2 commented Feb 4, 2023

xexyl commented Feb 4, 2023

lcn2 commented Feb 5, 2023 • edited Loading

UPDATE 0

lcn2 commented Feb 5, 2023 • edited Loading

xexyl commented Feb 5, 2023 • edited Loading

UPDATE 0

Important note for adding files to the bad subdirectory

UPDATE 0

xexyl commented Feb 5, 2023 • edited Loading

UPDATE 0

xexyl commented Feb 5, 2023

xexyl commented Feb 5, 2023 • edited Loading

UPDATE 0

xexyl commented Feb 5, 2023 • edited Loading

UPDATE 0

UPDATE 2

xexyl commented Feb 5, 2023 • edited Loading

UPDATE 0

UPDATE 0

UPDATE 1

UPDATE 2

xexyl commented Feb 5, 2023 • edited Loading

UPDATE 0

UPDATE 1

xexyl commented Feb 5, 2023 • edited Loading

QUESTION about txzchk and absolute paths (for abuse of tarball)

xexyl commented Feb 5, 2023

xexyl commented Feb 5, 2023 • edited Loading

UPDATE 0

xexyl commented Feb 5, 2023 • edited Loading

UPDATE 0

BSD

xexyl commented Feb 5, 2023

xexyl commented Feb 5, 2023 • edited Loading

xexyl commented Feb 5, 2023 • edited Loading

UPDATE 0

xexyl commented Feb 5, 2023 • edited Loading

lcn2 commented Feb 21, 2022 •

edited

Loading

xexyl commented Feb 21, 2022 •

edited

Loading

xexyl commented Feb 22, 2022 •

edited

Loading

xexyl commented Feb 22, 2022 •

edited

Loading

xexyl commented Apr 13, 2022 •

edited

Loading

xexyl commented Feb 4, 2023 •

edited

Loading

lcn2 commented Feb 5, 2023 •

edited

Loading

lcn2 commented Feb 5, 2023 •

edited

Loading

xexyl commented Feb 5, 2023 •

edited

Loading

xexyl commented Feb 5, 2023 •

edited

Loading

xexyl commented Feb 5, 2023 •

edited

Loading

xexyl commented Feb 5, 2023 •

edited

Loading

xexyl commented Feb 5, 2023 •

edited

Loading

xexyl commented Feb 5, 2023 •

edited

Loading

xexyl commented Feb 5, 2023 •

edited

Loading

xexyl commented Feb 5, 2023 •

edited

Loading

xexyl commented Feb 5, 2023 •

edited

Loading

xexyl commented Feb 5, 2023 •

edited

Loading

xexyl commented Feb 5, 2023 •

edited

Loading

xexyl commented Feb 5, 2023 •

edited

Loading