Stream output files instead of writing them in-memory first #46
Conversation
| @@ -23,6 +23,7 @@ module dummyPrinter_class | |||
|
|
|||
| contains | |||
| procedure :: init | |||
| procedure :: endFile | |||
| procedure :: extension | |||
| procedure :: writeToFile | |||
There was a problem hiding this comment.
I thought writeToFile was gone in the asciiOutput class?
There was a problem hiding this comment.
It is... I think this is an oversight that this got left in.
There was a problem hiding this comment.
Should be removed now.
| ! It causes buggy warnings to appear in compilation | ||
| allocate(self % outputFileName, source = trim(filename) // "." // self % output % extension()) | ||
|
|
||
| open(newunit = self % outputUnit, & |
There was a problem hiding this comment.
It looks like 'error' is gone from this list.
There was a problem hiding this comment.
😨 Ups... not great thing to forget
There was a problem hiding this comment.
Should be fixed.
| ! Close file | ||
| close(unitNum) | ||
| !! NOTE: | ||
| !! This subroutine WILL NOT be called if the `end program` statement is |
There was a problem hiding this comment.
This comment is not clear.
There was a problem hiding this comment.
!! NOTE:
!! Being marked `final`(a deconstructor) this subroutine will be normally implicitly called by
!! by the compiler whenever a variable of `type(outputFile)` goes out of scope (e.g. when program
!! execution reaches `end subroutine`, `end function` or `end block` statement. However, the rule in
!! Fortran is that the decostructors are NOT called at the end of the program `end program`. In that case
!! if one uses the output file as a module variable or defines it inside the program block, the `finalisation`
!! must be called explicitly to ensure or data from the buffer is written to the disk
!! Does this work better?
There was a problem hiding this comment.
Comment has been changed.
|
So I think I have addressed all the issues now... |
There was a problem hiding this comment.
Very nice, I think this all looks great and it's a much better alternative to write the output file than keep it all in memory!
I think there's just a small leftover typo that I spotted in the docs (soon SCONE will be typo free!!!) but then feel free to rebase and merge
Allows the outputFile to print to disk anytime between initialisation and finalisation. Thus, will allow to reduce memory usage.
A delay buffer is needed to be able to make small modifications to already written data (e.g. remove a trailing comma), which makes the implementation of output formats much easier.
Co-authored-by: valeriaRaffuzzi <108435337+valeriaRaffuzzi@users.noreply.github.com>
Mikolaj-A-Kowalski
force-pushed
the
stream-output
branch
from
29fb90c to
2df41b2
Compare
This resolves Issue #38. However, I cannot help but to think it is a bit filthy... that being said I am not sure how exactly to make it nicer.
The problem with streaming to the output file directly is that it is convenient to have a 'backtrace' capability, when writing the printers for different output formats. This allows, for example, to remove trailing commas with relative ease (crucial for JSON).
To 'have a cake and eat it' this implements a
delayedStreamwhich works a bit like a buffered output, but when the buffer is flushed it retainsMIN_SIZE=16last characters [as opposed to printing everything to the disk].What I don't like about the changes is the way the file ownership is handled. It is made to be a responsibility of the
outputFileclass, which opens a file ininitand closes it infinalisation(which is also the destructor). However, the printing is implemented by theasciiOutput_interwhich owns an instance ofdelayedStream. The appropriate output unit is set by a subroutine.The other thing I fear may be confusing is that if one does not supply
filenametooutputFileno file will be produced (this is required for unit testing).In short this PR may still require some iterations [in addition to squashing the fixups!]