diff --git a/publishing/article/discussion.tex b/publishing/article/discussion.tex index 9cd0e0b1..3dd119c2 100644 --- a/publishing/article/discussion.tex +++ b/publishing/article/discussion.tex @@ -69,14 +69,9 @@ \subsection{Outlook} We propose a few key considerations for the further development of article reexecution — though we note that practical reuse of this system might better identify promising enhancements better than theoretical analysis. -% concerns on this paragraph on what actually to leave in... may be it is just a matter of provisioning easy entry points (as was hinted on in Challenges) -In particular, we find that reexecutable article debugging in a container environment can be a significant challenge, and one which will only be more severe if such an environment is already implemented in an incipient state of the article, when analysis processes are not fully fleshed out. -To alleviate this we consider two possible solutions. -The first, high granularity, prescribes dedicated execution entry-points for individual substeps, and persistent logging of outputs at each step. -The second, pure virtualization, consists in usage of containers simply to provide access to the software management capabilities required by the target article. -High granularity maximizes container image reuse, but constrains the extent to which researchers can manipulate software tools during development. -Pure virtualization seamlessly integrates with development on a fully interactive system, but leads to bloated containers less suited to rebuilding. -In light of this we propose a combination of the above, consisting in a Make system which breaks processes down to granular steps, which may be executed by using applications either directly, or as provided via a container image. +In particular, we find that reexecutable article debugging in a container environment can be a significant challenge, and one which will only be more severe if such an environment is already implemented in the development stage of an article. +In order to provide seamless integration of both flexible development and portable reexecution, we envision a workflow system which, for each analysis step, permits either usage of locally present executables, or entry points to a container. +We implement a version of this concept for the meta-article generation, where the \texttt{make article} target which generates this article will use the local environment, and the \texttt{make container-article} executes the same code via an entry point to a \TeX{} container. % let's try to go across different levels how results should be provided for maximum re-interpretability and reuse: % e.g. a full paper text, and differences, might be needed or desired for decision making about validity of statements over the results presented in the paper diff --git a/publishing/data/paperdiff.csv b/publishing/data/paperdiff.csv index 6ca375cf..bca156ef 100644 --- a/publishing/data/paperdiff.csv +++ b/publishing/data/paperdiff.csv @@ -1,4 +1,25 @@ Page,Pixels Differ [N],Differing Pixels Proportion,Date,Environment +1,685,7.9e-05,20230920004416,podman +2,1027,0.000118,20230920004416,podman +3,685,7.9e-05,20230920004416,podman +4,7663,0.000881,20230920004416,podman +5,22483,0.002584,20230920004416,podman +6,685,7.9e-05,20230920004416,podman +7,685,7.9e-05,20230920004416,podman +8,685,7.9e-05,20230920004416,podman +9,685,7.9e-05,20230920004416,podman +10,685,7.9e-05,20230920004416,podman +11,4259,0.00049,20230920004416,podman +12,94357,0.010846,20230920004416,podman +13,612555,0.07041,20230920004416,podman +14,898519,0.10328,20230920004416,podman +15,685,7.9e-05,20230920004416,podman +16,685,7.9e-05,20230920004416,podman +17,58106,0.006679,20230920004416,podman +18,547764,0.062963,20230920004416,podman +19,543483,0.06247,20230920004416,podman +20,878267,0.100952,20230920004416,podman +21,655532,0.07535,20230920004416,podman 1,901,0.000104,20230906012936,singularity 2,1243,0.000143,20230906012936,singularity 3,901,0.000104,20230906012936,singularity @@ -20,27 +41,6 @@ Page,Pixels Differ [N],Differing Pixels Proportion,Date,Environment 19,541193,0.062207,20230906012936,singularity 20,867794,0.099748,20230906012936,singularity 21,653043,0.075064,20230906012936,singularity -1,915,0.000105,20230908162849,podman -2,1257,0.000144,20230908162849,podman -3,915,0.000105,20230908162849,podman -4,10230,0.001176,20230908162849,podman -5,10916,0.001255,20230908162849,podman -6,915,0.000105,20230908162849,podman -7,915,0.000105,20230908162849,podman -8,915,0.000105,20230908162849,podman -9,915,0.000105,20230908162849,podman -10,915,0.000105,20230908162849,podman -11,4489,0.000516,20230908162849,podman -12,84631,0.009728,20230908162849,podman -13,521230,0.059913,20230908162849,podman -14,922538,0.106041,20230908162849,podman -15,915,0.000105,20230908162849,podman -16,915,0.000105,20230908162849,podman -17,58743,0.006752,20230908162849,podman -18,534588,0.061448,20230908162849,podman -19,538606,0.06191,20230908162849,podman -20,898888,0.103322,20230908162849,podman -21,653038,0.075063,20230908162849,podman 1,685,7.9e-05,20230920004219,podman 2,1027,0.000118,20230920004219,podman 3,685,7.9e-05,20230920004219,podman @@ -62,69 +62,27 @@ Page,Pixels Differ [N],Differing Pixels Proportion,Date,Environment 19,537563,0.06179,20230920004219,podman 20,924606,0.106279,20230920004219,podman 21,460224,0.0529,20230920004219,podman -1,685,7.9e-05,20230920004416,podman -2,1027,0.000118,20230920004416,podman -3,685,7.9e-05,20230920004416,podman -4,7663,0.000881,20230920004416,podman -5,22483,0.002584,20230920004416,podman -6,685,7.9e-05,20230920004416,podman -7,685,7.9e-05,20230920004416,podman -8,685,7.9e-05,20230920004416,podman -9,685,7.9e-05,20230920004416,podman -10,685,7.9e-05,20230920004416,podman -11,4259,0.00049,20230920004416,podman -12,94357,0.010846,20230920004416,podman -13,612555,0.07041,20230920004416,podman -14,898519,0.10328,20230920004416,podman -15,685,7.9e-05,20230920004416,podman -16,685,7.9e-05,20230920004416,podman -17,58106,0.006679,20230920004416,podman -18,547764,0.062963,20230920004416,podman -19,543483,0.06247,20230920004416,podman -20,878267,0.100952,20230920004416,podman -21,655532,0.07535,20230920004416,podman -1,960,0.00011,20231019234913,singularity -2,1302,0.00015,20231019234913,singularity -3,960,0.00011,20231019234913,singularity -4,9871,0.001135,20231019234913,singularity -5,1981,0.000228,20231019234913,singularity -6,960,0.00011,20231019234913,singularity -7,960,0.00011,20231019234913,singularity -8,960,0.00011,20231019234913,singularity -9,960,0.00011,20231019234913,singularity -10,960,0.00011,20231019234913,singularity -11,4534,0.000521,20231019234913,singularity -12,85188,0.009792,20231019234913,singularity -13,557405,0.064071,20231019234913,singularity -14,938265,0.107849,20231019234913,singularity -15,960,0.00011,20231019234913,singularity -16,960,0.00011,20231019234913,singularity -17,14472,0.001663,20231019234913,singularity -18,530785,0.061011,20231019234913,singularity -19,539065,0.061963,20231019234913,singularity -20,906463,0.104193,20231019234913,singularity -21,654367,0.075216,20231019234913,singularity -1,922,0.000106,20230906053037,podman -2,1264,0.000145,20230906053037,podman -3,922,0.000106,20230906053037,podman -4,9519,0.001094,20230906053037,podman -5,11042,0.001269,20230906053037,podman -6,922,0.000106,20230906053037,podman -7,922,0.000106,20230906053037,podman -8,922,0.000106,20230906053037,podman -9,922,0.000106,20230906053037,podman -10,922,0.000106,20230906053037,podman -11,4496,0.000517,20230906053037,podman -12,84952,0.009765,20230906053037,podman -13,525931,0.060453,20230906053037,podman -14,577110,0.066336,20230906053037,podman -15,922,0.000106,20230906053037,podman -16,922,0.000106,20230906053037,podman -17,57308,0.006587,20230906053037,podman -18,533954,0.061375,20230906053037,podman -19,538338,0.061879,20230906053037,podman -20,881736,0.101351,20230906053037,podman -21,461270,0.053021,20230906053037,podman +1,915,0.000105,20230908162849,podman +2,1257,0.000144,20230908162849,podman +3,915,0.000105,20230908162849,podman +4,10230,0.001176,20230908162849,podman +5,10916,0.001255,20230908162849,podman +6,915,0.000105,20230908162849,podman +7,915,0.000105,20230908162849,podman +8,915,0.000105,20230908162849,podman +9,915,0.000105,20230908162849,podman +10,915,0.000105,20230908162849,podman +11,4489,0.000516,20230908162849,podman +12,84631,0.009728,20230908162849,podman +13,521230,0.059913,20230908162849,podman +14,922538,0.106041,20230908162849,podman +15,915,0.000105,20230908162849,podman +16,915,0.000105,20230908162849,podman +17,58743,0.006752,20230908162849,podman +18,534588,0.061448,20230908162849,podman +19,538606,0.06191,20230908162849,podman +20,898888,0.103322,20230908162849,podman +21,653038,0.075063,20230908162849,podman 1,915,0.000105,20230908122618,singularity 2,1257,0.000144,20230908122618,singularity 3,915,0.000105,20230908122618,singularity @@ -146,6 +104,27 @@ Page,Pixels Differ [N],Differing Pixels Proportion,Date,Environment 19,498795,0.057334,20230908122618,singularity 20,928814,0.106762,20230908122618,singularity 21,675041,0.077592,20230908122618,singularity +1,922,0.000106,20230906053037,podman +2,1264,0.000145,20230906053037,podman +3,922,0.000106,20230906053037,podman +4,9519,0.001094,20230906053037,podman +5,11042,0.001269,20230906053037,podman +6,922,0.000106,20230906053037,podman +7,922,0.000106,20230906053037,podman +8,922,0.000106,20230906053037,podman +9,922,0.000106,20230906053037,podman +10,922,0.000106,20230906053037,podman +11,4496,0.000517,20230906053037,podman +12,84952,0.009765,20230906053037,podman +13,525931,0.060453,20230906053037,podman +14,577110,0.066336,20230906053037,podman +15,922,0.000106,20230906053037,podman +16,922,0.000106,20230906053037,podman +17,57308,0.006587,20230906053037,podman +18,533954,0.061375,20230906053037,podman +19,538338,0.061879,20230906053037,podman +20,881736,0.101351,20230906053037,podman +21,461270,0.053021,20230906053037,podman 1,712,8.2e-05,20230921131127,singularity 2,1054,0.000121,20230921131127,singularity 3,712,8.2e-05,20230921131127,singularity @@ -167,3 +146,24 @@ Page,Pixels Differ [N],Differing Pixels Proportion,Date,Environment 19,541833,0.062281,20230921131127,singularity 20,884524,0.101671,20230921131127,singularity 21,678229,0.077959,20230921131127,singularity +1,960,0.00011,20231019234913,singularity +2,1302,0.00015,20231019234913,singularity +3,960,0.00011,20231019234913,singularity +4,9871,0.001135,20231019234913,singularity +5,1981,0.000228,20231019234913,singularity +6,960,0.00011,20231019234913,singularity +7,960,0.00011,20231019234913,singularity +8,960,0.00011,20231019234913,singularity +9,960,0.00011,20231019234913,singularity +10,960,0.00011,20231019234913,singularity +11,4534,0.000521,20231019234913,singularity +12,85188,0.009792,20231019234913,singularity +13,557405,0.064071,20231019234913,singularity +14,938265,0.107849,20231019234913,singularity +15,960,0.00011,20231019234913,singularity +16,960,0.00011,20231019234913,singularity +17,14472,0.001663,20231019234913,singularity +18,530785,0.061011,20231019234913,singularity +19,539065,0.061963,20231019234913,singularity +20,906463,0.104193,20231019234913,singularity +21,654367,0.075216,20231019234913,singularity