Collection of experiments on programming and source code

This repo contains some N-of-1 experiments. You can start each experiment just by clicking on the links in the list below.

In case you are interested in N-of-1 experiments in programming and software science, you might be interested in the following paper (open access):

Hanenberg, Mehlhorn, "Two N-of-1 self-trials on readability differences between anonymous inner classes (AICs) and lambda expressions (LEs) on Java code snippets", Empir. Softw. Eng. 27(2): 33 (2022)

Each experiment generates a csv file at the end of the experiment. The present code does not do the analysis automatically (just load the csv into your preferred stats program and run an ANOVA).

Note that this is NOT the source code repository for the software used in each experiment (which can be found here) - the experiments just use some versions of the mentioned repository in their file lib.js (probably in different version - I will not keep the experiments in sync with the code repo in order not to change experiments that were already executed). The present repository is the source code repository for the experiment definitions. However, if you still want to write your own experiments, take a look at this link.

In case you have questions or comments, just drop me an email.

-Stefan

1. Indentation in Source Code

1.1 Nested ifs

[click here to start the experiment] The experiment should take (without training) about 20-30 minutes. It is is a remake of the experiment described in:

Hanenberg, Morzeck, Gruhn, "Indentation and reading time: a randomized control trial on the differences between generated indented and non-indented if-statements", Empirical Software Engineering, Vol 29, No 5, 2024.

There is a predecessor of that experiment where manually chosen tasks were given to participants:

Morzeck, Hanenberg, Werger, Gruhn, "Intendation in Source Code: A Randomized Control Trial on the Readability of Control Flows in Java Code with Large Effects", ICSoft 2023

1.2 JSON-Objects

The experiment should take (without training) about 20-30 minutes. click here start the experiment The experiment tests the readability time of JSON-objects by asking the single quesion "how many fields does the outermost object have?". The experiment is described in:

Hanenberg,Morzeck, Werger, Gries, Gruhn, "Indentation and Reading Time: A Controlled Experiment on the Differences Between Generated Indented and Non-indented JSON Objects", Communications in Computer and Information Science 2104, Springer, 2024, pp. 50-75.

3. Type Systems - constructor calls

• Flat experiment: click here to start the experiment
• Hierarchical experiment: click here to start the experiment

4. Type System DSL

click here to start the experiment This experiment is a follow-up study of the one described in

Klanten, Hanenberg, Gries, Gruhn: Readability of Domain-Specific Languages: A Controlled Experiment Comparing (Declarative) Inference Rules with (Imperative) Java Source Code in Programming Language Design. ICSOFT 2024: 492-503

The results of the experiment are described in the paper

Hanenberg, Klanten, Gries, Gruhn, Readability of Domain-Specific Languages in Language Design: A Controlled Experiment on Declarative Inference Rules and Imperative Java Source Code (accepted for publication in Communications in Computer and Information Science -- to appear).

5. Strings with Questionmarks: Backslash, Composition, Containment

click here start the experiment

Checks how strings should be composed.

The experiment randomly generated expressions and it is up to the participant to decide whether the (Boolean) expression is valid. The different string representations are

• Escaping: strings might have escaped quotation marks)
• Containment: the outer string uses ' while inner string use " - similar to JavaScript)
• Composition: a string with quotation marks is a string composition (using + as a concat operator) where a variable QUOTE represents the quotation mark.

It is a 3x3 experiment with 4 repetitions per treatment combination - the first variable is the style (Excaping, etc.), the second one is the position where one decides whether or not the expression is correct (-1 means that the whole string needs to be read).

Probably the big result of the study is: string composition is probably the worst choice to do for string generation.

4. CamelCase, Snake_Case

click here start the experiment