Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added a buffer for unfinished lines during progress bar printing. the lines will be completed by the next print, or way at the end when all progress bars are done. #1951

Merged
merged 8 commits into from
May 24, 2024
14 changes: 14 additions & 0 deletions src/org/rascalmpl/library/util/Monitor.rsc
Original file line number Diff line number Diff line change
Expand Up @@ -199,4 +199,18 @@ test bool simpleAsyncPrintTest() {
println("d");
jobEnd("job");
return true;
}

test bool unfinishedInputTest() {
jobStart("job", totalWork=26);
for (/<l:[a-z]>/ := "abcdefghijklmnopqrstuwvxyz") {
print(l); // no newline!
jobStep("job", "letter <l>", work=1);
if (arbInt(10) == 0) {
println(); // break it
}
}
// println(); // flush it
jobEnd("job");
return true;
}
135 changes: 131 additions & 4 deletions src/org/rascalmpl/repl/TerminalProgressBarMonitor.java
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
import java.nio.charset.Charset;
import java.time.Duration;
import java.time.Instant;
import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;

Expand Down Expand Up @@ -36,6 +37,13 @@
*/
private List<ProgressBar> bars = new LinkedList<>();

/**
* We also keep a list of currently unfinished lines (one for each thread).
* Since there are generally very few threads a simple list beats a hash-map in terms
* of memory allocation and possibly also lookup efficiency.
*/
private List<UnfinishedLine> unfinishedLines = new ArrayList<>(3);

Check warning on line 45 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L45

Added line #L45 was not covered by tests

/**
* This writer is there to help with the encoding to what the terminal needs. It writes directly to the
* underlying stream.
Expand Down Expand Up @@ -109,6 +117,96 @@
}
}

private class UnfinishedLine {
jurgenvinju marked this conversation as resolved.
Show resolved Hide resolved
final long threadId;
int curCapacity = 512;
byte[] buffer = new byte[curCapacity];
int curEnd = 0;
int lastNewLine = 0;

Check warning on line 125 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L122-L125

Added lines #L122 - L125 were not covered by tests
jurgenvinju marked this conversation as resolved.
Show resolved Hide resolved

public UnfinishedLine() {
this.threadId = Thread.currentThread().getId();
}

Check warning on line 129 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L127-L129

Added lines #L127 - L129 were not covered by tests

/**
* Adding input combines previously unfinished sentences with possible
* new sentences. A number of cases come together here that otherwise
* should be diligently separated.
*
* - An unfinished line can already exist or not
* - The new input can contain newlines or not
* - The new input can end with a newline character or not
*
* By concatenating the previous unfinished line with the new input
* all we have to do now is figure out where the last newline character is.
*
*/
private void add(byte[] newInput, int offset, int len) {
// first ensure capacity of the array
if (curEnd + len >= curCapacity) {
var oldCapacity = curCapacity;
curCapacity *= 2; // this should not happen to often. we're talking a few lines of text.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we should grow by 50%?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the chance of having to grow again is large if we only grow by 50%. This way we quickly grow and we don't grow again on short notice.

byte[] tmp = new byte[curCapacity];
System.arraycopy(buffer, 0, tmp, 0, oldCapacity);
jurgenvinju marked this conversation as resolved.
Show resolved Hide resolved
buffer = tmp;

Check warning on line 151 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L147-L151

Added lines #L147 - L151 were not covered by tests
}

System.arraycopy(newInput, offset, buffer, curEnd, len);
curEnd += len;
lastNewLine = startOfLastLine();
jurgenvinju marked this conversation as resolved.
Show resolved Hide resolved
}

Check warning on line 157 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L154-L157

Added lines #L154 - L157 were not covered by tests

public void write(byte[] n, OutputStream out) throws IOException {
add(n, 0, n.length);
flushToLastLine(out);
}

Check warning on line 162 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L160-L162

Added lines #L160 - L162 were not covered by tests

public void write(byte[] n, int offset, int len, OutputStream out) throws IOException {
add(n, offset, len);
flushToLastLine(out);
}

Check warning on line 167 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L165-L167

Added lines #L165 - L167 were not covered by tests

private void flushToLastLine(OutputStream out) throws IOException {
if (lastNewLine != -1) {
// write everything out (except the last unfinished line), but including the last newline
out.write(buffer, 0, lastNewLine + 1);

Check warning on line 172 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L172

Added line #L172 was not covered by tests

if (lastNewLine + 1 == curEnd) {
// nothing left
curEnd = 0;
lastNewLine = -1;

Check warning on line 177 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L176-L177

Added lines #L176 - L177 were not covered by tests
}
else {
// copy the last line that does not end with a newline
curEnd -= lastNewLine;
System.arraycopy(buffer, lastNewLine + 1, buffer, 0, curEnd);
lastNewLine = -1; // no newline anymore

Check warning on line 183 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L181-L183

Added lines #L181 - L183 were not covered by tests
}
}

// otherwise we wait until the next input comes to be able to complete a line.
}

Check warning on line 188 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L188

Added line #L188 was not covered by tests

public void writeLeftOvers(OutputStream out) throws IOException {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: rename to flush?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not the same as a flush because it also adds newlines. So I have renamed it.

if (curEnd != 0) {
out.write(buffer, 0, curEnd);
out.write('\n');
curEnd = 0;
lastNewLine = -1;

Check warning on line 195 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L192-L195

Added lines #L192 - L195 were not covered by tests
}
}

Check warning on line 197 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L197

Added line #L197 was not covered by tests

private int startOfLastLine() {
for (int i = curEnd - 1; i >= 0; i--) {
if (buffer[i] == '\n') {
return i;

Check warning on line 202 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L202

Added line #L202 was not covered by tests
}
}

return -1;

Check warning on line 206 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L206

Added line #L206 was not covered by tests
}
}

/**
* Represents one currently running progress bar
*/
Expand Down Expand Up @@ -234,7 +332,7 @@
+ " "
;

writer.println("\r" + line); // note this puts us one line down
writer.println(line); // note this puts us one line down

Check warning on line 335 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L335

Added line #L335 was not covered by tests
}

private String threadLabel() {
Expand Down Expand Up @@ -395,6 +493,21 @@
.filter(b -> b.threadId == Thread.currentThread().getId())
.filter(b -> b.name.equals(name)).findFirst().orElseGet(() -> null);
}

private UnfinishedLine findUnfinishedLine() {
UnfinishedLine before = unfinishedLines.stream()

Check warning on line 498 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L498

Added line #L498 was not covered by tests
.filter(l -> l.threadId == Thread.currentThread().getId())
.findAny()
.orElse(null);

Check warning on line 501 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L500-L501

Added lines #L500 - L501 were not covered by tests

if (before == null) {
UnfinishedLine l = new UnfinishedLine();
jurgenvinju marked this conversation as resolved.
Show resolved Hide resolved
unfinishedLines.add(l);
before = l;

Check warning on line 506 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L504-L506

Added lines #L504 - L506 were not covered by tests
}

return before;

Check warning on line 509 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L509

Added line #L509 was not covered by tests
}

@Override
public synchronized void jobStart(String name, int workShare, int totalWork) {
Expand Down Expand Up @@ -483,12 +596,18 @@
* Here we make sure the progress bars are gone just before
* someone wants to print in the console. When the printing
* is ready, we simply add our own progress bars again.
*
* Special cases handle when there are unfinished lines in play.
* This code guarantees that printBars is only called when the cursor
* is before the first character of a line.
*/
@Override
public synchronized void write(byte[] b) throws IOException {
if (!bars.isEmpty()) {
eraseBars();
out.write(b);

findUnfinishedLine().write(b, out);

Check warning on line 609 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L609

Added line #L609 was not covered by tests

printBars();
}
else {
Expand All @@ -505,7 +624,7 @@
public synchronized void write(byte[] b, int off, int len) throws IOException {
if (!bars.isEmpty()) {
eraseBars();
out.write(b, off, len);
findUnfinishedLine().write(b, off, len, out);

Check warning on line 627 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L627

Added line #L627 was not covered by tests
printBars();
}
else {
Expand All @@ -522,7 +641,7 @@
public synchronized void write(int b) throws IOException {
if (!bars.isEmpty()) {
eraseBars();
out.write(b);
findUnfinishedLine().write(new byte[] { (byte) b }, out);

Check warning on line 644 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L644

Added line #L644 was not covered by tests
printBars();
}
else {
Expand All @@ -541,6 +660,14 @@
}

bars.clear();
for (UnfinishedLine l : unfinishedLines) {
try {
l.writeLeftOvers(out);

Check warning on line 665 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L665

Added line #L665 was not covered by tests
}
catch (IOException e) {

Check warning on line 667 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L667

Added line #L667 was not covered by tests
// might happen if the terminal crashes before we stop running
}
}

Check warning on line 670 in src/org/rascalmpl/repl/TerminalProgressBarMonitor.java

View check run for this annotation

Codecov / codecov/patch

src/org/rascalmpl/repl/TerminalProgressBarMonitor.java#L669-L670

Added lines #L669 - L670 were not covered by tests
writer.write(ANSI.showCursor());
writer.flush();
}
Expand Down
1 change: 0 additions & 1 deletion src/org/rascalmpl/test/infrastructure/TestFramework.java
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@
import java.util.Set;

import org.junit.After;
import org.rascalmpl.debug.IRascalMonitor;
import org.rascalmpl.interpreter.Evaluator;
import org.rascalmpl.interpreter.env.GlobalEnvironment;
import org.rascalmpl.interpreter.env.ModuleEnvironment;
Expand Down
2 changes: 0 additions & 2 deletions src/org/rascalmpl/uri/file/UNCResolver.java
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@

import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.regex.Pattern;

import io.usethesource.vallang.ISourceLocation;

/**
Expand Down
2 changes: 0 additions & 2 deletions test/org/rascalmpl/MatchFingerprintTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,6 @@
import org.rascalmpl.interpreter.env.GlobalEnvironment;
import org.rascalmpl.interpreter.env.ModuleEnvironment;
import org.rascalmpl.test.infrastructure.RascalJUnitTestRunner;
import org.rascalmpl.test.infrastructure.TestFramework;

import io.usethesource.vallang.IConstructor;
import io.usethesource.vallang.IInteger;
import io.usethesource.vallang.ISourceLocation;
Expand Down
Loading