Output in jupyter cells gets truncated

[ARF1]

Issue Type: Bug

Python environment set up with: `conda -n numba47 -c numba python=3 numba jupyter

The output of very long jupyter output cells gets truncated at the beginning.

Code to reproduce, run the following in "Python Interactive":

# %%
from numba import njit, jit
import numpy as np
from llvmlite import binding as llvm

llvm.set_option("", "--debug-only=loop-vectorize")


@njit(fastmath=True)
def foo(x):
    acc = 0
    for v in np.nditer(x):
        acc += v.item()
    return acc


a = np.ones((100,), dtype=np.float32)
foo(a)

print(foo.inspect_llvm(foo.signatures[0]))

# %%

The output should have 695 lines with 40279 characters:

LV: Checking a loop in "_ZN8__main__7foo$241E5ArrayIfLi1E1C7mutable7alignedE" from foo
LV: Loop hints: force=? width=0 unroll=0
LV: Found a loop: B20

<... snipped ...>

!7 = distinct !{!7, !"_ZN8__main__7foo$241E5ArrayIfLi1E1C7mutable7alignedE"}
!8 = distinct !{!8, !1}
!9 = distinct !{!9, !3, !1}

Output has "only" 371 lines with 20368 characters:

i8, i8* %29, i64 %sunkaddr81
  %sunkaddr83 = getelementptr i8, i8* %sunkaddr82, i64 112
  %30 = bitcast i8* %sunkaddr83 to <4 x float>*

<... snipped ...>

!7 = distinct !{!7, !"_ZN8__main__7foo$242E5ArrayIfLi1E1C7mutable7alignedE"}
!8 = distinct !{!8, !1}
!9 = distinct !{!9, !3, !1}

Note that it gets truncated at the beginning.

Also: I verified that jupyter used without vscode DOES show the complete output.

Extension version: 2019.11.50794
VS Code version: Code 1.41.1 (26076a4de974ead31f97692a0d32f90d735645c0, 2019-12-18T14:58:56.166Z)
OS version: Windows_NT x64 10.0.17763

System Info

Item	Value
CPUs	Intel(R) Core(TM) i5-5300U CPU @ 2.30GHz (4 x 2295)
GPU Status	2d_canvas: enabled flash_3d: enabled flash_stage3d: enabled flash_stage3d_baseline: enabled gpu_compositing: enabled metal: disabled_off multiple_raster_threads: enabled_on oop_rasterization: disabled_off protected_video_decode: unavailable_off rasterization: enabled skia_renderer: disabled_off surface_control: disabled_off surface_synchronization: enabled_on video_decode: enabled viz_display_compositor: enabled_on viz_hit_test_surface_layer: disabled_off webgl: enabled webgl2: enabled
Load (avg)	undefined
Memory (System)	7.88GB (2.58GB free)
Process Argv
Screen Reader	no
VM	0%

[IanMatthewHuff]

Hey @ARF1 . Looking into this now. Currently with your repro I'm getting the same results in both VSCode and in Jupyter. But my results are a bit shorter.

181 lines long

; ModuleID = 'foo'
source_filename = "<string>"
target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-pc-windows-msvc"

...

!0 = !{!"branch_weights", i32 1, i32 99}
!1 = !{!"branch_weights", i32 99, i32 1}
!2 = !{!3}
!3 = distinct !{!3, !4, !"_ZN8__main__7foo$241E5ArrayIfLi1E1C7mutable7alignedE: %retptr"}
!4 = distinct !{!4, !"_ZN8__main__7foo$241E5ArrayIfLi1E1C7mutable7alignedE"}

However I also tried with just a random big string (the text of a novel) and I am seeing truncation here that I'm not seeing in Jupyter. We'll look into this. Thanks for reporting.

[ARF1]

@IanMatthewHuff Thanks for taking the time to let me know.

Not sure if it is related but the output in the vscode integrated shell (cmd) also gets truncated, though at the end if I remember correctly. (Have not tried with powershell.)

I have been meaning to file a separate report but have not gotten around to it yet. Thought I would let you know just in case it helps...

[IanMatthewHuff]

Quick notes for triage:

Jupyter output with big_string = text of novel Moby Dick

VSCode Notebook editor (Interactive Window with same issue). Massive horz scrollbar on raw variable + truncation. Truncation on print of raw variable

Looks like 384 lines in this case, close to the amount @ARF1 also reported.

[IanMatthewHuff]

@ARF1 . My apologies. I totally forgot that we did have a setting to limit text output to help with performance issues.

In the VS Code settings can you change Python->Data Science->Text Output Limit to 0?

Does that work for you?

[ARF1]

@IanMatthewHuff Sorry it took a while. I was hit with the bug in the new version of the python extension where jupyter was refusing to connect.

The setting Python->Data Science->Text Output Limit to 0 fixes the issue.

That said, I think it is bad form to drop output without any warning to the user. It took me ages to figure out what the problem was: I was studying assembler code that I could not figure out how it could possibly work. - Turns out, it could not work as it was displayed. Let's say it was a painful experience. ;-)

Would it be possible to print a warning to the user that some output was dropped? E.g. as a line in the text output cell at the beginning where the text is actually missing? That way, those users that do scroll back are never bothered but those that are searching for something at the beginning are warned.

[IanMatthewHuff]

@ARF1 Yeah I'll leave this issue open and we will discuss in triage. Honestly I had forgotten the setting myself and was initially confused as well. This was to work with some specific performance issues so we might not be able to fix full output right away, but as you mentioned it we could tweak our messaging some.

[ARF1]

@IanMatthewHuff
For what it's worth, I think that fixing full output is not urgent IF users get a message that their output was truncated and how to "fix" it themselves in the settings.

[greazer]

@IanMatthewHuff , please file a bug for the horizontal scrollbar issue in your image above.

[IanMatthewHuff]

@greazer microsoft/vscode-jupyter#1548

[rchiodo]

Validated