Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Monthly Performance Meetings: 2022 #88

Closed
fcollonval opened this issue Jan 19, 2022 · 10 comments
Closed

Monthly Performance Meetings: 2022 #88

fcollonval opened this issue Jan 19, 2022 · 10 comments
Labels
documentation Improvements or additions to documentation

Comments

@fcollonval
Copy link
Member

fcollonval commented Jan 19, 2022 •
edited
Loading

Here is a record of the agenda/minutes for our monthly meeting. Everyone is welcome to join.

Let's avoid using this thread for discussion. If you'd like to discuss something in the minutes, open a separate issue and reference this thread.

Meeting Notes
@fcollonval fcollonval added the documentation Improvements or additions to documentation label Jan 19, 2022
@fcollonval fcollonval pinned this issue Jan 19, 2022
@fcollonval
Copy link
Member Author

19 Jan 2022 Monthly meeting

Attendees

Name affiliation github username
Frederic Collonval QuantStack @fcollonval
William Stein CoCalc/SageMath @williamstein
Marc Udoff DeShaw
Paul Ivanov Noteable @ivanov

Notes

  • Marc

    • Limit outputs work done
      It will probably become open-source in the near future
      Make long notebook more manageable

  • Frederic

    • Search extension: PR - Rework to carry search on the model
      • Document decision done: like ignoring the link markdown
      • Keep the GenericSearch for not supported outputs
      • william: encourage you to document the choices related to the subtle things like links, math in markdown, ansi output messages. I would be thrilled if there was an official jupyter specification of what search should do, and then client implementors could try to follow that spec.
      • Output messages is by far the hardest part, due to lack of control over them.
      • Regular expressions supported? Yes.
      • Look at other tools like VSCode to see what they are doing

@fcollonval fcollonval mentioned this issue Jan 19, 2022
Closed
@fcollonval
Copy link
Member Author

16 Feb 2022 Monthly meeting

Attendees

Name affiliation github username
Eric Charles @echarles
Frederic Collonval QuantStack @fcollonval
William Stein SageMath/CoCalc @williamstein

Notes

@fcollonval
Copy link
Member Author

fcollonval commented Mar 17, 2022 •
edited
Loading

16 March 2022

Attendees

Name affiliation github username
Frederic Collonval QuantStack @fcollonval
Eric Charles Datalayer @echarles
Marc Udoff D. E. Shaw @mlucool
William Stein SageMath/CoCalc @williamstein

Notes

  • Frederic

    • Final version of search from model
      At the end the search on output is still on the HTML. To hard to figure out what will be in the view (after testing mainly on Markdown).

  • Marc

    • ad blocker hit performance in Chrome really bad - we should document and shout out that information
    • Works by rtrg: 
    %%javascript
function addCss(cssCode) {
    const style = document.createElement('style');
    style.innerHTML = cssCode;
    document.head.appendChild(style);
}

addCss(`
    .jp-NotebookPanel.jp-Document {
        will-change: transform;
        transform: none !important;
    }
    .jp-NotebookPanel.jp-Document.lm-Widget.lm-mod-hidden {
        transform: scaleX(0) !important;
        display: block !important;
    }
`);

// This uses experimental CSS that is not supported by all browsers.
// By setting content-visibility: auto, when the cell is off-screen,
// style checks are avoided. We set an intrinsic height so that
// scrolling is not too jittery.
addCss(`
    .jp-Cell {
        content-visibility: auto !important;
        contain-intrinsic-height: auto 70px;
    }
`);

    The above shows great improvements.

    • Try hidding the panels with CSS containment for speeding-up tab switching. It makes rendering of new tabs slower but then switch faster.
    • jupyterlab-limit-output can help user for with those long outputs
      User can blow their browser by memory size. There may be a memory leak somewhere - but enable to isolate it for now.
    • Loading static assets Jupyter Server issue
      In Cocalc the assets are loaded using NGinx

  • Eric

    • Work on the kernel: On kernel area, recent work on kernel message parsing is merged Protocol alignment jupyter-server/jupyter_server#657
    • Virtual deferred rendering:
      • renderCellOnIdle improvements by waiting for long enough idle time #12166
      • Working on a PR to block cell list edition when the notebook is not yet fully rendered.
      • Probably gonna revert the setting to not activate virtual rendering by default

@fcollonval
Copy link
Member Author

16 April 2022

Attendees

Name affiliation github username
Frederic Collonval QuantStack @fcollonval
Marc Udoff D. E. Shaw & Co. @mlucool
William Stein CoCalc @williamstein

Notes

@fcollonval
Copy link
Member Author

11 May 2022

Attendees

Name affiliation github username
Jason Grout Databricks @jasongrout
WilliamStein CoCalc @williamstein
Marc Udoff D. E. Shaw & Co @mlucool
Pete Blois Google
Frederic Collonval QuantStack @fcollonval

Notes

  • Frederic

    • Windowing notebooks PR
      • To deal with outputs internal state, the current envision solution is to keep the outputs in the DOM.
      • iframe does not support to being moved in the DOM tree.

  • William

    • Windowing using Virtuoso is live in cocalc, and also used everywhere in cocalc that involves lists.
    • Solved the iframe state problem
      The solution used is to place those elements in the header element (Virtuoso feature) and move it at the right position
      when the cell is in the view port.
    • Also solved for ipywidgets - k3d at least.
    • [Pete] Do you deal with jumping to link within the document? It is not supported in Cocalc. (but should be...)

@fcollonval
Copy link
Member Author

8 June 2022

Attendees

Name affiliation github username
Frederic Collonval QuantStack @fcollonval
Jason Grout Databricks @jasongrout
William Stein CoCalc/SageMath @williamstein

Notes

  • Virtualization in CoCalc:

    • I finally got one user complaint about my windowing implementation. %%html blocks with styles... the style isn't global and disappears when the style block leaves the DOM.
    • Solution: scan the cells to move those in the header hidden cell to be available globally.
      • Currently styles are accumulated, and not updated as the html block changes. This does seem to satisfy users, though.
      • But, that could change, actually. Just an issue of time.
      • Other comment: i didn't worry about style blocks in markdown html... yet.

  • Frederic

    • Windowing notebooks PR

    Drag-and-drop with copy: we should not detach the source cell
    What about the case of %%html
    What about RTC structure change like added cell
    Listing to DOM events to see if the iframe is pulled out when scrolling into a specific position

    • william UX question: how do you enable/disable windowing; thoughts about scope of this?
      Currently there is a single user settings for all documents.
      But it is propagated to all notebook documents. So it will be
      easy to set it per document (same pattern as line numbers e.g.).
      About storing it within the document (aka notebook metadata);
      I did not thought about it and I am a bit reluctant to store
      view parameters in the document model.
      [william] The windowing parameter is usually conditioned by
      the document content (e.g. the type of outputs used). So it
      would make sense to store it in the document.

    • william question 2: any relations/issues between RTC and your virtualization implimentation?
      It has not been tested in RTC context.

    • william: I didn't know about "scroll to input"...

    • Links for the browser supporting virtualization:

    • Memory leak benchmark
      Start using fuite
      Trying BLeak - failed to make it works

@fcollonval
Copy link
Member Author

6 July 2022

Attendees

Name affiliation github username
Frederic Collonval QuantStack @fcollonval
William Stein CoCalc/SageMath @williamstein

Notes

  • Frederic
    • Performance:
      • CodeMirror 6 alone will bring a great boost: see benchmark
    • Memory leaks:

Discussion of how to deal with memory leaks happened.

What cocalc does is this. But WeakRef's can be even better...

export function close(obj: object, omit?: Set<string>): void {
  if (omit != null) {
    Object.keys(obj).forEach(function (key) {
      if (omit.has(key)) return;
      if (typeof obj[key] == "function") return;
      delete obj[key];
    });
  } else {
    Object.keys(obj).forEach(function (key) {
      if (typeof obj[key] == "function") return;
      delete obj[key];
    });
  }
}

This is similar to the disposable interface in Lumino. But with a more radical approach as it deletes all keys.

@fcollonval
Copy link
Member Author

3 August 2022

Attendees

Name affiliation github username
Frederic Collonval QuantStack @fcollonval
A. T. Darian QuantStack @afshin
M Krassowski @krassowski

Notes

@fcollonval
Copy link
Member Author

28 September 2022

Attendees

Name affiliation github username
Frederic Collonval QuantStack @fcollonval
M Krassowski @krassowski
William Stein CoCalc/SageMath @williamstein

Notes

  • Frederic
    • Windowing notebooks PR is merged
      It can be tested on the pre release pip install --pre jupyterlab
    • Revert using transform scale for switching tabs JupyterLab PR
      It was crashing the UI with tabs containing iframe (such as documentation tab) in Chrome-based browser. So this is now adviced to be used only on Firefox (not tested on webkit)
      To be released in 3.4.8
  • Mike
    • looking at menu performance

@fcollonval
Copy link
Member Author

26 October 2022

Attendees

Name affiliation github username
Frederic Collonval QuantStack @fcollonval
A. T. Darian QuantStack @afshin
M Krassowski Quansight @krassowski
William Stein CoCalc/SageMath @williamstein
Vidar T Fauske JP Morgan Chase @vidartf

Notes

  • [Mike] UI benchmark tool: PR
  • [Frederic] Benchmarks project is currently broken due to unsupported syntax (nullish assignment operator) in vega-lite by older typescript version
  • [Darian] Note about MessageLoop internal schedule implementation in Lumino 2
    In Lumino 2, the messages are gonna be processed in micro-tasks (no more in macro-tasks like in Lumino 1 due to setImmediate or requestAnimationFrame usage). So in pathologic case, this can create infinite loop (e.g. if an widget update method calls itself recursively).
  • [Vidar] Suggestion for adding more realistic benchmarking case with multiple large files open: Having other large notebooks open while tab switching #124
  • [Mike]

@fcollonval fcollonval mentioned this issue Jan 19, 2023
Open
@fcollonval fcollonval unpinned this issue Jan 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@fcollonval