URL reshaping and proxying for Positron localhost URLs on Posit Workbench #4274

jmcphers · 2024-08-07T22:45:47Z

Positron currently uses localhost URLs to serve content in many places.

The Help pane, which serves R and Python help pages over a localhost HTTP server
The Plots pane, which serves interactive plots and HTML content over a localhost HTTP server
The Viewer pane, which serves interactive applications (Streamlit, Shiny, etc.) as well as HTML content over a localhost HTTP server

These URLs won't be visible to browsers by default in Workbench; they will need to get transformed into proxied Posit Workbench URLs at some point so that they can be loaded in the browser.

One way to do this might be to use the resolver API, which is part of VS Code upstream and is defined here:

https://github.com/posit-dev/positron/blob/main/src/vscode-dts/vscode.proposed.resolvers.d.ts

Using this API allows us to request that any registered resolvers process the URL before we load it in Positron. Here's an example of where we call the API on the Positron side:

positron/src/vs/workbench/services/languageRuntime/common/languageRuntimeUiClient.ts

Lines 155 to 161 in de62b4e

    
           let uri = URI.parse(url); 
        
           try { 
        
           	const resolvedUri = await this._openerService.resolveExternalUri(uri); 
        
           	uri = resolvedUri.resolved; 
        
           } catch { 
        
           	// Noop; use the original URI 
        
           }

So a solution could look like this. Here I'm using "Help Server" as an example, but this is akin to what we need to do for each local URL.

graph TD
he[Help Pane] -- localhost URL --> p[Positron]
hs[Help Server] -- localhost URL --> he
p[Positron] -- localhost URL --> pw[Posit Workbench Extension]
pw -- external URL --> p
p -- external URL --> b[Browser] 
b -- HTTP request --> w[Posit Workbench]
w -- HTTP proxy --> hp
hp[Positron Help Proxy] --> hs
hs -- help content --> hp
hp -- help content --> w
w -- help content --> b

jmcphers · 2024-08-07T22:52:01Z

There's definitely some overlap here with the existing logic in the Posit Workbench extension, which already has logic for discovering localhost URLs and creating proxies for them. We could choose to re-use some of that code, or refactor that code to use the resolver APIs.

- Part of #4274 - Adds missing files to the includes: a new file (rsLoginCheck.js) from Workbench via #4655 and the help.html help container page ### QA Notes When running Positron in Workbench, there should be no server or dev console errors around missing these files. Positron should load in Workbench. --------- Signed-off-by: sharon <sharon-wang@users.noreply.github.com> Co-authored-by: sharon wang <25834218+sharon-wang@users.noreply.github.com>

This PR adds support for local Python application development for the following frameworks: * Streamlit * Dash * Gradio * FastAPI * Flask This is facilitated by a new extension: `positron-run-app`. Addresses #4557, #4555, #3023, #3027, #3024. https://github.com/user-attachments/assets/d72e5028-a8c9-4aab-b927-4413c4e74b24 ### QA notes 1. Open an application developed using one of the supported frameworks. 2. Run the relevant "Python: Run X App in Terminal" command. It should also be the default action if you click the run button in the editor menu. 3. It should create a terminal named after the framework, and automatically show the app in the Viewer pane once its ready. Here are example apps for all of the supported frameworks: #### Streamlit ```python import streamlit as st x = st.slider('x') st.write(x, 'squared is', x * x) ``` #### Dash ```python from dash import Dash, html app = Dash() app.layout = [html.Div(children='Hello World')] if __name__ == '__main__': app.run(debug=True) ``` #### Gradio ```python import gradio as gr def image_classifier(inp): return {'cat': 0.3, 'dog': 0.7} demo = gr.Interface(fn=image_classifier, inputs="image", outputs="label") demo.launch() ``` #### FastAPI ```python from fastapi import FastAPI app = FastAPI() @app.get("/") async def root(): return {"message": "Hello World"} ``` #### Flask ```python from flask import Flask app = Flask(__name__) @app.route("/") def hello_world(): return "<p>Hello, World!</p>" ``` ### Next steps: Workbench integration To support Workbench, we'll need to update `runApplication` to detect if we're in Workbench, and if so, find a free port and corresponding proxied URL (possibly related to #4274) and pass them both to `getTerminalOptions`.

### Description - addresses part of #4274 - fixes R help docs not loading in Positron Server and Positron on Workbench #### Implementation Notes - this adds an additional content rewriting step to the positronProxy extension, to replace any root-relative urls in the help HTML (urls that start with `/`) with the proxied path, so that those resources are loaded from the correct location - the rewriting step unfortunately uses a simple regex which does not account for the full range of ways to specify a url in HTML -- it simply replaces the slash `\` in instances of `src="\` and `href="\` with the proxyPath - this rewriting seems sufficient for what R help outputs in some limited testing, and we don't currently have a way to properly parse and modify the HTML aside from string replacement, so we'll go with this for now and keep an eye on it for improvements/fixes ### QA Notes In Positron Server Web and Positron on Workbench: 1. Select an R interpreter 2. In the Console, get help for something, e.g. `?iris` and hit Enter 3. What to expect in the Help Pane: - the pane should display the help content with styling that matches the current IDE theme - links to other help pages should load and work when clicked - there shouldn't be `404 Not Found` errors in the Developer Console or Network tab for paths referenced in the help HTML The Help Pane in Positron Desktop should continue to work. Note that Python help issues on Server Web and Workbench are not addressed in this PR. --------- Co-authored-by: sharon wang <25834218+sharon-wang@users.noreply.github.com>

### Description - addresses part of #4274 - a follow-up to #4814 to fix Python help docs in Positron Server and Positron on Workbench #### Screenshots of the issue <img width="865" alt="image" src="https://github.com/user-attachments/assets/631488c4-cee6-49ec-8c9a-314830c925e7"> ![image](https://github.com/user-attachments/assets/b0594155-f357-46e6-8a61-4a3ac211f600) #### What it looks like now! <img width="1727" alt="image" src="https://github.com/user-attachments/assets/313a33a1-4698-47ac-8df2-314a6cc034df"> #### Implementation Notes - adds back in the `sourceUrl.pathname` fix that I should have had in #4814, but accidentally deleted - appends the search query to the target path, which was getting dropped in the proxied request in `src/vs/server/node/webClientServer.ts` Note: the change to `src/vs/server/node/webClientServer.ts` needs to be contributed to our upstream. ### QA Notes In Positron Server Web and Positron on Workbench: 1. Select a Python interpreter 2. In the Console, get help for something, e.g. `import time` then `?time` and hit Enter 3. What to expect in the Help Pane: - the pane should display the help content with styling that matches the current IDE theme - links to other help pages should load and work when clicked - there shouldn't be 404 Not Found errors in the Developer Console or Network tab for paths referenced in the help HTML --------- Co-authored-by: Brian Lambert <brianlambert@gmail.com> Co-authored-by: Jonathan <jonathan@rstudio.com>

### Description - part of #4274 - addresses #4804 - should unblock #4806, where the new plotly rendering method relies on a proxy server instead of an HTML file proxy #### Changes - add a new command to create a generic proxy server `positronProxy.startHttpProxyServer` - rename the command `positronProxy.stopHelpProxyServer` to `positronProxy.stopProxyServer` since it is not help-specific - rename `resources/scripts.html` to `resources/scripts_help.html` since it is help-specific - move the src/href rewriting to a private reusable function `rewriteUrlsWithProxyPath`, which is now used by the generic http proxy and the help proxy `contentRewriter` - update `src/vs/code/browser/workbench/workbench.ts` to resolve the uri while maintaining the uri's original path, query string and fragment strings (NOTE: needs to be contributed upstream) - update `src/vs/workbench/services/languageRuntime/common/languageRuntimeUiClient.ts` to choose between starting an HTML file proxy if a file is being served or a generic http proxy if the content is not a file ### QA Notes On Server Web and Workbench, running the following in the corresponding consoles: ##### Python `pip install plotly nbformat pandas` ```python import plotly.express as px fig = px.bar(x=["a", "b", "c"], y=[1, 3, 2]) fig.show() ``` ##### R `install.packages('plotly')` ```r library(plotly) fig <- plot_ly(data = iris, x = ~Sepal.Length, y = ~Petal.Length) fig ``` #### Expected Result The corresponding interactive plots should display in the plots pane and be interact-able!

### Description A follow-up to #4806 to revert `src/vs/workbench/services/languageRuntime/common/languageRuntimeUiClient.ts` back to using a proxy for interactive plots. A generic HTTP proxy was added via #4855, so server plots can be proxied appropriately in Server Web and Workbench. Related to - #4804 - #4274 - #4245 ### QA Notes On Server Web and Workbench, running the following in the corresponding consoles: ##### Python `pip install plotly nbformat pandas` ```python import plotly.express as px fig = px.bar(x=["a", "b", "c"], y=[1, 3, 2]) fig.show() ``` ##### R `install.packages('plotly')` ```r library(plotly) fig <- plot_ly(data = iris, x = ~Sepal.Length, y = ~Petal.Length) fig ``` #### Expected Result The corresponding interactive plots should display in the plots pane and be interact-able!

sharon-wang · 2024-10-02T18:54:08Z

QA Notes

Please verify interactive plots, help content and the viewer in Positron Server Web and Positron on Workbench. Python app framework proxy support is WIP, logged as #4769.

Interactive Plots

Relevant PRs and issues

Help Content

Relevant PRs and issues

Viewer

Relevant PRs and issues

No specific PRs/issues for the Viewer. Proxying issues may have been fixed as part of the help proxy fixes.

Test case I used

A more complex case would be ideal!

Open https://github.com/posit-dev/qa-example-content/tree/main/workspaces/quarto_basic in Positron. Ensure quarto is already installed.
“Preview” the quarto_basic.qmd file so that the corresponding html and files are generated, and the html is previewed in the Viewer Pane.

testlabauto · 2024-10-03T18:23:56Z

Verified Fixed

Positron Version(s) : 2024.10.0-5
OS Version          : OSX

Test scenario(s)

Quarto and shiny from qa-example-content look good!

Link(s) to TestRail test cases run or created:

### Description - addresses part of #4274 - a follow-up to #4814 to fix Python help docs in Positron Server and Positron on Workbench #### Screenshots of the issue <img width="865" alt="image" src="https://github.com/user-attachments/assets/631488c4-cee6-49ec-8c9a-314830c925e7"> ![image](https://github.com/user-attachments/assets/b0594155-f357-46e6-8a61-4a3ac211f600) #### What it looks like now! <img width="1727" alt="image" src="https://github.com/user-attachments/assets/313a33a1-4698-47ac-8df2-314a6cc034df"> #### Implementation Notes - adds back in the `sourceUrl.pathname` fix that I should have had in #4814, but accidentally deleted - appends the search query to the target path, which was getting dropped in the proxied request in `src/vs/server/node/webClientServer.ts` Note: the change to `src/vs/server/node/webClientServer.ts` needs to be contributed to our upstream. ### QA Notes In Positron Server Web and Positron on Workbench: 1. Select a Python interpreter 2. In the Console, get help for something, e.g. `import time` then `?time` and hit Enter 3. What to expect in the Help Pane: - the pane should display the help content with styling that matches the current IDE theme - links to other help pages should load and work when clicked - there shouldn't be 404 Not Found errors in the Developer Console or Network tab for paths referenced in the help HTML --------- Co-authored-by: Brian Lambert <brianlambert@gmail.com> Co-authored-by: Jonathan <jonathan@rstudio.com>

### Description - part of #4274 - addresses #4804 - should unblock #4806, where the new plotly rendering method relies on a proxy server instead of an HTML file proxy #### Changes - add a new command to create a generic proxy server `positronProxy.startHttpProxyServer` - rename the command `positronProxy.stopHelpProxyServer` to `positronProxy.stopProxyServer` since it is not help-specific - rename `resources/scripts.html` to `resources/scripts_help.html` since it is help-specific - move the src/href rewriting to a private reusable function `rewriteUrlsWithProxyPath`, which is now used by the generic http proxy and the help proxy `contentRewriter` - update `src/vs/code/browser/workbench/workbench.ts` to resolve the uri while maintaining the uri's original path, query string and fragment strings (NOTE: needs to be contributed upstream) - update `src/vs/workbench/services/languageRuntime/common/languageRuntimeUiClient.ts` to choose between starting an HTML file proxy if a file is being served or a generic http proxy if the content is not a file ### QA Notes On Server Web and Workbench, running the following in the corresponding consoles: ##### Python `pip install plotly nbformat pandas` ```python import plotly.express as px fig = px.bar(x=["a", "b", "c"], y=[1, 3, 2]) fig.show() ``` ##### R `install.packages('plotly')` ```r library(plotly) fig <- plot_ly(data = iris, x = ~Sepal.Length, y = ~Petal.Length) fig ``` #### Expected Result The corresponding interactive plots should display in the plots pane and be interact-able!

### Description A follow-up to #4806 to revert `src/vs/workbench/services/languageRuntime/common/languageRuntimeUiClient.ts` back to using a proxy for interactive plots. A generic HTTP proxy was added via #4855, so server plots can be proxied appropriately in Server Web and Workbench. Related to - #4804 - #4274 - #4245 ### QA Notes On Server Web and Workbench, running the following in the corresponding consoles: ##### Python `pip install plotly nbformat pandas` ```python import plotly.express as px fig = px.bar(x=["a", "b", "c"], y=[1, 3, 2]) fig.show() ``` ##### R `install.packages('plotly')` ```r library(plotly) fig <- plot_ly(data = iris, x = ~Sepal.Length, y = ~Petal.Length) fig ``` #### Expected Result The corresponding interactive plots should display in the plots pane and be interact-able!

### Description - addresses part of #4274 - a follow-up to #4814 to fix Python help docs in Positron Server and Positron on Workbench #### Screenshots of the issue <img width="865" alt="image" src="https://github.com/user-attachments/assets/631488c4-cee6-49ec-8c9a-314830c925e7"> ![image](https://github.com/user-attachments/assets/b0594155-f357-46e6-8a61-4a3ac211f600) #### What it looks like now! <img width="1727" alt="image" src="https://github.com/user-attachments/assets/313a33a1-4698-47ac-8df2-314a6cc034df"> #### Implementation Notes - adds back in the `sourceUrl.pathname` fix that I should have had in #4814, but accidentally deleted - appends the search query to the target path, which was getting dropped in the proxied request in `src/vs/server/node/webClientServer.ts` Note: the change to `src/vs/server/node/webClientServer.ts` needs to be contributed to our upstream. ### QA Notes In Positron Server Web and Positron on Workbench: 1. Select a Python interpreter 2. In the Console, get help for something, e.g. `import time` then `?time` and hit Enter 3. What to expect in the Help Pane: - the pane should display the help content with styling that matches the current IDE theme - links to other help pages should load and work when clicked - there shouldn't be 404 Not Found errors in the Developer Console or Network tab for paths referenced in the help HTML --------- Co-authored-by: Brian Lambert <brianlambert@gmail.com> Co-authored-by: Jonathan <jonathan@rstudio.com>

### Description - part of #4274 - addresses #4804 - should unblock #4806, where the new plotly rendering method relies on a proxy server instead of an HTML file proxy #### Changes - add a new command to create a generic proxy server `positronProxy.startHttpProxyServer` - rename the command `positronProxy.stopHelpProxyServer` to `positronProxy.stopProxyServer` since it is not help-specific - rename `resources/scripts.html` to `resources/scripts_help.html` since it is help-specific - move the src/href rewriting to a private reusable function `rewriteUrlsWithProxyPath`, which is now used by the generic http proxy and the help proxy `contentRewriter` - update `src/vs/code/browser/workbench/workbench.ts` to resolve the uri while maintaining the uri's original path, query string and fragment strings (NOTE: needs to be contributed upstream) - update `src/vs/workbench/services/languageRuntime/common/languageRuntimeUiClient.ts` to choose between starting an HTML file proxy if a file is being served or a generic http proxy if the content is not a file ### QA Notes On Server Web and Workbench, running the following in the corresponding consoles: ##### Python `pip install plotly nbformat pandas` ```python import plotly.express as px fig = px.bar(x=["a", "b", "c"], y=[1, 3, 2]) fig.show() ``` ##### R `install.packages('plotly')` ```r library(plotly) fig <- plot_ly(data = iris, x = ~Sepal.Length, y = ~Petal.Length) fig ``` #### Expected Result The corresponding interactive plots should display in the plots pane and be interact-able!

### Description A follow-up to #4806 to revert `src/vs/workbench/services/languageRuntime/common/languageRuntimeUiClient.ts` back to using a proxy for interactive plots. A generic HTTP proxy was added via #4855, so server plots can be proxied appropriately in Server Web and Workbench. Related to - #4804 - #4274 - #4245 ### QA Notes On Server Web and Workbench, running the following in the corresponding consoles: ##### Python `pip install plotly nbformat pandas` ```python import plotly.express as px fig = px.bar(x=["a", "b", "c"], y=[1, 3, 2]) fig.show() ``` ##### R `install.packages('plotly')` ```r library(plotly) fig <- plot_ly(data = iris, x = ~Sepal.Length, y = ~Petal.Length) fig ``` #### Expected Result The corresponding interactive plots should display in the plots pane and be interact-able!

jmcphers added the area: workbench Issues related to Workbench category. label Aug 7, 2024

jmcphers added this to the 2024.10.0 Pre-Release milestone Aug 7, 2024

jmcphers mentioned this issue Aug 7, 2024

Epic: Positron Server on Posit Workbench #1617

Closed

29 tasks

sharon-wang self-assigned this Aug 27, 2024

seeM mentioned this issue Sep 13, 2024

Support local Python application development #4662

Merged

jmcphers mentioned this issue Sep 4, 2024

2024.10 Pre-Release Plan #4524

Closed

sharon-wang mentioned this issue Sep 19, 2024

Include missing files in reh-web binary #4761

Merged

seeM mentioned this issue Sep 20, 2024

Integrate Python web apps with Workbench #4769

Closed

sharon-wang mentioned this issue Sep 25, 2024

Fix R help url resolution on Server Web and Workbench #4814

Merged

sharon-wang mentioned this issue Sep 27, 2024

Fix Python help url resolution on Server Web and Workbench #4837

Merged

This was referenced Sep 30, 2024

Positron Web: Various R plot packages don't work (similar issue between packages) #4804

Closed

Positron Web: Help does not work (404) #4847

Closed

Fix interactive R and Python plots on Server Web and Workbench #4855

Merged

sharon-wang mentioned this issue Oct 2, 2024

always send html plot content to a proxy server #4876

Merged

sharon-wang added area: help Issues related to Help category. area: plots Issues related to Plots category. area: viewer Issues related to Viewer category. labels Oct 3, 2024

testlabauto closed this as completed Oct 3, 2024

github-actions bot locked as resolved and limited conversation to collaborators Oct 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

URL reshaping and proxying for Positron localhost URLs on Posit Workbench #4274

URL reshaping and proxying for Positron localhost URLs on Posit Workbench #4274

jmcphers commented Aug 7, 2024 •

edited

Loading

jmcphers commented Aug 7, 2024

sharon-wang commented Oct 2, 2024

testlabauto commented Oct 3, 2024

URL reshaping and proxying for Positron localhost URLs on Posit Workbench #4274

URL reshaping and proxying for Positron localhost URLs on Posit Workbench #4274

Comments

jmcphers commented Aug 7, 2024 • edited Loading

jmcphers commented Aug 7, 2024

sharon-wang commented Oct 2, 2024

QA Notes

Interactive Plots

Relevant PRs and issues

Help Content

Relevant PRs and issues

Viewer

Relevant PRs and issues

Test case I used

testlabauto commented Oct 3, 2024

Verified Fixed

Test scenario(s)

jmcphers commented Aug 7, 2024 •

edited

Loading