Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connect to jupyter kernels #2791

Merged
merged 162 commits into from
Apr 6, 2023
Merged
Changes from 1 commit
Commits
Show all changes
162 commits
Select commit Hold shift + click to select a range
9c417b7
update tool feeds (#2258)
brettfo Sep 14, 2022
d3cfde0
Merge pull request #2296 from shibbas/main
brettfo Sep 28, 2022
18651c5
Update all usages of NetCore* Pool providers to -Svc version since th…
MattGal Sep 29, 2022
6fb1e12
Merge pull request #2340 from shibbas/main
brettfo Oct 11, 2022
d46276d
Merge pull request #1 from shibbas/main
shibbas Oct 13, 2022
ea6382f
Merge pull request #2364 from shibbas/release/jupyter
brettfo Oct 19, 2022
79ae94b
Merge remote-tracking branch 'upstream/main' into merge-main-to-relea…
brettfo Oct 19, 2022
c457349
Merge pull request #2365 from dotnet/merge-main-to-release-jupyter
brettfo Oct 19, 2022
54c3c8e
add kernel name to Jupyter metadata (#2362)
jonsequitur Oct 20, 2022
0c88f8f
align TabularDataResource formatting for json to table-schema+json
jonsequitur Oct 20, 2022
df7a82b
update System.Security.Cryptography.Xml package
brettfo Oct 24, 2022
d0d1f84
introduce jupyter kernel and handle communication over the http
shibbas Jul 21, 2022
6507287
move to use Proxy kernel
shibbas Jul 22, 2022
0a2fbde
add more message observable extensions
shibbas Jul 22, 2022
6dc8e33
handle protocol serialization exceptions
shibbas Jul 26, 2022
98421b0
basic variable sharing support added.
shibbas Jul 26, 2022
0649e47
refactor to separate jupyter connection from jupyter kernel connectio…
shibbas Jul 29, 2022
41c6d85
make command handling using dynamic command handler registers to allo…
shibbas Jul 29, 2022
930f58e
add completion for juptyer kernels
shibbas Jul 30, 2022
786f7f9
add basic support for hover text. NOTE: currently the text is rendere…
shibbas Jul 30, 2022
9a77b5b
add support to set dataframe in python and refactor to initialize a v…
shibbas Aug 1, 2022
90f4fd1
fix set data frame in R
shibbas Aug 1, 2022
6e682b3
intialize value handling properly as the kernel is always expected to…
shibbas Aug 1, 2022
1e591dc
enable connections that require a bearer token over token auth
shibbas Aug 4, 2022
7f68fdd
python comm based value adapter
shibbas Aug 8, 2022
8689db6
enable R for data sharing via comm adapter
shibbas Aug 9, 2022
984cf12
add get request support
shibbas Aug 9, 2022
7e3ebe6
add support for get variables to python
shibbas Aug 9, 2022
e08bffa
add get variable for R
shibbas Aug 9, 2022
3f7e640
add request value info for all kernels
shibbas Aug 10, 2022
01645ec
initial zmq connection logic
shibbas Aug 19, 2022
747e73e
fix rebase issues.
shibbas Aug 25, 2022
8aa4c39
normalize line endings and change namings
shibbas Sep 30, 2022
f9d7cd5
update with new code changes.
shibbas Oct 12, 2022
4aed351
properly tag the new files
shibbas Oct 12, 2022
95a741d
fix R to python dataframe marshalling
shibbas Oct 17, 2022
888d568
remove value declarer and use sendvalue command instead.
shibbas Oct 19, 2022
3d15fb6
fix to use the TabularDataResourceFormatter instead of using default …
shibbas Oct 21, 2022
3046134
cleanup some of the message handling and fix kernel ids and URIs
shibbas Oct 25, 2022
c0ca5c4
refactor names to make them more understandable
shibbas Oct 25, 2022
6c9644a
fix normalized line endings to not be public contract
shibbas Oct 25, 2022
a8318b0
add completions for list kernel specs. Cannot parse shared variable t…
shibbas Oct 25, 2022
1d69639
fix so that we do null check for kernel specs.
shibbas Oct 26, 2022
995a1d1
set up ZMQ connections to look up directories correctly
shibbas Oct 26, 2022
818589a
pr feedback on names and type updates
shibbas Oct 27, 2022
f3668d7
Merge pull request #2404 from shibbas/main
brettfo Oct 31, 2022
cb4223b
Detect and report on invalid variable identifiers. Also inform of var…
shibbas Oct 27, 2022
03a5c7a
enable update_display_data
shibbas Oct 27, 2022
66775b3
rename exception to start
shibbas Oct 27, 2022
e663bcf
strip away unsupported formats coming from the kernel on documentation
shibbas Nov 1, 2022
17407df
refactor jupyter http connection
shibbas Oct 31, 2022
0988cd0
Refactor from Proxy kernel to inherit from kernel
shibbas Nov 2, 2022
7fbca7b
fix the api client names
shibbas Nov 2, 2022
30e6541
fix submit code to wait on executereply as well
shibbas Nov 2, 2022
35d55b4
refactor out unnecessary code
shibbas Nov 2, 2022
84d84c3
change run on kernel to async method
shibbas Nov 2, 2022
0ba659b
handle kernel interrupts on cancel
shibbas Nov 2, 2022
f6a52af
Merge pull request #2413 from shibbas/refactor-to-jupyterkernel
colombod Nov 2, 2022
f10e4d1
move process to kernel connection
shibbas Nov 3, 2022
34a953f
resolve merge conflicts with main
shibbas Nov 3, 2022
db5e5aa
Merge pull request #2421 from shibbas/merge-main-to-release-jupyter
brettfo Nov 3, 2022
1fa7f7b
add other channels to send to in zmq
shibbas Nov 3, 2022
2a945e2
use kernelspec list to locate local kernels. There is a perf impact t…
shibbas Nov 6, 2022
a9fcd9d
add comm channel lookup. Pending: differentiate between local comms v…
shibbas Nov 8, 2022
c2973cd
change local connection to a singleton and pre-retrieve all the avail…
shibbas Nov 9, 2022
14b57f7
enable stdin
shibbas Nov 9, 2022
8eee2b9
log zmq process stdio
shibbas Nov 9, 2022
0d3fa36
add images to root for the marketplace links
colombod Nov 3, 2022
5efc882
use absolute url for images
colombod Nov 3, 2022
a2d3eb4
js kernel can evaluate javascript code
colombod Nov 1, 2022
d32d194
use async evaluator
colombod Nov 3, 2022
bdddf38
add images to root for the marketplace links
colombod Nov 3, 2022
846d9d2
fixing case for variable explorer
colombod Nov 4, 2022
28d43fb
remove left over test code
colombod Nov 4, 2022
29f135c
add kusto notebook
colombod Nov 4, 2022
092f7b1
kql and sql fix
colombod Nov 6, 2022
0ed070b
".NET Interactive" "dotnet interactive"
colombod Nov 7, 2022
efaa6fd
Merge pull request #2453 from shibbas/zmq-kernel-updates
brettfo Nov 9, 2022
b865f3c
Merge pull request #7 from shibbas/main
shibbas Nov 14, 2022
dc5c2a7
decouple sub kernel handling command events from comms manager.
shibbas Nov 10, 2022
4e910df
add command to run script on initialization
shibbas Nov 11, 2022
201ae7b
create a mew command event adapter logic for value sharing
shibbas Nov 13, 2022
a9033af
rename adapter to channel and add support for request value and reque…
shibbas Nov 14, 2022
6301759
re-enable all value sharing commands using commandevents comm channel…
shibbas Nov 14, 2022
43663fb
delete all the DAP related protocol messages
shibbas Nov 14, 2022
3778f0b
add licensing
shibbas Nov 14, 2022
4087e39
fix API tests for jupyter with new changes
shibbas Nov 14, 2022
26db22e
fix stale naming
shibbas Nov 14, 2022
6c22628
update the scripts for python and r to handle specific message type
shibbas Nov 15, 2022
70647a7
Merge pull request #2462 from shibbas/refactor-jupyter-variable-sharing
colombod Nov 15, 2022
cd1e94d
Merge branch 'main' of https://github.com/dotnet/interactive into mer…
brettfo Nov 21, 2022
39600e8
Merge pull request #2491 from dotnet/merge-main-to-release-jupyter
brettfo Nov 21, 2022
0284eb0
Merge branch 'main' of https://github.com/dotnet/interactive into mer…
brettfo Nov 23, 2022
2069cd8
Merge pull request #2504 from dotnet/merge-main-to-release-jupyter
brettfo Nov 23, 2022
3039b61
merge changes from main
shibbas Jan 30, 2023
755f9b1
fix formatting from merge issues
shibbas Jan 30, 2023
5bc8f64
Merge pull request #2658 from shibbas/jupyter/mergewithmain
brettfo Jan 30, 2023
be77cb2
move extension loading to tool project, improve extension.dib loading…
jonsequitur Jan 30, 2023
468b55c
wip --from-result
colombod Jan 27, 2023
8915b26
wip
colombod Jan 27, 2023
e044bd2
more tests
colombod Jan 27, 2023
1a564b0
setting value from interpolation
colombod Jan 27, 2023
631a1d4
get value from another kernel
colombod Jan 27, 2023
d3e8843
wip
colombod Jan 27, 2023
39485d5
refactor set magic command as part of the value sharing gesture
colombod Jan 30, 2023
4c76413
specifying mimetype ignores reference value
colombod Jan 30, 2023
a5286e5
Merge remote-tracking branch 'upstream/main' into merge-main-to-relea…
brettfo Feb 2, 2023
c426ee4
Merge pull request #2675 from brettfo/merge-main-to-release-jupyter
brettfo Feb 2, 2023
07398d9
Add the test infra layer for running tests against Jupyter connection…
shibbas Feb 8, 2023
0ccfced
merge `main` into `release/jupyter` (#2705)
shibbas Feb 9, 2023
820e274
add tests for submit code and it's results (#2708)
shibbas Feb 10, 2023
703eade
Merge remote-tracking branch 'upstream/main' into merge-main-to-relea…
brettfo Feb 11, 2023
711c31f
Merge pull request #2720 from dotnet/merge-main-to-release-jupyter
brettfo Feb 11, 2023
f7f80bb
add kernel language tests (#2721)
shibbas Feb 13, 2023
045ebc4
Jupyter/fix completion kind (#2730)
shibbas Feb 14, 2023
29236c2
[release/jupyter] fix tool source (#2748)
brettfo Feb 17, 2023
3fad00c
unit tests for the python command handler
shibbas Feb 19, 2023
c7da64b
add error scenarios for commands
shibbas Feb 19, 2023
41ca756
add tests for request all and bat file to run the tests
shibbas Feb 19, 2023
cb79442
add test for invalid json
shibbas Feb 19, 2023
5a374e5
r tests
shibbas Feb 19, 2023
e3091f4
add more r tests and fix a bug in r handler
shibbas Feb 20, 2023
468d034
test dataframe set and get in python
shibbas Feb 20, 2023
19a96b1
fix a bug in r that caused data frame to fail
shibbas Feb 20, 2023
ef8149d
cleanup and add more r unit tests
shibbas Feb 20, 2023
7356d6f
merge from main (#2754)
shibbas Feb 21, 2023
e8181f2
Jupyter/developer guide updates (#2761)
shibbas Feb 22, 2023
e37a809
Jupyter/variable sharing e2e tests pt1 (#2764)
shibbas Feb 22, 2023
1e60a53
update license info
shibbas Feb 22, 2023
e2cd3ab
add tests for different use cases
shibbas Feb 24, 2023
c157475
add request value info tests and error cases for requestvalue
shibbas Feb 24, 2023
398c19c
fix header
shibbas Feb 24, 2023
510459d
fix the random chars
shibbas Feb 24, 2023
62b17c9
go back to the original wording
shibbas Feb 24, 2023
7598d86
Merge pull request #2767 from shibbas/jupyter/variable-sharing-e2e-te…
AbhitejJohn Feb 24, 2023
3b56a68
shutdown active sessions when kernel is disposed.
shibbas Feb 25, 2023
1306942
fix bug where interrupt was called on a disposed object
shibbas Feb 25, 2023
04a6407
adopt the contract change for requestvalueinfos
shibbas Feb 28, 2023
142fe1f
update the test messages
shibbas Feb 28, 2023
50ffa54
make sure that kernelspecs can be returned in the completions
shibbas Feb 28, 2023
4ca9633
Jupyter/merge from main (#2774)
shibbas Mar 1, 2023
00df46d
merge main to release/jupyter
shibbas Mar 1, 2023
0296fda
Merge pull request #2783 from shibbas/jupyter/merge-main-no-squash
jonsequitur Mar 1, 2023
4ff13b2
revert tool source changes to not use experiemental package source
shibbas Mar 3, 2023
9d6a7b9
cleanup unneeded changes from merges and address some PR feedback
shibbas Mar 3, 2023
4f5d13d
rename to appropriately represent environment
shibbas Mar 3, 2023
1fe4b2b
use shorter name
shibbas Mar 3, 2023
0614591
merge from main
shibbas Mar 21, 2023
d342be0
adapt to send back an empty kernelinfo list for kernel ready. We only…
shibbas Mar 21, 2023
9c0e7e1
update the test records to include the fix for the contract change. S…
shibbas Mar 22, 2023
18c09e3
PR feedback on the developer guide
shibbas Mar 22, 2023
a7c1f69
use are more specific kernel uri
shibbas Mar 22, 2023
92870ec
Give the jupyter kernel a display name with "Preview" tag in it.
shibbas Mar 22, 2023
2584168
move the logic for the current connection from JupyterConnection to t…
shibbas Mar 22, 2023
1c640cf
fix data frame sharing tests
shibbas Mar 22, 2023
1c4ec1d
add checks for messages being sent to the kernel for value sharing
shibbas Mar 22, 2023
fce7d3f
add preview languages via jupyter added to extension readmes
shibbas Mar 23, 2023
a185923
update to clarify better in readme
shibbas Mar 23, 2023
c993837
fix R kernel tests to enable them to run from command line.
shibbas Mar 23, 2023
a829f2c
fix the experience for newly installed conda where there is no runtim…
shibbas Mar 23, 2023
f619012
Don't set URI when creating a kernel since we are not a proxy kernel …
shibbas Mar 24, 2023
4eb1390
Merge branch 'main' into connect-to-jupyter-kernels
shibbas Mar 24, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
PR feedback on the developer guide
shibbas committed Mar 22, 2023
commit 18c09e34514f4c518b8e27e0078ac912b7806c2d
8 changes: 4 additions & 4 deletions DEVELOPER-GUIDE.md
Original file line number Diff line number Diff line change
@@ -207,15 +207,15 @@ Some tests require additional setup or will be skipped. `JupyterKernel` tests fo
### Run tests with a local Jupyter Server

1. Install [Jupyter server](https://docs.jupyter.org/en/latest/install.html) or [Anaconda](https://www.anaconda.com/products/distribution)
2. [Install R kernel](https://docs.anaconda.com/anaconda/user-guide/tasks/using-r-language/) for R tests by calling
2. [Install R kernel](https://docs.anaconda.com/anaconda/user-guide/tasks/using-r-language/) for R tests by calling the following in Anaconda Prompt (Windows) or the terminal (Mac/Linux)
```
conda install -c r r-irkernel
shibbas marked this conversation as resolved.
Show resolved Hide resolved
```
3. Start the server locally as mentioned [here](https://docs.jupyter.org/en/latest/running.html)
3. Start the server locally as mentioned [here](https://docs.jupyter.org/en/latest/running.html). You can use any random string or guid for your_token value.
```
jupyter notebook --no-browser --NotebookApp.token=<your_token> --port=8888
shibbas marked this conversation as resolved.
Show resolved Hide resolved
```
4. Set an environment variable `TEST_DOTNET_JUPYTER_HTTP_CONN` pointing to the server and the token generated for the Jupyter server as
4. Set an environment variable `TEST_DOTNET_JUPYTER_HTTP_CONN` pointing to the server and the token you are using for the Jupyter server as
```
--url http://localhost:8888 --token <your_token>
```
@@ -224,7 +224,7 @@ jupyter notebook --no-browser --NotebookApp.token=<your_token> --port=8888
### Run tests with a Jupyter Kernel over ZMQ

1. Install [Anaconda](https://www.anaconda.com/products/distribution)
2. [Install R kernel](https://docs.anaconda.com/anaconda/user-guide/tasks/using-r-language/) for R tests by calling
2. [Install R kernel](https://docs.anaconda.com/anaconda/user-guide/tasks/using-r-language/) for R tests by calling the following in Anaconda Prompt (Windows) or the terminal (Mac/Linux)
```
conda install -c r r-irkernel
shibbas marked this conversation as resolved.
Show resolved Hide resolved
```