-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Poll faculty for spring changes + improvements #30
Comments
Unless something changes, I will be the only one teaching 141 in the spring. A few points of friction that come to mind quickly, for me:
OK, now here are some dreams of how this whole system could work better. I don't expect these to be easily implementable, but I figure I'll share them anyway:
|
Another point of friction--to post a notebook on my JupyterHub to the class, I have to download it and commit it to our course repository. I'd like this to be simpler. It would be nice if I could maintain a folder (e.g. |
This is great feedback! Lots of things that @anesepark and @walidabualafia can look through here. I'll offer some initial thoughts.
There's a few things on the critical path to being dropped into the server:
If you mean physical server: this scales to about 1-2 classes max and requires a big box (e.g., 192GB of RAM, 32 cores, etc.). Running in the cloud, this is a HUGE spend, since memory dominates vCPU cost. I've proposed on-prem, but then we run into... But more importantly this is essential for isolation. There's some notes about this in the docs, but capping CPU, memory, and disk without containerizing is a pain. e.g., OOM killer will kill all processes, not just the culprit server. If we run containers, that's this solution.
We choose the OAuth response field for username. Issue here is ensuring IDs are unique--email provides that. We could explore concatenating different fields.
This happens when the user is active and not idle? I assume this is a OneLogin integration issue (maybe a timeout? spurious invalidation?). We should investigate. These are in the server logs and we can root-cause. Please ping the slack when this next happens and they can investigate.
Hacky solution-enhance our wrapper to Okpy and either submit to okpy OR just ping an endpoint (e.g., our own or the sheets api or...).
Again, a hack: This would be a cool add-on to the library: upload a cell's contents to a pastebin (or our own endpoint). A more robust solution breaks the JupyterHub security model, I believe.
This would be covered by the above idea. Or something similar. We could also explore a plug-in to use the admin API to authenticate against a bunch of user servers.
If we are using nbgitpuller+GitHub to distribute files, this can be done with a periodic poll mechanism, or by moving to the link-based sync in the docs. We could also add something to the library or the UI (if we want to start writing JupyterHub plugins) to save the current notebook and run nbgitpuller.
That would be cool. Fwiw Collab (doing exactly your use case for paper graphs right now!) and other IPython front ends (I believe) support this, but not JupyterHub. If we want to consider moving to Collab or something like it, I would not be opposed, assuming we could get feature parity, especially for admin access. |
So, disk isolation prevents this. Non-isolated disk was a huge issue but is 100% simpler to do this sort of stuff. I could imagine a back-channel, but that's kind of a hack. The link-based pull is a nice middle ground. Or other ideas above. But fwiw I did all of my course development on the server. There's a CLI with git (just choose new shell vs. notebook). I didn't install ssh and use 2fa for GitHub (which only allows ssh), so had to generate a token, but that was the most annoying part. There's a small note in the docs about that. |
Again this is awesome feedback and we can start to address this or at least gather some telemetry. @anesepark and @walidabualafia can you ping the Slack and I can point you to some things? |
Another thought: we could probably do a global shared read-only file system. Kubernetes supports it. That might facilitate some reduction in friction. Ultimately we use nbgitpuller because it will do smart diff resolving on a per file basis and guarantees not to clobber student work and get the latest version. We could point it at a shared volume instead of GitHub. When we started with this project, we (mostly me and Catie) used script-based distribution, but that had issues that nbgitpuller addresses. Also, isolating disk is a pretty important requirement. |
Just random thoughts:
|
While the updated cs1.graphics library is great in that it allows for clicking on the canvas and students don't have to call anything extra to have their image show up, one of the issues we're having this semester is that the notebooks no longer save the images when they're submitted to okpy. So, when we need to grade projects that include graphics, our graders have to re-run all the projects on the JupyterHub server to see the images that the programs create. Before the changes were made to the graphics library, these projects were very easy to grade since you only had to look at the notebook to see the drawing. This currently affects Project 3 (Potato Heads), Project 6 (Connect the Dots), and Project 8 (PPM Images). Is there anyway to modify cs1.graphics library to make the images persist when a notebook is submitted? |
I think this is probably do-able -- either as part of the submit or as part of the library. Fwiw for the programs that output images, I asked my grader to download the zip of the submissions, unzip it on their server, and then run this from a notebook:
That gives them the user + the output image. |
Re: saving images... I think the best thing we can do is wait for this feature to make it into the underlying library (tracked in this issue), which recently saw some progress with this PR. I can imagine other paths, like using the widget's Ultimately, there's not going to be really clean way to get the previous behavior back. The main reason is that we were previously having the cell output be a png. Animation/image updates worked in a hacky way, by updating the output of the cell. The underlying library that we using now facilitates interaction, as well as making animation much smoother by being an HTML5 canvas. So, the cell output is a widget object that knows how to display itself, rather than a png. If we want image files to be part of submissions, I think the best way to do this is to explicitly save them and submit them. I think reverting to the previous implementation would be regression in other dimensions. We already have Thoughts? |
Rhodes-CS-Department/comp141-libraries#10 is a POC of what I meant, obvious needs a better UX. |
If I were to choose two issues to resolve for the upcoming spring semester, it would be these: (1) Addressing the "Forbidden" error I mentioned above. @matthewlang, sorry for never pinging you about this, but I'm pretty sure I could reproduce it without trouble. It happens even while active, and it seems to consistently happen a certain amount of time after initial authentication. (For example, it typically happens once during my 11a class, after first opening JupyterHub shortly before my 10a class.) It affects students as well as me. My standard fix is to open rhodes-notebook.org in another tab, which will open immediately without requiring a OneLogin authentication; then I immediately close this tab, return to my original tab, and everything's fine. If you'd like, I could try to reproduce it and give time of initial login & time of error. (2) This is one I haven't mentioned above--usually among 50 submissions for a lab or project, I have about two that are just empty on okpy--the student's notebook says "submitted successfully," but no files are actually sent to okpy. If the student clicks the URL, they catch the issue and submit again. But my less conscientious students don't bother clicking the URL, and I've even had instances where a student repeatedly tries to submit and can't get anything to actually submit. Any insight into why this happens? The ideal fix would be to make this not happen; a perhaps more attainable fix is to give the student a specific warning if the files don't actually go through. |
I figured this out, I think. I mean I should verify the following is correct by looking at the logs, but that seems like work... ;) Here's what I think is happening: JupyterHub has two OAuth flows -- one to authenticate the user against an external OAuth for access to the hub (this is OneLogin in our case) and the other is an internal OAuth, where the hub is an OAuth provider for user notebook servers. When you do the first flow, the hub will generate an OAuth token that you pass to the server in a cookie, and the server validates the token against the hub. The default timeout for this used to be an hour but was changed to be configurable and default to one day in this commit. A subsequent commit updated this to the cookie max age, which is 14 days. We are currently using version 0.11.1 of the helm chart, which is version 1.3.0 of JupyterHub. If you look at the release tags for those commits, they are in everything after 1.4.0. Updating to at least 1.0.0 will pick up those changes, and a 14 day timeout seems reasonable and I don't think we need to further configure it. Action item here is to update our helm chart version and pick up all of the post 1.3.0 changes. @anesepark @walidabualafia
Yea, this is an ongoing issue with okpy. I haven't debugged why this happens. It looks like the okpy maintainers haven't either, looking at their issues. |
As to why you were getting a 403, it looks like that would happen if the page is long-lived. Refreshing the page or going to the root url (and then being redirected to the server) generates a new token. |
#34 updates the jupyterhub version. Leaving open for the time being in order to capture suggestions |
@anesepark and @walidabualafia -- can you chat with @msuperdock @betswms @CatieWelsh @pkirlin about any improvements they might imagine for the environment and libraries for the spring semester? It looks like @msuperdock is the only one teaching 141 in the spring.
Some friction points are noted in issues already (e.g., trusting notebooks, making github pulling less error-prone, etc.). Can you see what else has been points of friction for students and faculty? Send out an email?
The text was updated successfully, but these errors were encountered: