[EVENT] OceanHackWeek February 27 - March 3

[jmunroe]

Summary

There will be an OceanHackWeek-related event running in late February.

This event will be run in Spanish.

Reference: https://2i2c.freshdesk.com/a/contacts/80068629049

Event Info

• Community Representative: Emilio Mayorga emiliom@uw.edu
• Event begin: February 27
  • In your timezone:
• Event end: March 3
  • In your timezone:
• Active times:
  • In your timezone:
• Number of attendees:
• Hub Events Calendar

Hub info

• Hub URL:
• Hub decommisioned after event?:

Task List

Before the event

• Dates confirmed with the community representative and added to Hub Events Calendar.
• Quotas from the cloud provider are high-enough to handle expected usage.
• One week before event Hub is running.
• Confirm with Community Representative that their workflows function as expected.
  • 👉Template message to send to community representative

    Hey {{ COMMUNITY REPRESENTATIVE }}, the date of your event is getting close!
    
    Could you please confirm that your hub environment is ready-to-go, and matches your hub's infrastructure setup, by ensuring the following things:
    - [ ] Confirm that the "Event Info" above is correct
    - [ ] On your hub: log-in and authentication works as-expected
    - [ ] `nbgitpuller` links you intend to use resolve properly
    - [ ] Your notebooks and content run as-expected
    

• 1 day before event, either a separate nodegroup is provisioned for the event or the cluster is scaled up.

During and after event

• Confirm event is finished.
• Nodegroup created for the hub is decommissioned / cluster is scaled down.
• Hub decommissioned (if needed).
• Debrief with community representative.
  • 👉Template debrief to send to community representative

    Hey {{ COMMUNITY REPRESENTATIVE }}, your event appears to be over 🎉
    
    We hope that your hub worked out well for you! We are trying to understand where we can improve our hub infrastructure and setup around events, and would love any feedback that you're willing to give. Would you mind answering the following questions? If not, just let us know and that is no problem!
    
    - Did the infrastructure behave as expected?
    - Anything that was confusing or could be improved?
    - Any extra functionality you wish you would have had?
    - Could you share a story about how you used the hub?
    
    - Any other feedback that you'd like to share?
    
    

[emiliom]

Hub info

• Hub URL: https://oceanhackweek.2i2c.cloud
• Hub decommisioned after event?: No

[emiliom]

Some specific next tasks:

• Add new github teams. Should I submit a PR myself? See Add new github teams from the Spanish event oceanhackweek/Hub-Management#5
• Add JupyterLab Spanish language pack, jupyterlab-language-pack-es-ES. There's also an additional setting that makes a particular language the default. See Add jupyterlab-language-pack-es-ES to Python image and set Spanish to be the default language oceanhackweek/Hub-Management#3 , though I haven't added the steps for changing that default setting; I'll do that very soon
• Customize hub landing page. We'd like to edit the text, links and logo in the landing page (https://oceanhackweek.2i2c.cloud/). Here's what @consideRatio has said about this, and my follow up:

  /hub/login is when you are logged out and arrive initially, its the landing page.
  This page is dynamically updated based on https://github.com/2i2c-org/default-hub-homepage/blob/uwhackweeks-prod/templates/login.html. You can absolutely create a GitHub pull request to change the uwhackweeks-prod branch of that GitHub repository, and once its merged it will take up to ~15 minutes, and then visitors of the landing page should see something different.

  Great! This is not the sort of thing that we would change often or multiple times. We can insert the text we have in mind here, in a prominent place.

  One note about the login.html page you've shared: It's on a "uwhackweeks-prod" branch. I'm guessing that "uwhackweeks" refers to the hackweeks run by the UW eScience Institute, especially SnowEX and ICESAT-2 hackweeks. We collaborate with that group but it's distinct from OceanHackWeek. So, we'd need to make sure that this login template only applies to OceanHackWeek.

[emiliom]

I've submitted a PR (#2169) to address mainly the first task I listed above. In that PR I also added small, minor edits to homepage values. I'd like to enable particularly the users in https://github.com/orgs/Intercoonecta/teams/ohwes23-organizers to access the OHW hub ASAP.

[emiliom]

@consideRatio could you comment on the topic of customization of the landing page, above? Like I said, my main question is whether the template page you pointed to is used by other deployments and not just OceanHackWeek. If you can confirm that it's just OHW, I can go ahead and submit a PR with edits.

[GeorgianaElena]

@emiliom, if there is no branch in https://github.com/2i2c-org/default-hub-homepage, that matches the <cluster-name>-<hub-name> pattern, then the contents of the default main branch will be used, which is the case for most hubs right now, including the oceanhackweek hub. There are more details and instructions about this in the readme here.

Like I said, my main question is whether the template page you pointed to is used by other deployments and not just OceanHackWeek.

If we create a branch 2i2c-ohw in the template repo mentioned above, then all changes that happen there, will only apply to OceanHackWeek hub.

Because I don't think you have permissions to create a new branch in that repository, I believe the workflow should be the following:

• We create the appropriate 2i2c-ohw branch in the template repository
• You can then create a GitHub pull request with the desired changes and open a PR against that branch

EDIT BY ERIK: Updated the branch name to 2i2c-ohw

[GeorgianaElena]

@emiliom, I've just created a 2i2c-ohw branch in https://github.com/2i2c-org/default-hub-homepage/tree/2i2c-ohw.

Changing the contents of https://github.com/2i2c-org/default-hub-homepage/blob/2i2c-ohw/templates/login.html template there through a pull request as per @consideRatio's suggestion should now be possible.

[consideRatio]

@emiliom I think I've communicated confused between uwhackweeks and oceanhackweeks. If you modify 2i2c-ohw, that should only influence people visiting oceanhackweek.2i2c.cloud!

[emiliom]

Thank you @GeorgianaElena and @consideRatio! I'll submit a PR to the new 2iwc-ohw branch.

[emiliom]

I've submitted a PR (2i2c-org/default-hub-homepage#20) to the 2i2c-ohw branch with initial edits to the login page.

[emiliom]

We've updated the Python image for this hub just now. According to @abkfenris:

2i2c has an action that detects new images that runs periodically. It bumps the images in

infrastructure/config/clusters/2i2c/ohw.values.yaml

Line 38 in 1c88f42

image: "ghcr.io/oceanhackweek/python:99b6ab5"

They had us on a fast cycle for the last event, but they may have turned things down to weekly for us.

What's the image update frequency now? I anticipate doing several image updates between now and the start of the event.

[emiliom]

Following up on my question above, about the image update frequency. As the event starts in one week, we'll need to have at least a daily update frequency. What is it now, and is there a way to see when was the last time the image used was updated?

[yuvipanda]

hey @emiliom. the image can be updated anytime you want with a PR like #2100. The automatic one can be treated as a backup, and you can see its activity as it just makes PRs like #2070. So you can always make PRs as frequently as you want updating the values and we'll try to merge them asap.

We're hoping to make this fully self serve in the future...

[emiliom]

Thanks @yuvipanda! That's great, and easy.

I've created PR #2233 with the changes to the Python image plus a couple of minor tweaks.

[emiliom]

I've created PR #2252 with updates to the Python and R images plus a couple of minor tweaks to the metadata. (Update: there was a problem with the R image, but it was fixed in PR #2253).

As a reminder: the hackweek event starts on Monday (2/27). So, we've already kicked into high gear.

FYI, there'll be a total of around 40 users during the week.

[damianavila]

@emiliom, can you please confirm the timezone and active hours for the event? Thanks!!

[emiliom]

Start Date: Monday 2023-02-27
End Date: Friday 2023-03-03

Most active hours:

• Mon - Thu: 08:00 - 17:00 UTC-6
• Fri: 08:00 - 11:30 UTC-6

I'm sure there'll be activity at all hours, but those are the defined event hours.

[emiliom]

I have a question about quitting an RStudio session. We had a problem with this in our August 2022 event, and the problem is still there. Basically, there's no obvious way to actually quit and stop the server. When one clicks on the "Quit the current R session" button on the upper right, or the equivalent menu item under File, we get this screen:

But that's misleading, because the session doesn't seem to have ended. If we then try to go to the hub login page at https://oceanhackweek.2i2c.cloud, we're taken straight to https://oceanhackweek.2i2c.cloud/user/emiliom/rstudio/

The only solution we found was to go to https://oceanhackweek.2i2c.cloud/hub/home and then click on "Stop my Server". But that's difficult, because event participants need to remember where to find that link!

If on RStudio we click on Sign out instead, no real change happens. It's like a refresh, and you're right back in the session.

Any advice to address this problem?

[consideRatio]

There is no short term fix for this as i see it. The root issue is that rstudio isnt the main server, a jupyter server is, so shutting down rstudio isnt whats relevant to shut down - so the user experience is confusing if you default to rstudio for example.

I believe if you quit like that, the underlying jupyter server will shut down in due time automatically. For now, visiting /hub/home to shut down is what needs to be done to shut down the server directly instead of waiting for the automation which should take ~1 hour.

I think it isnt clear what the desired behavior would be if we consider a server both using jupyterlab and rstudio. Pressing shut down on ratudio shouldnt shut down lab i figure.

Tricky situation without onboarding users of the complexity.

[consideRatio]

If on RStudio we click on Sign out instead, no real change happens. It's like a refresh, and you're right back in the session.

Similarly, sign out is meaningless rstudio dummy as authentication is handled by jupyterhub decoupled from rstudio.

I suggest for now saying that shut down/sign out in rstudio should be avoided as irrelevant. Long term, maybe it could make sense to let those link somewhere if that is possible to change via configuration if we think that would be an improvement.

I'll open an issue among 2i2c engineers to deliberate further on this @emiliom!

[emiliom]

Thanks for the background @consideRatio . I wasn't hopeful of an easy solution, but you never know.

I'll add that in our context, where we support both Python and R via different images, the issue of not being able to truly stop the server and quit the session from RStudio has an additional, more immediate impact. If a user wants to quit the RStudio session in order to go back to the login and initiate a Python+JupyterLab session, the only option at this time is the link https://oceanhackweek.2i2c.cloud/hub/home. Otherwise, you're stuck in RStudio!

[emiliom]

Our event has started, and a question has come up. We have at least one participant from Cuba, who works at the Cuban Institute of Meteorology. She is having problems accessing the hub front page, https://oceanhackweek.2i2c.cloud. I'm waiting for her to send me a screenshot to know exactly what she means, but I wanted to start asking you. I didn't anticipate that there'd be access problems for someone from Cuba to just use the system, but I don't have any experience in the area of countries embargoed by the US. According to https://stackoverflow.com/questions/52763301/cuba-and-other-countries, there may be indeed limitations, but I'm not sure about the "personal" vs "business" use.

What do you know about this?

We also have a couple of participants from Venezuela, from a research institute and possibly one government agency. I don't know if Venezuela falls in the same embargo category as Cuba.

[emiliom]

Here's a screenshot (in Spanish) of what this user from Cuba is getting from her browser when she tries to access https://oceanhackweek.2i2c.cloud/
Basically, as you can see from "ERR_CONNECTION_TIMED_OUT", it hangs.

[emiliom]

Does any one have input on this issue of access from an embargoed country?? BTW, the person I'm working with was able to connect via a VPN, so it's clear this is the issue one way or another. But I'd really like to hear if there's anything that can be done on our end this week, or if this is a longer term challenge. Thanks.
Feel free to contact me via email if that makes more sense.

[yuvipanda]

@emiliom heya! I do think figuring out how to deal with embargos is a longer term challenge we would have to deal with (see what GitHub had to do for example: https://github.blog/2021-01-05-advancing-developer-freedom-github-is-fully-available-in-iran/). I don't entirely know what the next step here for us is, but given 2i2c wants to be doing more work in latin america, something to try figure out, perhaps in consultation with lawyers at CS&S.

Tagging @damianavila.

[emiliom]

Thank you @yuvipanda . Frankly I can't tell exactly what falls under the terms of the embargo and what doesn't. This user was able to create a GitHub account. Anyways, I take it that 2i2c hubs at this time are not accessible in any form from an IP address from an embargoed country, with no distinction between personal and other uses?

[yuvipanda]

@emiliom indeed, we can't tell either about what falls under and what does not. Clearly GitHub is accessible, and they don't make any mention of needing OFAC permission to allow it to be accessible in Cuba. I also can't seem to find any mention in Google Cloud of network restrictions being in place. @colliand has gratefully offered to chat with CS&S (our fiscal sponsors) to try figure out how to get clarity around this.

[emiliom]

I got a confirmation that they also couldn't access something as simple as a dataset page on Google Cloud, like this one: https://console.cloud.google.com/storage/browser/world-ocean-database
I suspect that on-prem web servers hosting public data from federal servers, like NASA and NOAA, are accessible (but don't know for a fact). So if that's all true, it means that cloud-hosted data may be inaccessible even if they're running code on their own computer and not on a cloud-based JupyterHub, and even though the same data may be accessible from traditional agency-hosted on-prem servers.

Anyways. Thanks for looking into it, for the longer term.

[emiliom]

I (my github account) have write permission to the shared folder, via the shared-readwrite folder. This was already in place for the Aug '22 event, but I don't know where my account is granted that permission. I'm not listed among admin_users in https://github.com/2i2c-org/infrastructure/blob/master/config/clusters/2i2c/ohw.values.yaml#L124, and I don't see anywhere else where in that config file where that special permission could be granted. Is that something I can modify, somewhere?

I'd like to know if I can grant other organizers that permission, so they can add some data files to shared. Thanks.

[consideRatio]

I'd like to know if I can grant other organizers that permission, so they can add some data files to shared. Thanks.

@emiliom visit /hub/admin and make them admins, then they will get the shared-readwrite folder mounted as well!

[emiliom]

Ah, thanks!

[damianavila]

Hey @emiliom, your event appears to be over 🎉

We hope that your hub worked out well for you! We are trying to understand where we can improve our hub infrastructure and setup around events, and would love any feedback that you're willing to give. Would you mind answering the following questions? If not, just let us know and that is no problem!

• Did the infrastructure behave as expected?
• Anything that was confusing or could be improved?
• Any extra functionality you wish you would have had?
• Could you share a story about how you used the hub?
• Any other feedback that you'd like to share?

Thanks!

[emiliom]

Hi @damianavila

Did the infrastructure behave as expected?

Yeah. We didn't have any hiccups or notable latencies that I'm aware of.

Anything that was confusing or could be improved?

I think the main source of confusion was around the steps to close RStudio (which I noted earlier, and @consideRatio opened issue 2i2c-org/features#21 for future exploration). It's confusing and misleading. In the end we tried to mitigate it by posting prominent instructions on our web site front page. I know it's not due to something that 2i2c has done or failed to do, but it'd be great if a solution or workaround could be found. As I mentioned earlier, for us it was extra challenging because our event supported both R and Python, so there was a need to go back and forth between R and Python images. BTW, there was an additional source of confusion that we noticed, though I'm not sure if it happened with everyone and all the time. When logging back into an R session, the warning below would appear. As far as we could tell, nothing was actually wrong, though.

Closing out of JupyterLab is also not crystal clear, what with the choice between "Hub Control Panel" and "Log Out" File menu items. Again, I know that's a JupyterHub characteristic, not a 2i2c setting. But it'd nice if there were a more straightforward, single route for fully closing out.

Any extra functionality you wish you would have had?

Can't think of any. On a minor note (call it a personal pet peeve), it'd be great to have syntax highlighting on the JupyterLab terminal! I've seen it on other JupyterHub deployments.

Could you share a story about how you used the hub?

Gosh, our use was fairly pedestrian as hackweeks go. We used it for our tutorials and for group projects. I don't have a stand-out story. I'll just mention one project on the use of water column sonar (echosounder) data for biological applications. The project mentor had the instrument raw binary files, and he uploaded them to (about 2 GB) the hub for access by project team members via the shared folder (after I gave him admin access). Anyone could then access them from the hub. The team members were all R users and I believe they felt more comfortable working locally, so in the end most of them simply downloaded the data files from the hub. But they also needed to use a Python package to process the files into a standard, open format (netcdf). The package was already on the hub Python image. They were having problems trying to set up Python locally, so I helped them by writing a Jupyter notebook on the hub and running it there, to process all the files on the hub and sharing the netcdf files via the shared folder.

Any other feedback that you'd like to share?

It'd be really helpful to have a web page with quick instructions or links about configuration changes most likely to be used by event hub admins (ie, people in my role). Ultimately I received answers for all of these, after asking about them. For example:

• How to add other users to have "admin" privilege (in our case, mainly so they could add files to the shared folder)
• Where to add github teams of users
• How image updates are pushed to the hub, including how quickly the hub deployment is typically rebuilt after the PR is approved and how to monitor that rebuild
• How to customize the hub front page

[damianavila]

This is really great feedback @emiliom.
Thanks for sharing it!!

[colliand]

Hi @emiliom! Can we shut down this hub now? Our agreement was to keep the hub operational until March. We can also work out an extended agreement if you wish to keep this in operation. I will also follow up via email.