Enable Real Time Collaboration in our Hubs

[yuvipanda]

JupyterLab and Retrolab have realtime collaboration features now. They are a bit experimental, but work reasonably well. We should enable them in our hubs.

[yuvipanda]

Coming soon to data100

[yuvipanda]

Step 1: Find the 'Share' menu item in JupyterLab

Step 2: Read the warning, make sure you understand it, and copy the link to be shared with someone else

Step 3: Profit? Real Time Collaboration 'works' now!

[balajialg]

@yuvipanda @ericvd-ucb and I found an interesting behavior of the RTC feature when we were exploring this functionality a few days ago in the Data 8 hub. I had a notebook file in the root folder of the tree. So I created a shareable link and then shared it with @ericvd-ucb and Elias. Once they accessed the notebook, any folder created in each of our directories was accessible for all of us.

Am I making the right assumption here - Whenever a file within a specific directory gets shared with other users, it automatically updates the directory where it resides as a shared directory? Any additions/deletions to the shared directory gets updated for users with access to shareable links. If that be the case, I would like to add this behavior to our existing documentation

[balajialg]

@yuvipanda Need to enable this functionality in Ischool hub too!

[yuvipanda]

@balajialg so you're really giving full access to your entire server, rather than any particular file. This is fully equivalent to sharing an 'admin access' link with another user, although you can't shut down your own server with this link. So anything anyone with the link does is treated exactly like you are doing it yourself.

[balajialg]

This documentation captures the essence of today's discussion and the potential next steps! Pasting it below for future reference!

Security Considerations for adopting RTC

• there is a share menu that generates a link with a token that can be sent to other users
• a user shares a link to the next user, that user can then see and modify all the files in the user's home directory for this particular hub - including other assignments and homework
• this permission goes away when the user server stops - either explicitly from the control panel, or after non-activity for 60 minutes
• there is no ability to revoke the link, but you can stop the server and the link becomes invalid
• anyone can forward the link to the next person - there is no link-level user control. Users would have to be careful where they share links while that server is active since anyone on the internet could get control.
• currently, the Data 8 hub has its own file directory, only that class is on it - the main Datahub has all of the user's classes ( eg connectors and modules on it)
• a user needs to leave the session to go to work on their own files

Minimum requirements for use by students in a collaborative 'project'

Tier 1

• Only allow users who already have an account on the hub
• Add the ability to revoke access once the link is out
• Be 'clearer' who is the host. A guest of the share might want to visit their own server and/or share it, so this could be confusing.
• Display informed consent - the link share dialog's text should be customizable
• a read-only collaboration that is time-bound (aspirational usecase)

Tier 2

• Only allow users whose names are explicitly chosen by the user initiating the share
• Have a concept of 'projects', only people who can be given access to particular projects

Maybe useful

• time limits for each share link?

• readonly is very helpful

• Must be limited by

• limit by username

• projects or directories

[yuvipanda]

Based on suggestions by @ericvd-ucb, I'm going to disable jupyterlab-link-share and RTC on all hubs except the dlab hub. Let's write out a doc that lays out the security posture, and let individual instructors make calls on when to enable this for now. Once we have better access control, we can enable it more broadly.

[yuvipanda]

I'm also going to leave it enabled in the ischool hub, as @balajialg is talking to those folks about how they can trial it.

[balajialg]

RTC is currently disabled across all hubs and the reasons are outlined in this issue #3517!

[fperez]

Quick question @afshin - are any of the recent changes made on the YJS side in Lab had any impact on these data loss issues? I know the big refactoring is due for 4.0, but I was wondering if some of the recent work has made its way into 3.4.5 in a manner that would change our risk assessment?

I'd very much love to have RTC available, but we can't really take any chances (this semester I'm teaching a huge class of ~ 1,200+, not my smaller one from the spring where I could afford to be a bit more risk-tolerant and conduct experiments).

Thx for any input you might have.

[afshin]

@fperez @yuvipanda @balajialg, That bug still exists in every current release and pre-release of JupyterLab. It will be resolved when we merge this PR which refactors RTC in JupyterLab.

This PR is backward-incompatible, so it'll ship in a 4.0 alpha but it will not land in a 3.x release. We're pushing hard on merging it so we can iterate and get back to truly testing RTC in the wild.

[fperez]

Understood, thx for the clarification @afshin! It helps us plan on our end, this means we're probably going to hold off for this semester on RTC, at least in very large courses like D100 where flexibility/risk have to be tightly controlled (smaller courses have a different dynamic).

[VictoriaHoll]

@aculich and I are interested in enabling this feature on the general DataHub as well as the D-Lab DataHub! Do you have an estimate of when this might be available? Thank you!

[balajialg]

Thanks, @VictoriaHoll for raising the request! We are waiting for Jupyter Lab 4.0 release to actually test the RTC feature.

@afshin Do you have any inputs on when we can expect to test the RTC feature in our hubs? We have had multiple requests to enable RTC during the past 4-5 months (like the comment above).

[fperez]

If we can get 3.6.0 out the door with the RTC machinery enabled soon, I'm inclined to test it again in Stat 159. 159 is a safer environment to test this in, as I have "only" ~70 students, a very knowledgeable GSI, and the focus is precisely collaborative research.

We did last time (sp22), and that's how we found the data loss issues! It was a bit scary, but ultimately it led to the right fixes going in.

[balajialg]

@fperez Thanks for both your willingness to test RTC as part of Stat 159 and the update that RTC is part of the minor release 3.6 which I assumed was part of the 4.0 release earlier.

[fperez]

Sure! All the big changes are meant for 4.0, but 3.6 has the key backports that should (that's why we need to test! :) fix the bugs we encountered in the spring.

[balajialg]

@fperez Got it. That makes sense.