Pin ipykernel<6.0.0

[devstein]

Closes #492

Description

Changes to the async behavior in IPykernel >=6.0.0 causes connection issues with the Sparkmagic (Py)Spark kernels. This pins the dependency while I find the root cause of the breaking change.

• Pins ipykernel<6.0.0
• Removes more references to Python 2
• Runs black formatter

Checklist

• Wrote a description of my changes above
• Added a bullet point for my changes to the top of the CHANGELOG.md file
• Manually tested with a notebook

[itamarst]

Seems fine assuming tests pass.

[devstein]

@itamarst CI is broken because of a Travis CI migration (see below). I've reached out to the maintainer of the pip package, so I can migrate to GH Actions

Since June 15th, 2021, the building on travis-ci.org is ceased. Please use travis-ci.com from now on.

[Carreau]

Thanks seem fine to me, I do have access to PyPI as a maintainer, though I believe I can only upload as I don't have access to any settings panels.

One caveat for this fix: now that pip has a resolver, once this get published, pip can decide that the solution is to not upgrade sparkmagic; I've seen similar behavior in the wild, so we might have to yank

[itamarst]

Yeah, probably need to:

1. Switch to Github Actions (GHA).
2. Generate new PyPI access token, and add it to GHA for auto-release-on-tag, like Travis currently does.

[devstein]

@itamarst Agreed.

Thanks seem fine to me, I do have access to PyPI as a maintainer, though I believe I can only upload as I don't have access to any settings panels.

@Carreau Do you have access to generate a PyPI access token? If so, can you generate a new token and set it as an environment variable for the sparkmagic repo? I

[Carreau]

I believe this is now done with a repository secret.

Do you know who hdinsightmsftagu on PyPI is ? Or how to contact them ? Otherwise we can make a request to PyPI maintainer to transfer ownership – it should not be too hard as I'm already maintainer.

[devstein]

@Carreau Thank you! I'll create another PR in the next few days to migrate to GH actions and tag you.

Do you know who hdinsightmsftagu on PyPI is ? Or how to contact them ? Otherwise we can make a request to PyPI maintainer to transfer ownership – it should not be too hard as I'm already maintainer.

I do not. This is the only maintainer I couldn't find information about. It seems like it's a service account. If you can reach out to PyPI that would be great. It's the simplest option.

[Carreau]

I've personally Pinged Brian and Min to see if they have access, otherwise I'll ask PyPI.

…

On Thu, 12 Aug 2021 at 12:05, Devin Stein ***@***.***> wrote: @Carreau <https://github.com/Carreau> Thank you! I'll create another PR in the next few days to migrate to GH actions and tag you. Do you know who hdinsightmsftagu on PyPI is ? Or how to contact them ? Otherwise we can make a request to PyPI maintainer to transfer ownership – it should not be too hard as I'm already maintainer. I do not. This is the only maintainer I couldn't find information about. It seems like it's a service account. If you can reach out to PyPI that would be great. It's the simplest option. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#723 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACR5T5E5VB32YQN2M7AQE3T4QLQLANCNFSM5A4I3QYQ> .

[Carreau]

I have an update and we might be able to get everyone as an owner in a week or so - some people are travelling and put their work notification off.

[devstein]

@Carreau Amazing, thank you for all the help on this.

[Carreau]

@Carreau Amazing, thank you for all the help on this.

No problem, thank you for working on this. We are also working on making sure that access to resources is managed in a better way.

Do you also have a PyPI username, I supposed it will not be a problem to add you once we get back ownership.

[devstein]

@Carreau https://pypi.org/user/devstein/

[mrinmoyprasad]

Hi @devstein - do you think my issue here could be related to the bug your PR is addressing?

[devstein]

@mrinmoyprasad Hard to say without investigating further. Would you mind trying to reproduce to verify after the next release?

[devstein]

@Carreau I'm having issues publishing the packages. The error message indicates an invalid token, here are the probably root causes

Any ideas what could be wrong with the token? Let me know if I can help.

Upload failed (403): Invalid API Token: InvalidMacaroon('invalid macaroon signature')
error: Upload failed (403): Invalid API Token: InvalidMacaroon('invalid macaroon signature')
Error: Process completed with exit code 1.

[Carreau]

I've updated the token, and you should have received an invite.

[mrinmoyprasad]

@mrinmoyprasad Hard to say without investigating further. Would you mind trying to reproduce to verify after the next release?

Sure thing, as soon as 0.19.1 is published, I can verify and let you know! Thank you

[devstein]

@Carreau Thanks! I now have access tosparkmagic; however, I discovered the source of the error 403 is that the token is only valid for sparkmagic and not hdijupyterutils nor autovizwidget.

@ellisonbg is the only one with access to hdijupyterutils and autovizwidget. Are you able to reach out to him?

[Carreau]

I'll see what I can do, though I'll be offline until next Wednesday.

[Carreau]

There should now be 3 new tokens : PYPI_TOKEN_AUTOVIZWIDGET, PYPI_TOKEN_HDIJUPYTERUTILS, PYPI_TOKEN_SPARKMAGIC with upload access to corresponding repos.

[mrinmoyprasad]

@devstein It looks like 0.19.1 fixed our issues with sparkmagic that I had logged in here! Thank you so much for identifying this, this had plagued us for over 2 months at this point. Really appreciate your help on this.

[devstein]

There should now be 3 new tokens : PYPI_TOKEN_AUTOVIZWIDGET, PYPI_TOKEN_HDIJUPYTERUTILS, PYPI_TOKEN_SPARKMAGIC with upload access to corresponding repos.

@Carreau Amazing, thanks for all the support on this! Hope you enjoyed your the time offline.