From 7d4a027232e654407baa93de75e8d8ccde979429 Mon Sep 17 00:00:00 2001 From: Chris Holdgraf Date: Fri, 5 Jun 2020 12:03:02 -0700 Subject: [PATCH] updating with new JEP process --- .github/pull_request_template.md | 12 + README.md | 11 +- .../JEP-TEMPLATE.md | 84 +++++++ ...jupyter-enhancement-proposal-guidelines.md | 226 ++++++++++++++---- 4 files changed, 287 insertions(+), 46 deletions(-) create mode 100644 .github/pull_request_template.md create mode 100644 jupyter-enhancement-proposal-guidelines/JEP-TEMPLATE.md diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 00000000..aefb40d1 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,12 @@ +✨Proposing a new JEP?✨ + +Make sure that you have at least done the following: + +1. **Create a GitHub issue** in the Jupyter Enhancement Proposals repo that + 1. Briefly outlines the proposal + 2. Suggests a review team (optional) + 3. Declares why it should be a JEP (See the “JEP / Not-a-JEP Rubric” below.) +2. **Identify a Shepherd** to see the process through. (Shepherds are assigned + on a round-robin basis from a set of interested engaged volunteers). +3. **Decide if it's a JEP** according to the criteria listed above. The Shepherd decides if the JEP criteria have been met. +4. **Used the new JEP markdown template** to help structure your thoughts. You can find it in `jupyter-enhancement-proposal-guidelines/TEMPLATE.md` in this repository. diff --git a/README.md b/README.md index 681b1c7c..9def4fc8 100644 --- a/README.md +++ b/README.md @@ -25,9 +25,14 @@ Below is a list of JEPs that have been submitted. To view the discussion around | 0025 | Completed | [Enterprise Gateway](https://github.com/jupyter/enhancement-proposals/pull/25) | | 0026 | **Submitted** | [Add Language Server Support to Jupyter Server and jupyterlab-monaco](https://github.com/jupyter/enhancement-proposals/pull/26) | | 0028 | **Submitted** | [Standalone Jupyter Server](https://github.com/jupyter/enhancement-proposals/pull/28) | -| 0029 | **Submitted** | [Jupyter Enhancement Proposal updates](https://github.com/jupyter/enhancement-proposals/pull/29) +| 0029 | **Accepted** | [Jupyter Enhancement Proposal updates](https://github.com/jupyter/enhancement-proposals/pull/29) | 0042 | **Accepted** | [Voila Incorporation](https://github.com/jupyter/enhancement-proposals/pull/42) -## How do I submit a JEP? +## To submit a JEP -In order to submit a JEP, please read the [Jupyter Enhancement Proposal Submission Guidelines](jupyter-enhancement-proposal-guidelines/jupyter-enhancement-proposal-guidelines.md). You can copy this file and use it as the starting point for outlining your own JEP. +In order to submit a JEP, first read the [Jupyter Enhancement Proposal Submission Guidelines](jupyter-enhancement-proposal-guidelines/jupyter-enhancement-proposal-guidelines.md) which describes the JEP process. + +In addition, read +[the JEP proposal template](jupyter-enhancement-proposal-guidelines/JEP-TEMPLATE.md) +for guidance on the questions you should answer before officially submitting +the JEP. diff --git a/jupyter-enhancement-proposal-guidelines/JEP-TEMPLATE.md b/jupyter-enhancement-proposal-guidelines/JEP-TEMPLATE.md new file mode 100644 index 00000000..5ce71610 --- /dev/null +++ b/jupyter-enhancement-proposal-guidelines/JEP-TEMPLATE.md @@ -0,0 +1,84 @@ +--- +title: +authors: +issue-number: +pr-number: +date-started: +--- + +# Summary + +One paragraph explanation of the proposal. + +# Motivation + +Why are we doing this? What use cases does it support? What is the expected outcome? + +# Guide-level explanation + +Explain the proposal as if it was already implemented and you were +explaining it to another community member. That generally means: + +- Introducing new named concepts. +- Adding examples for how this proposal affects people's experience. +- Explaining how others should *think* about the feature, and how it should impact the experience using Jupyter tools. It should explain the impact as concretely as possible. +- If applicable, provide sample error messages, deprecation warnings, or migration guidance. +- If applicable, describe the differences between teaching this to existing Jupyter members and new Jupyter members. + +For implementation-oriented JEPs, this section should focus on how other Jupyter +developers should think about the change, and give examples of its concrete impact. For policy JEPs, this section should provide an example-driven introduction to the policy, and explain its impact in concrete terms. + +# Reference-level explanation + +This is the technical portion of the JEP. Explain the design in +sufficient detail that: + +- Its interaction with other features is clear. +- It is reasonably clear how the feature would be implemented. +- Corner cases are dissected by example. + +The section should return to the examples given in the previous section, and explain more fully how the detailed proposal makes those examples work. + +# Rationale and alternatives + +- Why is this choice the best in the space of possible designs? +- What other designs have been considered and what is the rationale for not choosing them? +- What is the impact of not doing this? + +# Prior art + +Discuss prior art, both the good and the bad, in relation to this proposal. +A few examples of what this can include are: + +- Does this feature exist in other tools or ecosystems, and what experience have their community had? +- For community proposals: Is this done by some other community and what were their experiences with it? +- For other teams: What lessons can we learn from what other communities have done here? +- Papers: Are there any published papers or great posts that discuss this? If you have some relevant papers to refer to, this can serve as a more detailed theoretical background. + +This section is intended to encourage you as an author to think about the lessons from other languages, provide readers of your JEP with a fuller picture. +If there is no prior art, that is fine - your ideas are interesting to us whether they are brand new or if it is an adaptation from other languages. + + +# Unresolved questions + +- What parts of the design do you expect to resolve through the JEP process before this gets merged? +- What related issues do you consider out of scope for this JEP that could be addressed in the future independently of the solution that comes out of this JEP? + +# Future possibilities + +Think about what the natural extension and evolution of your proposal would +be and how it would affect the Jupyter community at-large. Try to use this section as a tool to more fully consider all possible +interactions with the project and language in your proposal. +Also consider how the this all fits into the roadmap for the project +and of the relevant sub-team. + +This is also a good place to "dump ideas", if they are out of scope for the +JEP you are writing but otherwise related. + +If you have tried and cannot think of any future possibilities, +you may simply state that you cannot think of anything. + +Note that having something written down in the future-possibilities section +is not a reason to accept the current or a future JEP; such notes should be +in the section on motivation or rationale in this or subsequent JEPs. +The section merely provides additional information. diff --git a/jupyter-enhancement-proposal-guidelines/jupyter-enhancement-proposal-guidelines.md b/jupyter-enhancement-proposal-guidelines/jupyter-enhancement-proposal-guidelines.md index a5bef181..0af808fa 100644 --- a/jupyter-enhancement-proposal-guidelines/jupyter-enhancement-proposal-guidelines.md +++ b/jupyter-enhancement-proposal-guidelines/jupyter-enhancement-proposal-guidelines.md @@ -1,61 +1,201 @@ # Jupyter Enhancement Proposals (JEP) Guidelines -## Problem -Enhancements to the Jupyter ecosystem are currently presented across a multitude of platforms without any centralized management or discussion. +**Note**: This JEP repo is the **canonical "source of truth"** for individual JEPs, +the JEP process, and activity on JEPs. -## Proposed Enhancement +## Goals of the JEP process -Jupyter Enhancement Proposals will be used when presenting changes or additions that affect multiple components of the Jupyter ecosystem. These changes can include things like: -* Additions/changes to the message spec -* API design that is consumed and produced in several contexts (e.g. the notebook kernel API) -* The notebook format +Project Jupyter uses the Jupyter Enhancement Proposal process (JEP) +to address distributed collaboration and experimentation. +The primary guiding principle of the JEP process is to +encourage collaboration and discussion as early as possible in the lifecycle +of a major proposed change in Jupyter, with the goal of preventing costly rework, +competing ideas, and unnecessary forking or fragmentation of ideas. -The format of this README is itself a JEP and can be duplicated for the creation of further JEPs. +Several sub-goals exist for this process: -## Detail Explanation +- **Maximize success of large proposals** that get stalled in the wrong venue (e.g. a single PR comment thread) +- Provide a **better alternative to “piecemeal” development** where multiple PRs + to build an end-to-end set of functionality are split across multiple GitHub + project without broad consensus, context, or guidance. +- Provide a **clear, time-limited, and authoritative process for work proposals**, + to facilitate funding conversations. (e.g. provide a concrete artifact to reference + in a grant proposal, roadmap item, etc.) +- Provide a **consolidated “stream” of all proposals across the entire Jupyter community** + that contributors of all levels can monitor and selectively engage in. -### JEP Titles +## Tenets of the JEP process -Jupyter Enhancement Proposals will be submitted with a title that is no longer than 12-words long. A JEP is uniquely identified by its title and the pull request number associated with it. +The JEP process operates under the following tenets: -### JEP Labels +- **The JEP process is intended for changes of non-trivial scope.** + “Non-trivial” is addressed below in the “JEP / Not-a-JEP Rubric” of this document. -The pull-request submitted with each JEP will be labeled with the following labels for easy searching: -* `accepted` — this JEP has been accepted and is currently being implemented -* `implemented` — this JEP has been implemented -* `rejected` - this JEP has been rejected and will not be implemented -* `withdrawn` - this JEP has been withdrawn by the submitter but can be re-submitted if someone is willing to champion it -* `active` - this JEP is currently under active discussion within the community +- **The JEP process naturally complements the PR process, but does not replace it.** + A thoroughly-reviewed and approved JEP is a valuable reference during a PR to + reduce friction, reduce time-consuming context sharing, and encapsulate decisions + and other discussions. Moving a premature PR into a JEP should be a lightweight + process that doesn’t cause friction for the contributor. -### JEP Structure + For example, GitHub issue and PR templates across the entire Jupyter project should have + references to the JEP process as a possible outcome of a given PR. -When submitting an enhancement proposal, individuals will include the following information in their submission. +- **There is one JEP repository, all Jupyter-governed projects must use it.** + To faciliate the easiest possible adoption and high visibility of ideas, a + single JEP repository will be used. Even if a JEP only applies to a single organization. -1. The problem that this enhancement addresses. If possible include code or anecdotes to describe this problem to readers. -2. A brief (1-2 sentences) overview of the enhancement you are proposing. If possible include hypothetical code sample to describe how the solution would work to readers. -3. A detailed explanation covering relevant algorithms, data structures, an API spec, and any other relevant technical information -4. A list of pros that this implementation has over other potential implementations. -5. A list of cons that this implementation has. -6. A list of individuals who would be interested in contributing to this enhancement should it be accepted. +- The JEP process **has multiple valid use cases**. Each use case might have a + slightly different expected workflow. Some expected use cases include: -### JEP Submission Process -1. Create a [Markdown](https://help.github.com/articles/github-flavored-markdown/) write up of the problem, proposed enhancement, detailed technical explanation, pros and cons, and interested contributors of the enhancement you are proposing. -2. Create a fork of this repository. -3. Create a folder with its name set to the JEP title in lower snake-case. -3. Place the markdown file created in step 1 and any supplemental materials in that folder. -4. Submit a pull request to the main repository with your JEP. -5. Once your PR is accepted, it will be labeled `active` per the guidelines above. -6. Your JEP will be added to the JEP Index file in this repository. + - Non-trivial feature proposals within a single component that would benefit from + process. (e.g., a non-trivial change to JupyterLab that would benefit from + formal process within the JupyterLab project) + - Non-trivial features or improvements that span multiple projects. + - Any proposed changes to published APIs or core specifications (e.g., nbformat) + - Changes to the JEP process itself. + - Launching a major project in one of the Jupyter GitHub organizations. + + +## JEP Submission Workflow + +### Phase 1: Pre-proposal + +This is the least formal stage of any jupyter enhancement proposals. During this +phase, discussions on the mailing list, community forum, in-person, on github issues +are all fine to gauge community interest, feasibility, consequences, and to +scope out technical impact and implementation details. -## Pros and Cons +In order to transition out of the pre-proposal stage, the following checklist must be complete: -Pros associated with this implementation include: -* A higher quality discussion around enhancement proposals -* Individuals are encourage to put more thought into an enhancement proposal before submitting it -* Precedence exists in the form of PEPs (Python Enhancement Proposals) and IPEPs (IPython Enhancement Proposals) +1. **Create a GitHub issue** in the Jupyter Enhancement Proposals repo that + 1. Briefly outlines the proposal + 2. Suggests a review team (optional) + 3. Declares why it should be a JEP (See the “JEP / Not-a-JEP Rubric” below.) +2. **Identify a Shepherd** to see the process through. (Shepherds are assigned + on a round-robin basis from a set of interested engaged volunteers). +3. **Decide if it's a JEP** according to the criteria listed above. The Shepherd decides if the JEP criteria have been met. -Cons associated with this implementation include: -* Existing IPEPs (IPython Enhancement Proposals) will not be included in this migrated repository +**If it's a JEP**. The JEP author creates a new PR with the contents of the JEP proposal. See [this markdown template](JEP-TEMPLATE.md) for a suggested structure of the pull-request. + +Subsequent discussion and decisions about the JEP will happen in that PR, and +we now **Move to the RFC phase**. + +**If it’s not a JEP**. The shepherd provides a reason for why the issue +doesn't meet JEP criteria, and closes the issue. + +### Phase 2: Request for Comments for the JEP + +Phase two begins when **the JEP author submits a draft of the JEP as a PR to the JEP repository**. +The Shepherd assigns a number to the JEP according to the pull-request number, and updates +the README of the JEP repository with information about the now in-progress JEP. +The Shepherd then works with the JEP author to +**identify and notify relevant individuals in the Jupyter community to review and comment**. + +Once this group has been notified, the JEP process enters the RFC phase. + +During the **Request For Comment (RFC) phase**, the proposal is iterated on with +feedback from the Review Team and the community at large. The Shepherd helps engage +the Review Team and encourage productive discussion. After the Review Team members +have signed off on the JEP, with the criteria that there are no major objections, +and at least some of the Review Team are in favor, the Shepherd initiates the Final Comment Period. + +In the **Final Comment Period (FCP)**, the community at-large has at least 10 calendar days +to provide any final comments on the JEP. A JEP may revert back to RFC if +objections from the community are supported by the Review Team. + +At the end of the FCP, the JEP is either: + +* **approved**, in which case the PR is merged and work may commence on implementing the JEP (see Phase 3, below) +* **rejected**, in which case the PR is closed +* asked to return to the RFC phase. + +In each case, the JEP's status should be updated in the README of the +`enhancement-proposals` repository. + +### Phase 3: Work Commences + +Once a JEP has been merged into the `jupyter/enhancement-proposal` repository, +implementation can commence on the JEP. The JEP author does not need to implement the JEP themselves. + +If the JEP requires one or more pull-requests to be implemented, the author of the PRs should +provide a reference to the JEP so that reviewer has background context on the JEP. +As the JEP is being implemented, the JEP text should be amended with with addendums to +denote the progress of the implementation using the following stages. + +1. In progress implementation via (list of PRs). +2. Fully implemented via (list of PRs). + +If in the course of implementation, it is discovered that the implementation needs to +be radically different from what was defined in the original JEP, then a pull +request is submitted to modify the original JEP with the new necessary implementation, +and a note citing the need for a modification to the JEP. +This pull request should be re-approved by the original review team. + +### JEP Pathways and Status + +Below is a rough guide that describes the JEP process and its possible pathways. + +``` + +-----------+ + | | + +-----------+ | withdrawn | + | | | | + | rejected | +-----------+ + | | + +-----^-----+ + | + | ++--------------+ +-----------+ +-----+-----+ +-------------+ +-----------+ +| | | | | | | | | | +| pre-proposal +---> submitted +--------------> RFC/FCP +-------------> in progress +---> completed | +| | | | identify | | approved | | | | ++--------------+ +-----------+ review +-----+-----+ +-------------+ +-----------+ + team | + | + +-----v-----+ + | | + | postponed | + | | + +-----------+ + +``` + + +## What qualifies as a JEP? + +This section contains a set of principles to help determine when something is a JEP. +These should be used to determine when something becomes a PR during the JEP +pre-proposal stage, as well as to determine when a PR becomes a JEP at an individual repo level. + +**Principles to follow** + +Below are a few example guidelines to follow when deciding if an idea should include +a JEP (If yes, it requires a JEP). Under each question is a relevant example proposal. + +- Does the proposal/implementation require PRs across multiple orgs? + - **Example:** Defining a unique cell identifier +- Does the proposal/implementation PR impact multiple orgs, or have widespread community impact? + - **Example:** Updating nbformat +- Does the proposal/implementation change an invariant in one or more orgs? + - **Example:** Defining a unique cell identifier + - **Example:** Deferred kernel startup +- Does the proposal/implementation create a new concept that will impact multiple repositories? + - **Example:** Sandboxed cell outputs +- Does the proposal involve creating a new repository or sub-project? + +## The JEP public archive + +A public website contains a readable archive of all JEP proposals. +It contains list of all JEPs that have entered a "final" state +(e.g., "Completed", "Withdrawn", "Rejected"). The content of each JEP will +be displayed in a readable fashion. When a JEP enters into a final state, it +is added to this website. + +Note that the JEPs themselves contain the content, while the website is just a +quick way to display them in a reading-friendly format. + +## Background + +For a background of the JEP process, and recent efforts to improve it, see +[the meta-JEP readme](../29-jep-process/README.md). -## Interested Contributors -@captainsafia, @rgbkrk