Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CEP 4 - Plugins Mechanism Implementation #32

Merged
merged 9 commits into from
Aug 24, 2022
137 changes: 137 additions & 0 deletions cep-4.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
<table>
<tr><td> Title </td><td> Implement initial conda plugin mechanism </td>
<tr><td> Status </td><td> Accepted </td></tr>
<tr><td> Author(s) </td><td> Bianca Henderson &lt;bhenderson@anaconda.com&gt;</td></tr>
<tr><td> </td><td>Jannis Leidel &lt;jleidel@anaconda.com&gt; </td></tr>
<tr><td> Created </td><td> July 5, 2022 </td></tr>
<tr><td> Updated </td><td> August 22, 2022 </td></tr>
<tr><td> Discussion </td><td>https://github.com/conda-incubator/ceps/pull/32</td></tr>
<tr><td> Implementation </td><td>https://github.com/conda/conda/pull/11435</td></tr>
</table>

## Abstract

In order to enable customization and extra features that are compatible with and discoverable by `conda` (but do not necessarily ship as a default part of the `conda` codebase), we would like to implement an official `conda` plugin mechanism.

A plugin mechanism in `conda` would provide many benefits, including (but not limited to) the coverage of underserved use cases, the added ability to extend `conda` internals via official APIs, as well as lowering the barrier of entry for contributions from other stakeholders in the `conda` community and ecosystem.

The [initial proposal to add this plugin architecture](https://github.com/conda-incubator/ceps/blob/main/cep-2.md) has been [officially approved](https://github.com/conda-incubator/ceps/issues/23); this current CEP will discuss the specific way that the plugin mechanism will actually be implemented.

## Specification

Plugins in `conda` will integrate the "hook + entry point" structure by utilizing the [`pluggy`](https://pluggy.readthedocs.io/en/stable/index.html) Python framework. This implementation can be broken down via the following two steps:

- Define the hook(s) to be registered
- Register the plugin under the `conda` entrypoint namespace


### Hook

Below is an example of a very basic plugin "hook":

_my_plugin.py_
```python
import conda.plugins


@conda.plugins.register
def conda_subcommands():
...
```

### Packaging via a `pyproject.toml` file


Below is an example that configures `setuptools` using a `pyproject.toml` file (note that the `setup.py` file is optional if a `pyproject.toml` file is defined):

_pyproject.toml_
```toml
[build-system]
requires = ["setuptools", "setuptools-scm"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on my personal experience, I would avoid explicitly defining a full example of how to package a project like this and instead focus on how to specify an entry point. Otherwise I could see people copy-and-pasting either example for all their plugins as the way to package conda plugins.

I would also list the pyproject.toml example first as that's the standard.

build-backend = "setuptools.build_meta"

[project]
name = "my-conda-plugin"
version = "1.0.0"
description = "My conda plugin"
requires-python = ">=3.7"
dependencies = ["conda"]

[project.entry-points."conda"]
my-conda-plugin = "my_plugin"
```


### Packaging via a `setup.py` file

Below is an example of an entry point namespace for the custom plugin function, decorated with the plugin hook shown in the "Hook" section above:

_setup.py_
```python
from setuptools import setup

setup(
name="my-conda-plugin",
install_requires="conda",
entry_points={"conda": ["my-conda-plugin = my_plugin"]},
py_modules=["my_plugin"],
)
```


## Rationale

This new `conda` plugin API ecosystem will bring about many benefits and possibilities, including but not limited to:

- Custom subcommands
- Support for packaging-related topics (_e.g._, virtual packages)
- Development environment integrations (_e.g._, shells)
- Alternative dependency solver backends
- Experimental features that are not currently covered by `conda`

As mentioned previously, an official plugin ecosystem will also enable contributors across the `conda` community to develop and share new features, thus bringing about more functionality and focus on the user experience.

## Backwards Compatibility

There are no expected backwards compatibility issues for this new feature.
<!-- ??? -->

## Alternatives

The following lists alternative Python frameworks, libraries, and packages that were considered for use in this proposed `conda` plugin implementation:

- [Hookman](https://github.com/ESSS/hookman)
marcelotrevisani marked this conversation as resolved.
Show resolved Hide resolved
- [`stevedore`](https://docs.openstack.org/stevedore/latest/)
- [`importlib-resources`](https://pypi.org/project/importlib-resources/)
- [`importlib-metadata`](https://pypi.org/project/importlib-metadata/)

Ultimately, `pluggy` was decided on as the ideal framework for this project due to its extensive documentation and relatively straightforward "hook + entry point" configuration.
marcelotrevisani marked this conversation as resolved.
Show resolved Hide resolved

## Implementation

The [pull request for the initial `conda` plugin mechanism implementation](https://github.com/conda/conda/pull/11435) includes extensive documentation as well as a tutorial on how to implement a basic custom subcommand. Please add any implementation-related suggestions directly to this pull request.

The method of how these plugins will be shared/distributed is currently undecided and will be discussed in a future CEP.

## Resolution

This was a standard, non-timed-out vote.

- Among [Steering Council members](https://github.com/conda-incubator/governance/blob/eaf59a5779dc1f678bee4453ceb92fd733e7306a/steering.csv) there are 10 "yes", 0 "no", and no abstentions.
- No [Emeritus Steering member](https://github.com/conda-incubator/governance/blob/eaf59a5779dc1f678bee4453ceb92fd733e7306a/emeritus.csv) voted.

This vote has reached quorum (10 + 0 = 10 which is at least 60% of 16).

It has also passed since it recorded 10 "yes" votes and 0 "no" votes giving 10/10 which is greater than 60% of 15.

It should be noted that a request for change was recorded in the pull request about minor implementation details that do not invalidate the previous votes. The author made the requested change.

## Reference

- [`pluggy` Documentation](https://pluggy.readthedocs.io/en/stable/index.html)
- [CEP 2: "Add plugin architecture to conda"](https://github.com/conda-incubator/ceps/blob/main/cep-2.md)
- [`conda` Issue #11112: Introduce a Plugin System](https://github.com/conda/conda/issues/11112)

## Copyright

All CEPs are explicitly [CC0 1.0 Universal](https://creativecommons.org/publicdomain/zero/1.0/).