Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Document working locally + Conan 2.0 commands #13102

Merged
merged 13 commits into from
Oct 6, 2022
30 changes: 13 additions & 17 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,27 +11,28 @@ The following summarizes the process for contributing to the CCI (Conan Center I

## Community

Conan Center Index is an Open Source MIT licensed project.
Conan Center Index is developed by the Conan maintainers and a great community of contributors.
ConanCenterIndex is an Open Source MIT licensed project; it is developed by the Conan maintainers and a great community of contributors.

## Dev-flow & Pull Requests

CCI follows the ["GitFlow"](https://datasift.github.io/gitflow/IntroducingGitFlow.html) branching model.
Issues are triaged and categorized mainly by type (package request, bug...) and priority (high, medium...) using GitHub
labels.
labels.

To contribute follow the next steps:

1. Comment in the corresponding issue that you want to contribute the package/fix proposed. If there is no open issue, we strongly suggest
to open one to gather feedback.
2. Check the [how_to_add_packages.md](docs/how_to_add_packages.md) if are
contributing for a new package.
3. Fork the [CCI repository](https://github.com/conan-io/conan-center-index) and create a `package/xxx` branch from the `master` branch and develop
opening one to gather feedback.
2. Get setup by following [Developing Recipes](docs/developing_recipes_locally.md) guide and learn the basic commands.
3. Check the [How To Add Packages](docs/how_to_add_packages.md) for the break down ConanCenterIndex specific conventions and practices.
4. In your fork create a `package/xxx` branch from the `master` branch and develop
your fix/packages as discussed in previous step.
4. Try to keep your branch updated with the `master` branch to avoid conflicts.
5. Add the text (besides other comments): "fixes #IssueNumber" in the body of the PR, referring to the issue of step 1.
5. [Submit a pull request](docs/how_to_add_packages.md#submitting-a-package) once you are ready. This can be when you
got everything working or even if you need help. Add the text to the issue body (besides other comments): "fixes #IssueNumber"
in the body of the PR, referring to the issue of step 1.

The ``conan-io`` organization maintainers will review and help with the packaging.
The Conan Community works hard to review all the pull requests and provided assistance where need.
The [Review Process](docs/review_process.md) is partially automated with the help of @conan-center-index-bot :rocket:

## Issues

Expand All @@ -50,10 +51,5 @@ For any suggestion, feature request or question open an issue indicating the fol
- Try to explain the motivation, what are you trying to do, what is the pain it tries to solve.
- What do you expect from CCI.

We use the following tags to control the status of the issues:

- **infrastructure**: Waiting on tools or services belonging to the infra.
- **library request**: Request a new package to be created.
- **question**: Further information is requested .
- **upstream update**: Bump a new package version.
- **conan.io/center**: Issues and features related to Web UI .
We use the following tags to control the status of the issues and pull requests, you can learn more in [Labels](docs/labels.md) document
which details the important one and their roles.
17 changes: 10 additions & 7 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,23 @@
<img src="../assets/JFrogConanCenter.png" width="600"/>

This repository, ConanCenterIndex, contains recipes for the remote [JFrog ConanCenter](https://conan.io/center/).
This remote is added by default to a clean installation of the Conan client. Recipes are contributed by opening pull requests as explained in the sections below.
When pull requests are merged, the CI will upload the generated packages to the [ConanCenter](https://conan.io/center/) remote.

# Documentation

* [Code of conduct](code_of_conduct.md)
* [Changelog](changelog.md)

* User documentation
+ [Contributing to Conan Center Index](../CONTRIBUTING.md)
+ [Adding Packages to ConanCenter](how_to_add_packages.md)
+ [Policy about patching](policy_patching.md)
+ [Developing Recipes Locally](developing_recipes_locally.md)
+ [Adding Packages to ConanCenter](how_to_add_packages.md) :point_left: Best place to learn how to contribute
+ [Errors from the conan-center hook (KB-Hxxx)](error_knowledge_base.md)
+ [Review Process](review_process.md)
+ [Packaging policy](packaging_policy.md)
+ [Labels](labels.md)
+ [Supported platforms and configurations](supported_platforms_and_configurations.md)
+ [Errors from the conan-center hook (KB-Hxxx)](error_knowledge_base.md)
+ [FAQs](faqs.md)
+ [Consuming Recipes](consuming_recipes.md) :information_source: Learn how to limit the impact of recipe changes
+ [Community Resources](community_resources.md)
+ [Review Guidelines](reviewing.md)
+ [Labels](labels.md)
+ [Preparing recipes for Conan 2.0](v2_migration.md)
+ [FAQs](faqs.md)
8 changes: 4 additions & 4 deletions docs/consuming_recipes.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ There can be several causes if a recipe (a new revision) might stopped to work i

This use case is covered by the [`required_conan_version`](https://docs.conan.io/en/latest/reference/conanfile/other.html?highlight=required_conan_version#requiring-a-conan-version-for-the-recipe) feature. It will
substitute the syntax error by one nicer error provided by Conan client.

To be sure that people using these new experimental features are using the required Conan version and testing the actual behavior
of those features (feedback about them is very important to Conan).

Expand All @@ -47,8 +47,8 @@ Here are a few choices:
- [Running your own Conan Server](https://docs.conan.io/en/latest/uploading_packages/running_your_server.html) - great for local ad-hoc setups
- [Cache recipes in your own ArtifactoryCE](https://docs.conan.io/en/latest/uploading_packages/using_artifactory.html) - recommended for production environments

Using your own ArtifactoryCE instance is easy. You can [deploy it on-premise](https://conan.io/downloads.html) or use a
[cloud provided solution](https://jfrog.com/community/start-free) for **free**. Your project should
Using your own ArtifactoryCE instance is easy. You can [deploy it on-premise](https://conan.io/downloads.html) or use a
[cloud provided solution](https://jfrog.com/community/start-free) for **free**. Your project should
[use only this remote](https://docs.conan.io/en/latest/reference/commands/misc/remote.html?highlight=add%20new) and new recipe
revisions are only pushed to your Artifactory after they have been validated in your project.

Expand All @@ -59,7 +59,7 @@ The minimum solution, if still choosing to rely on ConanCenter directly, involve
This feature needs to be enabled in Conan 1.x, see the [Activation Instructions](https://docs.conan.io/en/latest/versioning/revisions.html#how-to-activate-the-revisions) for details.
- [Lockfiles](https://docs.conan.io/en/latest/versioning/lockfiles.html) can be created with the `conan lock create` and read with by
adding `--lockfile=conan.lock` to `conan install` or `conan create` commands. See the [lockfile introduction](https://docs.conan.io/en/latest/versioning/lockfiles/introduction.html#) for more information.

> **Warning** Please, be aware there are some known bugs related to lockfiles that are not being fixed in Conan v1.x - we are really excited for the 2.0 improvements to be widely used.

Both of these give you better control and will allow you to choose when to upgrade your Conan client.
Expand Down
214 changes: 214 additions & 0 deletions docs/developing_recipes_locally.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,214 @@
# Developing Recipes Locally

Before you can contribute any code changes, you'll need to make sure you are familiar with the Conan client and have an environment that is conducive to developing recipes.

This file is intended to provide all the commands you need to run in order to be an expert ConanCenter contributor.

<!-- toc -->
## Contents

* [Clone your fork](#clone-your-fork)
* [Setup your environment](#setup-your-environment)
* [Installing the ConanCenter Hooks](#installing-the-conancenter-hooks)
* [Updating conan hooks on your machine](#updating-conan-hooks-on-your-machine)
* [Basic Commands](#basic-commands)
* [Try it yourself](#try-it-yourself)
* [Debugging Failed Builds](#debugging-failed-builds)
* [Running the Python Linters](#running-the-python-linters)
* [Testing the different `test__package`](#testing-the-different-test__package)
* [Testing more environments](#testing-more-environments)
* [Using Conan 2.0](#using-conan-20)<!-- endToc -->

## Clone your fork

1. Follow the GitHub UI to [fork this repository](https://github.com/conan-io/conan-center-index/fork)
2. [Clone your fork](https://docs.github.com/en/repositories/creating-and-managing-repositories/cloning-a-repository)

## Setup your environment

1. Install a C++ development toolchain - ConanCenter's [build images](#testing-more-environments) are available
2. [Install the Conan client](https://docs.conan.io/en/latest/installation.html) - make sure to keep it up to date!
prince-chrismc marked this conversation as resolved.
Show resolved Hide resolved
3. Install CMake - this is the only tool which is assumed to be present
[see FAQ](faqs.md#why-recipes-that-use-build-tools-like-cmake-that-have-packages-in-conan-center-do-not-use-it-as-a-build-require-by-default) for details.

> **Note**: It's recommended to use a dedicated Python virtualenv when installing with `pip`.

### Installing the ConanCenter Hooks

The system will use the [conan-center hooks](https://github.com/conan-io/hooks) to perform some quality checks. You can install the hooks by running:

```sh
conan config install https://github.com/conan-io/hooks.git -sf hooks -tf hooks
conan config set hooks.conan-center
```

The hooks will show error messages but the `conan create` won’t fail unless you export the environment variable `CONAN_HOOK_ERROR_LEVEL=40`.
All hooks checks will print a similar message:

```txt
[HOOK - conan-center.py] post_source(): [LIBCXX MANAGEMENT (KB-H011)] OK
[HOOK - conan-center.py] post_package(): ERROR: [PACKAGE LICENSE] No package licenses found
```

#### Updating conan hooks on your machine

The hooks are updated from time to time, so it's worth keeping your own copy of the hooks updated regularly. To do this, simply run:

```sh
conan config install
```

## Basic Commands

We recommend working from the `recipes/project` folder itself. You can learn about the [recipe file structure](how_to_add_packages.md#recipe-files-structure) to understand the folder and files located there.

> **Note**: You can only change one recipe per pull request, and working from the [_recipe folder_](how_to_add_packages.md#the-recipe-folder-conanfilepy) will help prevent making a few mistakes. The default for this folder is `all`, follow the link above to learn more.

The [entire workflow of a recipe](https://docs.conan.io/en/latest/developing_packages/package_dev_flow.html) can be execute with the [`conan create`](https://docs.conan.io/en/latest/reference/commands/creator/create.html). This should look like:

* `conan create all/conanfile.py 0.0.0@ -pr:b=default -pr:h=default`

ConanCenter also has a few [support settings and options](supported_platforms_and_configurations.md) which highly recommend to test. For example
`conan create all/conanfile.py 0.0.0@ -o project:shared=True -s build_type=Debug` is a easy way to test more configurations ensuring the package is correct.

### Try it yourself

For instance you can create packages for `fmt` in various supported configurations by running:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to purpose fmt as the standard example... its quick to build and has all the configuration supported by CCI and no deps so it should always work 🤞

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fmt is a good choice.


```sh
cd recipes/fmt
conan create all/conanfile.py fmt/9.0.0@ -pr:b=default -pr:h=default
conan create all/conanfile.py fmt/9.0.0@ -o fmt:header_only=True -pr:b=default -pr:h=default
conan create all/conanfile.py fmt/9.0.0@ -s build_type=Debug -o fmt:shared=True -pr:b=default -pr:h=default
```

## Debugging Failed Builds

Some common errors related to Conan can be found on [troubleshooting](https://docs.conan.io/en/latest/faq/troubleshooting.html) section.
For ConanCenter Hook errors, go to the [Error Knowledge Base](error_knowledge_base.md) page to know more about those.

To test with the same environment, the [build images](supported_platforms_and_configurations.md#build-images) are available.
Instructions for using these images can be found in [Testing more environments](#testing-more-environments) section.

In ConanCenterIndex, the most common failure point is upstream
build script that are tailored to their specific use cases. It's not uncommon to [patch build scripts](policy_patching.md#policy-about-patching) but make sure to read the [patch policy](policy_patching.md). You are encouraged to submit pull requests upstream.

## Running the Python Linters

Linters are always executed by Github actions to validate parts of your recipe, for instance, if it uses migrated Conan tools imports.

It is possible to run the linter locally the same way it is being run [using Github actions](../.github/workflows/linter-conan-v2.yml) by:

* (Recommended) Use a dedicated Python virtualenv.
* Ensure you have required tools installed: `conan` and `pylint` (better to uses fixed versions)

```sh
pip install conan~=1.0 pylint==2.14
```

* Set environment variable `PYTHONPATH` to the root of the repository

```sh
export PYTHONPATH=your/path/conan-center-index
```

* Now you just need to execute the `pylint` commands:

```sh
# Lint a recipe:
pylint --rcfile=linter/pylintrc_recipe recipes/fmt/all/conanfile.py

# Lint the test_package
pylint --rcfile=linter/pylintrc_testpackage recipes/fmt/all/test_package/conanfile.py
```

## Testing the different `test_*_package`

This can be selected when calling `conan create` or separately with `conan test`

```sh
# By adding the `-tf` argument
conan create recipes/fmt/all/conanfile.py 9.0.0@ -tf test_v1_package/ -pr:b=default -pr:h=default
prince-chrismc marked this conversation as resolved.
Show resolved Hide resolved
```

```sh
# Passing test package's conanfile directly (make sure to export first)
conan test recipes/fmt/all/test_v1_package/conanfile.py fmt/9.0.0@ -pr:h=default -pr:b=default
```

## Testing more environments

This can be difficult for some platforms given virtualization support.

For Windows and MacOS users, you can test the Linux build environments with the Docker build images.

Assuming you've already tested it locally and it's been successfully exported to your cache, you can:

1. Creating a new profile.
* You can also download them from CCI build summary
2. Build missing packages

Example.

```sh
docker run -v/Users/barbarian/.conan:/home/conan/.conan conanio/gcc8 bash -c "conan profile new --detect gcc8"
docker run -v/Users/barbarian/.conan:/home/conan/.conan conanio/gcc8 bash -c "conan install -pr gcc8 fmt/9.0.0@ --build missing"
```

prince-chrismc marked this conversation as resolved.
Show resolved Hide resolved
> **Note**: If you are running on Mac M1, the follow Docker argument is required: `--platform=linux/amd64`

## Using Conan 2.0

Everything you need to know about the methods, commands line, outputs can be found in the
[Conan 2.0 Migrations](https://docs.conan.io/en/latest/conan_v2.html) docs.

This should be non-intrusive. Conan 2.0 by default has a different `CONAN_USER_HOME` location, which means that it has separate caches, profiles, and settings.
This will leave your Conan 1.0 setup completely intact when using Conan 2.0.

> **Note**: There are substantial changes to the CLI so very few of the commands will remain the same.
> The new [Unified Command Pattern](https://docs.conan.io/en/latest/migrating_to_2.0/commands.html#unified-patterns-in-command-arguments),
> as an example, changes how settings and options are passed.

### Installing Conan 2.0 beta

Simply install Conan 2.0 with `pip install conan --upgrade --pre`.

You can confirm the installation with:

```sh
$ conan --version
Conan version 2.0.0-beta3
$ conan config home
Current Conan home: /Users/barbarian/.conan2
```

> **Note**: You will most likely see
>
> ```sh
> Initialized file: '/Users/barbarian/.conan2/settings.yml'
> Initialized file: '/Users/barbarian/.conan2/extensions/plugins/compatibility/compatibility.py'
> Initialized file: '/Users/barbarian/.conan2/extensions/plugins/compatibility/app_compat.py'
> Initialized file: '/Users/barbarian/.conan2/extensions/plugins/compatibility/cppstd_compat.py'
> Initialized file: '/Users/barbarian/.conan2/extensions/plugins/profile.py'
> ```
>
> When running the client for the first time.

You will need to setup profiles. This is one of the changes in 2.0. The default profile is now opt-in and no longer generated automatically.

```sh
conan profile detect
```

> **Warning**: This is a best guess, you need to make sure it's correct.

### Trying it out

Try building an existing recipe. We'll repeat the 1.x example with `fmt` to build the same configurations:

```sh
cd recipes/fmt
conan create all/conanfile.py --version 9.0.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is not dual profile mandatory for Conan 2.0?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the defaults that changed. But you do not need to manually specify it

the defaults match what the 1.x docs are showing

conan create all/conanfile.py --version 9.0.0 -o fmt/9.0.0:header_only=True
conan create all/conanfile.py --version 9.0.0 -s build_type=Debug -o fmt/9.0.0:shared=True
```
Loading