Skip to content

Git and Github Guide

Jump to bottom
Jordan Padams edited this page Jun 29, 2023 · 43 revisions

Getting Started with Github

Instead of reinventing the wheel, there are countless trainings online. Here are few good ones we have found:

GitHub Workflow for Developers

The PDS generally follows the GitHub flow, which is a lightweight, branch-based workflow that supports teams and projects where deployments are made regularly. The guide here explains how and why GitHub flow works. Depending upon the application, we may skip the official "deploy" step, but there is usually some level of deployment, at least locally, in order to properly test the application.

Creating a New Repo

Creating a new repo under NASA-PDS org is restricted to the @NASA-PDS/pds-software-pmc team.

New repositories can be of 3 types:

  • library: a re-usable piece of code. The CICD must publish stable versions on artifactories such as maven, pypi, npm...
  • micro-service: an elementary component which will run as a service. The CICD must publish stable version on a docker image repository (e.g. docker hub, AWS/ECR)
  • application: a user facing application. The repository contains, user's manual, integration tests and deployment scripts configuration (e.g. docker-compose, terraform...). Applications can either refer to libraries and microservices but can also contain their own code.

When creating a new repo, here are some things we should set by default for each repo:

  • Use template repo whenever possible:
  • Update CODEOWNERS with the applicable team that will manage the code, if different than @NASA-PDS/pds-software-committers
  • Assign teams:
    • Always assign the @NASA-PDS/pds-software-committers team and give it write access
    • Always assign the @NASA-PDS/pds-software-pmc team and give it admin access
    • Always assign the @NASA-PDS/pds-operations team and give it admin access
    • Always assign the CODEOWNERS team(s) and give it admin access
  • Enable GitHub pages:
    • From Settings → Pages → Branch gh-pages, 📁 / Root, then click Save
  • Setup main branch protections:
    • Go to Settings → Branches → Branch Protections → Branch Protections
    • Enter Branch name pattern = main
    • Select:
      • Require pull request reviews before merging with Required approving reviews:1
      • Require review from Code Owners
      • Require status checks to pass before merging
      • Require branches to be up to date before merging
      • Restrict who can push to matching branches
      • Allow force pushes
      • Allow deletions
    • Click Create to submit
  • Add the DOI assignment webhook (only for repositories that should have digital object identifiers assigned to releases):
    • Go to Zenodo and click Login
    • Log in with GitHub; use the PDS Engineering Node Continuous Integration account, pdsen-ci. You can use your own account, however, Zenodo makes frequent use of the GitHub API and those calls will be charged to your own account. If you are concerned about API limits, use the pdsen-ci account instead.
    • From the profile menu (upper right), click the ▼ and choose GitHub.
    • Find the new repository under "Repositories" and slide the switch from Off to On.
    • On GitHub or in a clone of the new repository, create a new file .zenodo.json with this single line—{"license": "Apache-2.0", "communities": [{"identifier": "nasa-pds"}]}—and commit this change. `
  • Update the README.md and remove/replace any placeholder information.

Python Repos

After you create the repo:

  1. Create the Pypi.org and test.pypi.org projects
  2. Add, at minimum, the following people to the project with Role = Owner:
    1. pdsen-ci
    2. jpadams
    3. tloubrieu
    4. nutjob
    5. viviant
    6. collinss-jpl
    7. alexdunnjpl

DOI Assignment

If the DOI assignment webhook was set up above (and the .zenodo.json file created), then each time a release is made (whether automatically or by hand), a new digital object identifier will be assigned to the release (and one assigned one time only to the entire repository, called a "concept DOI"). Once a DOI is assigned, you can add a badge to the repository's README by adding (using Markdown syntax) the following:

[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.XXXXXXX.svg)](https://doi.org/10.5281/zenodo.XXXXXXX)

Replace XXXXXXX with the concept DOI that got assigned.

Github Notifications

Github email notifications can become very difficult to parse through without proper email management and setup. The following section helps you pull high priority emails out to ensure you do not miss out on various activities that require your more immediate attention.

Setup Notifications

To ensure you are up-to-date with all things happening in repositories, check your notification settings send you emails mobile notifications when needed. For PDS development, at minimum, please check the following:

  • Participating:
    ✅ Email ✅ Web and Mobile

  • Watching:
    ✅ Email ✅ Web and Mobile

Managing Repository Notifications

You can manage the notifications you receive for specific repositories from here: https://github.com/watching . From this page, you can determine what types of notifications you would like to receive from repositories you are watching.

For more information on managing notifications, see the Github documentation

@mentions Email Filter

Think of a Github @mention as a direct message, versus other issue and commit notifications being group messages. It is highly recommended you setup an email filter to pull out these @mentions (aka direct messages) so you are able to respond in a timely manner.

Here is an example email filter from Microsoft Exchange:

When a new message arrives that meets all these conditions:
  Body     Contains    @my_username
  Sent To  Contains    mention@noreply.github.com
  From     Contains    github.com

Do the following:
  Move to Folder      Github-@mentions (you will need to create this folder)

[checked] Do not apply other rules to message that meet the same conditions

[checked] Enabled

Pull Requests Review Requested Self Email Filter

In order to prioritize pull request reviews where you are explicitly called out above other Github email notifications, this email filter will help you pull out emails where your review is requested.

Note: This applies to both personal reviews and reviews requested of a team you belong to.

Here is an example email filter from Microsoft Exchange:

When a new message arrives that meets all these conditions:
  Body     Contains    requested your review
  Sent To  Contains    review_requested@noreply.github.com
  From     Is          notifications@github.com

Do the following:
  Move to Folder      Github-@reviews (you will need to create this folder)

[checked] Do not apply other rules to message that meet the same conditions

[checked] Enabled

Pull Requests Review Requested Team Email Filter

In order to prioritize pull request reviews for Github teams that you are apart of above other Github email notifications, this email filter will help you pull out emails where your review is requested.

Note: This applies to both personal reviews and reviews requested of a team you belong to.

Here is an example email filter from Microsoft Exchange:

When a new message arrives that meets all these conditions:
  Sent To  Contains    review_requested@noreply.github.com
  From     Is          notifications@github.com

Do the following:
  Move to Folder      Github-@reviews-team (you will need to create this folder)

[checked] Do not apply other rules to message that meet the same conditions

[checked] Enabled

Github Discussions Email Filter

This email filter will pull all discussion-related notifications into a folder.

NOTE: Make sure the @mentions filter comes before this one.

When a new message arrives that meets all these conditions:
  Subject     Contains    "(Discussion #"
  From     Contains    notifications@github.com

Do the following:
  Move to Folder      Github-@discussions (you will need to create this folder)

[checked] Do not apply other rules to message that meet the same conditions

[checked] Enabled

Github Notifications Email Filter

This email filter is intended to be a catchall for remaining github notifications for a particular organization. This can be adapted to handle individual repositories as you see fit.

NOTE: Make sure this is the last email filter in your list, otherwise the filters created above will be ignored.

Here is an example email filter for the NASA-PDS Github Org through Microsoft Exchange:

When a new message arrives that meets all these conditions:
  Subject  Contains    NASA-PDS/
  From     Contains    "noreply@github.com" or "notifications@github.com"

Do the following:
  Move to Folder      PDS-Github (you will need to create this folder)

[checked] Do not apply other rules to message that meet the same conditions

[checked] Enabled

Here is an example email filter for the PDS-Data-Dictionaries Github Org through Microsoft Exchange:

When a new message arrives that meets all these conditions:
  Subject  Contains    pds-data-dictionaries/
  From     Contains    "noreply@github.com" or "notifications@github.com"

Do the following:
  Move to Folder      pds-ldd-github (you will need to create this folder)

[checked] Do not apply other rules to message that meet the same conditions

[checked] Enabled

Git Secrets

Accidentally committing a change that contains a credential—such as a password or API key—is regrettable but avoidable thanks to Git Secrets. The Engineering Node's template repositories include support for Git Secrets in their pre-commit hooks.

However, one extra step must be done one time only: setting up your Git global configuration to understand the patterns of secrets to avoid and those to allow. Doing so is easy:

  1. Install Git Secrets
  2. Run git secrets --register-aws --global

This adds patterns to your ~/.gitconfig so that commits and commit messages that contain AWS secrets will be aborted. You can edit your ~/.gitconfig (or use git secrets --add) to include more patterns to watch out for.

Commit Message Best Practices

For commit message best practices, the PDS tries to follow these Github Commit Messages Guidelines where possible.

Once you have read through the guidelines, here are some highlights to touch on and some additional notes:

A properly formed git commit subject line should always be able to complete the following sentence

If applied, this commit will <your subject line here>
  • Rules for a great commit message style
    • Separate subject from body with a blank line
    • Do not end the subject line with a period
    • Capitalize the subject line and each paragraph
    • Use the imperative mood in the subject line
    • Wrap lines at 72 characters
    • Use the body to explain what and why you have done something. In most cases, you can leave out details about how a change has been made.
  • References in commit messages
    • If the commit refers to an issue, add this information to the commit message header or body.
    • See how to autolink references and urls for more information on linking to other issues, repos, commits, etc.
    • Do this for ticket references, people, etc.
    • Be sure use the correct referencing method when going across repositories.
Short (72 chars or less) summary

More detailed explanatory text. Wrap it to 72 characters. The blank
line separating the summary from the body is critical (unless you omit
the body entirely).

Write your commit message in the imperative: "Fix bug" and not "Fixed
bug" or "Fixes bug." This convention matches up with commit messages
generated by commands like git merge and git revert.

Further paragraphs come after blank lines.

- Bullet points are okay, too.
- Typically a hyphen or asterisk is used for the bullet, followed by a
  single space. Use a hanging indent.

For example:

$ git add my_file.txt

# Try to avoid using `git commit -m` unless it is a very trivial update/change
$ git commit

# Vim or Emacs window will open. To change editor that opens, see https://help.github.com/en/github/using-git/associating-text-editors-with-git .
Created new my_file with text in it

This new file includes:
* Part 1
* Part 2
* Part 3

Issue Tracking

See more information at PDS Issue Tracking Guide

Advanced Help

Help With Rebasing

Rebasing can get messy and be painful. We highly recommend reading this article on The Golden Rule of Rebasing.

References

Copyright © 2021-2024 California Institute of Technology.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Clone this wiki locally