Skip to content

Commit

Permalink
feat(cli): Change app name to snap2bucket
Browse files Browse the repository at this point in the history
1. Change the app name to snap2bucket.
2. Write Actions to build PyPi packages.
3. Update version to 1.0.4

Signed-off-by: Gaurav Mishra <mishra.gaurav@siemens.com>
  • Loading branch information
GMishx committed Jun 29, 2022
1 parent 39f693b commit f200739
Show file tree
Hide file tree
Showing 19 changed files with 115 additions and 35 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
image: python:3.8-slim

steps:
- uses: actions/checkout@v1
- uses: actions/checkout@v3
- name: Install host dependencies
run: |
apt-get -qq update
Expand Down
11 changes: 7 additions & 4 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,21 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Set up Python 3.9
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: 3.9
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pylint pipenv
pipx install pylint
pipx install pipenv
- name: Install package
run: |
pipenv install --dev --editable .
- name: Analysing the code with pylint
run: |
pipenv run pylint src/snap_to_bucket
- name: Check package
run: |
pipenv run check_package
37 changes: 37 additions & 0 deletions .github/workflows/release-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: Publish Release Packages

on:
release:
types:
- published

jobs:
release:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3

- name: Setup python
uses: actions/setup-python@v4
with:
python-version: '3.8'
architecture: 'x64'

- name: Install build dependencies
run: |
pipx install pipenv
pipenv install
- name: Check package
run: pipenv run check_package

- name: Build packages
run: pipenv run build_packages

- name: Upload Packages to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
user: __token__
password: ${{ secrets.PYPI_API_TOKEN }}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,4 @@ tags
*__pycache__/
build
*.egg-info
dist
4 changes: 4 additions & 0 deletions Pipfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,7 @@ snap-to-bucket = {path = "."}
sphinx = "*"
snap-to-bucket = {editable = true, path = "."}
pylint = "*"

[scripts]
build_packages = "python3 ./setup.py sdist bdist_wheel"
check_package = "python3 ./setup.py check"
22 changes: 15 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,25 @@
# snap_to_bucket

![snap_to_bucket docs](https://github.com/siemens/snap-to-bucket/workflows/snap_to_bucket%20docs/badge.svg)
[![PyPI version](https://badge.fury.io/py/snap-to-bucket.svg)](https://badge.fury.io/py/snap-to-bucket)

This tool allows to move data from AWS snapshots to S3 buckets.

### Installation

#### Local installation
```console
$ git clone https://github.com/siemens/snap-to-bucket.git
$ cd snap-to-bucket
$ python3 -m pip install -U pipenv
$ pipenv install --dev --editable .
```

#### PyPi
```console
$ python3 -m pip install -U snap-to-bucket
```

### Requirements

1. The script needs to be running on an EC2 instance.
Expand Down Expand Up @@ -95,11 +104,11 @@ script needs to mount/unmount volumes.

- Runing from source
```console
# pipenv run snap_to_bucket --bucket <bucket>
# pipenv run snap2bucket --bucket <bucket>
```
- Runing from install
```console
# snap_to_bucket --bucket <bucket>
# snap2bucket --bucket <bucket>
```

If you have used different tags on snapshots, use `-t\--tag` option.
Expand Down Expand Up @@ -133,10 +142,9 @@ Since the S3 has a limit on object size, a single split can not be larger than
### Options

```
Usage: snap_to_bucket [OPTIONS]
Usage: snap2bucket [OPTIONS]
snap_to_bucket is a simple tool based on boto3 to move snapshots to S3
buckets.
snap2bucket is a simple tool based on boto3 to move snapshots to S3 buckets.
Options:
--version Show the version and exit.
Expand Down Expand Up @@ -239,11 +247,11 @@ for i in /sys /proc /run /dev; do sudo umount /mnt/snapshot$i; done
Run the script with `-r\--restore` flag and provide the bucket and the key.
- Runing from source
```console
# pipenv run snap_to_bucket --restore --bucket <bucket> --key <key>
# pipenv run snap2bucket --restore --bucket <bucket> --key <key>
```
- Runing from install
```console
# snap_to_bucket --restore --bucket <bucket> --key <key>
# snap2bucket --restore --bucket <bucket> --key <key>
```

**Note:** The script will create new volume of size 25% more than the size of
Expand Down
4 changes: 2 additions & 2 deletions doc-sources/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,11 @@
# -- Project information -----------------------------------------------------

project = 'snap_to_bucket'
copyright = '2020-2021, Siemens AG'
copyright = '2020-2022, Siemens AG'
author = 'Gaurav Mishra <mishra.gaurav@siemens.com>'

# The full version, including alpha/beta/rc tags
release = '1.0.1'
release = '1.0.4'


# -- General configuration ---------------------------------------------------
Expand Down
14 changes: 11 additions & 3 deletions doc-sources/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,20 @@ This tool allows to move data from AWS snapshots to S3 buckets.
Installation
=======================

* From source code

.. code-block:: bash
$ git clone https://github.com/siemens/snap-to-bucket.git
$ cd snap-to-bucket
$ python3 -m pip install -U pipenv
$ pipenv install --dev --editable .
* From PyPi

.. code-block:: bash
$ python3 -m pip install -U snap-to-bucket
Requirements
===============

Expand Down Expand Up @@ -95,10 +104,9 @@ Options

.. code-block::
Usage: snap_to_bucket [OPTIONS]
Usage: snap2bucket [OPTIONS]
snap_to_bucket is a simple tool based on boto3 to move snapshots to S3
buckets.
snap2bucket is a simple tool based on boto3 to move snapshots to S3 buckets.
Options:
--version Show the version and exit.
Expand Down
4 changes: 2 additions & 2 deletions doc-sources/recovery.rst
Original file line number Diff line number Diff line change
Expand Up @@ -60,13 +60,13 @@ Run the script with ``-r\--restore`` flag and provide the bucket and the key.

.. code-block:: bash
pipenv run snap_to_bucket --restore --bucket <bucket> --key <key>
pipenv run snap2bucket --restore --bucket <bucket> --key <key>
* Runing from install

.. code-block:: bash
snap_to_bucket --restore --bucket <bucket> --key <key>
snap2bucket --restore --bucket <bucket> --key <key>
**Note:** The script will create new volume of size 25% more than the size of
tar or ``x-amz-meta-disc-size`` metadata (if available).
Expand Down
4 changes: 2 additions & 2 deletions doc-sources/setupmigrate.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,13 @@ script needs to mount/unmount volumes.

.. code-block:: bash
pipenv run snap_to_bucket --bucket <bucket>
pipenv run snap2bucket --bucket <bucket>
* Runing from install

.. code-block:: bash
snap_to_bucket --bucket <bucket>
snap2bucket --bucket <bucket>
If you have used different tags on snapshots, use ``-t/--tag`` option.

Expand Down
6 changes: 3 additions & 3 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# -*- coding: utf-8 -*-

"""
SPDX-FileCopyrightText: Siemens AG, 2020-2021 Gaurav Mishra <mishra.gaurav@siemens.com>
SPDX-FileCopyrightText: Siemens AG, 2020-2022 Gaurav Mishra <mishra.gaurav@siemens.com>
SPDX-License-Identifier: MIT
"""
Expand Down Expand Up @@ -32,7 +32,7 @@ def read(fname):

metadata = dict(
name="snap_to_bucket",
version="1.0.2",
version="1.0.4",
author="Gaurav Mishra",
author_email="mishra.gaurav@siemens.com",
description=("Move AWS EBS Snapshots to S3 Buckets"),
Expand Down Expand Up @@ -65,7 +65,7 @@ def read(fname):
],
entry_points = {
'console_scripts': [
'snap_to_bucket = snap_to_bucket.run:main'
'snap2bucket = snap_to_bucket.run:main'
]
},
)
Expand Down
5 changes: 0 additions & 5 deletions src/snap_to_bucket/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,3 @@
SPDX-License-Identifier: MIT
"""
__author__ = 'Siemens AG'

from .ec_2_handler import Ec2Handler
from .fs_handler import FsHandler
from .s3_handler import S3Handler
from .snap_handler import SnapToBucket
13 changes: 13 additions & 0 deletions src/snap_to_bucket/handlers/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-

"""
SPDX-FileCopyrightText: Siemens AG, 2022 Gaurav Mishra <mishra.gaurav@siemens.com>
SPDX-License-Identifier: MIT
"""
__author__ = 'Siemens AG'

from .ec_2_handler import Ec2Handler
from .s3_handler import S3Handler
from .fs_handler import FsHandler
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ def __get_object_count(self, key):
partition_size = int(
response["Metadata"]["x-amz-meta-disc-size"])
if partition_size < 2:
partition_size = sum([int(o["Size"]) for o in objects])
partition_size = sum((int(o["Size"]) for o in objects))
self.restore_partition_size = partition_size
return len(objects)
except Exception as ex:
Expand Down
4 changes: 2 additions & 2 deletions src/snap_to_bucket/run.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
import click
from pkg_resources import get_distribution

from snap_to_bucket import SnapToBucket
from snap_to_bucket.runner import SnapToBucket


class VolSize(click.ParamType):
Expand Down Expand Up @@ -122,7 +122,7 @@ def main(verbose, proxy, noproxy, bucket, tag, type, storage_class, mount,
delete, split, gzip, restore, key, boot, restore_dir, iops,
throughput):
"""
snap_to_bucket is a simple tool based on boto3 to move snapshots to S3
snap2bucket is a simple tool based on boto3 to move snapshots to S3
buckets.
"""
if type not in ["gp3", "io1", "io2"] and iops is not None:
Expand Down
11 changes: 11 additions & 0 deletions src/snap_to_bucket/runner/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-

"""
SPDX-FileCopyrightText: Siemens AG, 2022 Gaurav Mishra <mishra.gaurav@siemens.com>
SPDX-License-Identifier: MIT
"""
__author__ = 'Siemens AG'

from .snap_handler import SnapToBucket
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@
import os
import sys

from snap_to_bucket import Ec2Handler
from snap_to_bucket import S3Handler
from snap_to_bucket import FsHandler
from snap_to_bucket.handlers import Ec2Handler
from snap_to_bucket.handlers import S3Handler
from snap_to_bucket.handlers import FsHandler


class SnapToBucket:
Expand Down

0 comments on commit f200739

Please sign in to comment.