Skip to content

Commit

Permalink
Unix script url (#480)
Browse files Browse the repository at this point in the history
* Updated unix-install.sh to pass URLs

* Add documentation around new mirrior functionality

* Add suggestion for autosync

* Change QSG link text

* Documentation feedback from JS on the PR

* revert gosec version to fix false pos

* disable arm64 for now, circle ci plan does not include it

* Updated script to accept versions as just numeric (1.2.12) and automatically prepend the v (v1.2.12) needed for GH URLs

* Clean up mirrors doc further, and implement changes suggested by JS again.

* Fix tired typing typo

Co-authored-by: jsirianni <joe.sirianni@observiq.com>
  • Loading branch information
Dylan-M and jsirianni authored Nov 2, 2021
1 parent 5466439 commit adad941
Show file tree
Hide file tree
Showing 5 changed files with 113 additions and 39 deletions.
56 changes: 28 additions & 28 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -203,29 +203,29 @@ jobs:
- codecov/upload:
file: '**/coverage.txt'

test-linux-arm64:
machine:
image: ubuntu-2004:202101-01
resource_class: arm.large

steps:
- checkout
- run:
name: Download golang
command: curl -SL https://golang.org/dl/go1.17.1.linux-arm64.tar.gz -O
- run:
name: Extract golang
command: tar -C ~ -xzf go1.17.1.linux-arm64.tar.gz
- run:
name: Add Golang to Path
command: echo 'export PATH=~/go/bin:$PATH' >> $BASH_ENV
- run:
# call go vet and go test directly, as `make test` will attempt
# to run with GOOS=windows GOARCH=arm64, which is an invalid combination
name: Run Unit Tests
command: go vet ./... && go test -race -coverprofile coverage.txt -coverpkg ./... ./...
- codecov/upload:
file: '**/coverage.txt'
#test-linux-arm64:
# machine:
# image: ubuntu-2004:202101-01
# resource_class: arm.large
#
# steps:
# - checkout
# - run:
# name: Download golang
# command: curl -SL https://golang.org/dl/go1.17.1.linux-arm64.tar.gz -O
# - run:
# name: Extract golang
# command: tar -C ~ -xzf go1.17.1.linux-arm64.tar.gz
# - run:
# name: Add Golang to Path
# command: echo 'export PATH=~/go/bin:$PATH' >> $BASH_ENV
# - run:
# # call go vet and go test directly, as `make test` will attempt
# # to run with GOOS=windows GOARCH=arm64, which is an invalid combination
# name: Run Unit Tests
# command: go vet ./... && go test -race -coverprofile coverage.txt -coverpkg ./... ./...
# - codecov/upload:
# file: '**/coverage.txt'

test-macos:
executor: mac
Expand Down Expand Up @@ -517,10 +517,10 @@ workflows:
filters:
tags:
only: /^v\d+\.\d+\.\d+.*/
- test-linux-arm64:
filters:
tags:
only: /^v\d+\.\d+\.\d+.*/
#- test-linux-arm64:
# filters:
# tags:
# only: /^v\d+\.\d+\.\d+.*/
- test-macos:
filters:
tags:
Expand Down Expand Up @@ -550,7 +550,7 @@ workflows:
- wait-for-validation:
requires:
- test-linux
- test-linux-arm64
#- test-linux-arm64
- test-macos
- test-windows
- build
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/gosec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,6 @@ jobs:
uses: actions/checkout@v2

- name: Run Gosec Security Scanner
uses: securego/gosec@master
uses: securego/gosec@v2.8.1
with:
args: './...'
args: './...'
58 changes: 58 additions & 0 deletions docs/MIRRORS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Creating & Using Mirrors for Stanza Releases

## Creating a Mirror
Mirrors for Stanza can come in two forms:
1. Hosted websites
2. Local filesystem mirrors

The only requirements for either are creating a directory layout that
mirrors that of the GitHub releases, such as in the visualization below.
It is suggested to use an automated synchronization process to manage
keeping ths up to date, including rewriting the symlink for the latest
to the highest version number.

### Mirror Tree Visualization
➜ stanza_mirror tree
.
├── download
│ └── v1.2.12
│ ├── stanza-plugins.tar.gz
│ ├── stanza-plugins.zip
│ ├── stanza_darwin_amd64
│ ├── stanza_linux_amd64
│ ├── stanza_linux_arm64
│ ├── stanza_windows_amd64
│ ├── unix-install.sh
│ ├── version.json
│ └── windows-install.ps1
└── latest
└── download
├── stanza-plugins.tar.gz
├── stanza-plugins.zip
├── stanza_darwin_amd64
├── stanza_linux_amd64
├── stanza_linux_arm64
├── stanza_windows_amd64
├── unix-install.sh
├── version.json
└── windows-install.ps1

## Usage Syntax with the Install Script

### Web URL
```shell
# Latest
./unix-install -l http://dl.example.com/some/path
# Specific Version 1.2.12
./unix-install -l http://dl.example.com/some/path -v 1.2.12
```

### File URL
```shell
./unix-install -l file:///Users/username/Downloads/stanza_local
# Specific Version 1.2.12
./unix-install -l file:///Users/username/Downloads/stanza_local -v 1.2.12
```

## Further Information
For further usage information, and other supported flags, please see the [Quick Start Guide](README.md)
10 changes: 5 additions & 5 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,7 @@

We recommend using our single-line installer provided with each release:

### Linux
```shell
sh -c "$(curl -fsSlL https://github.com/observiq/stanza/releases/latest/download/unix-install.sh)" unix-install.sh
```
### macOS
### Linux/macOS
```shell
sh -c "$(curl -fsSlL https://github.com/observiq/stanza/releases/latest/download/unix-install.sh)" unix-install.sh
```
Expand All @@ -20,6 +16,10 @@ sh -c "$(curl -fsSlL https://github.com/observiq/stanza/releases/latest/download

#### Alternately, feel free to download the [latest release](https://github.com/observIQ/stanza/releases) directly.

## Local File System Stanza Mirrior
For Linux and macOS it is possible to run the script from a local mirror,
passing in the URL. See the [Local Mirror](MIRRORS.md) documentation.


## Running Stanza

Expand Down
24 changes: 20 additions & 4 deletions scripts/unix-install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -135,14 +135,22 @@ Usage:
$(fg_yellow '-v, --version')
Defines the version of the agent.
If not provided, this will default to the latest version.
Example: '-v 1.2.12' will download 1.2.12.
$(fg_yellow '-i, --install-dir')
Defines the install directory of the agent.
If not provided, this will default to an OS specific location.
Example: '-i /mnt/agent/' will install to that directory.
$(fg_yellow '-u, --service-user')
Defines the service user that will run the agent as a service.
If not provided, this will default to root.
Example: '-u agent' will run the script as user 'agent'.
$(fg_yellow '-l, --url')
Defines the URL that the components will be downloaded from.
If not provided, this will default to Stanza\'s GitHub releases.
Example: '-l http://my.domain.org/stanza' will download from there.
EOF
)
Expand Down Expand Up @@ -251,19 +259,25 @@ set_os()
# This will set the urls to use when downloading the agent and its plugins.
# These urls are constructed based on the --version flag or STANZA_VERSION env variable.
# If not specified, the version defaults to "latest".
# 2021-11-02: Added the ability to specify URL to anything that cURL supports, so long as
# it follows the directory structure paradigm.
set_download_urls()
{
if [ -z "$version" ] ; then
# shellcheck disable=SC2153
version=$STANZA_VERSION
fi

if [ -z "$url" ] ; then
url=$DOWNLOAD_BASE
fi

if [ -z "$version" ] ; then
agent_download_url="$DOWNLOAD_BASE/latest/download/${BINARY_NAME}_${os}_amd64"
plugins_download_url="$DOWNLOAD_BASE/latest/download/${PLUGINS_PACKAGE}"
agent_download_url="$url/latest/download/${BINARY_NAME}_${os}_amd64"
plugins_download_url="$url/latest/download/${PLUGINS_PACKAGE}"
else
agent_download_url="$DOWNLOAD_BASE/download/$version/${BINARY_NAME}_${os}_amd64"
plugins_download_url="$DOWNLOAD_BASE/download/$version/${PLUGINS_PACKAGE}"
agent_download_url="$url/download/v$version/${BINARY_NAME}_${os}_amd64"
plugins_download_url="$url/download/v$version/${PLUGINS_PACKAGE}"
fi
}

Expand Down Expand Up @@ -1030,6 +1044,8 @@ main()
install_dir=$2 ; shift 2 ;;
-u|--service-user)
service_user=$2 ; shift 2 ;;
-l|--url)
url=$2 ; shift 2 ;;
-h|--help)
usage
force_exit
Expand Down

0 comments on commit adad941

Please sign in to comment.