Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix non legacy build/docker issues plus dev to rpm install mechanism. Fixes #1989 #1990

Conversation

phillxnet
Copy link
Member

Fix a number of the remaining build and docker issues concerning the proposed move to openSUSE as an upstream linux ditro base. Includes a fix to facilitate moving from a developer (source) install to an rpm based one: currently this only supports our existing legacy CentOS base, pending the instantiation of an openSUSE rpm build backend along with related distro aware repository config code changes.

Docker specific notes:
Many more modern dockerd invocations require a number of command line arguments. Previously we passed, from the docker.service file, only one: our --data-root target. The included modifications allows for more custom or distro specific requirements to be met via accommodation of any number of arguments (unfiltered). All prior Rockstor specific dockerd arguments are preserved and applied as before.

Summary:

  • Add dependency on python ‘distro’ library.
  • Store build system distro info in django settings, the assumption here is we build on our target distro: normally the case.
  • Add distro UI element, uses prior 2 items.
  • Normalise prior UI subheader linux info formatting.
  • Remove prior incorrect data_collector code comment.
  • Selectively run postgresql-setup (legacy) or initdb (non legacy) in initrock.
  • Update psycopg2 from 2.6 to 2.7.4.
  • Normalise on direct paths for commands: avoids redundant fs redirection ie in CentOS root we have “/bin -> /usr/bin" and "/sbin - > /usr/sbin"; as these dir links are not found in our non legacy base move all hard wired command paths using them to their canonical reference.
  • Use Django settings for a selection of variably located (distro specific) command paths: again with the assumption that we build on our target distro.
  • Fix version indicator and software update page display for dev (source) installs.
  • Fix dev (source install) to rpm install transition mechanism - necessarily considered as a re-install so db is wiped during the transition. Note that this, in part, involved the addition of an explicit 'yum install rockstor' command during update, along with ensuring that initrock is re-run on next rockstor.service start.
  • Add distro aware docker.service template file selection based on distro.id(); moving fully to a live edit (during Rock-on service enable) rather than build time customization: ie to accommodate for our docker wrapper redirect and it’s consequent requirement for NofityAccess=all for Type=notify docker configs. Both included openSUSE templates are taken from their respective distro default installs of docker-ce.
  • Establish docker-generic.service failover config for unknown distro ids taken from default upstream docker-ce 18.09 CentOS example.
  • Enhance docker wrapper to pass additional arguments to dockerd.
  • Minor additional rock-ons-root config exception logging.
  • Catch and log harmless reboot/shutdown command exceptions with rc=-15. The exception log reports from these Web-UI initiated events are misleading as they suggest malfunction where there is none as both commands execute as expected with: out='', err='', and rc=-15.

Fixes #1989
See issue text for context and forum reference.

@suman Ready for review. Please note the additional ‘python-distro’ package dependency required of the rpm. My understanding is that the included entry in base-buildout.cfg [rpm-deps] section is sufficient, but I am not certain of this. N.B. my experience so far is that the build system will also require this package installed due to its use in settings.conf.in and it’s invocation from within initrock. Apologies for the inconvenience here. Also note the use of subprocess to run ‘which’ within settings.conf.in: this makes the assumption that the build system is as per the target system, ie CentOS (currently), with regard to binary locations of the commands in question (‘udevadm’, ‘shutdown’, and ‘chkconfig’ (see #1986 )). The package name of ‘python-distro’ works across all 3 current target distros, CentOS (currently only python2 variant) and both python2 and python3 variants in the openSUSE distros: both currently defaulting / resolving to their python2 versions.

Testing:

Source to rpm install transition (source install assumed not to be in /opt/rockstor).
Worked on CentOS distro base (only rpms currently available), with caveat that browser (Firefox) prompted for new cert exception upon transition as this is treated as a re-install / fresh install: db is wiped due to unknown state from development / source install. N.B. re-activation of the chosen update channel is also require, again due to this transition having a re-install status (prior db wiped). However the prior yum repo config is unaffected until a selection is made.

Distro adaptive docker config.
Tested on an original legacy CentOS (‘rockstor’ distro.id()) install as well as on current openSUSE Leap 15.0 (opensuse-leap distro.id()) and Tumbleweed (opensuse-tumbleweed distro.id()) installs. In all cases a successful Rock-ons-root config was achieved with the consequent test of installing a Plex Rock-on and ensuring that it auto started on reboot. The Rock-ons service was also successfully enabled and disabled on all systems via the Web-UI services page.

Images to follow in comments to illustrate the functioning Rock-on system and the added distro UI element.

…ockstor#1989

Fix a number of the remaining build and docker issues concerning the
proposed move to openSUSE as an upstream linux ditro base. Includes a
fix to facilitate moving from a developer (source) install to an rpm
based one: currently this only supports our existing legacy CentOS base,
pending the instantiation of an openSUSE rpm build backend along with
related distro aware repository config code changes.

Docker specific notes:
Many more modern dockerd invocations require a number of command line
arguments. Previously we passed, from the docker.service file, only one:
our --data-root target. The included modifications allows for more
custom or distro specific requirements to be met via accommodation of
any number of arguments (unfiltered). All prior Rockstor specific
dockerd arguments are preserved and applied as before.

Summary:
- Add dependency on python ‘distro’ library.
- Store build system distro info in django settings, the assumption here
is we build on our target distro: normally the case.
- Add distro UI element, uses prior 2 items.
- Normalise prior UI subheader linux info formatting.
- Remove prior incorrect data_collector code comment.
- Selectively run postgresql-setup (legacy) or initdb (non legacy) in
initrock.
- Update psycopg2 from 2.6 to 2.7.4.
- Normalise on direct paths for commands: avoids redundant fs
redirection ie in CentOS root we have “/bin -> /usr/bin" and
"/sbin - > /usr/sbin"; as these dir links are not found in our non
legacy base move all hard wired command paths using them to their
canonical reference.
- Use Django settings for a selection of variably located
(distro specific) command paths: again with the assumption that we build
on our target distro.
- Fix version indicator and software update page display for dev
(source) installs.
- Fix dev (source install) to rpm install transition mechanism -
necessarily considered as a re-install so db is wiped during the
transition. Note that this, in part, involved the addition of an
explicit 'yum install rockstor' command during update, along with
ensuring that initrock is re-run on next rockstor.service start.
- Add distro aware docker.service template file selection based on
distro.id(); moving fully to a live edit (during Rock-on service enable)
rather than build time customization: ie to accommodate for our docker
wrapper redirect and it’s consequent requirement for NofityAccess=all
for Type=notify docker configs. Both included openSUSE templates are
taken from their respective distro default installs of docker-ce.
- Establish docker-generic.service failover config for unknown distro
ids taken from default upstream docker-ce 18.09 CentOS example.
- Enhance docker wrapper to pass additional arguments to dockerd.
- Minor additional rock-ons-root config exception logging.
- Catch and log harmless reboot/shutdown command exceptions with rc=-15.
The exception log reports from these Web-UI initiated events are
misleading as they suggest malfunction where there is none as both
commands execute as expected with: out='', err='', and rc=-15.
@phillxnet
Copy link
Member Author

Examples of functioning Rock-ons and added distro Web-UI indicator element with mouse over version activated:

openSUSE Leap 15.0 with Plex Rock-on installed and running:
leap15-version-info-and-kernel-version-plus-rockon-running

openSUSE Tumbleweed 20181112 snapshot running both Netdata and Plex Rock-ons:
tumbleweed-snapshot-and-kernel-version-plus-rockon-running

@schakrava
Copy link
Member

Thanks @phillxnet . This is pretty substantial. At one point though (may be better to do it sooner than later), it may be better to completely decouple the next gen(suse based) Rockstor from the current one. Lets discuss more about this separately.

@schakrava schakrava merged commit 33917f1 into rockstor:master Nov 21, 2018
@phillxnet phillxnet deleted the 1989_fix_non_legacy_build/docker_issues_plus_dev_to_rpm_install_mechanism branch November 21, 2018 13:57
@phillxnet
Copy link
Member Author

@schakrava Cheers. And agreed re decoupling next gen, or the like.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

fix non legacy build/docker issues plus dev to rpm install mechanism
2 participants