Skip to content

Commit

Permalink
zfs-import: Use cache file to reimport pools at boot
Browse files Browse the repository at this point in the history 
This change modifies the import service to use the default cache file
to reimport pools at boot.  This fixes code that exhaustively searched
the entire system and imported all visible pools.  Using the cache
file is in keeping with the way ZFS has always worked, and is how it is
written in the man page (zpool(1M,8)):

    All pools  in  this  cache  are  automatically imported when the
    system boots.

Importantly, the cache contains important information for importing
multipath devices, and helps control which pools get imported in more
dynamic environments like SANs, which may have thousands of visible
and constantly changing pools, which the ZFS_POOL_EXCEPTIONS variable
is not equipped to handle.

The change also stops the service from exporting pools at shutdown.
Exporting pools is only meant to be performed by the administrator
of the system.

Closes #3777
Closes #3526
  • Loading branch information
@jameslikeslinux
jameslikeslinux committed Sep 18, 2015
1 parent 7a27ad0 commit f135bb1
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 264 deletions.
264 changes: 11 additions & 253 deletions etc/init.d/zfs-import.in
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
#!@SHELL@
#
# zfs-import This script will import/export zfs pools.
# zfs-import This script will reimport ZFS pools.
#
# chkconfig: 2345 01 99
# description: This script will import/export zfs pools during system
# boot/shutdown.
# It is also responsible for all userspace zfs services.
# description: This script will reimport previously imported ZFS pools
# during system boot.
# probe: true
#
### BEGIN INIT INFO
Expand All @@ -16,8 +15,8 @@
# Default-Stop: 0 1 6
# X-Start-Before: checkfs
# X-Stop-After: zfs-mount
# Short-Description: Import ZFS pools
# Description: Run the `zpool import` or `zpool export` commands.
# Short-Description: Reimport ZFS pools
# Description: Run the `zpool import` command.
### END INIT INFO
#
# NOTE: Not having '$local_fs' on Required-Start but only on Required-Stop
Expand All @@ -43,246 +42,14 @@ do_depend()
keyword -lxc -openvz -prefix -vserver
}

# Support function to get a list of all pools, separated with ';'
find_pools()
{
local CMD="$*"
local pools

pools=$($CMD 2> /dev/null | \
grep -E "pool:|^[a-zA-Z0-9]" | \
sed 's@.*: @@' | \
sort | \
while read pool; do \
echo -n "$pool;"
done)

echo "${pools%%;}" # Return without the last ';'.
}

# Import all pools
# Reimport all previously imported pools
do_import()
{
local already_imported available_pools pool npools
local exception dir ZPOOL_IMPORT_PATH RET=0 r=1

# In case not shutdown cleanly.
[ -n "$init" ] && rm -f /etc/dfs/sharetab

# Just simplify code later on.
if [ -n "$USE_DISK_BY_ID" -a "$USE_DISK_BY_ID" != 'yes' ]
then
# It's something, but not 'yes' so it's no good to us.
unset USE_DISK_BY_ID
fi

# Find list of already imported pools.
already_imported=$(find_pools "$ZPOOL" list -H -oname)
available_pools=$(find_pools "$ZPOOL" import)

# Just in case - seen it happen (that a pool isn't visable/found
# with a simple "zpool import" but only when using the "-d"
# option or setting ZPOOL_IMPORT_PATH).
if [ -d "/dev/disk/by-id" ]
then
npools=$(find_pools "$ZPOOL" import -d /dev/disk/by-id)
if [ -n "$npools" ]
then
# Because we have found extra pool(s) here, which wasn't
# found 'normaly', we need to force USE_DISK_BY_ID to
# make sure we're able to actually import it/them later.
USE_DISK_BY_ID='yes'

if [ -n "$available_pools" ]
then
# Filter out duplicates (pools found with the simpl
# "zpool import" but which is also found with the
# "zpool import -d ...").
npools=$(echo "$npools" | sed "s,$available_pools,,")

# Add the list to the existing list of
# available pools
available_pools="$available_pools;$npools"
else
available_pools="$npools"
fi
fi
fi

# Filter out any exceptions...
if [ -n "$ZFS_POOL_EXCEPTIONS" ]
then
local found=""
local apools=""
OLD_IFS="$IFS" ; IFS=";"

for pool in $available_pools
do
for exception in $ZFS_POOL_EXCEPTIONS
do
[ "$pool" = "$exception" ] && continue 2
found="$pool"
done

if [ -n "$found" ]
then
if [ -n "$apools" ]
then
apools="$apools;$pool"
else
apools="$pool"
fi
fi
done

IFS="$OLD_IFS"
available_pools="$apools"
fi

# For backwards compability, make sure that ZPOOL_IMPORT_PATH is set
# to something we can use later with the real import(s). We want to
# make sure we find all by* dirs, BUT by-vdev should be first (if it
# exists).
if [ -n "$USE_DISK_BY_ID" -a -z "$ZPOOL_IMPORT_PATH" ]
if [ -f "$ZPOOL_CACHE" ]
then
local dirs
dirs="$(for dir in $(echo /dev/disk/by-*)
do
# Ignore by-vdev here - we wan't it first!
echo "$dir" | grep -q /by-vdev && continue
[ ! -d "$dir" ] && continue

echo -n "$dir:"
done | sed 's,:$,,g')"

if [ -d "/dev/disk/by-vdev" ]
then
# Add by-vdev at the beginning.
ZPOOL_IMPORT_PATH="/dev/disk/by-vdev:"
fi

# Help with getting LUKS partitions etc imported.
if [ -d "/dev/mapper" ]; then
if [ -n "$ZPOOL_IMPORT_PATH" ]; then
ZPOOL_IMPORT_PATH="$ZPOOL_IMPORT_PATH:/dev/mapper:"
else
ZPOOL_IMPORT_PATH="/dev/mapper:"
fi
fi

# ... and /dev at the very end, just for good measure.
ZPOOL_IMPORT_PATH="$ZPOOL_IMPORT_PATH$dirs:/dev"
zfs_action "Reimporting ZFS pool(s)" \
"$ZPOOL" import -c "$ZPOOL_CACHE" -N -a
fi

# Needs to be exported for "zpool" to catch it.
[ -n "$ZPOOL_IMPORT_PATH" ] && export ZPOOL_IMPORT_PATH

# Mount all availible pools (except those set in ZFS_POOL_EXCEPTIONS.
#
# If not interactive (run from init - variable init='/sbin/init')
# we get ONE line for all pools being imported, with just a dot
# as status for each pool.
# Example: Importing ZFS pool(s)... [OK]
#
# If it IS interactive (started from the shell manually), then we
# get one line per pool importing.
# Example: Importing ZFS pool pool1 [OK]
# Importing ZFS pool pool2 [OK]
# [etc]
[ -n "$init" ] && zfs_log_begin_msg "Importing ZFS pool(s)"
OLD_IFS="$IFS" ; IFS=";"
for pool in $available_pools
do
[ -z "$pool" ] && continue

# We have pools that haven't been imported - import them
if [ -n "$init" ]
then
# Not interactive - a dot for each pool.
# Except on Gentoo where this doesn't work.
zfs_log_progress_msg "."
else
# Interactive - one 'Importing ...' line per pool
zfs_log_begin_msg "Importing ZFS pool $pool"
fi

# Import by using ZPOOL_IMPORT_PATH (either set above or in
# the config file) _or_ with the 'built in' default search
# paths. This is the prefered way.
"$ZPOOL" import -N "$pool" 2> /dev/null
r="$?" ; RET=$((RET + r))
if [ "$r" -eq 0 ]
then
# Output success and process the next pool
[ -z "$init" ] && zfs_log_end_msg 0
continue
fi
# We don't want a fail msg here, we're going to try import
# using the cache file soon and that might succeed.
[ ! -f "$ZPOOL_CACHE" ] && zfs_log_end_msg "$RET"

if [ "$r" -gt 0 -a -f "$ZPOOL_CACHE" ]
then
# Failed to import without a cache file. Try WITH...
if [ -z "$init" -a "$VERBOSE_MOUNT" = 'yes' ]
then
# Interactive + Verbose = more information
zfs_log_progress_msg " using cache file"
fi

"$ZPOOL" import -c "$ZPOOL_CACHE" -N "$pool" 2> /dev/null
r="$?" ; RET=$((RET + r))
if [ "$r" -eq 0 ]
then
[ -z "$init" ] && zfs_log_end_msg 0
continue 3 # Next pool
fi
zfs_log_end_msg "$RET"
fi
done
[ -n "$init" ] && zfs_log_end_msg "$RET"

IFS="$OLD_IFS"
[ -n "$already_imported" -a -z "$available_pools" ] && return 0

return "$RET"
}

# Export all pools
do_export()
{
local already_imported pool root_pool RET r
RET=0

root_pool=$(get_root_pool)

[ -n "$init" ] && zfs_log_begin_msg "Exporting ZFS pool(s)"

# Find list of already imported pools.
already_imported=$(find_pools "$ZPOOL" list -H -oname)

OLD_IFS="$IFS" ; IFS=";"
for pool in $already_imported; do
[ "$pool" = "$root_pool" ] && continue

if [ -z "$init" ]
then
# Interactive - one 'Importing ...' line per pool
zfs_log_begin_msg "Exporting ZFS pool $pool"
else
# Not interactive - a dot for each pool.
zfs_log_progress_msg "."
fi

"$ZPOOL" export "$pool"
r="$?" ; RET=$((RET + r))
[ -z "$init" ] && zfs_log_end_msg "$r"
done
IFS="$OLD_IFS"

[ -n "$init" ] && zfs_log_end_msg "$RET"

return "$RET"
}

# Output the status and list of pools
Expand Down Expand Up @@ -324,14 +91,6 @@ do_start()
fi
}

do_stop()
{
# Check to see if the module is even loaded.
check_module_loaded "zfs" || exit 0

do_export
}

# ----------------------------------------------------

if [ ! -e /etc/gentoo-release ]
Expand All @@ -341,7 +100,7 @@ then
do_start
;;
stop)
do_stop
# no-op
;;
status)
do_status
Expand All @@ -351,7 +110,7 @@ then
;;
*)
[ -n "$1" ] && echo "Error: Unknown command $1."
echo "Usage: $0 {start|stop|status}"
echo "Usage: $0 {start|status}"
exit 3
;;
esac
Expand All @@ -361,6 +120,5 @@ else
# Create wrapper functions since Gentoo don't use the case part.
depend() { do_depend; }
start() { do_start; }
stop() { do_stop; }
status() { do_status; }
fi
11 changes: 0 additions & 11 deletions etc/init.d/zfs.in
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,6 @@ ZFS_SHARE='yes'
# Run `zfs unshare -a` during system stop?
ZFS_UNSHARE='yes'

# Specify specific path(s) to look for device nodes and/or links for the
# pool import(s). See zpool(8) for more information about this variable.
# It supersedes the old USE_DISK_BY_ID which indicated that it would only
# try '/dev/disk/by-id'.
# The old variable will still work in the code, but is deprecated.
#ZPOOL_IMPORT_PATH="/dev/disk/by-vdev:/dev/disk/by-id"

# Should the datasets be mounted verbosely?
# A mount counter will be used when mounting if set to 'yes'.
VERBOSE_MOUNT='no'
Expand Down Expand Up @@ -73,10 +66,6 @@ ZFS_INITRD_POST_MODPROBE_SLEEP='0'
# Example: If root FS is 'rpool/ROOT/rootfs', this would make sense.
#ZFS_INITRD_ADDITIONAL_DATASETS="rpool/ROOT/usr rpool/ROOT/var"

# List of pools that should NOT be imported at boot?
# This is a space separated list.
#ZFS_POOL_EXCEPTIONS="test2"

# Optional arguments for the ZFS Event Daemon (ZED).
# See zed(8) for more information on available options.
#ZED_ARGS="-M"

3 comments on commit f135bb1

@ryao
Copy link
Contributor

@ryao ryao commented on f135bb1 Sep 19, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am okay with leaving @FransUrbo's code as an optional function. Some people might want things to work this way. That said, it should not be done this way by default.

@jameslikeslinux
Copy link
Contributor Author

@jameslikeslinux jameslikeslinux commented on f135bb1 Sep 19, 2015 via email

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@FransUrbo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If (when?) we've decided to change the default of cachefile, it would make sense to change the default import way to. Or vice versa if you like (which is what I'm doing with #3526).

Considering that there's a good, functional way to make sure one ONLY uses a cache file to do the import, there is no point in changing the init script.

Please sign in to comment.