-
Notifications
You must be signed in to change notification settings - Fork 1
Troubleshooting
You can list the Umpire help text by running:
umpire --help
Make sure you have the latest umpire is installed! This is the number one issue seen. Use the following command to upgrade umpire:
sudo pip install umpire --upgrade
Enable debug logging and tracebacks by calling Umpire with the debug flag.
Ex: umpire --debug ./deployment.json
Create a compressed package dependency file as either ...
- A compressed tar file using the command
tar czvf package_name.tar.gz [files](Linux/OS X) - A compressed zip file using the command
zip -r package_name.zip [files](Linux/OS X)
On OS X a number of default python packages are protected at a system level. One of these is the "six" package. If you receive an error like the following:
OSError: [Errno 1] Operation not permitted
Attempt to find the package it's failing on, and execute the upgrade command with the following:
sudo pip install umpire --upgrade --ignore-installed $PACKAGE
ex: sudo pip install umpire --upgrade --ignore-installed six
The best way to diagnose cache errors is to enable debugging as explained above, and to manually inspect the cache for anomalies.
If you're receiving md5 errors, you should check the .umpire file within a the dependency's folder. The .umpire file contains some metadata including the md5 hash. An overview of the info contained in this file is available in the Design wiki page. Verify the md5 stored matches the md5 in the Amazon ETag for the file.
If you're receiving lock file errors, such as "Umpire is waiting 300s to unlock", this is likely because an umpire process failed with an unrecoverable exception. The simplest solution is to remove the .umplock file. You can do that manually, or by running the following command:
umpire -r
These are generally file permission errors, and often happen when multiple OSes share a cache. There is a mysterious problem that we've seen when a cache is shared with Alpine Linux, and the solution was to have Alpine Linux use a separate cache.
Delete it! Umpire will just download the files it needs again. You can strategically delete entries, so you don't need to re-download the larger dependencies or you can run the following command to delete the entire cache:
umpire -c
The default location for the umpire cache is under the user's home directory under a folder called ".umpire".
AWS errors bubble up from the boto3 library. You can Google these errors, and they often don't have anything to do with Umpire but are configuration errors on the system that Umpire is running on. Check the following items when a boto3 error is encountered
Check the permissions on the file in the bucket you're attempting to access. If you require anonymous access, the dependency should have permissions for "Everyone" to "Open/Download File". Otherwise the default permissions are sufficient for authenticated access. If the file has NO permissions, neither authenticated or anonymous access will work.
See Missing or Misconfigured Credentials for more troubleshooting options.
Since Umpire uses the boto3 library, it shares its authentication schemes. Umpire will always first try an authenticated connection to AWS and will then fallback to anonymous. If the file you're attempting to access should be available anonymously and the connection is still failing -- ensure you have the correct permissions set on the file and that you are connected to the internet.
Ensure you are connected to the internet from the machine running umpire. Simply ping an external server to check:
ping 8.8.8.8