Skip to content

Commit

Permalink
More documentation fixes (#560)
Browse files Browse the repository at this point in the history
* Fix grammar issues resulting from the SRF->MRC rename 
* Update docs to reflect that stages no longer run in a fiber
* Third-party isn't one word
* Set up is two words when its a verb, one word when a noun
* Replace usage of the word "look"
* Misc spelling & grammar fixes

Authors:
  - David Gardner (https://github.com/dagardner-nv)
  - https://github.com/bsuryadev
  - Bhargav Suryadevara (https://github.com/bsuryadevara)

Approvers:
  - Christopher Harris (https://github.com/cwharris)
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: #560
  • Loading branch information
dagardner-nv authored Dec 19, 2022
1 parent 987e6bf commit 4c511ae
Show file tree
Hide file tree
Showing 13 changed files with 265 additions and 156 deletions.
6 changes: 3 additions & 3 deletions docs/source/cloud_deployment_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,7 +225,7 @@ kubectl -n $NAMESPACE exec -it deploy/mlflow -- bash
`Important`: When (mlflow) is present, commands are directly within the container.

First let's examine the syntax of the commands we will be using to communicate with the MLflow Triton plugin before we start deploying models.
Publish models to MLflow server looks like:
Publish models to MLflow server is in the form of:

```bash
(mlflow) root@mlflow-6d98:/mlflow# python publish_model_to_mlflow.py \
Expand Down Expand Up @@ -415,7 +415,7 @@ Refer to the [Using Morpheus to Run Pipelines](#using-morpheus-to-run-pipelines)
- Replace **<YOUR_RELEASE_NAME>** with the name you want.


For reference, the Morpheus SDK Client install pipeline command template is provided. Let's take a closer look at this when running [example workflows](#example-workflows), but for now, let's proceed to the next step.
For reference, the Morpheus SDK Client install pipeline command template is provided. We will examine this further in the [example workflows](#example-workflows) section, but for now, let's proceed to the next step.

```bash
helm install --set ngc.apiKey="$API_KEY" \
Expand Down Expand Up @@ -788,7 +788,7 @@ For more information on how to use the Morpheus CLI to customize and run your ow


## Troubleshooting
This section lists solutions to problems you might encounter with Morpheus or from it's supporting components.
This section lists solutions to problems you might encounter with Morpheus or from its supporting components.

### Common Problems

Expand Down
22 changes: 11 additions & 11 deletions docs/source/developer_guide/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Contributions to Morpheus fall into the following three categories.
2. To propose and implement a new Feature, please file a new feature request
[issue](https://github.com/nv-morpheus/Morpheus/issues/new/choose). Describe the
intended feature and discuss the design and implementation with the team and
community. Once the team agrees that the plan looks good, go ahead and
community. Once the team agrees that the plan is good, go ahead and
implement it, using the [code contributions](#code-contributions) guide below.
3. To implement a feature or bug-fix for an existing outstanding issue, please
follow the [code contributions](#code-contributions) guide below. If you
Expand All @@ -42,7 +42,7 @@ More information can be found at: [Contributor Code of Conduct](CODE_OF_CONDUCT.

### Your first issue

1. Find an issue to work on. The best way is to look for issues with the [good first issue](https://github.com/nv-morpheus/Morpheus/issues) label.
1. Find an issue to work on. The best way is to search for issues with the [good first issue](https://github.com/nv-morpheus/Morpheus/issues) label.
2. Comment on the issue stating that you are going to work on it.
3. Code! Make sure to update unit tests! Ensure the [license headers are set properly](#Licensing).
4. When done, [create your pull request](https://github.com/nv-morpheus/Morpheus/compare).
Expand All @@ -53,15 +53,15 @@ Remember, if you are unsure about anything, don't hesitate to comment on issues

### Seasoned developers

Once you have gotten your feet wet and are more comfortable with the code, you can look at the prioritized issues for our next release in our [project boards](https://github.com/nv-morpheus/Morpheus/projects).
Once you have gotten your feet wet and are more comfortable with the code, you can review the prioritized issues for our next release in our [project boards](https://github.com/nv-morpheus/Morpheus/projects).

> **Pro Tip:** Always look at the release board with the highest number for issues to work on. This is where Morpheus developers also focus their efforts.
> **Pro Tip:** Always review the release board with the highest number for issues to work on. This is where Morpheus developers also focus their efforts.
Look at the unassigned issues, and find an issue to which you are comfortable contributing. Start with _Step 2_ above, commenting on the issue to let others know you are working on it. If you have any questions related to the implementation of the issue, ask them in the issue instead of the PR.
Review the unassigned issues, and find an issue to which you are comfortable contributing. Start with _Step 2_ above, commenting on the issue to let others know you are working on it. If you have any questions related to the implementation of the issue, ask them in the issue instead of the PR.

## Building from Source

The following instructions are for developers who are getting started with the Morpheus repository. The Morpheus development environment is flexible (Docker, Conda and bare metal workflows) but has a high number of dependencies that can be difficult to setup. These instructions outline the steps for setting up a development environment inside a Docker container or on a host machine with Conda.
The following instructions are for developers who are getting started with the Morpheus repository. The Morpheus development environment is flexible (Docker, Conda and bare metal workflows) but has a high number of dependencies that can be difficult to set up. These instructions outline the steps for setting up a development environment inside a Docker container or on a host machine with Conda.

All of the following instructions assume several variables have been set:
- `MORPHEUS_ROOT`: The Morpheus repository has been checked out at a location specified by this variable. Any non-absolute paths are relative to `MORPHEUS_ROOT`.
Expand Down Expand Up @@ -106,7 +106,7 @@ This workflow utilizes a Docker container to set up most dependencies ensuring a
```
1. Note: When debugging Python code, you just need to add `ci/conda/recipes/python-dbg/source` to your debugger's
source path.
1. Once created, you will be able to introspect Python objects from within GDB. For example, if we were to break within a generator setup call and examine its PyFrame_Object `f`, it might look like this:
1. Once created, you will be able to introspect Python objects from within GDB. For example, if we were to break within a generator setup call and examine its PyFrame_Object `f`, it might be similar to:
```shell
#4 0x000056498ce685f4 in gen_send_ex (gen=0x7f3ecc07ad40, arg=<optimized out>, exc=<optimized out>, closing=<optimized out>) at Objects/genobject.c:222
(gdb) pyo f
Expand Down Expand Up @@ -178,7 +178,7 @@ Note: These instructions assume the user is using `mamba` instead of `conda` sin

- **Note:** `mamba` should only be installed once in the base environment

1. Setup env variables and clone the repo:
1. Set up env variables and clone the repo:
```bash
export PYTHON_VER=3.8
export RAPIDS_VER=22.08
Expand Down Expand Up @@ -255,7 +255,7 @@ Launching a full production Kafka cluster is outside the scope of this project.
- "0.0.0.0::9092"
```
This will prevent the containers from attempting to map IPv6 ports.
1. Change the value of `KAFKA_ADVERTISED_HOST_NAME` to match the value of the `KAFKA_ADVERTISED_HOST_NAME` environment variable from the previous step. For example, the line should look like:
1. Change the value of `KAFKA_ADVERTISED_HOST_NAME` to match the value of the `KAFKA_ADVERTISED_HOST_NAME` environment variable from the previous step. For example, the line should be similar to:

```yaml
environment:
Expand Down Expand Up @@ -383,8 +383,8 @@ Morpheus is licensed under the Apache v2.0 license. All new source files includi
*/
```
### Thirdparty code
Thirdparty code included in the source tree (that is not pulled in as an external dependency) must be compatible with the Apache v2.0 license and should retain the original license along with a url to the source. If this code is modified, it should contain both the Apache v2.0 license followed by the original license of the code and the url to the original code.
### Third-party code
Third-party code included in the source tree (that is not pulled in as an external dependency) must be compatible with the Apache v2.0 license and should retain the original license along with a url to the source. If this code is modified, it should contain both the Apache v2.0 license followed by the original license of the code and the url to the original code.
Ex:
```
Expand Down
12 changes: 6 additions & 6 deletions docs/source/developer_guide/guides/1_simple_python_stage.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ limitations under the License.

## Background

Morpheus makes use of the MRC graph-execution framework. Morpheus pipelines are built on top of MRC pipelines. Pipelines in MRC are made up of segments; however, in many common cases, a MRC pipeline will consist of only a single segment. Our Morpheus stages will interact with the MRC segment to build nodes and add them to the MRC graph. In the common case, a Morpheus stage will add a single node to the graph, but in some cases it will add multiple nodes to the graph.
Morpheus makes use of the MRC graph-execution framework. Morpheus pipelines are built on top of MRC pipelines, which are comprised of collections of nodes and edges, called segments (think sub-graphs), which can in turn be connected by ingress/egress ports. In many common cases, an MRC pipeline will consist of only a single segment. Our Morpheus stages interact with the MRC segment to define, build, and add nodes to the MRC graph; the stages themselves can be thought of as packaged units of work to be applied to data flowing through the pipeline. These work units comprising an individual Morpheus stage may consist of a single MRC node, a small collection of nodes, or an entire MRC subgraph.

## The Pass Through Stage

Expand Down Expand Up @@ -54,7 +54,7 @@ There are four methods that need to be defined in our new subclass to implement
return "pass-thru"
```

The `accepted_types` method returns a tuple of message classes that this stage accepts as valid inputs. Morpheus uses this to validate that the parent of this stage emits a message that this stage can accept. Since our stage is a pass through, we will declare that we can accept any incoming message type. Note that production stages will often declare only a single Morpheus message class such as `MessageMeta` or `MultiMessage` (refer to the message classes defined in `morpheus.pipeline.messages` for a complete list).
The `accepted_types` method returns a tuple of message classes that this stage is able to accept as input. Morpheus uses this to validate that the parent of this stage emits a message that this stage can accept. Since our stage is a pass through, we will declare that we can accept any incoming message type. Note that production stages will often declare only a single Morpheus message class such as `MessageMeta` or `MultiMessage` (refer to the message classes defined in `morpheus.pipeline.messages` for a complete list).
```python
def accepted_types(self) -> typing.Tuple:
return (typing.Any,)
Expand All @@ -66,14 +66,14 @@ The `supports_cpp_node` method returns a boolean indicating if the stage has a C
return False
```

Our `on_data` method accepts the incoming message and returns a message. The returned message can be the same message instance that we received as our input or it could be a new message instance. The method is named `on_data` by convention; however, it is not part of the API. In the next section, we will register it as a callback in Morpheus.
Our `on_data` method accepts an incoming message and returns a message. The returned message can be the same message instance that we received as our input or it could be a new message instance. The method is named `on_data` by convention; however, it is not part of the API. In the next section, we will register it as a callback in Morpheus.
```python
def on_data(self, message: typing.Any):
# Return the message for the next stage
return message
```

Finally, the `_build_single` method will be used at build time to wire our stage into the pipeline. `_build_single` receives an instance of the MRC pipeline segment along with a `StreamPair` instance. We will be using the segment instance to build a node from our stage and add it to the pipeline segment. The `StreamPair` argument is a tuple; the first element is our parent node, and the second is our parent node's output type. The return type of this method is also a `StreamPair`. Typically, we will be returning our newly constructed node along with our output type.
Finally, the `_build_single` method will be used at stage build time to construct our node and wire it into the pipeline. `_build_single` receives an instance of an MRC segment builder (`mrc.Builder`) along with a `StreamPair` instance, which is a tuple consisting of our parent node and its output type. We will be using the builder instance to construct a node from our stage and connecting it to the Morpheus pipeline. The return type of `_build_single` is also a `StreamPair` which will be comprised of our node along with its data type.
```python
def _build_single(self, builder: mrc.Builder, input_stream: StreamPair) -> StreamPair:
node = builder.make_node(self.unique_name, self.on_data)
Expand All @@ -82,7 +82,7 @@ Finally, the `_build_single` method will be used at build time to wire our stage
return node, input_stream[1]
```

In most cases, a Morpheus stage will define and build a single MRC node. In some advanced cases, a stage can construct more than one node. For our purposes, a Morpheus _stage_ defines information about the type of node(s) it builds, while the _node_ is the instance of the stage that is wired into the MRC pipeline. To build the node, we will call the `make_node` method of the segment instance, passing to it our name and our `on_data` method. We used the `unique_name` property, which will take the name property which we already defined and append a unique id to it.
For our purposes, a Morpheus _stage_ defines the input data type the stage will accept, the unit of work to be performed on that data, and the output data type. In contrast each individual node or nodes comprising a _stage_'s unit of work are wired into the underlying MRC execution graph. To build the node, we will call the `make_node` method of the builder instance, passing it our `unique_name` and `on_data` methods. We used the `unique_name` property, which will take the `name` property which we already defined and append a unique id to it.
```python
node = builder.make_node(self.unique_name, self.on_data)
```
Expand Down Expand Up @@ -137,7 +137,7 @@ class PassThruStage(SinglePortStage):

## Testing our new Stage
To start testing our new pass through stage, we are going to construct a simple pipeline and add our new stage to it. This pipeline will do the minimum work necessary to verify our pass through stage. Data will flow through our simple pipeline as follows:
1. A source stage will produce data and inject it into the pipeline.
1. A source stage will produce data and emit it into the pipeline.
1. This data will be read and processed by our pass through stage, in this case simply forwarding on the data.
1. A monitoring stage will record the messages from our pass through stage and terminate the pipeline.

Expand Down
Loading

0 comments on commit 4c511ae

Please sign in to comment.