Skip to content

Commit

Permalink
Scenario Authoring documentation (#1900)
Browse files Browse the repository at this point in the history 
* initial changes to existing docs

* Update Spel.md

* Update Spel.md

* Update Spel.md

Removed term key in the area of JSON like name:value (key:value) pairs, other parts of documentation avoid using term key to avoid confusion with Kafka key, Flink (partitioning) key, etc.

* Applied mproch review comments.

* applied mproch review comments - removed SQLVariable description

Co-authored-by: ZbyszekMM <zml@touk.pl>
Co-authored-by: ZbyszekMM <50945192+ZbyszekMM@users.noreply.github.com>
  • Loading branch information
@arkadius @ZbyszekMM
3 people authored Jul 20, 2021
1 parent 394a9ad commit 7789665
Show file tree
Hide file tree
Showing 2 changed files with 65 additions and 63 deletions.
109 changes: 52 additions & 57 deletions docs/designingProcesses/BasicNodes.md
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,14 @@
# Introduction

Node works with a data stream. It can produce, fetch, send, collect data or organize data flow. Each node has at least two parameters: `Name` and `Description`. Name has to be unique in a scenario. Description is a narrative of your choice.

Most of the nodes, with source and sink nodes being notable exceptions, have input and at least one output flow.

Sinks and filters can be disabled by selecting `Disable` checkbox.

# Filter
Filter passes records which satisfies its condition. It can have one or two outputs.
First output is for records satisfying the condition, second (optional) is for others.
Filter passes records which satisfies filtering condition. It can have one or two outputs.
![filter graph](../img/filter_graph.png)

Records from the `source` which meet filter's condition go to the `true sink`, and others go to the `false sink`.
Expand All @@ -9,94 +17,81 @@ Records from the `source` which meet filter's condition go to the `true sink`, a

Records from the `source` which meets condition go to the `blue sink`, and others are filtered out.

### Parameters
![filter window](../img/filter_window.png)
There are two parameters: `Expression`, and `Disabled`.

Expression is written in SpEL and it is evaluated for each record to logical value. Outgoing data flow depends on expression result.

When the node is Disabled, expression isn't evaluated, and always returns `true`.

### Flow
Filter has one input and one or two outputs.

If there is one output, only outgoing pipe is named `true`, and each record which expression evaluates to true passes. Otherwise record is gone.

If there are two outputs, one pipe is named `true` and another `false`. Each record which expression evaluates to `true` goes to `true` pipe, and other record goes to `false` pipe.


The Expression field should contain the SpEL expression for the filtering conditiona and should produce a boolean value

# Split
Each output receives all records and processes them independently.

Split node logically splits processing into two or more parallel branches. Each branch receives all records and processes them independently.

![split graph](../img/split_graph.png)

Every record from the `source` gos to `sink 1` and `sink 2`.

### Parameters
Split doesn't have additional parameters.

![split window](../img/split_window.png)

### Flow
Split has one or more outputs. Outputs have the same record as input, so all outputs are identical.
Every record from the `source` gos to `sink 1` and `sink 2`. Split node doesn't have additional parameters.



# Switch
Distributes incoming records between outputs.

Switch distributes incoming records among output branches in accordance with the criteria configured in those branches

![switch graph](../img/switch_graph.png)

Each record form the `source` is tested against condition defined on the edge. If `color` is `blue` record goes to the `blue sink`. If `color` is `green` record goes to the `green sink`. For every other value record goes to the `sink for others`.
Each record form the `source` is tested against condition defined on the edge. If `#color` is `blue` record goes to the `blue sink`. If `#color` is `green` record goes to the `green sink`. For every other value record goes to the `sink for others`.

### Parameters
![switch window](../img/switch_window.png)

There are two parameters `Expression` and `exprVal`. `Expression` contains expression which is evaluated for each record, and evaluated value is assigned to variable named in `exprVal`.

Edges outgoing from `Switch` node can be of one of two types.
There are two parameters `Expression` and `exprVal`. `Expression` contains expression which is evaluated for each record, and result is assigned to variable named in `exprVal` - `#color` in the example above.

![switch_edge_condition](../img/switch_edge_condition.png)

Edge of the `Condition` type has an `Expression`. Record go to the first output with matching condition. Order of matching outgoing edges is not guaranteed.
Eeach edge outgoing from `Switch` node has a boolean expression attached to it; if the expression evaluates to true the record is allowed to pass through this edge. Record go to the first output with matching condition. *Order of matching outgoing edges is not guaranteed.*

![switch_edge_default](../img/switch_edge_default.png)

There can be at most one edge of type `Default`, and it gets all records that don't match any `Condition` edge.

### Flow
For each incoming record `Expression` is evaluated and result is assigned to `exprVal` variable. After that, records are tested against condition `Expressions` from output edges one by one. Record goes to first output which condition it satisfies. If record doesn't match any conditional output, and default output exists, record goes to default output. Otherwise mismatching record is filtered out.



# Variable
Evaluates `Expression`, and assigns to `Variable Name.`
### Flow
For each incoming record `Expression` is evaluated and result is assigned to `exprVal` variable. After that, records are tested against condition `Expressions` from output edges one by one. Record goes to the first output where the edge condition is evaluated to `true`. If record doesn't match any conditional output, and default output exists, record goes to default output. Otherwise mismatching record is filtered out.

![variable graph](../img/variable_graph.png)

### Parameters
## Union

![variable_window](../img/variable_window.png)
![union_window](../img/union_window.png)

### Flow
Doesn't change records flow. Have to have exactly one output. Variable once defined cannot be overwritten.
Union merges multiple branches into one stream. For each incoming branch two parameters are configured:
- key - it's value should be of type `String`, definex how elements from branches will be matched together
- value - this is the output value which will be put the field with name the same as branch id

# Map Variable
Defines a new variable with one or more fields. Each field value is evaluated by using `Expression`.
Union node defines new stream which is union of all branches. In this new stream there is only one variable, it's name
is defined by 'Output' parameter, it's value is:
```$json
{
"key": `value of key expression for given event`,
"branch1": `value expression when event comes from branch1, otherwise null`,
"branch2": `value expression when event comes from branch2, otherwise null`,
...
}
```
Currently branches are identified by id of last node in this branch before union.

## UnionMemo

### Parameters
![map_variable_window](../img/map_variable_window.png)
Works exactly like Union, but also memoize values for each branches. Memoized state will be cleared when there was no
subsequent events from any branch during `stateTimeout`. Produced object has values from all branches.

### Flow
Doesn't change records flow. Once defined value of this type cannot be overwritten.

## PreviousValue

## Sql Variable
Evaluates sql `Expression`, and assigns to `Variable Name.`
![previous_value_window](../img/previous_value_window.png)

### Parameters
![sql_variable_window](../img/sql_variable_window.png)
Previousvalue stores arbitrary value for the given key. This element has two parameters:
- keyBy - expression defining key for which we compute aggregate, e.g. `#input.userId`
- value - stored value

### Flow
Doesn't change records flow. Have to have exactly one output. Variable once defined cannot be overwritten.
For example, given stream of events which contain users with their current location, when we set
- keyBy is `#input.userId`
- value is `#input.location`
then the value of output variable is the previous location for current user. If this is the first appearance of this user,
**current** location will be returned.
19 changes: 13 additions & 6 deletions docs/designingProcesses/Spel.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ On the other hand, `map` describes "generic" structure - Nussknacker tacitly ass

Nussknacker usually infers structure of record from external source (e.g. AVRO schema), but it can also detect it from map literals.


### Arrays/lists

In Nussknacker (e.g. in code completion) JSON / Avro arrays are refered to as `Lists`;
Expand Down Expand Up @@ -117,7 +118,7 @@ There are a few notable exceptions:
|`true` |true | Boolean |
|`null` | null | Null |
|`{1,2,3,4}` | a list of integers from 1 to 4 | List[Integer] |
|`{john:300, alex:400}` |a map (key-value collection) | Map[String, Integer] |
|`{john:300, alex:400}` |a map (name-value collection) | Map[String, Integer] |
| `#input` | variable | |
## Arithmetic Operators
Expand All @@ -131,11 +132,17 @@ There are a few notable exceptions:

| Expression | Result | Type |
| ------------|--------|--------|
| `2 == 2` | true | boolean |
| `2 > 1` | true | boolean |
| `true AND false` | false | boolean |
| `true && false` | false | boolean |
| `true OR false` | true | boolean |
| `true OR false` | true | boolean |
| `2 > 1 ? 'a' : 'b'` | "a" | String |
| `2 < 1 ? 'a' : 'b'` | "b" String |
| `#nonNullVar == null ? 'Success'` : 'Unknown' | "Success" | String |
| `#nullVar == null ? 'Success'` : 'Unknown' | "Unknown" | String |
| `#nullVar?:'Unknown` | "Unknown" | String |
| `2 < 1 ? 'a' : 'b'` | "b" | String |
| `#nonNullVar == null ? 'Unkown' : 'Success'` | "Success" | String |
| `#nullVar == null ? 'Unknown' : 'Success'` | "Unknown" | String |
| `#nullVar?:'Unknown'` | "Unknown" | String |
| `'john'?:'Unknown'` | "john" | String |

## Method invocations
Expand All @@ -154,7 +161,7 @@ are shown in code completion hints.
| Expression | Result | Type |
| ------------|--------|--------|
|`{1,2,3,4}[0]` | 1 | Integer |
|`{jan:300, alex:400}[alex]` | a value of key 'alex', which is 400 | Integer |
|`{jan:300, alex:400}[alex]` | a value of field 'alex', which is 400 | Integer |

## Filtering lists
Expand Down

0 comments on commit 7789665

Please sign in to comment.