Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Issue #417] gRPC design doc and protobuf models #594

Merged
merged 37 commits into from
Nov 22, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
ba67c5c
Merge pull request #1 from apache/develop
jinrongluo May 10, 2021
d638ec4
[Issue #337] Fix HttpSubscriber startup issue
May 10, 2021
5ebfb54
[Issue #337] test commit
jinrongluo May 10, 2021
a3afff3
[Issue #337] revert test commit
jinrongluo May 10, 2021
50f959d
[Issue #337] Enhance Http Demo Subscriber by using ExecutorService, C…
jinrongluo May 11, 2021
7adc322
Merge remote-tracking branch 'origin/develop' into develop
jinrongluo May 11, 2021
d48ead5
[Issue #337] Enhance Http Demo Subscriber by using ExecutorService, C…
jinrongluo May 11, 2021
c9021fe
[Issue #337] Address code review comment for Subscriber Demo App
jinrongluo May 12, 2021
c6d732e
Merge branch 'apache:develop' into develop
jinrongluo May 14, 2021
c613be8
Merge branch 'apache:develop' into develop
jinrongluo May 18, 2021
66ac95e
Merge branch 'apache:develop' into develop
jinrongluo May 19, 2021
9e636c0
Merge branch 'apache:develop' into develop
jinrongluo May 26, 2021
a0f44b0
Merge branch 'apache:develop' into develop
jinrongluo Jun 4, 2021
37f5d7a
Merge branch 'apache:develop' into develop
jinrongluo Jun 12, 2021
93e711c
Merge branch 'apache:develop' into develop
jinrongluo Jun 15, 2021
e75d67a
Merge branch 'apache:develop' into develop
jinrongluo Jun 21, 2021
2fb485b
Merge branch 'apache:develop' into develop
jinrongluo Jul 5, 2021
80ab9af
Merge branch 'apache:develop' into develop
jinrongluo Sep 14, 2021
87d1747
Merge branch 'apache:develop' into develop
jinrongluo Sep 17, 2021
c9d3537
Merge branch 'apache:develop' into develop
jinrongluo Sep 26, 2021
c9e38f5
Merge branch 'apache:develop' into develop
jinrongluo Sep 27, 2021
63728fc
Merge branch 'apache:develop' into develop
jinrongluo Oct 12, 2021
ad18b29
Merge branch 'apache:develop' into develop
jinrongluo Oct 16, 2021
0c58188
Merge branch 'apache:develop' into develop
jinrongluo Oct 18, 2021
0141006
Merge branch 'apache:develop' into develop
jinrongluo Nov 12, 2021
bf879e4
Merge branch 'apache:develop' into develop
jinrongluo Nov 16, 2021
d4c4810
Merge branch 'apache:develop' into develop
jinrongluo Nov 16, 2021
51e9f6d
adding license headers
jinrongluo Nov 16, 2021
9ebe2f0
adding grpc build file
jinrongluo Nov 16, 2021
8845f62
[Issue#417] update settings.gradle
jinrongluo Nov 16, 2021
f7be96d
[Issue#417] fix grpc generated code styles
jinrongluo Nov 16, 2021
9a9848d
[Issue#417] fix grpc generated code styles
jinrongluo Nov 16, 2021
bdb43e1
[Issue#417] fix grpc generated code styles
jinrongluo Nov 16, 2021
26b5a9d
[Issue#417] fix grpc generated code styles and license issue
jinrongluo Nov 16, 2021
c84f01b
[Issue#417] fix license issue
jinrongluo Nov 16, 2021
1f14710
[Issue #417] ignore checkstyle for generated files
jinrongluo Nov 18, 2021
0ef7783
[Issue #417] adding allow licensed for grpc protobuf
jinrongluo Nov 18, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,8 @@ allprojects {
maxWarnings = 0
configFile = new File("${rootDir}/style/checkStyle.xml")
}

checkstyleMain.exclude '**/org/apache/eventmesh/client/grpc/protos**'
}

task tar(type: Tar) {
Expand Down
120 changes: 120 additions & 0 deletions docs/cn/instructions/eventmesh-runtime-protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -258,3 +258,123 @@ public class LiteMessage {
| 场景 | Server向Client发送消息请求码 | Client回复Server消息响应码 | 说明 |
| ------------------ | ---------------------------- | -------------------------- | ---------------------- |
| 客户端接收异步事件 | HTTP_PUSH_CLIENT_ASYNC(105) | retCode | retCode值为0时代表成功 |


## gRPC 协议文档

#### 1. protobuf

在 `eventmesh-protocol-gprc` 模块有 Eventmesh gRPC 客户端的 protobuf 文件. the protobuf 文件路径是 `/src/main/proto/eventmesh-client.proto`.

用gradle build 生成 gRPC 代码在 `/build/generated/source/proto/main`. 生成代码用于 `eventmesh-sdk-java` 模块.

#### 2. gRPC 数据模型

- 消息

以下消息数据模型用于 `publish()`, `requestReply()` 和 `broadcast()` APIs.

```
message RequestHeader {
string env = 1;
string region = 2;
string idc = 3;
string ip = 4;
string pid = 5;
string sys = 6;
string username = 7;
string password = 8;
string version = 9;
string language = 10;
string seqNum = 11;
}

message Message {
RequestHeader header = 1;
string productionGroup = 2;
string topic = 3;
string content = 4;
string ttl = 5;
string uniqueId = 6;
}

message Response {
string respCode = 1;
string respMsg = 2;
string respTime = 3;
string seqNum = 4;
}
```

- 订阅

以下订阅数据模型用于 `subscribe()` 和 `unsubscribe()` APIs.

```
message Subscription {
RequestHeader header = 1;
string consumerGroup = 2;

message SubscriptionItem {
string topic = 1;
string mode = 2;
string type = 3;
string url = 4;
}

repeated SubscriptionItem subscriptionItems = 3;
}
```

- 心跳

以下心跳数据模型用于 `heartbeat()` API.

```
message Heartbeat {
RequestHeader header = 1;
string clientType = 2;
string producerGroup = 3;
string consumerGroup = 4;

message HeartbeatItem {
string topic = 1;
string url = 2;
}

repeated HeartbeatItem heartbeatItems = 5;
}
```

#### 3. gRPC 服务接口

- 事件生产端服务 APIs

```
# 异步事件生产
rpc publish(Message) returns (Response);

# 同步事件生产
rpc requestReply(Message) returns (Response);

# 事件广播
rpc broadcast(Message) returns (Response);
```

- 事件消费端服务 APIs

```
# 所消费事件通过 HTTP Webhook推送事件
rpc subscribe(Subscription) returns (Response);

# 所消费事件通过 TCP stream推送事件
rpc subscribeStream(Subscription) returns (stream Message);

rpc unsubscribe(Subscription) returns (Response);
```

- 客户端心跳服务 API

```
rpc heartbeat(Heartbeat) returns (Response);
```
122 changes: 122 additions & 0 deletions docs/en/instructions/eventmesh-runtime-protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,3 +259,125 @@ same with RequestHeader of Heartbeat Msg
| Scene | Server Send | Client Reply | Remark |
| ------------------ | ---------------------------- | -------------------------- | ---------------------- |
| Push async msg to client | HTTP_PUSH_CLIENT_ASYNC(105) | retCode | retCode=0,send success |

## gRPC Protocol Document In Eventmesh-Runtime

#### 1. protobuf

The `eventmesh-protocol-gprc` module contains the protobuf file of the evenmesh client. the protobuf file
is located as `/src/main/proto/eventmesh-client.proto`.

Run the gradle build to generate the gRPC codes. The generated codes are located at `/build/generated/source/proto/main`.

These generated grpc codes will be used in `eventmesh-sdk-java` module.

#### 2. data models

- message

The following is the message data model, used by `publish()`, `requestReply()` and `broadcast()` APIs.

```
message RequestHeader {
string env = 1;
string region = 2;
string idc = 3;
string ip = 4;
string pid = 5;
string sys = 6;
string username = 7;
string password = 8;
string version = 9;
string language = 10;
string seqNum = 11;
}

message Message {
RequestHeader header = 1;
string productionGroup = 2;
string topic = 3;
string content = 4;
string ttl = 5;
string uniqueId = 6;
}

message Response {
string respCode = 1;
string respMsg = 2;
string respTime = 3;
string seqNum = 4;
}
```

- subscription

The following data model is used by `subscribe()` and `unsubscribe()` APIs.

```
message Subscription {
RequestHeader header = 1;
string consumerGroup = 2;

message SubscriptionItem {
string topic = 1;
string mode = 2;
string type = 3;
string url = 4;
}

repeated SubscriptionItem subscriptionItems = 3;
}
```

- heartbeat

The following data model is used by `heartbeat()` API.

```
message Heartbeat {
RequestHeader header = 1;
string clientType = 2;
string producerGroup = 3;
string consumerGroup = 4;

message HeartbeatItem {
string topic = 1;
string url = 2;
}

repeated HeartbeatItem heartbeatItems = 5;
}
```

#### 3. service operations

- event publisher service APIs

```
# Async event publish
rpc publish(Message) returns (Response);

# Sync event publish
rpc requestReply(Message) returns (Response);

# event broadcast
rpc broadcast(Message) returns (Response);
```

- event consumer service APIs

```
# The subscribed event will be delivered by invoking the webhook url in the Subscription
rpc subscribe(Subscription) returns (Response);

# The subscribed event will be delivered through stream of Message
rpc subscribeStream(Subscription) returns (stream Message);

rpc unsubscribe(Subscription) returns (Response);
```

- client heartbeat service API

```
rpc heartbeat(Heartbeat) returns (Response);
```
53 changes: 53 additions & 0 deletions eventmesh-protocol-plugin/eventmesh-protocol-grpc/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

plugins {
id 'java'
id 'com.google.protobuf' version '0.8.17'
}

repositories {
mavenCentral()
}

def grpcVersion = '1.15.0' // CURRENT_GRPC_VERSION
def protobufVersion = '3.5.1'
def protocVersion = protobufVersion

dependencies {
implementation "io.grpc:grpc-protobuf:${grpcVersion}"
implementation "io.grpc:grpc-stub:${grpcVersion}"
implementation "com.google.protobuf:protobuf-java-util:${protobufVersion}"
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.6.0'
testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine'
}

protobuf {
protoc { artifact = "com.google.protobuf:protoc:${protocVersion}" }
plugins {
grpc { artifact = "io.grpc:protoc-gen-grpc-java:${grpcVersion}" }
}
generateProtoTasks {
all()*.plugins {
grpc {}
}
}
}

test {
useJUnitPlatform()
}
Loading