Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: topic management #86

Merged
merged 1 commit into from
Jun 11, 2020
Merged

feat: topic management #86

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
172 changes: 92 additions & 80 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,12 @@
- [x] Basic [Consumer/Producer API use](#consumer-api-and-producer-api)
- [x] Several [prompt options](#prompt-options-tree) (`polling.timeout`, `auto.offset.reset.policy`)
- [x] [AKHQ (KafkaHQ)](#akhq) support
- [x] Topic management

## 🛠 To do or doing...

You can have more details about work in progress in [issues](https://github.com/fdelbrayelle/generator-jhipster-kafka/issues):

- [ ] Topic management
- [ ] Producer API (ordered messages, high throughput...)
- [ ] Deserialization alternatives (JacksonSerde) as a prompt option
- [ ] Security (SSL protocol as a prompt option, safe mode...)
Expand Down Expand Up @@ -112,85 +112,97 @@ Choose your own adventure module!

The **END** represents the end of the prompts below, when files are written after confirmation (you can use the `--force` option with `yo jhipster-kafka` to overwrite all files).

```
.
├── Big Bang Mode (build a configuration from scratch) (default)
│ ├── Consumer
│ │ ├── No entity (will be typed String) (default)
│ │ │ └── componentPrefix
│ │ │ └── pollingTimeoutValue (default = 10000)
│ │ │ ├── earliest (automatically reset the offset to the earliest offset) (default)
│ │ │ │ └── END
│ │ │ ├── latest (automatically reset the offset to the latest offset)
│ │ │ │ └── END
│ │ │ └── none (throw exception to the consumer if no previous offset is found for the consumer group)
│ │ │ └── END
│ │ ├── FooEntity
│ │ │ └── pollingTimeoutValue (default = 10000)
│ │ │ ├── earliest (automatically reset the offset to the earliest offset) (default)
│ │ │ │ └── END
│ │ │ ├── latest (automatically reset the offset to the latest offset)
│ │ │ │ └── END
│ │ │ └── none (throw exception to the consumer if no previous offset is found for the consumer group)
│ │ │ └── END
│ │ └── BarEntity
│ │ └── pollingTimeoutValue (default = 10000)
│ │ ├── earliest (automatically reset the offset to the earliest offset) (default)
│ │ ├── latest (automatically reset the offset to the latest offset)
│ │ └── none (throw exception to the consumer if no previous offset is found for the consumer group)
│ └── Producer
│ ├── No entity (will be typed String)
│ │ └── componentPrefix
│ │ └── END
│ ├── FooEntity
│ │ └── END
│ └── BarEntity
│ └── END
└── Incremental Mode (upgrade an existing configuration)
├── No entity (will be typed String) (default)
│ └── componentPrefix
│ ├── Consumer
│ │ └── pollingTimeoutValue (default = 10000)
│ │ ├── earliest (automatically reset the offset to the earliest offset) (default)
│ │ │ ├── Continue adding consumers or producers (default = N)
│ │ │ └── END
│ │ ├── latest (automatically reset the offset to the latest offset)
│ │ │ ├── Continue adding consumers or producers (default = N)
│ │ │ └── END
│ │ └── none (throw exception to the consumer if no previous offset is found for the consumer group)
│ │ ├── Continue adding consumers or producers (default = N)
│ │ └── END
│ └── Producer
│ └── END
├── FooEntity
│ ├── Consumer
│ │ └── pollingTimeoutValue (default = 10000)
│ │ ├── earliest (automatically reset the offset to the earliest offset) (default)
│ │ │ ├── Continue adding consumers or producers (default = N)
│ │ │ └── END
│ │ ├── latest (automatically reset the offset to the latest offset)
│ │ │ ├── Continue adding consumers or producers (default = N)
│ │ │ └── END
│ │ └── none (throw exception to the consumer if no previous offset is found for the consumer group)
│ │ ├── Continue adding consumers or producers (default = N)
│ │ └── END
│ └── Producer
│ └── END
└── BarEntity
├── Consumer
│ └── pollingTimeoutValue (default = 10000)
│ ├── earliest (automatically reset the offset to the earliest offset) (default)
│ │ ├── Continue adding consumers or producers (default = N)
│ │ └── END
│ ├── latest (automatically reset the offset to the latest offset)
│ │ ├── Continue adding consumers or producers (default = N)
│ │ └── END
│ └── none (throw exception to the consumer if no previous offset is found for the consumer group)
│ ├── Continue adding consumers or producers (default = N)
│ └── END
└── Producer
└── END
```
<ul>
<li>Big Bang Mode (build a configuration from scratch) <strong>(default)</strong>
<ul>
<li>Which components would you like to generate?
<ul>
<li>Consumer</li>
<li>Producer</li>
</ul>
</li>
<li>For which entity (class name)?
<ul>
<li>No entity (will be typed String) <strong>(default)</strong>
<ul>
<li>How would you prefix your objects (no entity, for instance: [SomeEventType]Consumer|Producer...)?</li>
</ul>
</li>
<li>Foo</li>
<li>Bar</li>
<li>...</li>
</ul>
</li>
<li><em>If "Consumer" was selected:</em> What is the consumer polling timeout (in ms)? <strong>(default = 10000)</strong></li>
<li><em>If "Consumer" was selected:</em> Define the auto offset reset policy (what to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server)?
<ul>
<li>earliest (automatically reset the offset to the earliest offset) <strong>(default)</strong></li>
<li>latest (automatically reset the offset to the latest offset)</li>
<li>none (throw exception to the consumer if no previous offset is found for the consumer group)</li>
</ul>
</li>
<li>For which topic?
<ul>
<li>Default topic name following this convention: message_type.application_type.entity_name <strong>(default)</strong></li>
<li>Custom topic name
<ul>
<li>What is the topic name?</li>
</ul>
</li>
<li>queuing.application_name.existing_topic_name</li>
<li>...</li>
</ul>
</li>
<li>Do you want to continue to the next entity/prefix or exit? <strong>(default = Y)</strong></li>
<li><strong>END</strong></li>
</ul>
</li>
<li>Incremental Mode (upgrade an existing configuration)
<ul>
<li>For which entity (class name)?
<ul>
<li>No entity (will be typed String) <strong>(default)</strong>
<ul>
<li>How would you prefix your objects (no entity, for instance: [SomeEventType]Consumer|Producer...)?</li>
</ul>
</li>
<li>Foo</li>
<li>Bar</li>
<li>...</li>
</ul>
</li>
<li>Which components would you like to generate?
<ul>
<li>Consumer</li>
<li>Producer</li>
</ul>
</li>
<li>For which topic?
<ul>
<li>Default topic name following this convention: message_type.application_type.entity_name <strong>(default)</strong></li>
<li>Custom topic name
<ul>
<li>What is the topic name?</li>
</ul>
</li>
<li>queuing.application_name.existing_topic_name</li>
<li>...</li>
</ul>
</li>
<li><em>If "Consumer" was selected:</em> What is the consumer polling timeout (in ms)? <strong>(default = 10000)</strong></li>
<li><em>If "Consumer" was selected:</em> Define the auto offset reset policy (what to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server)?
<ul>
<li>earliest (automatically reset the offset to the earliest offset) <strong>(default)</strong></li>
<li>latest (automatically reset the offset to the latest offset)</li>
<li>none (throw exception to the consumer if no previous offset is found for the consumer group)</li>
</ul>
</li>
<li>Do you want to continue adding consumers or producers? <strong>(default = N)</strong></li>
<li><em>If "N" was selected:</em> <strong>END</strong></li>
</ul>
</li>
</ul>
</code>

## Skip prompts

Expand Down
60 changes: 36 additions & 24 deletions USE_CASES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,10 @@ After following the first 3 steps of the [basic usage](README.md#basic-usage) ab
5. "For which entity (class name)?" - Foo (the available entities are retrieved in the `.jhipster` folder as `.json`)
6. "What is the consumer polling timeout (in ms)?" - Your answer or '10000' by default (global for all consumers)
7. "Define the auto offset reset policy?" - Your answer or 'earliest' by default (global for all consumers)
8. Overwrite all files in conflict
9. `FooConsumer` (consumes `Foo`) is available with a `FooDeserializer`
8. "Which topic for Foo?" - Any choice (choosing "Custom topic name" will add another question "What is the topic name for Foo?")
9. Loop on each entity with step 8
10. Overwrite all files in conflict
11. `FooConsumer` (consumes `Foo`) is available with a `FooDeserializer`

#### Incremental Mode

Expand All @@ -39,11 +41,12 @@ After following the first 3 steps of the [basic usage](README.md#basic-usage) ab
3. "Which type of generation do you want?" - Incremental Mode (upgrade an existing configuration)
4. "For which entity (class name)?" - Foo (the available entities are retrieved in the `.jhipster` folder as `.json`)
5. "Which components would you like to generate?" - Consumer
6. "What is the consumer polling timeout (in ms)?" - Your answer or '10000' by default (global for all consumers)
7. "Define the auto offset reset policy?" - Your answer or 'earliest' by default (global for all consumers)
8. "Do you want to continue adding consumers or producers?" - Your answer or 'N' par default
9. Overwrite all files in conflict
10. `FooConsumer` (consumes `Foo`) is available with a `FooDeserializer`
6. "For which topic?" - Any choice (choosing "Custom topic name" will add another question "What is the topic name?")
7. "What is the consumer polling timeout (in ms)?" - Your answer or '10000' by default (global for all consumers)
8. "Define the auto offset reset policy?" - Your answer or 'earliest' by default (global for all consumers)
9. "Do you want to continue adding consumers or producers?" - Your answer or 'N' par default
10. Overwrite all files in conflict
11. `FooConsumer` (consumes `Foo`) is available with a `FooDeserializer`

### Create a producer linked to an entity

Expand All @@ -56,8 +59,10 @@ After following the first 3 steps of the [basic usage](README.md#basic-usage) ab
3. "Which type of generation do you want?" - Big Bang Mode (build a configuration from scratch)
4. "Which components would you like to generate?" - Producer
5. "For which entity (class name)?" - Foo (the available entities are retrieved in the `.jhipster` folder as `.json`)
6. Overwrite all files in conflict
7. `FooProducer` (produces `Foo`) is available with a `FooSerializer` and a `FooKafkaResource` to [help testing](README.md#test-consumers-and-producers)
6. "Which topic for Foo?" - Any choice (choosing "Custom topic name" will add another question "What is the topic name for Foo?")
7. Loop on each entity with step 6
8. Overwrite all files in conflict
9. `FooProducer` (produces `Foo`) is available with a `FooSerializer` and a `FooKafkaResource` to [help testing](README.md#test-consumers-and-producers)

#### Incremental Mode

Expand All @@ -66,9 +71,10 @@ After following the first 3 steps of the [basic usage](README.md#basic-usage) ab
3. "Which type of generation do you want?" - Incremental Mode (upgrade an existing configuration)
4. "For which entity (class name)?" - Foo (the available entities are retrieved in the `.jhipster` folder as `.json`)
5. "Which components would you like to generate?" - Producer
6. "Do you want to continue adding consumers or producers?" - Your answer or 'N' par default
7. Overwrite all files in conflict
8. `FooProducer` (produces `Foo`) is available with a `FooSerializer` and a `FooKafkaResource` to [help testing](README.md#test-consumers-and-producers)
6. "For which topic?" - Any choice (choosing "Custom topic name" will add another question "What is the topic name?")
7. "Do you want to continue adding consumers or producers?" - Your answer or 'N' par default
8. Overwrite all files in conflict
9. `FooProducer` (produces `Foo`) is available with a `FooSerializer` and a `FooKafkaResource` to [help testing](README.md#test-consumers-and-producers)

### Create a consumer NOT linked to an entity

Expand All @@ -84,8 +90,10 @@ After following the first 3 steps of the [basic usage](README.md#basic-usage) ab
6. "How would you prefix your objects (no entity, for instance: [SomeEventType]Consumer|Producer...)?" - someEventType
7. "What is the consumer polling timeout (in ms)?" - Your answer or '10000' by default (global for all consumers)
8. "Define the auto offset reset policy?" - Your answer or 'earliest' by default (global for all consumers)
9. Overwrite all files in conflict
10. `SomeEventTypeConsumer` (consumes `String`) is available with a `SomeEventTypeDeserializer`
9. "Which topic for Foo?" - Any choice (choosing "Custom topic name" will add another question "What is the topic name for Foo?")
10. Loop on each entity with step 9
11. Overwrite all files in conflict
12. `SomeEventTypeConsumer` (consumes `String`) is available with a `SomeEventTypeDeserializer`

#### Incremental Mode

Expand All @@ -95,11 +103,12 @@ After following the first 3 steps of the [basic usage](README.md#basic-usage) ab
4. "For which entity (class name)?" - No entity (will be typed String)
5. "How would you prefix your objects (no entity, for instance: [SomeEventType]Consumer|Producer...)?" - someEventType
6. "Which components would you like to generate?" - Consumer
7. "What is the consumer polling timeout (in ms)?" - Your answer or '10000' by default (global for all consumers)
8. "Define the auto offset reset policy?" - Your answer or 'earliest' by default (global for all consumers)
9. "Do you want to continue adding consumers or producers?" - Your answer or 'N' par default
10. Overwrite all files in conflict
11. `SomeEventTypeConsumer` (consumes `String`) is available with a `SomeEventTypeDeserializer`
7. "For which topic?" - Any choice (choosing "Custom topic name" will add another question "What is the topic name?")
8. "What is the consumer polling timeout (in ms)?" - Your answer or '10000' by default (global for all consumers)
9. "Define the auto offset reset policy?" - Your answer or 'earliest' by default (global for all consumers)
10. "Do you want to continue adding consumers or producers?" - Your answer or 'N' par default
11. Overwrite all files in conflict
12. `SomeEventTypeConsumer` (consumes `String`) is available with a `SomeEventTypeDeserializer`

### Create a producer NOT linked to an entity

Expand All @@ -113,8 +122,10 @@ After following the first 3 steps of the [basic usage](README.md#basic-usage) ab
4. "Which components would you like to generate?" - Producer
5. "For which entity (class name)?" - No entity (will be typed String)
6. "How would you prefix your objects (no entity, for instance: [SomeEventType]Consumer|Producer...)?" - someEventType
7. Overwrite all files in conflict
8. `SomeEventTypeProducer` (produces `String`) is available with a `SomeEventTypeSerializer` and a `SomeEventTypeKafkaResource` to [help testing](README.md#test-consumers-and-producers)
7. "Which topic for Foo?" - Any choice (choosing "Custom topic name" will add another question "What is the topic name for Foo?")
8. Loop on each entity with step 7
9. Overwrite all files in conflict
10. `SomeEventTypeProducer` (produces `String`) is available with a `SomeEventTypeSerializer` and a `SomeEventTypeKafkaResource` to [help testing](README.md#test-consumers-and-producers)

#### Incremental Mode

Expand All @@ -124,6 +135,7 @@ After following the first 3 steps of the [basic usage](README.md#basic-usage) ab
4. "For which entity (class name)?" - No entity (will be typed String)
5. "How would you prefix your objects (no entity, for instance: [SomeEventType]Consumer|Producer...)?" - someEventType
6. "Which components would you like to generate?" - Producer
7. "Do you want to continue adding consumers or producers?" - Your answer or 'N' par default
8. Overwrite all files in conflict
9. `SomeEventTypeProducer` (produces `String`) is available with a `SomeEventTypeSerializer` and a `SomeEventTypeKafkaResource` to [help testing](README.md#test-consumers-and-producers)
7. "For which topic?" - Any choice (choosing "Custom topic name" will add another question "What is the topic name?")
8. "Do you want to continue adding consumers or producers?" - Your answer or 'N' par default
9. Overwrite all files in conflict
10. `SomeEventTypeProducer` (produces `String`) is available with a `SomeEventTypeSerializer` and a `SomeEventTypeKafkaResource` to [help testing](README.md#test-consumers-and-producers)
Loading