Skip to content

Commit

Permalink
feat: topic management (#86)
Browse files Browse the repository at this point in the history
  • Loading branch information
@fdelbrayelle
fdelbrayelle authored Jun 11, 2020
1 parent e76ba89 commit 4195115
Show file tree
Hide file tree
Showing 16 changed files with 486 additions and 138 deletions.
172 changes: 92 additions & 80 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,12 @@
- [x] Basic [Consumer/Producer API use](#consumer-api-and-producer-api)
- [x] Several [prompt options](#prompt-options-tree) (`polling.timeout`, `auto.offset.reset.policy`)
- [x] [AKHQ (KafkaHQ)](#akhq) support
- [x] Topic management

## 🛠 To do or doing...

You can have more details about work in progress in [issues](https://github.com/fdelbrayelle/generator-jhipster-kafka/issues):

- [ ] Topic management
- [ ] Producer API (ordered messages, high throughput...)
- [ ] Deserialization alternatives (JacksonSerde) as a prompt option
- [ ] Security (SSL protocol as a prompt option, safe mode...)
Expand Down Expand Up @@ -112,85 +112,97 @@ Choose your own adventure module!

The **END** represents the end of the prompts below, when files are written after confirmation (you can use the `--force` option with `yo jhipster-kafka` to overwrite all files).

```
.
├── Big Bang Mode (build a configuration from scratch) (default)
│ ├── Consumer
│ │ ├── No entity (will be typed String) (default)
│ │ │ └── componentPrefix
│ │ │ └── pollingTimeoutValue (default = 10000)
│ │ │ ├── earliest (automatically reset the offset to the earliest offset) (default)
│ │ │ │ └── END
│ │ │ ├── latest (automatically reset the offset to the latest offset)
│ │ │ │ └── END
│ │ │ └── none (throw exception to the consumer if no previous offset is found for the consumer group)
│ │ │ └── END
│ │ ├── FooEntity
│ │ │ └── pollingTimeoutValue (default = 10000)
│ │ │ ├── earliest (automatically reset the offset to the earliest offset) (default)
│ │ │ │ └── END
│ │ │ ├── latest (automatically reset the offset to the latest offset)
│ │ │ │ └── END
│ │ │ └── none (throw exception to the consumer if no previous offset is found for the consumer group)
│ │ │ └── END
│ │ └── BarEntity
│ │ └── pollingTimeoutValue (default = 10000)
│ │ ├── earliest (automatically reset the offset to the earliest offset) (default)
│ │ ├── latest (automatically reset the offset to the latest offset)
│ │ └── none (throw exception to the consumer if no previous offset is found for the consumer group)
│ └── Producer
│ ├── No entity (will be typed String)
│ │ └── componentPrefix
│ │ └── END
│ ├── FooEntity
│ │ └── END
│ └── BarEntity
│ └── END
└── Incremental Mode (upgrade an existing configuration)
├── No entity (will be typed String) (default)
│ └── componentPrefix
│ ├── Consumer
│ │ └── pollingTimeoutValue (default = 10000)
│ │ ├── earliest (automatically reset the offset to the earliest offset) (default)
│ │ │ ├── Continue adding consumers or producers (default = N)
│ │ │ └── END
│ │ ├── latest (automatically reset the offset to the latest offset)
│ │ │ ├── Continue adding consumers or producers (default = N)
│ │ │ └── END
│ │ └── none (throw exception to the consumer if no previous offset is found for the consumer group)
│ │ ├── Continue adding consumers or producers (default = N)
│ │ └── END
│ └── Producer
│ └── END
├── FooEntity
│ ├── Consumer
│ │ └── pollingTimeoutValue (default = 10000)
│ │ ├── earliest (automatically reset the offset to the earliest offset) (default)
│ │ │ ├── Continue adding consumers or producers (default = N)
│ │ │ └── END
│ │ ├── latest (automatically reset the offset to the latest offset)
│ │ │ ├── Continue adding consumers or producers (default = N)
│ │ │ └── END
│ │ └── none (throw exception to the consumer if no previous offset is found for the consumer group)
│ │ ├── Continue adding consumers or producers (default = N)
│ │ └── END
│ └── Producer
│ └── END
└── BarEntity
├── Consumer
│ └── pollingTimeoutValue (default = 10000)
│ ├── earliest (automatically reset the offset to the earliest offset) (default)
│ │ ├── Continue adding consumers or producers (default = N)
│ │ └── END
│ ├── latest (automatically reset the offset to the latest offset)
│ │ ├── Continue adding consumers or producers (default = N)
│ │ └── END
│ └── none (throw exception to the consumer if no previous offset is found for the consumer group)
│ ├── Continue adding consumers or producers (default = N)
│ └── END
└── Producer
└── END
```
<ul>
<li>Big Bang Mode (build a configuration from scratch) <strong>(default)</strong>
<ul>
<li>Which components would you like to generate?
<ul>
<li>Consumer</li>
<li>Producer</li>
</ul>
</li>
<li>For which entity (class name)?
<ul>
<li>No entity (will be typed String) <strong>(default)</strong>
<ul>
<li>How would you prefix your objects (no entity, for instance: [SomeEventType]Consumer|Producer...)?</li>
</ul>
</li>
<li>Foo</li>
<li>Bar</li>
<li>...</li>
</ul>
</li>
<li><em>If "Consumer" was selected:</em> What is the consumer polling timeout (in ms)? <strong>(default = 10000)</strong></li>
<li><em>If "Consumer" was selected:</em> Define the auto offset reset policy (what to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server)?
<ul>
<li>earliest (automatically reset the offset to the earliest offset) <strong>(default)</strong></li>
<li>latest (automatically reset the offset to the latest offset)</li>
<li>none (throw exception to the consumer if no previous offset is found for the consumer group)</li>
</ul>
</li>
<li>For which topic?
<ul>
<li>Default topic name following this convention: message_type.application_type.entity_name <strong>(default)</strong></li>
<li>Custom topic name
<ul>
<li>What is the topic name?</li>
</ul>
</li>
<li>queuing.application_name.existing_topic_name</li>
<li>...</li>
</ul>
</li>
<li>Do you want to continue to the next entity/prefix or exit? <strong>(default = Y)</strong></li>
<li><strong>END</strong></li>
</ul>
</li>
<li>Incremental Mode (upgrade an existing configuration)
<ul>
<li>For which entity (class name)?
<ul>
<li>No entity (will be typed String) <strong>(default)</strong>
<ul>
<li>How would you prefix your objects (no entity, for instance: [SomeEventType]Consumer|Producer...)?</li>
</ul>
</li>
<li>Foo</li>
<li>Bar</li>
<li>...</li>
</ul>
</li>
<li>Which components would you like to generate?
<ul>
<li>Consumer</li>
<li>Producer</li>
</ul>
</li>
<li>For which topic?
<ul>
<li>Default topic name following this convention: message_type.application_type.entity_name <strong>(default)</strong></li>
<li>Custom topic name
<ul>
<li>What is the topic name?</li>
</ul>
</li>
<li>queuing.application_name.existing_topic_name</li>
<li>...</li>
</ul>
</li>
<li><em>If "Consumer" was selected:</em> What is the consumer polling timeout (in ms)? <strong>(default = 10000)</strong></li>
<li><em>If "Consumer" was selected:</em> Define the auto offset reset policy (what to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server)?
<ul>
<li>earliest (automatically reset the offset to the earliest offset) <strong>(default)</strong></li>
<li>latest (automatically reset the offset to the latest offset)</li>
<li>none (throw exception to the consumer if no previous offset is found for the consumer group)</li>
</ul>
</li>
<li>Do you want to continue adding consumers or producers? <strong>(default = N)</strong></li>
<li><em>If "N" was selected:</em> <strong>END</strong></li>
</ul>
</li>
</ul>
</code>

## Skip prompts

Expand Down
60 changes: 36 additions & 24 deletions USE_CASES.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,10 @@ After following the first 3 steps of the [basic usage](README.md#basic-usage) ab
5. "For which entity (class name)?" - Foo (the available entities are retrieved in the `.jhipster` folder as `.json`)
6. "What is the consumer polling timeout (in ms)?" - Your answer or '10000' by default (global for all consumers)
7. "Define the auto offset reset policy?" - Your answer or 'earliest' by default (global for all consumers)
8. Overwrite all files in conflict
9. `FooConsumer` (consumes `Foo`) is available with a `FooDeserializer`
8. "Which topic for Foo?" - Any choice (choosing "Custom topic name" will add another question "What is the topic name for Foo?")
9. Loop on each entity with step 8
10. Overwrite all files in conflict
11. `FooConsumer` (consumes `Foo`) is available with a `FooDeserializer`

#### Incremental Mode

Expand All @@ -39,11 +41,12 @@ After following the first 3 steps of the [basic usage](README.md#basic-usage) ab
3. "Which type of generation do you want?" - Incremental Mode (upgrade an existing configuration)
4. "For which entity (class name)?" - Foo (the available entities are retrieved in the `.jhipster` folder as `.json`)
5. "Which components would you like to generate?" - Consumer
6. "What is the consumer polling timeout (in ms)?" - Your answer or '10000' by default (global for all consumers)
7. "Define the auto offset reset policy?" - Your answer or 'earliest' by default (global for all consumers)
8. "Do you want to continue adding consumers or producers?" - Your answer or 'N' par default
9. Overwrite all files in conflict
10. `FooConsumer` (consumes `Foo`) is available with a `FooDeserializer`
6. "For which topic?" - Any choice (choosing "Custom topic name" will add another question "What is the topic name?")
7. "What is the consumer polling timeout (in ms)?" - Your answer or '10000' by default (global for all consumers)
8. "Define the auto offset reset policy?" - Your answer or 'earliest' by default (global for all consumers)
9. "Do you want to continue adding consumers or producers?" - Your answer or 'N' par default
10. Overwrite all files in conflict
11. `FooConsumer` (consumes `Foo`) is available with a `FooDeserializer`

### Create a producer linked to an entity

Expand All @@ -56,8 +59,10 @@ After following the first 3 steps of the [basic usage](README.md#basic-usage) ab
3. "Which type of generation do you want?" - Big Bang Mode (build a configuration from scratch)
4. "Which components would you like to generate?" - Producer
5. "For which entity (class name)?" - Foo (the available entities are retrieved in the `.jhipster` folder as `.json`)
6. Overwrite all files in conflict
7. `FooProducer` (produces `Foo`) is available with a `FooSerializer` and a `FooKafkaResource` to [help testing](README.md#test-consumers-and-producers)
6. "Which topic for Foo?" - Any choice (choosing "Custom topic name" will add another question "What is the topic name for Foo?")
7. Loop on each entity with step 6
8. Overwrite all files in conflict
9. `FooProducer` (produces `Foo`) is available with a `FooSerializer` and a `FooKafkaResource` to [help testing](README.md#test-consumers-and-producers)

#### Incremental Mode

Expand All @@ -66,9 +71,10 @@ After following the first 3 steps of the [basic usage](README.md#basic-usage) ab
3. "Which type of generation do you want?" - Incremental Mode (upgrade an existing configuration)
4. "For which entity (class name)?" - Foo (the available entities are retrieved in the `.jhipster` folder as `.json`)
5. "Which components would you like to generate?" - Producer
6. "Do you want to continue adding consumers or producers?" - Your answer or 'N' par default
7. Overwrite all files in conflict
8. `FooProducer` (produces `Foo`) is available with a `FooSerializer` and a `FooKafkaResource` to [help testing](README.md#test-consumers-and-producers)
6. "For which topic?" - Any choice (choosing "Custom topic name" will add another question "What is the topic name?")
7. "Do you want to continue adding consumers or producers?" - Your answer or 'N' par default
8. Overwrite all files in conflict
9. `FooProducer` (produces `Foo`) is available with a `FooSerializer` and a `FooKafkaResource` to [help testing](README.md#test-consumers-and-producers)

### Create a consumer NOT linked to an entity

Expand All @@ -84,8 +90,10 @@ After following the first 3 steps of the [basic usage](README.md#basic-usage) ab
6. "How would you prefix your objects (no entity, for instance: [SomeEventType]Consumer|Producer...)?" - someEventType
7. "What is the consumer polling timeout (in ms)?" - Your answer or '10000' by default (global for all consumers)
8. "Define the auto offset reset policy?" - Your answer or 'earliest' by default (global for all consumers)
9. Overwrite all files in conflict
10. `SomeEventTypeConsumer` (consumes `String`) is available with a `SomeEventTypeDeserializer`
9. "Which topic for Foo?" - Any choice (choosing "Custom topic name" will add another question "What is the topic name for Foo?")
10. Loop on each entity with step 9
11. Overwrite all files in conflict
12. `SomeEventTypeConsumer` (consumes `String`) is available with a `SomeEventTypeDeserializer`

#### Incremental Mode

Expand All @@ -95,11 +103,12 @@ After following the first 3 steps of the [basic usage](README.md#basic-usage) ab
4. "For which entity (class name)?" - No entity (will be typed String)
5. "How would you prefix your objects (no entity, for instance: [SomeEventType]Consumer|Producer...)?" - someEventType
6. "Which components would you like to generate?" - Consumer
7. "What is the consumer polling timeout (in ms)?" - Your answer or '10000' by default (global for all consumers)
8. "Define the auto offset reset policy?" - Your answer or 'earliest' by default (global for all consumers)
9. "Do you want to continue adding consumers or producers?" - Your answer or 'N' par default
10. Overwrite all files in conflict
11. `SomeEventTypeConsumer` (consumes `String`) is available with a `SomeEventTypeDeserializer`
7. "For which topic?" - Any choice (choosing "Custom topic name" will add another question "What is the topic name?")
8. "What is the consumer polling timeout (in ms)?" - Your answer or '10000' by default (global for all consumers)
9. "Define the auto offset reset policy?" - Your answer or 'earliest' by default (global for all consumers)
10. "Do you want to continue adding consumers or producers?" - Your answer or 'N' par default
11. Overwrite all files in conflict
12. `SomeEventTypeConsumer` (consumes `String`) is available with a `SomeEventTypeDeserializer`

### Create a producer NOT linked to an entity

Expand All @@ -113,8 +122,10 @@ After following the first 3 steps of the [basic usage](README.md#basic-usage) ab
4. "Which components would you like to generate?" - Producer
5. "For which entity (class name)?" - No entity (will be typed String)
6. "How would you prefix your objects (no entity, for instance: [SomeEventType]Consumer|Producer...)?" - someEventType
7. Overwrite all files in conflict
8. `SomeEventTypeProducer` (produces `String`) is available with a `SomeEventTypeSerializer` and a `SomeEventTypeKafkaResource` to [help testing](README.md#test-consumers-and-producers)
7. "Which topic for Foo?" - Any choice (choosing "Custom topic name" will add another question "What is the topic name for Foo?")
8. Loop on each entity with step 7
9. Overwrite all files in conflict
10. `SomeEventTypeProducer` (produces `String`) is available with a `SomeEventTypeSerializer` and a `SomeEventTypeKafkaResource` to [help testing](README.md#test-consumers-and-producers)

#### Incremental Mode

Expand All @@ -124,6 +135,7 @@ After following the first 3 steps of the [basic usage](README.md#basic-usage) ab
4. "For which entity (class name)?" - No entity (will be typed String)
5. "How would you prefix your objects (no entity, for instance: [SomeEventType]Consumer|Producer...)?" - someEventType
6. "Which components would you like to generate?" - Producer
7. "Do you want to continue adding consumers or producers?" - Your answer or 'N' par default
8. Overwrite all files in conflict
9. `SomeEventTypeProducer` (produces `String`) is available with a `SomeEventTypeSerializer` and a `SomeEventTypeKafkaResource` to [help testing](README.md#test-consumers-and-producers)
7. "For which topic?" - Any choice (choosing "Custom topic name" will add another question "What is the topic name?")
8. "Do you want to continue adding consumers or producers?" - Your answer or 'N' par default
9. Overwrite all files in conflict
10. `SomeEventTypeProducer` (produces `String`) is available with a `SomeEventTypeSerializer` and a `SomeEventTypeKafkaResource` to [help testing](README.md#test-consumers-and-producers)
Loading

0 comments on commit 4195115

Please sign in to comment.