Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cloudant.since defaults to 0 which means the connector always starts from the beginning on restarts #124

Closed
tomblench opened this issue Oct 26, 2022 · 0 comments · Fixed by #125
Closed

cloudant.since defaults to 0 which means the connector always starts from the beginning on restarts #124

tomblench opened this issue Oct 26, 2022 · 0 comments · Fixed by #125
Milestone
0.200.0

Comments

@tomblench
Copy link
Contributor

tomblench commented Oct 26, 2022 •
edited
Loading

Bug Description

1. Steps to reproduce and the simplest code sample possible to demonstrate the issue

  • Create document in cloudant
  • Start source connector
  • Shut down connector
  • Start source connector again

2. What you expected to happen

One event created on sink topic

3. What actually happened

Two events created on sink topic

Environment details

main latest
tomblench added a commit that referenced this issue Oct 26, 2022
@tomblench
fix: don't default cloudant.since to 0
88d66fa 
This was preventing the connector from resuming properly on restarts.

If the last sequence number is not present in offset storage, _then_ 0
will be used.

Tested manually be restarting and observing log message and lack of
event duplication.

fixes #124
@tomblench tomblench mentioned this issue Oct 26, 2022
Merged
@ricellis ricellis added this to the 0.200.0 milestone Oct 27, 2022
tomblench added a commit that referenced this issue Oct 27, 2022
@tomblench @ricellis
fix: change cloudant.since logic to prefer saved offset, if present (…
223b60c 
…#125)

* fix: don't default `cloudant.since` to 0

This was preventing the connector from resuming properly on restarts.

If the last sequence number is not present in offset storage, _then_ 0
will be used.

Tested manually be restarting and observing log message and lack of
event duplication.

fixes #124

* doc: PR feedback: update documentation message

* fix: change `cloudant.since` logic and documentation

- Prefer value from offset storage if present
- Clarify docs

* fix: broken test

* Update src/main/resources/message_en_US.properties

Co-authored-by: Rich Ellis <ricellis@users.noreply.github.com>

* test: add new test case for revised `cloudant.since` logic

also remove redundant constant `DEFAULT_CLOUDANT_LAST_SEQ`

* Apply suggestions from code review

Co-authored-by: Rich Ellis <ricellis@users.noreply.github.com>

Co-authored-by: Thomas Blench <tomblench@users.noreply.github.com>
Co-authored-by: Rich Ellis <ricellis@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
0.200.0
Development

Successfully merging a pull request may close this issue.

fix: change cloudant.since logic to prefer saved offset, if present IBM/cloudant-kafka-connector
2 participants
@tomblench @ricellis