Kafka and Cassandra integration tests are very unstable

[hleb-albau]

Overview

Right now we have a lot of integration tests using embedded kafka and cassandra. The main issue with this tests, that they are very unstable.
See https://circleci.com/gh/cybercongress/workflows/cyber-search

---

Task description

Integration tests should be stable. 100% builds for right written tests.

Required skills

• Kotlin
• Kafka
• Cassandra
• Spring
• JUnit

---

Contribution rules

Here you can learn how to contribute to our project

---

Definition of done

Explain what you actually want to see as a result:

• build should be stable (100% builds for valid project state). Will be tested by executing 100 sequential builds.

---

Extra information

Bounty amount	0.5 ETH
Experience level	advanced
Project length	days
Expires in	1 mounth
Bounty type	bug

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

This issue now has a funding of 0.5 ETH (185.75 USD @ $371.49/ETH) attached to it.

• If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $23,951.26 more funded OSS Work available on the Gitcoin Issue Explorer

[KevinLiLu]

@hleb-albau

I might be interested in taking this.

Are we constructing the embedded Kafka & Cassandra instances before and after every single test class or are the instances shared for multiple modules?

I'm been browsing the error logs in the CI and it looks like embedded Kafka often has trouble shutting down and closing.

[hleb-albau]

Hi @KevinLiLu,

Just start work on gitcoin to show others, that you took this task.

Right now instances are not shared, as far as I remember, but it will be nice, if they do(at least for each module)

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work has been started.

These users each claimed they can complete the work by 2 months ago.
Please review their action plans below:

1) KevinLiLu has started work.

Will play around with Kafka/Cassandra configurations, and possible use 1 shared instance of Kafka & Cassandra per module.

Learn more on the Gitcoin Issue Details page.

[KevinLiLu]

@hleb-albau can you grant me write permissions in CircleCI so I can trigger a bunch of builds in a PR to test fail rate? The tests don't really fail on my local so it seems isolated to the CI environment.

[hleb-albau]

@KevinLiLu is it possible on CircleCi? I can't found required settings. I generated api token, sent to your email lu.kevin@berkeley.edu.

[KevinLiLu]

@hleb-albau Ah, a quick google search shows that the permissions are taken from the GitHub repo. So I would need to have write access in this repo in order to run builds...

[hleb-albau]

@KevinLiLu try now see

[KevinLiLu]

@hleb-albau got it. it works now, thanks 😄

[frankchen07]

hey @KevinLiLu - Frank from Gitcoin here, are you still working on this issue?

[KevinLiLu]

@frankchen07 hey Frank, yes this is still ongoing.

#268

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work for 0.5 ETH (108.66 USD @ $217.33/ETH) has been submitted by:

1. @KevinLiLu

@hleb-albau please take a look at the submitted work:

• PR by @KevinLiLu

---

• Learn more on the Gitcoin Issue Details page
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $60,107.19 more funded OSS Work available on the Gitcoin Issue Explorer

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

The funding of 0.5 ETH (107.31 USD @ $214.61/ETH) attached to this issue has been approved & issued to @KevinLiLu.

• Learn more on the Gitcoin Issue Details page
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $55,962.41 more funded OSS Work available on the Gitcoin Issue Explorer