Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: itests: Don't hang on exit in MineBlocksMustPost #8345

Merged
merged 1 commit into from
Mar 18, 2022

Conversation

magik6k
Copy link
Contributor

@magik6k magik6k commented Mar 18, 2022

Proposed Changes

MineBlocksMustPost could hang during test cleanup, causing tests to randomly time out. The issue seems to be that we're waiting for a block to be mined, but the block mining logic has already shut down (this is quite racy, but happens a lot more it tests with very low block delays).

Additional Info

Happened in https://app.circleci.com/pipelines/github/filecoin-project/lotus/20105/workflows/1c498222-3542-4b64-bb14-5667927379e6/jobs/409257

Relevant goroutines:

goroutine 85 [semacquire, 28 minutes]:
sync.runtime_Semacquire(0xc0038ad250)
	/usr/local/go/src/runtime/sema.go:56 +0x45
sync.(*WaitGroup).Wait(0xc0038ad248)
	/usr/local/go/src/sync/waitgroup.go:130 +0x65
github.com/filecoin-project/lotus/itests/kit.(*BlockMiner).Stop(0xc0038ad230)
	/home/circleci/project/itests/kit/blockminer.go:324 +0x9a
testing.(*common).Cleanup.func1()
	/usr/local/go/src/testing/testing.go:901 +0xef
testing.(*common).runCleanup(0xc000592a80, 0x0, 0x0, 0x0)
	/usr/local/go/src/testing/testing.go:1007 +0x8c
testing.tRunner.func2(0xc000592a80)
	/usr/local/go/src/testing/testing.go:1187 +0x48
testing.tRunner(0xc000592a80, 0x4ae9e68)
	/usr/local/go/src/testing/testing.go:1197 +0x10e
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:1238 +0x2b3



goroutine 894 [chan receive, 28 minutes]:
github.com/filecoin-project/lotus/itests/kit.(*BlockMiner).MineBlocksMustPost.func1(0xc0038ad230, 0x4d80178, 0xc001c42140, 0xf4240)
	/home/circleci/project/itests/kit/blockminer.go:195 +0x3c5
created by github.com/filecoin-project/lotus/itests/kit.(*BlockMiner).MineBlocksMustPost
	/home/circleci/project/itests/kit/blockminer.go:153 +0xd3

Checklist

Before you mark the PR ready for review, please make sure that:

  • All commits have a clear commit message.
  • The PR title is in the form of of <PR type>: <area>: <change being made>
    • example: fix: mempool: Introduce a cache for valid signatures
    • PR type: fix, feat, INTERFACE BREAKING CHANGE, CONSENSUS BREAKING, build, chore, ci, docs,perf, refactor, revert, style, test
    • area: api, chain, state, vm, data transfer, market, mempool, message, block production, multisig, networking, paychan, proving, sealing, wallet, deps
  • This PR has tests for new functionality or change in behaviour
  • If new user-facing features are introduced, clear usage guidelines and / or documentation updates should be included in https://lotus.filecoin.io or Discussion Tutorials.
  • CI is green

@magik6k magik6k requested a review from a team as a code owner March 18, 2022 09:46
@magik6k magik6k requested a review from ZenGround0 March 18, 2022 09:46
@magik6k magik6k added the impact/test-flakiness Impact: Test Flakiness label Mar 18, 2022
@codecov
Copy link

codecov bot commented Mar 18, 2022

Codecov Report

Merging #8345 (42b2add) into master (18987f9) will decrease coverage by 0.01%.
The diff coverage is 50.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #8345      +/-   ##
==========================================
- Coverage   40.21%   40.19%   -0.02%     
==========================================
  Files         682      682              
  Lines       74425    74428       +3     
==========================================
- Hits        29929    29920       -9     
- Misses      39256    39270      +14     
+ Partials     5240     5238       -2     
Impacted Files Coverage Δ
itests/kit/blockminer.go 69.56% <50.00%> (-0.61%) ⬇️
chain/actors/builtin/miner/diff.go 48.52% <0.00%> (-10.30%) ⬇️
markets/retrievaladapter/client_blockstore.go 62.50% <0.00%> (-6.25%) ⬇️
storage/wdpost_sched.go 75.49% <0.00%> (-3.93%) ⬇️
extern/storage-sealing/currentdealinfo.go 71.17% <0.00%> (-3.61%) ⬇️
chain/events/events_called.go 83.90% <0.00%> (-2.44%) ⬇️
chain/vm/vm.go 61.63% <0.00%> (-1.07%) ⬇️
extern/sector-storage/sched.go 84.77% <0.00%> (-0.83%) ⬇️
extern/storage-sealing/fsm.go 60.73% <0.00%> (-0.68%) ⬇️
extern/sector-storage/sched_worker.go 78.48% <0.00%> (-0.59%) ⬇️
... and 11 more

@magik6k magik6k merged commit a94e47c into master Mar 18, 2022
@magik6k magik6k deleted the fix/itest-minepost-hang branch March 18, 2022 10:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
impact/test-flakiness Impact: Test Flakiness
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants