Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky test: TestInterceptors/StreamClientSpans #1352

Closed
codeboten opened this issue Oct 12, 2021 · 5 comments · Fixed by #4535 or #4537
Closed

Flaky test: TestInterceptors/StreamClientSpans #1352

codeboten opened this issue Oct 12, 2021 · 5 comments · Fixed by #4535 or #4537
Assignees
@pellared
Labels
area: instrumentation Related to an instrumentation package bug Something isn't working good first issue Good for newcomers help wanted Extra attention is needed instrumentation: otelgrpc
Milestone
untracked

Comments

@codeboten
Copy link
Contributor

As per the output available here, a flaky test was causing CI failures with the following message:

=== RUN   TestInterceptors/StreamClientSpans
    grpc_test.go:181: 
        	Error Trace:	grpc_test.go:181
        	            				grpc_test.go:106
        	Error:      	"[%!s(*trace.snapshot=&{grpc.testing.TestService/StreamingInputCall {[155 73 36 223 179 3 93 25 
22 238 66 50 110 75 56 179] [234 158 25 193 181 253 113 135] 1 {[]} false} {[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 0 0 0 0 
0] 0 {[]} false} 3 {13858018644475018353 7069793 0x8860e20} {13858018644475405458 7456898 0x8860e20} 
[{rpc.system {4 0 grpc <nil>}} {rpc.service {4 0 grpc.testing.TestService <nil>}} {rpc.method {4 0 StreamingInputCall 
<nil>}} {rpc.grpc.status_code {2 0  <nil>}}] [{message [{message.type {4 0 SENT <nil>}} {message.id {2 1  <nil>}} 
{message.uncompressed_size {2 27190  <nil>}}] 0 {13858018644475097554 7148994 0x8860e20}} {message 
[{message.type {4 0 SENT <nil>}} {message.id {2 2  <nil>}} {message.uncompressed_size {2 12  <nil>}}] 0 
{13858018644475100754 7152194 0x8860e20}} {message [{message.type {4 0 SENT <nil>}} {message.id {2 3  <nil>}} 
{message.uncompressed_size {2 1834  <nil>}}] 0 {13858018644475105254 7156794 0x8860e20}} {message 
[{message.type {4 0 SENT <nil>}} {message.id {2 4  <nil>}} {message.uncompressed_size {2 45912  <nil>}}] 0 
{13858018644475116554 7168094 0x8860e20}}] [] {0 } 0 0 0 0 0xa80c9d0 {go.opentelemetry.io/contrib/instrumentation
/google.golang.org/grpc/otelgrpc semver:0.25.0 }}) %!s(*trace.snapshot=&{grpc.testing.TestService/StreamingOutputCall 
{[243 106 181 94 132 178 122 89 48 9 144 23 98 136 210 18] [69 26 172 62 99 229 191 196] 1 {[]} false} {[0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0] [0 0 0 0 0 0 0 0] 0 {[]} false} 3 {13858018644475358957 7410497 0x8860e20} {13858018644475716362 
7767902 0x8860e20} [{rpc.system {4 0 grpc <nil>}} {rpc.service {4 0 grpc.testing.TestService <nil>}} {rpc.method {4 0 
StreamingOutputCall <nil>}} {rpc.grpc.status_code {2 0  <nil>}}] [{message [{message.type {4 0 SENT <nil>}} {message.id 
{2 1  <nil>}} {message.uncompressed_size {2 21  <nil>}}] 0 {13858018644475426558 7478098 0x8860e20}} {message 
[{message.type {4 0 RECEIVED <nil>}} {message.id {2 1  <nil>}} {message.uncompressed_size {2 31423  <nil>}}] 0 
{13858018644475607460 7659000 0x8860e20}} {message [{message.type {4 0 RECEIVED <nil>}} {message.id {2 2  
<nil>}} {message.uncompressed_size {2 13  <nil>}}] 0 {13858018644475611360 7662900 0x8860e20}} {message 
[{message.type {4 0 RECEIVED <nil>}} {message.id {2 3  <nil>}} {message.uncompressed_size {2 2659  <nil>}}] 0 
{13858018644475627360 7678900 0x8860e20}} {message [{message.type {4 0 RECEIVED <nil>}} {message.id {2 4  
<nil>}} {message.uncompressed_size {2 58987  <nil>}}] 0 {13858018644475684261 7735701 0x8860e20}}] [] {0 } 0 0 0 0 
0xa80c9d0 {go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc semver:0.25.0 }})]" should have 
3 item(s), but has 2
        	Test:       	TestInterceptors/StreamClientSpans
@codeboten codeboten added area: instrumentation Related to an instrumentation package enhancement New feature or request labels Oct 12, 2021
@codeboten codeboten mentioned this issue Oct 12, 2021
Merged
@pellared
Copy link
Member

pellared commented Jun 7, 2023 •
edited
Loading

Same https://github.com/open-telemetry/opentelemetry-go-contrib/actions/runs/5196993840/jobs/9371298999?pr=3966

--- FAIL: TestInterceptors/StreamClientSpans (0.00s)
        grpc_test.go:183: 
            	Error Trace:	/home/runner/work/opentelemetry-go-contrib/opentelemetry-go-contrib/instrumentation/google.golang.org/grpc/otelgrpc/test/grpc_test.go:183
            	            				/home/runner/work/opentelemetry-go-contrib/opentelemetry-go-contrib/instrumentation/google.golang.org/grpc/otelgrpc/test/grpc_test.go:111
            	Error:      	"[%!s(*trace.snapshot=&{grpc.testing.TestService/StreamingInputCall {[[131](https://github.com/open-telemetry/opentelemetry-go-contrib/actions/runs/5196993840/jobs/9371298999?pr=3966#step:6:132) 60 121 34 85 14 29 25 176 143 168 131 114 191 10 249] [22 253 136 252 112 5 112 172] 1 {[]} false} {[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 0 0 0 0 0] 0 {[]} false} 3 {13913916118296001899 7328509 0x8ba6fe0} {13913916118296347805 7674415 0x8ba6fe0} [{rpc.system {4 0 grpc <nil>}} {rpc.service {4 0 grpc.testing.TestService <nil>}} {rpc.method {4 0 StreamingInputCall <nil>}} {rpc.grpc.status_code {2 0  <nil>}}] [{message [{message.type {4 0 SENT <nil>}} {message.id {2 1  <nil>}}] 0 {13913916118296086801 7413411 0x8ba6fe0}} {message [{message.type {4 0 SENT <nil>}} {message.id {2 2  <nil>}}] 0 {13913916118296090001 7416511 0x8ba6fe0}} {message [{message.type {4 0 SENT <nil>}} {message.id {2 3  <nil>}}] 0 {13913916118296093401 7419911 0x8ba6fe0}} {message [{message.type {4 0 SENT <nil>}} {message.id {2 4  <nil>}}] 0 {13913916118296109001 7435611 0x8ba6fe0}}] [] {0 } 0 0 0 0 0xa09ca70 {go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc 0.42.0 }}) %!s(*trace.snapshot=&{grpc.testing.TestService/StreamingOutputCall {[184 204 181 228 54 12 249 52 126 169 87 144 253 94 174 189] [242 244 146 63 182 83 165 226] 1 {[]} false} {[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 0 0 0 0 0] 0 {[]} false} 3 {13913916118296316804 7643414 0x8ba6fe0} {13913916118296711410 8038020 0x8ba6fe0} [{rpc.system {4 0 grpc <nil>}} {rpc.service {4 0 grpc.testing.TestService <nil>}} {rpc.method {4 0 StreamingOutputCall <nil>}} {rpc.grpc.status_code {2 0  <nil>}}] [{message [{message.type {4 0 SENT <nil>}} {message.id {2 1  <nil>}}] 0 {13913916118296391705 7718315 0x8ba6fe0}} {message [{message.type {4 0 RECEIVED <nil>}} {message.id {2 1  <nil>}}] 0 {13913916118296576908 7903418 0x8ba6fe0}} {message [{message.type {4 0 RECEIVED <nil>}} {message.id {2 2  <nil>}}] 0 {13913916118296580108 7906618 0x8ba6fe0}} {message [{message.type {4 0 RECEIVED <nil>}} {message.id {2 3  <nil>}}] 0 {13913916118296596708 7923318 0x8ba6fe0}} {message [{message.type {4 0 RECEIVED <nil>}} {message.id {2 4  <nil>}}] 0 {13913916118296666009 7992619 0x8ba6fe0}}] [] {0 } 0 0 0 0 0xa09ca70 {go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc 0.42.0 }})]" should have 3 item(s), but has 2
            	Test:       	TestInterceptors/StreamClientSpans

PS. It is annoying.

@pellared pellared changed the title flaky test: TestInterceptors/StreamClientSpans Flaky test: TestInterceptors/StreamClientSpans Jun 7, 2023
@pellared pellared added bug Something isn't working and removed enhancement New feature or request labels Jun 7, 2023
@pellared
Copy link
Member

Again https://github.com/open-telemetry/opentelemetry-go-contrib/actions/runs/5611209814/jobs/10267227488

--- FAIL: TestInterceptors (0.01s)
    --- FAIL: TestInterceptors/StreamClientSpans (0.00s)
        grpc_test.go:196: 
            	Error Trace:	/home/runner/work/opentelemetry-go-contrib/opentelemetry-go-contrib/instrumentation/google.golang.org/grpc/otelgrpc/test/grpc_test.go:196
            	            				/home/runner/work/opentelemetry-go-contrib/opentelemetry-go-contrib/instrumentation/google.golang.org/grpc/otelgrpc/test/grpc_test.go:124
            	Error:      	"[%!s(*trace.snapshot=&{grpc.testing.TestService/StreamingInputCall {[14 3 77 14 138 85 126 251 98 191 142 115 221 [133](https://github.com/open-telemetry/opentelemetry-go-contrib/actions/runs/5611209814/jobs/10267227488#step:6:134) 165 70] [216 195 34 150 223 167 148 102] 1 {[]} false} {[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 0 0 0 0 0] 0 {[]} false} 3 {13917927642706074921 7814597 0x8bd9780} {13917927642706405229 8144905 0x8bd9780} [{rpc.system {4 0 grpc <nil>}} {rpc.service {4 0 grpc.testing.TestService <nil>}} {rpc.method {4 0 StreamingInputCall <nil>}} {rpc.grpc.status_code {2 0  <nil>}}] [{message [{message.type {4 0 SENT <nil>}} {message.id {2 1  <nil>}}] 0 {13917927642706151123 7890799 0x8bd9780}} {message [{message.type {4 0 SENT <nil>}} {message.id {2 2  <nil>}}] 0 {13917927642706154223 7893899 0x8bd9780}} {message [{message.type {4 0 SENT <nil>}} {message.id {2 3  <nil>}}] 0 {13917927642706157723 7897399 0x8bd9780}} {message [{message.type {4 0 SENT <nil>}} {message.id {2 4  <nil>}}] 0 {13917927642706174523 7914199 0x8bd9780}}] [] {0 } 0 0 0 0 0x9c9cb80 {go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc 0.42.0 }}) %!s(*trace.snapshot=&{grpc.testing.TestService/StreamingOutputCall {[60 228 200 89 201 86 77 149 212 166 87 196 104 121 20 147] [106 250 55 27 208 86 246 84] 1 {[]} false} {[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 0 0 0 0 0] 0 {[]} false} 3 {13917927642706378028 8117704 0x8bd9780} {13917927642706792839 8532515 0x8bd9780} [{rpc.system {4 0 grpc <nil>}} {rpc.service {4 0 grpc.testing.TestService <nil>}} {rpc.method {4 0 StreamingOutputCall <nil>}} {rpc.grpc.status_code {2 0  <nil>}}] [{message [{message.type {4 0 SENT <nil>}} {message.id {2 1  <nil>}}] 0 {13917927642706458730 8198406 0x8bd9780}} {message [{message.type {4 0 RECEIVED <nil>}} {message.id {2 1  <nil>}}] 0 {13917927642706634135 8373711 0x8bd9780}} {message [{message.type {4 0 RECEIVED <nil>}} {message.id {2 2  <nil>}}] 0 {13917927642706638835 8378511 0x8bd9780}} {message [{message.type {4 0 RECEIVED <nil>}} {message.id {2 3  <nil>}}] 0 {13917927642706655735 8395311 0x8bd9780}} {message [{message.type {4 0 RECEIVED <nil>}} {message.id {2 4  <nil>}}] 0 {13917927642706748838 8488514 0x8bd9780}}] [] {0 } 0 0 0 0 0x9c9cb80 {go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc 0.42.0 }})]" should have 3 item(s), but has 2
            	Test:       	TestInterceptors/StreamClientSpans

@pellared pellared mentioned this issue Jul 20, 2023
Closed
@pellared
Copy link
Member

Duplicate #3373

@pellared pellared added instrumentation: otelgrpc help wanted Extra attention is needed good first issue Good for newcomers labels Jul 20, 2023
@pellared pellared mentioned this issue Jul 27, 2023
Merged
@pellared pellared mentioned this issue Oct 12, 2023
Closed
@pellared pellared mentioned this issue Nov 6, 2023
Merged
@pellared
Copy link
Member

pellared commented Nov 6, 2023

Should be fixed by be618e6

@pellared pellared closed this as completed Nov 6, 2023
@pellared pellared self-assigned this Nov 6, 2023
@pellared
Copy link
Member

pellared commented Nov 6, 2023

Still fails

@pellared pellared reopened this Nov 6, 2023
@github-project-automation github-project-automation bot moved this to Needs triage in Go: Triage Nov 6, 2023
@pellared pellared moved this from Needs triage to Low priority in Go: Triage Nov 6, 2023
@pellared pellared mentioned this issue Nov 7, 2023
Merged
@github-project-automation github-project-automation bot moved this from Low priority to Closed in Go: Triage Nov 8, 2023
@pellared pellared mentioned this issue Nov 9, 2023
Merged
@pellared pellared added this to the untracked milestone Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pellared pellared

Labels
area: instrumentation Related to an instrumentation package bug Something isn't working good first issue Good for newcomers help wanted Extra attention is needed instrumentation: otelgrpc
Projects
Go: Triage
Archived in project
Milestone
untracked
2 participants
@codeboten @pellared