[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

datastore: data race in transaction retryer #10387

Open
maros7 opened this issue Jun 17, 2024 · 0 comments
Open

datastore: data race in transaction retryer #10387

maros7 opened this issue Jun 17, 2024 · 0 comments
Assignees
@bhshkh
Labels
api: datastore Issues related to the Datastore API. triage me I really want to be triaged.

Comments

@maros7
Copy link
maros7 commented Jun 17, 2024 •
edited
Loading

Client

Datastore

Environment

Any

Go Environment

go version go1.22.4 darwin/arm64

Code

WARNING: DATA RACE
Read at 0x000102683318 by goroutine 383:
  github.com/googleapis/gax-go/v2.(*Backoff).Pause()
      /Users/myUser/go/pkg/mod/github.com/googleapis/gax-go/v2@v2.12.4/call_option.go:186 +0x60
  github.com/googleapis/gax-go/v2.(*boRetryer).Retry()
      /Users/myUser/go/pkg/mod/github.com/googleapis/gax-go/v2@v2.12.4/call_option.go:118 +0x108
  cloud.google.com/go/datastore.backoffBeforeRetry()
      /Users/myUser/go/pkg/mod/cloud.google.com/go/datastore@v1.17.1/transaction.go:290 +0x50
  cloud.google.com/go/datastore.(*Client).RunInTransaction()
      /Users/myUser/go/pkg/mod/cloud.google.com/go/datastore@v1.17.1/transaction.go:417 +0x454
  github.com/myOrg/sdk-wrapper/datastore.(*Client).RunInTransaction.func2()
      /Users/myUser/git/sdk-wrapper/datastore/client.go:394 +0x298
  github.com/myOrg/sdk-wrapper/internal/retry.Run.func1()
      /Users/myUser/go/pkg/mod/github.com/myOrg/sdk-wrapper@v0.24.2/internal/retry/retry.go:39 +0x68
  github.com/cenkalti/backoff/v4.RetryNotifyWithTimer.Operation.withEmptyData.func1()
      /Users/myUser/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.3.0/retry.go:18 +0x30
  github.com/cenkalti/backoff/v4.doRetryNotify[go.shape.struct {}]()
      /Users/myUser/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.3.0/retry.go:88 +0x15c
  github.com/cenkalti/backoff/v4.RetryNotifyWithTimer()
      /Users/myUser/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.3.0/retry.go:61 +0x80
  github.com/cenkalti/backoff/v4.RetryNotify()
      /Users/myUser/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.3.0/retry.go:49 +0x290
  github.com/cenkalti/backoff/v4.Retry()
      /Users/myUser/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.3.0/retry.go:38 +0x274
  github.com/myOrg/sdk-wrapper/internal/retry.Run()
      /Users/myUser/go/pkg/mod/github.com/myOrg/sdk-wrapper@v0.24.2/internal/retry/retry.go:71 +0x21c
  github.com/myOrg/sdk-wrapper/datastore.(*Client).RunInTransaction()
      /Users/myUser/git/sdk-wrapper/datastore/client.go:412 +0x1c8
  github.com/myOrg/sdk-wrapper/datastore.(*Client).put()
      /Users/myUser/git/sdk-wrapper/datastore/client.go:301 +0x540
  github.com/myOrg/sdk-wrapper/datastore.(*Client).PutMulti()
      /Users/myUser/git/sdk-wrapper/datastore/client.go:256 +0x15c
  github.com/myOrg/sdk-wrapper/datastore_test.IntegrationTestSuite.TestClient_PutMulti_concurrent_transactions.func1.1()
      /Users/myUser/git/sdk-wrapper/datastore/client_test.go:694 +0x120
  testing.tRunner()
      /Users/myUser/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.4.darwin-arm64/src/testing/testing.go:1689 +0x180
  testing.(*T).Run.gowrap1()
      /Users/myUser/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.4.darwin-arm64/src/testing/testing.go:1742 +0x40

Previous write at 0x000102683318 by goroutine 374:
  github.com/googleapis/gax-go/v2.(*Backoff).Pause()
      /Users/myUser/go/pkg/mod/github.com/googleapis/gax-go/v2@v2.12.4/call_option.go:200 +0x15c
  github.com/googleapis/gax-go/v2.(*boRetryer).Retry()
      /Users/myUser/go/pkg/mod/github.com/googleapis/gax-go/v2@v2.12.4/call_option.go:118 +0x108
  cloud.google.com/go/datastore.backoffBeforeRetry()
      /Users/myUser/go/pkg/mod/cloud.google.com/go/datastore@v1.17.1/transaction.go:290 +0x50
  cloud.google.com/go/datastore.(*Client).RunInTransaction()
      /Users/myUser/go/pkg/mod/cloud.google.com/go/datastore@v1.17.1/transaction.go:417 +0x454
  github.com/myOrg/sdk-wrapper/datastore.(*Client).RunInTransaction.func2()
      /Users/myUser/git/sdk-wrapper/datastore/client.go:394 +0x298
  github.com/myOrg/sdk-wrapper/internal/retry.Run.func1()
      /Users/myUser/go/pkg/mod/github.com/myOrg/sdk-wrapper@v0.24.2/internal/retry/retry.go:39 +0x68
  github.com/cenkalti/backoff/v4.RetryNotifyWithTimer.Operation.withEmptyData.func1()
      /Users/myUser/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.3.0/retry.go:18 +0x30
  github.com/cenkalti/backoff/v4.doRetryNotify[go.shape.struct {}]()
      /Users/myUser/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.3.0/retry.go:88 +0x15c
  github.com/cenkalti/backoff/v4.RetryNotifyWithTimer()
      /Users/myUser/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.3.0/retry.go:61 +0x80
  github.com/cenkalti/backoff/v4.RetryNotify()
      /Users/myUser/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.3.0/retry.go:49 +0x290
  github.com/cenkalti/backoff/v4.Retry()
      /Users/myUser/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.3.0/retry.go:38 +0x274
  github.com/myOrg/sdk-wrapper/internal/retry.Run()
      /Users/myUser/go/pkg/mod/github.com/myOrg/sdk-wrapper@v0.24.2/internal/retry/retry.go:71 +0x21c
  github.com/myOrg/sdk-wrapper/datastore.(*Client).RunInTransaction()
      /Users/myUser/git/sdk-wrapper/datastore/client.go:412 +0x1c8
  github.com/myOrg/sdk-wrapper/datastore.(*Client).put()
      /Users/myUser/git/sdk-wrapper/datastore/client.go:301 +0x540
  github.com/myOrg/sdk-wrapper/datastore.(*Client).PutMulti()
      /Users/myUser/git/sdk-wrapper/datastore/client.go:256 +0x15c
  github.com/myOrg/sdk-wrapper/datastore_test.IntegrationTestSuite.TestClient_PutMulti_concurrent_transactions.func1.1()
      /Users/myUser/git/sdk-wrapper/datastore/client_test.go:694 +0x120
  testing.tRunner()
      /Users/myUser/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.4.darwin-arm64/src/testing/testing.go:1689 +0x180
  testing.(*T).Run.gowrap1()
      /Users/myUser/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.4.darwin-arm64/src/testing/testing.go:1742 +0x40

Expected behavior

No data race like before #10349. For example, if I downgrade to v1.17.0, then go test -race doesn't report a data race.

Actual behavior

Data race as seen above. Due to global variable being accessed concurrently:

google-cloud-go/datastore/transaction.go

Line 52 in 0c17df2

txnRetryer = gax.OnCodes([]codes.Code{codes.Aborted, codes.Canceled, codes.Unknown, codes.DeadlineExceeded,

Screenshots

Additional context

Started after #10349.
@maros7 maros7 added the triage me I really want to be triaged. label Jun 17, 2024
@product-auto-label product-auto-label bot added the api: datastore Issues related to the Datastore API. label Jun 17, 2024
@maros7 maros7 changed the title datastore: data race in transaction datastore: data race in transaction retryer Jun 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bhshkh bhshkh

Labels
api: datastore Issues related to the Datastore API. triage me I really want to be triaged.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bhshkh @maros7