-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
release-24.2: colexec: fix type schema corruption in an edge case #133761
Open
blathers-crl
wants to merge
1
commit into
release-24.2
Choose a base branch
from
blathers/backport-release-24.2-133624
base: release-24.2
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+55
−0
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit fixes type schema corruption in the vectorized engine in an edge case. In particular, consider the following circumstances: - during the physical planning, when creating a new stage of processors, we often reuse the same type slice (stored in `InputSyncSpec.ColumnTypes`) that we get from the previous stage. In other words, we might have memory aliasing, but only on the gateway node because the remote nodes get their specs deserialized and each has its own memory allocation. - throughout the vectorized operator planning, as of 85fd4fb, for each newly projected vector we append the corresponding type to the type slice we have in scope. We also capture intermediate state of the type slice by some operators (e.g. `BatchSchemaSubsetEnforcer`). - as expected, when appending a type to the slice, if there is enough capacity, we reuse it, meaning that we often append to the slice that came to us via `InputSyncSpec.ColumnTypes`. - now, if we have two stages of processors that happened to share the same underlying type slice with some free capacity AND we needed to append vectors for each stage, then we might corrupt the type schema captured by an operator for the earlier stage when performing vectorized planning for the later stage. The bug is effectively the same as the comment deleted by 85fd4fb outlined: ``` // As an example, consider the following scenario in the context of // planFilterExpr method: // 1. r.ColumnTypes={types.Bool} with len=1 and cap=4 // 2. planSelectionOperators adds another types.Int column, so // filterColumnTypes={types.Bool, types.Int} with len=2 and cap=4 // Crucially, it uses exact same underlying array as r.ColumnTypes // uses. // 3. we project out second column, so r.ColumnTypes={types.Bool} // 4. later, we add another types.Float column, so // r.ColumnTypes={types.Bool, types.Float}, but there is enough // capacity in the array, so we simply overwrite the second slot // with the new type which corrupts filterColumnTypes to become // {types.Bool, types.Float}, and we can get into a runtime type // mismatch situation. ``` The only differences are: - aliasing of the type slice occurs via the `InputSyncSpec.ColumnTypes` that is often used as the starting points for populating `NewColOperatorResult.ColumnTypes` which is used throughout the vectorized operator planning - columns are "projected out" by sharing the type schema between two stages of DistSQL processors. This commit addresses this issue by capping the slice to its length right before we get into the vectorized planning. This will make it so that if we need to append a type, then we'll make a fresh allocation, and any possible memory aliasing with a different stage of processors will be gone. I haven't quite figured out the exact conditions that are needed for this bug to occur, but my intuition says that it should be quite rare in practice (otherwise we'd have seen this much sooner given that the offending commit was merged more than a year ago and was backported to older branches). Release note (bug fix): Previously, CockroachDB could encounter an internal error of the form `interface conversion: coldata.Column is` in an edge case and this is now fixed. The bug is present in versions 22.2.13+, 23.1.9+, 23.2+.
blathers-crl
bot
force-pushed
the
blathers/backport-release-24.2-133624
branch
from
October 30, 2024 00:26
bf0e606
to
1ed8233
Compare
blathers-crl
bot
added
blathers-backport
This is a backport that Blathers created automatically.
O-robot
Originated from a bot.
labels
Oct 30, 2024
Thanks for opening a backport. Please check the backport criteria before merging:
If your backport adds new functionality, please ensure that the following additional criteria are satisfied:
Also, please add a brief release justification to the body of your PR to justify this |
blathers-crl
bot
added
the
backport
Label PR's that are backports to older release branches
label
Oct 30, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
backport
Label PR's that are backports to older release branches
blathers-backport
This is a backport that Blathers created automatically.
O-robot
Originated from a bot.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport 1/1 commits from #133624 on behalf of @yuzefovich.
/cc @cockroachdb/release
This commit fixes type schema corruption in the vectorized engine in an edge case. In particular, consider the following circumstances:
InputSyncSpec.ColumnTypes
) that we get from the previous stage. In other words, we might have memory aliasing, but only on the gateway node because the remote nodes get their specs deserialized and each has its own memory allocation.BatchSchemaSubsetEnforcer
).InputSyncSpec.ColumnTypes
.The bug is effectively the same as the comment deleted by 85fd4fb outlined:
The only differences are:
InputSyncSpec.ColumnTypes
that is often used as the starting points for populatingNewColOperatorResult.ColumnTypes
which is used throughout the vectorized operator planningThis commit addresses this issue by capping the slice to its length right before we get into the vectorized planning. This will make it so that if we need to append a type, then we'll make a fresh allocation, and any possible memory aliasing with a different stage of processors will be gone.
I haven't quite figured out the exact conditions that are needed for this bug to occur, but my intuition says that it should be quite rare in practice (otherwise we'd have seen this much sooner given that the offending commit was merged more than a year ago and was backported to older branches).
Fixes: #130402.
Release note (bug fix): Previously, CockroachDB could encounter an internal error of the form
interface conversion: coldata.Column is
in an edge case and this is now fixed. The bug is present in versions 22.2.13+, 23.1.9+, 23.2+.Release justification: bug fix.