Hello Sébastien,
thanks for reaching out. The fix was not back ported to 3.x branches,
because it is very unlikely that there will be another 3.x bug fix release.
Kafka community usually only does bug-fix releases for the last minor
release, and there was already 3.9.1. With 4.1.0 being in the process or
getting release, we would only do another 3.9.2 bug-fix release if there
is some highly critical issue, and user cannot upgrade to a newer 4.x
version. A 3.8.2 or older version bug-fix release is even more unlikely.
Thus, my recommendation would be, to wait for 4.0.1
(https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+4.0.1)
or 4.1.0
(https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+4.1.0)
and upgrade your KS app to either one version to get the fix.
-Matthias
On 7/21/25 10:22 AM, Sébastien wrote:
> Hi everyone,
>
> I'm upgrading my old Kafka Stream application currently in 2.6. to 3.8.1
> version.
>
> But I'm facing a bug where some KStream-GlobalKTable left join doesn't work
> anymore and instead Drop records, and that's a regression from my 2.6.1
> version.
> What I have is a Kstream With K1,V1 Avro records (Key and value) and I want
> to leftJoin a GlobalKTable that has K2,V2 Avro records. K2 can be extracted
> from V1 record (a classic Foreign Key)
>
> What I am doing is that I use a KeyValueMapper to map V1 to K2
> If I can't build K2 because necessary values in V1 are null, since I can't
> build K2 (Avro Object) with null values I return null from the mapper and
> it just do a Left Outer Join and record is not dropped (actual behavior in
> 2.6).
>
> But code introduced in KIP-962 change that, and new behavior in 3.7+
> versions is to drop records if mappedKey is null
> I'm sure that is an unwanted regression since the KIP-692 was supposed to
> relax non-null requirements ?
>
> This regression was fixed in KAFKA-19208, but only in 4.x versions
>
> So my question is : Is there any fix planned in 3.x versions ?
> I initially planned a two way upgrade, first from 2.6 to 3.8.1 (that is the
> latest version included in actual spring boot version) , and later a
> migration from 3.8.1 to 4.x, should I directly go to 4.x with that
> regression ?
>
> Thank for your help
>
> Sébastien
>
thanks for reaching out. The fix was not back ported to 3.x branches,
because it is very unlikely that there will be another 3.x bug fix release.
Kafka community usually only does bug-fix releases for the last minor
release, and there was already 3.9.1. With 4.1.0 being in the process or
getting release, we would only do another 3.9.2 bug-fix release if there
is some highly critical issue, and user cannot upgrade to a newer 4.x
version. A 3.8.2 or older version bug-fix release is even more unlikely.
Thus, my recommendation would be, to wait for 4.0.1
(https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+4.0.1)
or 4.1.0
(https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+4.1.0)
and upgrade your KS app to either one version to get the fix.
-Matthias
On 7/21/25 10:22 AM, Sébastien wrote:
> Hi everyone,
>
> I'm upgrading my old Kafka Stream application currently in 2.6. to 3.8.1
> version.
>
> But I'm facing a bug where some KStream-GlobalKTable left join doesn't work
> anymore and instead Drop records, and that's a regression from my 2.6.1
> version.
> What I have is a Kstream With K1,V1 Avro records (Key and value) and I want
> to leftJoin a GlobalKTable that has K2,V2 Avro records. K2 can be extracted
> from V1 record (a classic Foreign Key)
>
> What I am doing is that I use a KeyValueMapper to map V1 to K2
> If I can't build K2 because necessary values in V1 are null, since I can't
> build K2 (Avro Object) with null values I return null from the mapper and
> it just do a Left Outer Join and record is not dropped (actual behavior in
> 2.6).
>
> But code introduced in KIP-962 change that, and new behavior in 3.7+
> versions is to drop records if mappedKey is null
> I'm sure that is an unwanted regression since the KIP-692 was supposed to
> relax non-null requirements ?
>
> This regression was fixed in KAFKA-19208, but only in 4.x versions
>
> So my question is : Is there any fix planned in 3.x versions ?
> I initially planned a two way upgrade, first from 2.6 to 3.8.1 (that is the
> latest version included in actual spring boot version) , and later a
> migration from 3.8.1 to 4.x, should I directly go to 4.x with that
> regression ?
>
> Thank for your help
>
> Sébastien
>
Comments
Post a Comment