Skip to main content

Re: Is SinkContext thread safe?

Ben,

My understanding is that you don't need to worry about any thread
synchronization. Each task has their own instance of the
`SinkTaskContext` and given Kafka Connect's behavior of spreading the
tasks over the cluster -- by definition it won't be the same instance.
This means that even if you try to lock onto any object -- it won't take
effect because each task is running in their own JVM/ClassLoader.
Moreover, each SinkTaskContext instance has a internal reference to a
Kafka consumer and any requests to its `pause()` method will delegate
the execution to the `pause()` method of this Kafka consumer. Finally
and as you may know, Kafka make sure that a given partition is read by
only one consumer group at a time.

Thanks,

-- Ricardo

On 6/23/20 4:38 PM, Ben Zhang wrote:
> Hi, I had read the doc and tried to google it but could find relavent questions/answers. I am curious that is SinkContext thread safe? Say I want to use the SinkContext to pause partitions in different threads, do I need to lock on the object?

Comments