Re: Kafka Connect

I don't think KStreams is a good option just by itself for inter site replication.
How about using a replication technology like MM2 to first replicate to a topic in the destination cluster and then run KStreams client there in the destination cluster to consume, transform and then produce to your final topic.

Regards,
Neeraj


> On 15 Nov 2024, at 1:04 PM, Surbhi Mungre <mungre.surbhi@gmail.com> wrote:
>
> Can Kafka Connect be used to read messages from one Kafka Cluster, apply
> some basic transformation and write messages to another Kafka Cluster? I
> did not find a Kafka Connect Connector in the list of connectors provided
> by Confluence[1]. I only found a Replicator[2] but for my use-case I want
> to apply some transformation on the messages.
>
> Instead of using Kafka Connect, does it make more sense to use Kafka
> Streams or Spark Streaming. I want to perform very simple transformations.
>
> [1] https://www.confluent.io/product/connectors/
> [2]
> https://docs.confluent.io/platform/current/multi-dc-deployments/replicator/
>
> Thanks,
> -Surbhi