For sink connectors, I believe you can scale up the tasks to match the
partitions on the topic. But I don't believe this is the case for source
connectors; the number of partitions on the topic you're producing to has
nothing to do with the number of connector tasks. It really depends on the
individual source connector and if the source data-type could benefit from
multiple tasks. For example, the JDBC source connector (a very popular
connector) only supports 1 task - even if you're querying multiple tables.
Bottom line: you'll need to check the documentation for the connector in
question to see if it supports multiple tasks.
Alex
On Thu, May 30, 2024 at 7:51 AM Sébastien Rebecchi < srebecchi@kameleoon.com >
wrote:
> Hello
>
> Confirmed. Partition is the minimal granularity level, so having more
> consumers than the number of partitions of a topic for a same consumer
> group is useless, having P partitions means maximu...