Hello!
I ask for your help in connection with the my recent task:
- Price lists are delivered to 20,000 points of sale with a frequency of <10
price lists per day.
- The order in which the price lists follow is important. It is also
important that the price lists are delivered to the point of sale online.
- At each point of sale, an agent application is deployed, which processes
the received price lists.
This task is not particularly difficult. Help in solving the task is not
required.
The difficulty is that Kafka in our company is a new "silver bullet", and
the project manager requires me to implement the following technical
decision:
deploy 20,000 Kafka consumer instances (one instance for each point of sale)
for one topic partitioned into 20,000 partitions - one partition per
consumer.
Technical problems obtained in experiments with this technical decision do
not convince him.
Please give me references to the books/documents/blogposts. which clearly
shows that Kafka not intended for this way to use (references to other
anti-patterns/pitfalls will be useful).
My own attempts to find such references were unsuccessful.
Thank you!
I ask for your help in connection with the my recent task:
- Price lists are delivered to 20,000 points of sale with a frequency of <10
price lists per day.
- The order in which the price lists follow is important. It is also
important that the price lists are delivered to the point of sale online.
- At each point of sale, an agent application is deployed, which processes
the received price lists.
This task is not particularly difficult. Help in solving the task is not
required.
The difficulty is that Kafka in our company is a new "silver bullet", and
the project manager requires me to implement the following technical
decision:
deploy 20,000 Kafka consumer instances (one instance for each point of sale)
for one topic partitioned into 20,000 partitions - one partition per
consumer.
Technical problems obtained in experiments with this technical decision do
not convince him.
Please give me references to the books/documents/blogposts. which clearly
shows that Kafka not intended for this way to use (references to other
anti-patterns/pitfalls will be useful).
My own attempts to find such references were unsuccessful.
Thank you!
Comments
Post a Comment