Skip to main content

Re: MirrorMaker 2.0 XDCR / KIP-382

Apologies, copy/paste issue. Config should look like:

In DC1:

DC1->DC2.enabled = true
DC2->DC1.enabled = false

In DC2:

DC1->DC2.enabled = false
DC2->DC1.enabled = true

Running 1 mm2 node in DC1 / DC2 each. If I start up the DC1 node first,
then DC1 data is replicated to DC2. DC2 data does not replicate.
Inverting the start order inverts the cluster that gets replicated.
Running the DC2 config locally and debugging it, it seems that that the
Source connector task is not started. I'm wondering if somehow the two DCs
are conflicting about what should be running since they share the same
group names/ connect name, etc. I tried overriding the groud.id and name
of the connectors which resulted in no replication. Not quite sure what
could be going wrong.



On Thu, May 30, 2019 at 11:26 AM Ryanne Dolan <ryannedolan@gmail.com> wrote:

> Hey Jeremy, it looks like you've got a typo or copy-paste artifact in the
> configuration there -- you've got DC1->DC2 listed twice, but not the
> reverse. That would result in the behavior you are seeing, as DC1 actually
> has nothing enabled. Assuming this is just a mistake in the email, your
> approach is otherwise correct.
>
> Ryanne
>
>
> On Thu, May 30, 2019 at 7:56 AM Jeremy Ford <jeremy.l.ford@gmail.com>
> wrote:
>
> > I am attempting to setup a simple cross data center replication POC using
> > the new mirror maker branch. The behavior is not quite what I was
> > expecting, so it may be that I have made some assumptions in terms of
> > deployment that are incorrect or my setup is incorrect (see below).
> When I
> > run the two MMs, it seems like replication will work for one DC but not
> for
> > the other. If I run MM2 on just one node and enable both pairs, then
> > replication works as expected. However, that deployment does not match
> the
> > described setup in the KIP-382 documentation.
> >
> > Should I be using the MM driver to deploy in both DCs? Or do I need to
> > use a connect cluster instead? Is my configuration (included below)
> > possibly incorrect?
> >
> > Thanks,
> >
> > Jeremy Ford
> >
> >
> >
> >
> > Setup:
> >
> > I have two data centers. I have MM2 deployed in both DCs on a single
> > node. I am using the MIrrorMaker driver for the deployment. The
> > configuration for both DCs is exactly the same, except the enabled flag.
> >
> >
> > Config File:
> >
> > clusters: DC1,DC2
> > DC1.boostrap.servers = kafka.dc1
> > DC2.boostrap.servers = kafka.dc2
> >
> > DC1->DC2.topics = test
> > DC2->DC1.topics = test
> >
> >
> > In DC1:
> >
> > DC1->DC2.enabled=true
> > DC1->DC2.enabled=false
> >
> > In DC2:
> >
> > DC1->DC2.enabled=false
> > DC1->DC2.enabled=true
>

Comments