Re: Stream msg using R

Yes

That piece of code is not working for me, because when I run it I am not
getting it.

I am trying to connect through Data Bricks, so I think I have to change the
"local" connection in master?

Also, how do I add in the read options the kafka username and psswd?

Notes: Kafka is hosted in confluent

Best Regards
Paulo

On Wed, Jul 28, 2021 at 8:52 PM Blake Miller <blak3mill3r@gmail.com> wrote:

> Paulo,
>
> Could you clarify what your question is? I wasn't able to understand it
> from your email.
>
> I understand that you want to read data from Kafka in R, and you shared
> some R code. What is the issue that you're having with this code?
>
> Regards,
> Blake
>
> On Tue, Jul 27, 2021 at 2:42 PM Paulo Ramos
> <paulo.ramos@3pillarglobal.com.invalid> wrote:
>
> > Hi All,
> >
> > I have a technical question, regarding reading stream msg from kafka
> using
> > R.
> >
> > This is what I am currently using:
> >
> > if (FALSE) { library <https://rdrr.io/r/base/library.html>(sparklyr
> > <https://spark.rstudio.com/>) sc <- spark_connect
> > <https://spark.rstudio.com/reference/spark-connections.html>(master =
> > "local", version = "2.3", packages = "kafka") read_options <- list
> > <https://rdrr.io/r/base/list.html>(kafka.bootstrap.servers =
> > "localhost:9092", subscribe = "topic1") write_options <- list
> > <https://rdrr.io/r/base/list.html>(kafka.bootstrap.servers =
> > "localhost:9092", topic = "topic2") stream <- stream_read_kafka(sc,
> options
> > = read_options) %>% stream_write_kafka
> > <https://spark.rstudio.com/reference/stream_write_kafka.html>(options =
> > write_options) stream_stop
> > <https://spark.rstudio.com/reference/stream_stop.html>(stream) }
> >
> > But I need to add
> > sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
> > required username="" password="" to connect to my kafka topic. Also what
> do
> > I have to add in master = "local" if I am using databrick. This is for
> > consumer part
> >
> > Any help regarding this issue will be a great help
> >
> > Best Regards
> >
> > --
> > Paulo Ramos
> > 3Pillar Global
> > Office:
> > North America | Asia Pacific | Europe | Latin America
> > www.3PillarGlobal.com
> >
> > <
> >
> https://pages.3pillarglobal.com/pm-ebook.html?utm_source=Email&utm_campaign=email-signature&utm_medium=Banner&utm_content=Ebook
> > >
> >
>


--
Paulo Ramos
3Pillar Global
Office:
North America | Asia Pacific | Europe | Latin America
www.3PillarGlobal.com

<https://pages.3pillarglobal.com/pm-ebook.html?utm_source=Email&utm_campaign=email-signature&utm_medium=Banner&utm_content=Ebook>