OK, I've made it work.
Steps to fix:
+ Build https://github.com/a0x8o/kafka <https://github.com/a0x8o/kafka>
+ Collect the "connect-file-3.5.0-SNAPSHOT.jar"
+ Update the "plugin.path" in "connect-distributed.properties" to the directory containing that jar file
+ Start Kafka Connect
+ The connector name is "FileStreamSource" instead of "FileStream-Source" (The hyphen comes from the book)
---
See https://docs.confluent.io/platform/current/connect/filestream_connector.html#kconnect-long-filestream-connectors <https://docs.confluent.io/platform/current/connect/filestream_connector.html#kconnect-long-filestream-connectors> to understand why "connect-file.jar" is not included
> On 28 Jan 2023, at 22:36, Hieu Nguyen <emerald.hieu@gmail.com> wrote:
>
> Hello,
>
> I'm reading the book "Kafka: The Definitive Guide: Real-Time Data and Stream Processing at Scale" by by Neha Narkhede, Gwen Shapira, and Todd Palino.
>
> In the chapter "Kafka Connect", section "Connector Example: File Source and File Sink", I run the below command and got the missing connector error.
>
> hieunguyenduc@hieugioi kafka_2.13-3.3.1 % echo '{"name":"load-kafka-config", "config":{"connector.class":"FileStream-Source","file":"config/server.properties","topic":"kafka-config-topic"}}' | curl -X POST -d @- http://localhost:8083/connectors <http://localhost:8083/connectors> --header "Content-Type:application/json"
> {"error_code":500,"message":"Failed to find any class that implements Connector and which name matches FileStream-Source, available connectors are: PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorCheckpointConnector, name='org.apache.kafka.connect.mirror.MirrorCheckpointConnector', version='3.3.1', encodedVersion=3.3.1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorHeartbeatConnector, name='org.apache.kafka.connect.mirror.MirrorHeartbeatConnector', version='3.3.1', encodedVersion=3.3.1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorSourceConnector, name='org.apache.kafka.connect.mirror.MirrorSourceConnector', version='3.3.1', encodedVersion=3.3.1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSinkConnector, name='org.apache.kafka.connect.tools.MockSinkConnector', version='3.3.1', encodedVersion=3.3.1, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSourceConnector, name='org.apache.kafka.connect.tools.MockSourceConnector', version='3.3.1', encodedVersion=3.3.1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.SchemaSourceConnector, name='org.apache.kafka.connect.tools.SchemaSourceConnector', version='3.3.1', encodedVersion=3.3.1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSinkConnector, name='org.apache.kafka.connect.tools.VerifiableSinkConnector', version='3.3.1', encodedVersion=3.3.1, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSourceConnector, name='org.apache.kafka.connect.tools.VerifiableSourceConnector', version='3.3.1', encodedVersion=3.3.1, type=source, typeName='source', location='classpath'}"}%
>
> How can I fix it?
>
> The part after that says "
>
> If all went well, you should see something along the lines of:
> {"schema":{"type":"string","optional":false},"payload":"# Licensed to the
> Apache Software Foundation (ASF) under one or more"}
>
> "
>
> I run the command and it shows nothing.
>
> Regards,
> Hieu
>
>
Steps to fix:
+ Build https://github.com/a0x8o/kafka <https://github.com/a0x8o/kafka>
+ Collect the "connect-file-3.5.0-SNAPSHOT.jar"
+ Update the "plugin.path" in "connect-distributed.properties" to the directory containing that jar file
+ Start Kafka Connect
+ The connector name is "FileStreamSource" instead of "FileStream-Source" (The hyphen comes from the book)
---
See https://docs.confluent.io/platform/current/connect/filestream_connector.html#kconnect-long-filestream-connectors <https://docs.confluent.io/platform/current/connect/filestream_connector.html#kconnect-long-filestream-connectors> to understand why "connect-file.jar" is not included
> On 28 Jan 2023, at 22:36, Hieu Nguyen <emerald.hieu@gmail.com> wrote:
>
> Hello,
>
> I'm reading the book "Kafka: The Definitive Guide: Real-Time Data and Stream Processing at Scale" by by Neha Narkhede, Gwen Shapira, and Todd Palino.
>
> In the chapter "Kafka Connect", section "Connector Example: File Source and File Sink", I run the below command and got the missing connector error.
>
> hieunguyenduc@hieugioi kafka_2.13-3.3.1 % echo '{"name":"load-kafka-config", "config":{"connector.class":"FileStream-Source","file":"config/server.properties","topic":"kafka-config-topic"}}' | curl -X POST -d @- http://localhost:8083/connectors <http://localhost:8083/connectors> --header "Content-Type:application/json"
> {"error_code":500,"message":"Failed to find any class that implements Connector and which name matches FileStream-Source, available connectors are: PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorCheckpointConnector, name='org.apache.kafka.connect.mirror.MirrorCheckpointConnector', version='3.3.1', encodedVersion=3.3.1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorHeartbeatConnector, name='org.apache.kafka.connect.mirror.MirrorHeartbeatConnector', version='3.3.1', encodedVersion=3.3.1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorSourceConnector, name='org.apache.kafka.connect.mirror.MirrorSourceConnector', version='3.3.1', encodedVersion=3.3.1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSinkConnector, name='org.apache.kafka.connect.tools.MockSinkConnector', version='3.3.1', encodedVersion=3.3.1, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSourceConnector, name='org.apache.kafka.connect.tools.MockSourceConnector', version='3.3.1', encodedVersion=3.3.1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.SchemaSourceConnector, name='org.apache.kafka.connect.tools.SchemaSourceConnector', version='3.3.1', encodedVersion=3.3.1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSinkConnector, name='org.apache.kafka.connect.tools.VerifiableSinkConnector', version='3.3.1', encodedVersion=3.3.1, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSourceConnector, name='org.apache.kafka.connect.tools.VerifiableSourceConnector', version='3.3.1', encodedVersion=3.3.1, type=source, typeName='source', location='classpath'}"}%
>
> How can I fix it?
>
> The part after that says "
>
> If all went well, you should see something along the lines of:
> {"schema":{"type":"string","optional":false},"payload":"# Licensed to the
> Apache Software Foundation (ASF) under one or more"}
>
> "
>
> I run the command and it shows nothing.
>
> Regards,
> Hieu
>
>
Comments
Post a Comment