Kafka Connect Made Convenient

Photo by Mika Baumeister on Unsplash
kcctl config set-context my-cluster --cluster http://localhost:8083 --username myusername --password mypasswordkcctl config set-context 
--cluster=https://user-name:password@localhost:443 my-context
URL:               https://<USERNAME>:<PASSWORD>@localhost:443
Version: 3.2.0
Commit: 54454035f8b22b67
Kafka Cluster ID: tArO_G21TQyIPS4A3BE1nw
kcctl get pluginsTYPE   CLASS                                                 VERSION
source com.couchbase.connect.kafka.CouchbaseSourceConnector 4.1.6
sink com.couchbase.connect.kafka.CouchbaseSinkConnector 4.1.6
...
{
"name": "debezium-connector",
"config": {
"database.server.name": "demo-pg",
"database.hostname": "demo-pg.host.com",
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"name": "debezium-connector",
"database.port": "28577",
"database.user": "USER",
"database.password": "PASSWORD",
"database.dbname": "DB-NAME", "database.history.kafka.topic": "DB-NAME-schema",
"database.history.kafka.bootstrap.servers": "demo-kafka.host.com:28579"
}
}
kcctl apply --dry-run --file register_debezium_connector.json
The configuration is valid!
kcctl apply --file register_debezium_connector.json
Created connector debezium-connector
NAME                 TYPE     STATE     TASKS
test-connector source RUNNING 0: RUNNING
debezium-connector source RUNNING 0: RUNNING
kcctl get connectorsNAME                 TYPE     STATE     TASKS
test-connector source RUNNING 0: RUNNING
debezium-connector source RUNNING 0: FAILED
Name:       debezium-connector
Type: source
State: RUNNING
Worker ID: demo-kafka-2.aiven.local:3000
Config:
connector.class: io.debezium.connector.postgresql.PostgresConnector
incrementing.column.name: id
...
Tasks:
0:
State: FAILED
Worker ID: demo-kafka-2.aiven.local:3000
Trace: io.debezium.DebeziumException: Couldn't obtain encoding for database star_trek
at io.debezium.connector.postgresql.connection.PostgresConnection.getDatabaseCharset(PostgresConnection.java:476)
at io.debezium.connector.postgresql.PostgresConnectorTask.start(PostgresConnectorTask.java:75)
at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:130)
at io.debezium.connector.common.BaseSourceTask.startIfNeededAndPossible(BaseSourceTask.java:207)
at io.debezium.connector.common.BaseSourceTask.poll(BaseSourceTask.java:148)
at org.apache.kafka.connect.runtime.WorkerSourceTask.poll(WorkerSourceTask.java:305)
at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:249)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:188)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:243)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.postgresql.util.PSQLException: Connection to demo-pg-cratliff1-demo.aivencloud.com:28577 refused. Check that the hostname and port are correct and that the postm
aster is accepting TCP/IP connections.
...

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Clay Ratliff

Clay Ratliff

I am a jack of all trades and a master of none. An all-around neophile, currently disguised as a Solution Architect.