In my local machine, I have created a Apache kafka producer connection and an adapter using it. When I send a message to the kafka topic from my local machine, it is working fine.
But when I create a docker image from this MSR instance and run it as a docker container (in same machine or from different machine), the adapter service run to push message to the kafka topic is throwing the below error
[ADA.507.4010] Error sending kafka message
org.apache.kafka.common.errors.TimeoutException: Topic not present in metadata after 60000 ms.
I can successfully telnet the kafka server and port from the machine where I am running the container. I did some research and saw that jar file version mismatches could cause this too. But same is working fine in my local. I build the image from this machine, so it would have all the necessary jar files. But for some reason this error occurs when it is run as a container.
The kafka I am connecting to is confluent kafka docker container version 7.6.1. I have placed the jar files wm-kafka-v9.jar, kafka-tools-3.6.2.jar, kafka-clients-3.6.2.jar in the WmKafkaAdapter/code/jars folder.
Do we need to follow anything addition when running it as a container. Does anyone have any inputs related to this?
Did you create the container yourself or downloaded it from containers.softwareag.com ?
In theory they should both work the same and you shouldn’t do anything for the containers, but it also depends on a lot of independent parameters like firewall configuration etc. Did you check if you can successfully connect to server from the container without using IS services? Please also share the rest of the logs and the flow as well. You can mask the confidential information.
I build my own image. I build it from the on prem MSR 10.15. I have to add the cumulocity cloudstream package to it and few other custom packages and rebuild it.
The firewall to the destination server is open. Infact, there is a database container running in the same destination machine in a different port to which I am able to connect from my MSR container. Also from inside the contaianer when I do curl to the destination server IP and the kafka port, I am not getting any connection refused or timed out issues.
I also have no issues enabling the producer connection. The test connection is successful and I am able to enable it too. But just when running the adapter service, this error comes.
If your MSR container and the Kafka containers reside in the same Docker host, then you should use the Docker network to connect them, not the IP address of the host network.
I suspect the error you’ve got here is the good old “Docker no route to host” error in disguise. 192.168.220.130:9092 cannot be addressed inside the MSR container.
Your containers need to be in the same Docker network and in the adapter connection properties you need to reference the Kafka container using the Docker dns name (which usually corresponds to the container name) instead of the Docker host IP address.
Please share your Docker commands (or docker-compose.yml file) if you need more detailed instructions.
My MSR container and Kafka container are in different hosts which are in the same network. The hosts can communicate between each other. I also have a database container in the same host as the kafka container, which I can connect from the MSR container. Also I can enable the kafka connection without any issues, only the adapter call times out.
Below is my MSR container run command and I have attached the docker-compose.yml file for the kafka-zookeeper containers (uploading is as text document, as I am not able to upload yml)
If MSR & kafka are in different Docker hosts, then this probably isn’t a “no route to host” issue.
I’ll see if I can find an hour or two to setup a lab in order to investigate the matter in the next few days. With the elements you just gave me, it should not be too difficult.
One question: do you inject a value for the DOCKER_HOST_IP? I see you use it in your declaration of KAFKA_ADVERTISED_LISTENER. If it’s not populated, then the EXTERNAL listener will default to 127.0.0.1:9092, which won’t work for your use case.
Thank you, that was an issue. I injected the DOCKER_HOST_IP environment variable into the docker container and my kafka producer adapter connection is working fine and I am able to publish messages to the topic. But still my listener is not working, it’s throwing the same timeout error.
