Kafka adapter in OIC

Kafka adapter in OIC

How to Create Connection.?

Navigate to Designer->Connections.

 Select the Apache Kafka adapter.

Name the Connection. And please note, role can be only selected as Invoke.

I used localhost:9092* – because the actual connectivity is handled by the agent, so in reality we are connecting to the Kafka server as if we were inside the machine where it runs.
*9092 is the default Kafka port, but you can verify the one you are using in <Kafka_Home>/config/server.properties

Note: Please visit my previous post on how to create Agent and run it in system where kafka is running.

 How to Produce Message into Kafka Topic.?

In this example, I have configured a scheduled orchestrated integration to read records using the FTP Adapter and publish them to an Apache Kafka topic using the Apache Kafka Adapter produce operation.

Final look of the integration will be something like below,

I have not mentioned the steps about reading the message using FTP adapter. Assume your FTP adapter will read the CSV file and we are going to use the same data for publishing into Kafka queue.
Now drag and drop the Kafka connection created in previous step.

You can visit my previous post to understand how to configure FTP adapters.

 Select the publish option.

Select the Topic name and partition. If we do not select a specific partition and use the Default selection, Kafka considers all available partitions and decides which one to use.
And select the Specify the Message structure as “Yes”.

You can specify the message structure with one of the below,
1. XML Schema (XSD) document
2. Sample XML document
3. Sample JSON document

Just have a simple Schema with 3 fields.

Select next and finish.
Below is the mapping details from File response to kafka request.

Now Activate the integration and test it. You can see the message in kafka Queue.

In my next post, we can see how to consume message from kafka topic.