What is Kafka Connect?
Answer
Kafka Connect is a scalable, reliable framework for streaming data between Kafka and other systems without writing custom code. It provides a plugin-based connector system. Source Connectors: read data from an external system and write it to Kafka topics. Examples: Debezium (CDC from databases), S3 Source, JDBC Source (read from any SQL database). Sink Connectors: read from Kafka topics and write to external systems. Examples: S3 Sink, Elasticsearch Sink, JDBC Sink (write to SQL databases), BigQuery Sink. Workers: Connect runs as distributed workers that execute connectors. Converters: serialize/deserialize data (Avro, JSON, Protobuf). Hundreds of connectors are available from Confluent Hub as open-source or commercial plugins. Connect eliminates most ETL pipeline boilerplate code.