🛠️ Prerequisites
Before getting started, ensure your system has the following:
Podman and Podman Compose installed (Installation Guide)
Confluent CLI (optional) – useful for managing Confluent services via the command line
Java 17 or 21 – required for running Confluent CLI
At least 6 GB of RAM allocated for containerized services
📦 OS-specific Installation Tips
macOS users are encouraged to install Podman directly from the official site
Windows users can follow this guide to set up Podman
Windows users can follow this guide to set up Podman
⚙️ Simplest Way to Spin Up Kafka with Podman Compose
The quickest way to launch Kafka and its ecosystem locally is by using Podman Compose with a Compose file.
📝 Step 1: Create the Compose File
Create a file named docker-compose.yaml and paste the following configuration:
# docker-compose.yaml
services:
broker:
image: confluentinc/cp-kafka:7.8.0
hostname: broker
container_name: broker
ports:
- "9092:9092"
- "9101:9101"
environment:
KAFKA_NODE_ID: 1
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT'
KAFKA_ADVERTISED_LISTENERS: 'PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092'
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
KAFKA_JMX_PORT: 9101
KAFKA_JMX_HOSTNAME: localhost
KAFKA_PROCESS_ROLES: 'broker,controller'
KAFKA_CONTROLLER_QUORUM_VOTERS: '1@broker:29093'
KAFKA_LISTENERS: 'PLAINTEXT://broker:29092,CONTROLLER://broker:29093,PLAINTEXT_HOST://0.0.0.0:9092'
KAFKA_INTER_BROKER_LISTENER_NAME: 'PLAINTEXT'
KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
KAFKA_LOG_DIRS: '/tmp/kraft-combined-logs'
CLUSTER_ID: 'MkU3OEVBNTcwNTJENDM2Qk'
schema-registry:
image: confluentinc/cp-schema-registry:7.8.0
hostname: schema-registry
container_name: schema-registry
depends_on:
- broker
ports:
- "8081:8081"
environment:
SCHEMA_REGISTRY_HOST_NAME: schema-registry
SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: 'broker:29092'
SCHEMA_REGISTRY_LISTENERS: http://0.0.0.0:8081
connect:
image: cnfldemos/cp-server-connect-datagen:0.6.4-7.6.0
hostname: connect
container_name: connect
depends_on:
- broker
- schema-registry
ports:
- "8083:8083"
environment:
CONNECT_BOOTSTRAP_SERVERS: 'broker:29092'
CONNECT_REST_ADVERTISED_HOST_NAME: connect
CONNECT_GROUP_ID: compose-connect-group
CONNECT_CONFIG_STORAGE_TOPIC: docker-connect-configs
CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: 1
CONNECT_OFFSET_FLUSH_INTERVAL_MS: 10000
CONNECT_OFFSET_STORAGE_TOPIC: docker-connect-offsets
CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: 1
CONNECT_STATUS_STORAGE_TOPIC: docker-connect-status
CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: 1
CONNECT_KEY_CONVERTER: org.apache.kafka.connect.storage.StringConverter
CONNECT_VALUE_CONVERTER: io.confluent.connect.avro.AvroConverter
CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: http://schema-registry:8081
CLASSPATH: /usr/share/java/monitoring-interceptors/monitoring-interceptors-7.8.0.jar
CONNECT_PRODUCER_INTERCEPTOR_CLASSES: "io.confluent.monitoring.clients.interceptor.MonitoringProducerInterceptor"
CONNECT_CONSUMER_INTERCEPTOR_CLASSES: "io.confluent.monitoring.clients.interceptor.MonitoringConsumerInterceptor"
CONNECT_PLUGIN_PATH: "/usr/share/java,/usr/share/confluent-hub-components"
control-center:
image: confluentinc/cp-enterprise-control-center:7.8.0
hostname: control-center
container_name: control-center
depends_on:
- broker
- schema-registry
- connect
- ksqldb-server
ports:
- "9021:9021"
environment:
CONTROL_CENTER_BOOTSTRAP_SERVERS: 'broker:29092'
CONTROL_CENTER_CONNECT_CONNECT-DEFAULT_CLUSTER: 'connect:8083'
CONTROL_CENTER_CONNECT_HEALTHCHECK_ENDPOINT: '/connectors'
CONTROL_CENTER_KSQL_KSQLDB1_URL: "http://ksqldb-server:8088"
CONTROL_CENTER_KSQL_KSQLDB1_ADVERTISED_URL: "http://localhost:8088"
CONTROL_CENTER_SCHEMA_REGISTRY_URL: "http://schema-registry:8081"
CONTROL_CENTER_REPLICATION_FACTOR: 1
CONTROL_CENTER_INTERNAL_TOPICS_PARTITIONS: 1
CONTROL_CENTER_MONITORING_INTERCEPTOR_TOPIC_PARTITIONS: 1
CONFLUENT_METRICS_TOPIC_REPLICATION: 1
PORT: 9021
ksqldb-server:
image: confluentinc/cp-ksqldb-server:7.8.0
hostname: ksqldb-server
container_name: ksqldb-server
depends_on:
- broker
- connect
ports:
- "8088:8088"
environment:
KSQL_CONFIG_DIR: "/etc/ksql"
KSQL_BOOTSTRAP_SERVERS: "broker:29092"
KSQL_HOST_NAME: ksqldb-server
KSQL_LISTENERS: "http://0.0.0.0:8088"
KSQL_CACHE_MAX_BYTES_BUFFERING: 0
KSQL_KSQL_SCHEMA_REGISTRY_URL: "http://schema-registry:8081"
KSQL_PRODUCER_INTERCEPTOR_CLASSES: "io.confluent.monitoring.clients.interceptor.MonitoringProducerInterceptor"
KSQL_CONSUMER_INTERCEPTOR_CLASSES: "io.confluent.monitoring.clients.interceptor.MonitoringConsumerInterceptor"
KSQL_KSQL_CONNECT_URL: "http://connect:8083"
KSQL_KSQL_LOGGING_PROCESSING_TOPIC_REPLICATION_FACTOR: 1
KSQL_KSQL_LOGGING_PROCESSING_TOPIC_AUTO_CREATE: 'true'
KSQL_KSQL_LOGGING_PROCESSING_STREAM_AUTO_CREATE: 'true'
ksqldb-cli:
image: confluentinc/cp-ksqldb-cli:7.8.0
container_name: ksqldb-cli
depends_on:
- broker
- connect
- ksqldb-server
entrypoint: /bin/sh
tty: true
ksql-datagen:
image: confluentinc/ksqldb-examples:7.8.0
hostname: ksql-datagen
container_name: ksql-datagen
depends_on:
- ksqldb-server
- broker
- schema-registry
- connect
command: "bash -c 'echo Waiting for Kafka to be ready... && \
cub kafka-ready -b broker:29092 1 40 && \
echo Waiting for Confluent Schema Registry to be ready... && \
cub sr-ready schema-registry 8081 40 && \
echo Waiting a few seconds for topic creation to finish... && \
sleep 11 && \
tail -f /dev/null'"
environment:
KSQL_CONFIG_DIR: "/etc/ksql"
STREAMS_BOOTSTRAP_SERVERS: broker:29092
STREAMS_SCHEMA_REGISTRY_HOST: schema-registry
STREAMS_SCHEMA_REGISTRY_PORT: 8081
rest-proxy:
image: confluentinc/cp-kafka-rest:7.8.0
depends_on:
- broker
- schema-registry
ports:
- 8082:8082
hostname: rest-proxy
container_name: rest-proxy
environment:
KAFKA_REST_HOST_NAME: rest-proxy
KAFKA_REST_BOOTSTRAP_SERVERS: 'broker:29092'
KAFKA_REST_LISTENERS: "http://0.0.0.0:8082"
KAFKA_REST_SCHEMA_REGISTRY_URL: 'http://schema-registry:8081'
docker-compose.yaml (Podman Compose Compatible)Confluent Platform local development environment
1. broker
Image: confluentinc/cp-kafka:7.8.0
Role: Kafka broker using KRaft (no ZooKeeper).
Key Env Config:
KAFKA_ADVERTISED_LISTENERS: Makes broker accessible both inside and outside the container.
CLUSTER_ID: Must be a valid base64 UUID, used for KRaft mode.
KAFKA_PROCESS_ROLES: Set to both broker and controller.
Ports:
9092: For clients
9101: JMX metrics (optional)
2. schema-registry
Image: confluentinc/cp-schema-registry:7.8.0
Role: Manages Avro/Protobuf/JSON schemas.
Depends on: broker
Port: 8081
Connects to Kafka via SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS.
3. connect
Image: cnfldemos/cp-server-connect-datagen
Role: Kafka Connect service pre-packaged with Datagen connector for demo data.
Depends on: broker, schema-registry
Port: 8083
Converts values using AvroConverter linked to the schema registry.
4. control-center
Image: confluentinc/cp-enterprise-control-center:7.8.0
Role: Web-based UI for managing Kafka clusters.
Depends on: broker, schema-registry, connect, ksqldb-server
Port: 9021
Monitors Kafka, Connect, KSQLDB, Schema Registry
5. ksqldb-server
Image: confluentinc/cp-ksqldb-server:7.8.0
Role: Stream processing engine using SQL-like syntax for Kafka topics.
Depends on: broker, connect
Port: 8088
6. ksqldb-cli
Image: confluentinc/cp-ksqldb-cli:7.8.0
Role: Command-line interface for interacting with KSQLDB.
This container is interactive (tty: true) and doesn’t expose ports.
7. ksql-datagen
Image: confluentinc/ksqldb-examples:7.8.0
Role: Prepares demo data for KSQLDB use.
Startup delay built in via cub scripts to ensure services are ready.
Runs indefinitely using tail -f /dev/null.
Image:
confluentinc/ksqldb-examples:7.8.0Role: Prepares demo data for KSQLDB use.
Startup delay built in via
cub scripts to ensure services are ready.Runs indefinitely using
tail -f /dev/null.8. rest-proxy
Image: confluentinc/cp-kafka-rest:7.8.0
Role: Exposes a REST API for Kafka (useful for clients that can’t use Kafka protocol).
Port: 8082
Connects to both Kafka and Schema Registry.
🔁 Summary
This compose file sets up a fully local Confluent stack that includes:
- Kafka broker (KRaft mode)
- Schema Registry
- Kafka Connect with Datagen
- REST Proxy
- KSQLDB (server and CLI)
- Control Center UI
✅ Next Steps
Once your docker-compose.yaml file is ready: podman-compose up -d
That's it! You now have a full Confluent stack running locally using Podman.
🧪 Optional: Validate with Confluent CLI
If you've installed the Confluent CLI, you can run: confluent local services list
To ensure everything is operational.




