How to Create and Manage Kafka Topics Using Command Line

Apache Kafka topics are where your producers write records and your consumers read them. If you need to create, inspect, resize, or delete a topic from a terminal, the main tool is kafka-topics.sh.

Graphical tools are useful, but CLI commands are still the fastest way to verify what the cluster is doing during an incident or deployment. This guide shows the topic commands you will use most often and calls out the places where Kafka can surprise you.

Prerequisites and Setup

To execute the commands in this guide, you must have access to a machine where the Kafka binaries are installed. All topic management operations are performed using the kafka-topics.sh utility, typically found in the bin directory of your Kafka installation.

All commands require the address of at least one Kafka broker, specified with --bootstrap-server. Older Kafka clusters may still show examples using --zookeeper, but broker-based administration is the current pattern.

For the examples below, we will assume the broker is running locally on the default port:

# Standard Broker Address Placeholder
BROKER_ADDRESS="localhost:9092"

---

1. Creating a New Kafka Topic

Creating a topic requires defining its name, along with two critical parameters that dictate its behavior and fault tolerance: the number of partitions and the replication factor.

Essential Parameters

• --topic <name>: The name of the topic.
• --partitions <N>: The number of partitions the topic will be split into. Partitions are the units of parallelism and ordering within a topic.
• --replication-factor <N>: The number of copies of the data that will be maintained across different brokers. A replication factor of 1 means no redundancy.

Command Example: Creating sales-data

This command creates a topic named sales-data with 3 partitions and a replication factor of 2 (meaning 2 copies of every partition will exist across the cluster).

kafka-topics.sh --create --topic sales-data \
  --bootstrap-server $BROKER_ADDRESS \
  --partitions 3 \
  --replication-factor 2

Tip: In production, a replication factor of 3 is common because it keeps multiple copies of each partition on different brokers. Real fault tolerance also depends on broker placement, min.insync.replicas, and producer acknowledgment settings.

2. Listing All Topics

To view all topics currently available in the Kafka cluster, use the --list flag.

Command Example

kafka-topics.sh --list --bootstrap-server $BROKER_ADDRESS

Output Example:

sales-data
logistics-stream
__consumer_offsets

3. Describing Topic Configuration

Checking the existing configuration, partition count, and broker assignment for a specific topic is essential for troubleshooting and verification. Use the --describe flag.

Command Example: Describing sales-data

kafka-topics.sh --describe --topic sales-data \
  --bootstrap-server $BROKER_ADDRESS

Output Interpretation:

The output shows the configuration at both the topic level and the partition level:

Topic: sales-data  PartitionCount: 3 ReplicationFactor: 2 Configs:
  Topic: sales-data  Partition: 0 Leader: 1 Replicas: 1,2 Isr: 1,2
  Topic: sales-data  Partition: 1 Leader: 2 Replicas: 2,0 Isr: 2,0
  Topic: sales-data  Partition: 2 Leader: 0 Replicas: 0,1 Isr: 0,1

• Leader: The broker currently responsible for handling reads/writes for that partition.
• Replicas: The list of brokers holding a copy of that partition.
• Isr (In-Sync Replicas): The replicas that are sufficiently caught up to be eligible for safe failover. If ISR shrinks, check broker health, disk latency, and network lag.

4. Altering Existing Topics

Kafka provides limited mechanisms for altering topics after creation. The two most common alteration tasks are increasing the partition count and overriding default broker configuration settings.

A. Increasing Partitions

Partitions can only be increased, never decreased. Increasing partitions helps scale out consumer parallelism.

Warning: Increasing partitions changes how messages are mapped (hashed) to partitions. If your producers rely on key-based ordering guarantees, increasing partitions may break ordered delivery for existing keys.

If sales-data currently has 3 partitions, we can increase it to 5:

kafka-topics.sh --alter --topic sales-data \
  --bootstrap-server $BROKER_ADDRESS \
  --partitions 5

B. Altering Topic-Specific Configuration

You may see older examples changing topic configs through kafka-topics.sh --alter --config. On current Kafka clusters, prefer kafka-configs.sh for topic configuration changes because it is the dedicated admin tool for dynamic configs.

Example: Setting a message retention time of 24 hours (86400000 milliseconds) for sales-data.

kafka-configs.sh --bootstrap-server $BROKER_ADDRESS \
  --alter --entity-type topics --entity-name sales-data \
  --add-config retention.ms=86400000

To remove a specific configuration override and revert to the broker default, use --delete-config:

kafka-configs.sh --bootstrap-server $BROKER_ADDRESS \
  --alter --entity-type topics --entity-name sales-data \
  --delete-config retention.ms

5. Deleting a Kafka Topic

Topics that are no longer in use should be properly deleted to reclaim disk space and maintain cluster hygiene.

Enabling Topic Deletion

Kafka brokers must allow deletion before this command can remove data. In many modern deployments it is enabled, but do not assume. Check the broker configuration for:

delete.topic.enable=true

Command Example: Deleting old-stream

Use the --delete flag to initiate the topic removal. Topic deletion is often asynchronous, meaning the command submits the request, and the deletion happens in the background.

kafka-topics.sh --delete --topic old-stream \
  --bootstrap-server $BROKER_ADDRESS

Confirmation Output:

Deletion of topic old-stream initiated successfully.

---

Summary of Topic Management Commands

Action	Flag(s)	Purpose	Example Parameters
Create	--create	Initialize a new topic.	--partitions 5 --replication-factor 3
List	--list	Show all topics in the cluster.	N/A
Describe	--describe	View current configuration and layout.	--topic my-topic
Alter (Partitions)	--alter	Increase the number of partitions.	--partitions N (N > current count)
Alter (Config)	kafka-configs.sh --alter	Override broker defaults for a specific topic.	--add-config retention.ms=...
Delete	--delete	Remove a topic permanently.	--topic my-topic

Next Steps

1. Practice these commands in a development or staging environment.
2. Use kafka-configs.sh --describe before and after config changes so you know which values are topic overrides.
3. Learn the corresponding CLI commands for producer and consumer testing (kafka-console-producer.sh and kafka-console-consumer.sh).