Apache Kafka has emerged as a robust open-source distributed event streaming platform, extensively utilized by numerous organizations for its ability to enable high-throughput and low-latency messaging. This article will guide you through the installation process of Apache Kafka on Fedora 39, assuming that you possess at least a basic understanding of Linux and shell usage.

Before proceeding with the installation, ensure that you meet the following prerequisites:

  • A server running Fedora 39
  • Fresh OS installation is recommended to avoid potential issues
  • Terminal access for command execution
  • Internet access for Apache Kafka repository download
  • Access to a non-root sudo user or root user. It is suggested to act as a non-root sudo user to prevent accidental system damage

Apache Kafka Installation on Fedora 39: Step-by-Step Guide

Step 1: System Update

Updating your Fedora system before installing Java ensures that you have the most recent packages. Use the following commands in your terminal:

sudo dnf clean all
sudo dnf update

Step 2: Java Installation

To install Java, use the dnf install command followed by the package name of the Java version you wish to install. For instance, to install OpenJDK 11, use the following command:

sudo dnf install java-11-openjdk.x86_64

To verify the successful installation of Java, check the Java version on your system using the following command:

java --version

Step 3: Apache Kafka Installation

Download the latest version of Apache Kafka from the official Apache Kafka website. Choose the binary download that matches your system’s architecture:

wget https://downloads.apache.org/kafka/3.6.1/kafka_2.13-3.6.1.tgz

Replace the version number in the URL with the version you intend to download.

Post-download, extract the Kafka tar.gz file using the tar command:

sudo tar -xzf /opt/kafka_2.13-3.6.1.tgz -C /opt

Rename the extracted directory to kafka:

sudo mv /opt/kafka_2.13-3.6.1 /opt/kafka

Assign ownership of the Kafka directory to your current user:

sudo chown -R $USER:$USER /opt/kafka

Step 4: Kafka Server Configuration

Kafka necessitates Apache ZooKeeper for coordination between nodes. By default, Kafka is designed to run ZooKeeper, but we will use the standalone mode. Open server.properties:

nano /opt/kafka/config/server.properties

To use standalone mode, set the following:

zookeeper.connect=localhost:2181

Edit the log directories to:

log.dirs=/tmp/kafka-logs

Save and close the file when done.

Step 5: Start Apache Kafka Server

You can start the Kafka server by running the following command:

cd /opt/kafka/bin
./kafka-server-start.sh ../config/server.properties

To start Kafka in the background, use:

./kafka-server-start.sh -daemon ../config/server.properties

Step 6: Create Kafka Topics

Kafka organizes data streams into topics. Create a test topic named test:

./kafka-topics.sh --create --topic test --partitions 1 --replication-factor 1 --bootstrap-server localhost:9092

This command creates a topic with one partition and one replica. List the topics to confirm:

./kafka-topics.sh --list --bootstrap-server localhost:9092

You should see the test topic listed.

Step 7: Test Kafka

To verify that Kafka is functioning correctly, start a console producer to publish some test messages:

./kafka-console-producer.sh --topic test --bootstrap-server localhost:9092

This will open an input prompt. Type a few messages and hit enter to publish them. Next, consume those messages:

./kafka-console-consumer.sh --topic test --from-beginning --bootstrap-server localhost:9092

You should see the published messages printed in the consumer output. Use Ctrl+C to stop the producer and consumer.

Step 8: Configure Systemd Service

To ensure Kafka starts automatically when the system boots up, create a systemd service file. Create a file named kafka.service:

sudo nano /etc/systemd/system/kafka.service

Add the following:

[Unit]
Description=Apache Kafka Server
Documentation=http://kafka.apache.org/documentation.html
Requires=network.target remote-fs.target
After=network.target remote-fs.target

[Service] 
Type=simple
User=kafka
ExecStart=/opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties
ExecStop=/opt/kafka/bin/kafka-server-stop.sh
Restart=on-abnormal

[Install]
WantedBy=multi-user.target

Save and close the file when done, then reload systemd to pick up the new service:

sudo systemctl daemon-reload

Now, start Kafka and enable it to start on boot:

sudo systemctl start kafka
sudo systemctl enable kafka

Check the status with:

sudo systemctl status kafka

Congratulations, you have successfully installed Apache Kafka on your Fedora 39 system!

Shape.host provides Linux SSD Vps services, which can be an excellent solution for your hosting needs, especially if you prefer a Linux environment. With a variety of package options to choose from, Shape.host can cater to a wide range of hosting requirements. Visit the official Shape.host website for more information.