Skip to content

Home

Jump to bottom
rgolder1 edited this page Feb 28, 2024 · 72 revisions

This wiki supports the Introduction to Kafka with Spring Boot online course.

Frequently Asked Questions in an Introduction To Kafka With Spring Boot

General Questions

Getting Help with the Course

You have a number of options for getting help with your Introduction to Kafka with Spring Boot course. See this page for how to get help.

Which IDE Should I Use

Please see this page for IDE related questions.

Which Java Version

See this page for questions related to which Java version to use.

Can Kafka Run on Microsoft Windows

Yes, but it will need to utilise WSL2 (Windows Sub-system for Linux). Details can be found here

Enable Lombok Annotations

Ensure annotation processing is turned on in Intellij. Without this the IDE will not be able to compile the project. To do so:

  1. In IntelliJ Preferences expand the Build, Execution, Deployment node.
  2. On the right pane of the Compiler / Annotation Processor dialog box, ensure Enable annotation processing check box is checked.

Kafka CLI Cheat Sheet

A convenient reference page for the CLI commands used on the course and more can be found here Kafka-CLI-Cheat-Sheet

UUID Tool

MacOS and Linux platform provide a cli tool for generating UUIDs. This is a great help when creating data for use in the CLI examples. Simply run the following command in your terminal window to generate a UUID:

uuidgen

Mockito

The Mockito test library is used throughout this course for unit testing. More information on Mockito can be found here Mockito

Where to get more information on Kafka

Lydtech has a wealth of articles digging deep into Kafka and Event Driven Architecture, which can be found here: Lydtech Blog

Common Problems and FAQs

FAQs

Error running Integration Tests With Spring Boot 3.2

The following error can be seen running the Spring Boot integration tests with Spring Boot 3.2.0, 3.2.1, and 3.2.2:

java.lang.IllegalStateException: Expected 2 but got 1 partitions

For compatibility with these versions, set the embedded broker partitions count to 1 (it defaults to 2):

@EmbeddedKafka(controlledShutdown = true, partitions = 1)

This change in behaviour in Spring Kafka was not intended, and Spring have rectified it in version 3.2.3, and the integration tests pass as before.

Getting error - Whitelabel Error Page - This application has no explicit mapping for /error...

This is a generic error page generated by Spring Boot. The root cause can be many different things.

If you see this web page, you need to view the Spring Boot console messages to determine the root cause.

Project will not start. Getting error: java.net.BindException: Address already in use

May see in console log:

***************************
APPLICATION FAILED TO START
***************************

Description:

The Tomcat connector configured to listen on port 8080 failed to start. The port may already be in use or the connector may be misconfigured.

Action:

Verify the connector's configuration, identify and stop any process that's listening on port 8080, or configure this application to listen on another port.

This means another application is already running on port 8080. You need to find and stop the other application.

How to Compare Branches

Getting an error, but cannot find what is different from lesson source code. Sometimes it can be a very small difference. You can use Git to report the differences between your code and the example source code for the lesson.

Note: This will only work if you have forked the original source code example to your GitHub repository.

  1. Commit all of your changes.
  2. Add remote for original repo - git remote add kafkaRepo github_url
  3. Fetch remote - git fetch kafkaRepo
  4. Run diff - git diff HEAD..kafkaRepo/main <- update branch name from main to desired branch for lesson

Forking the Git Repositories

When forking the Git repos, be sure to unselect the option Copy the main branch only. This ensures all branches are forked. The branches contain the start and end states of the codebase for each of the coding modules. These branches are linked to from the resources under each relevant module.

JUnit 5 Tests are not running from Maven

There are several reasons for this, and a lot of bad advice on the internet.

Please see this blog post.

Error 'No qualifying Bean of type [Java class name] found for dependency...'

Spring is looking for a bean of a specific type in the context, but is unable to find it. Error is because Spring has not been configured to create a bean for the Java class indicated in the message.

This may be due to several different reasons, but when troubleshooting - keep in mind Spring is telling you that Java class is missing from the context. You need to tell Spring to create a bean of that type.

Try the following:

Make sure you have a bean of the type configured. Is the missing Java class annotated with a Spring stereotype - @Component, @Controller, @RestController, @Service, @Repository? Or has the bean been declared in a Java configuration class using the @Bean annotation? Is the package of the Java class or configuration class being scanned? By default, Spring Boot will do a component scan of the package of the main application class, and all sub-packages. A common mistake is to create a package outside of this package structure. (thus, Spring does not 'see' the configuration) If the component is not under the package of the main application class, use the @ComponentScan annotation. See this link for additional information.

How to use the source code of the course?

Source code for this course is in Github. Code-centric lectures have a Resources Available link on the Udemy player.

On clicking it, you will find Starting Source and Ending Source links for that Lecture. Starting Source points to the Github repo branch that contains the starter code for the lecture. Ending Source points to the Github repo branch that contains the final code of the lecture. To get the best out of the course:

  • Fork/clone the repo.
  • Checkout the Starting Source branch
  • Write the code along with the lecture.
  • If you get stuck, refer the Ending Source

Video quality is not good or video is blur

Videos go through quality checks before getting uploaded to Udemy. Therefore, if your viewing experience is not good, it might be due to the following reasons:

  • Low Internet Bandwith: Check your Internet Connectivity because Udemy will serve video in lower resolution for low Internet bandwidth.
  • Incorrect settings in Udemy Video Player: Adjust video quality in Udemy Video Player. Click the gear icon on the bottom-right corner of the player, and select resolutions of 360p, 480p, 720p and 1080p for the video quality.

Red underlines appear on Spring Beans in Intellij IDE

Spring Beans such as the EmbeddedKafkaBroker and KafkaListenerEndpointRegistry in the integration tests can be marked by the IDE with red underlines. These can be safely ignored so long as the code still builds and the tests pass. There are a couple of things to try to remove them:

  1. From the Intellij menu, select Invalidate Caches
  2. Alter the IntelliJ settings to mark these as warnings: Incorrect autowiring in Spring bean components

Error Starting the Kafka Broker ‐ No meta.properties found

Problem

When starting the Kafka broker it fails with the following message:

ERROR Exiting Kafka due to fatal exception (kafka.Kafka$) org.apache.kafka.common.KafkaException: No meta.properties found in /tmp/kraft-combined-logs (have you run kafka-storage.sh to format the directory?)

By default the kafka log directory is created in /tmp e.g /tmp/kraft-combined-logs which isn't the best place for it as the /tmp directory often gets cleaned out.

Solution

The log location has probably become corrupt or cleaned down and should be re-initialised using

KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"

bin/kafka-storage.sh format -t $KAFKA_CLUSTER_ID -c config/kraft/server.properties

Then restart the kafka broker

A more permanent fix would be to change the location to somewhere other than /tmp This can be achieved by editing the config/kraft/server.properties file and changing the location specified by the log.dirs setting Then re-initialise the log directory and restart the server by using the commands above

Clone this wiki locally