Debugging Elasticsearch and Lucene with IntelliJ IDEA

Now posted on the Elastic blog

January 14, 2018 update: This article has been published on Elastic’s website as: https://www.elastic.co/blog/how-to-debug-elasticsearch-source-code-in-intellij-idea.

Introduction

IntelliJ IDEA is a Java integrated development environment (IDE) for developing computer software. In this blog post, I discuss how to setup an IntelliJ IDEA project that will allow interactive debugging of Elasticsearch and Lucene source code.

The instructions presented in this blog have been tested on Mac OSX with IntelliJ IDEA 2018.3 (Community Edition), and OpenJDK 11.

Download Elasticsearch

Get a copy of the Elasticsearch source code from github as follows:

git clone https://github.com/elastic/elasticsearch.git

Checkout the branch for the Elasticsearch release that you want to debug.

cd elasticsearch
git checkout --track origin/6.6

Review text files included with the distribution

Within the “elasticsearch” directory, there are several text files that should be reviewed. In particular, “CONTRIBUTING.md” includes a description of the process for importing Elasticsearch code into an IntelliJ IDEA project, and “TESTING.asciidoc” describes ways to build and debug the code. The remainder of this blog post is based on the instructions in these files.

Configure the code for use with IntelliJ IDEA

The build system used by Elasticsearch is gradle, and at least Java 11 is required to build Elasticsearch gradle tools. Before executing gradlew, ensure that your JAVA_HOME environment variable is set correctly. For example my JAVA_HOME (on OSX) is set as follows:

JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk-11.0.2.jdk/Contents/Home
export JAVA_HOME 

Finally, execute the following command to configure Elasticsearch for use in an IntelliJ IDEA project.

./gradlew idea

The above command may take a few minutes to execute, and once it is complete, your project is ready to be imported into IntelliJ IDEA.

Import Elasticsearch into an IntelliJ IDEA project

1. Open IntelliJ Idea, and if you don’t have any other projects open, you will see a screen that looks like the image below. Click on “Import project”

 

Screen Shot 2019-02-02 at 8.56.52 PM

2. Open the “elasticsearch” directory that was created by the previously executed “git clone” command.

Screen Shot 2019-02-02 at 9.22.19 PM

3. Select “Import project from external model” -> “Gradle”, and the click on “Next”

Screen Shot 2019-02-02 at 9.28.06 PM

4. Select “Use default gradle wrapper (recommended)” and set “Gradle JVM” to version 11, as shown below. Then click on “Finish”.

Screen Shot 2019-02-02 at 9.28.36 PM

5. After completing the above steps, IntelliJ IDEA will start building the source code. The IntelliJ IDEA window should look similar to the image below once the build has completed.

Screen Shot 2019-02-02 at 9.36.27 PM

Start Elasticsearch for debugging

One way to debug Elasticsearch is to start the project in debug mode from the command line with the following command:

./gradlew run --debug-jvm

It may take a few minutes for the above process to fully start, at which point you can connect to the process from IntelliJ IDEA  by clicking on “Run” -> “Attach to Process” as shown below:

Screen Shot 2019-02-02 at 9.49.47 PM.png

This will allow you to select the process to attach to, which should look similar to the following:

Screen Shot 2019-02-02 at 9.53.27 PM

You should now be able to set breakpoints and step through both Elasticsearch and Lucene code.

Conclusion

In this blog post, I have demonstrated how to setup a project in IntelliJ IDEA that will allow interactive debugging of Elasticsearch and Lucene source code. You are now ready to dig deep into the internal workings of Elasticsearch!