How to Set Up Your First Elasticsearch Cluster for Development

Setting up your first Elasticsearch cluster is mostly about choosing a safe development configuration and verifying that the node actually joined the cluster you expected. A single-node setup is enough for basic indexing and search tests; a local multi-node setup helps you learn discovery and shard allocation.

The focus here is local development. Do not reuse these relaxed settings for a public or production cluster.

Prerequisites

Before you begin, ensure you have the following prerequisites met:

Java runtime: Recent Elasticsearch distributions include a bundled JDK. If you use a package or distribution that does not, install the Java version required by that Elasticsearch release.
Download Elasticsearch: Obtain the binary distribution of your desired Elasticsearch version from the official Elastic website.
System Resources: For basic development testing, 2GB of RAM is generally sufficient, though more is recommended for multi-node testing.

Step 1: Downloading and Extracting Elasticsearch

Once downloaded (usually as a .zip or .tar.gz file), extract the archive to a directory where you want to host your cluster files (e.g., ~/elasticsearch-8.12.0). This directory is referred to as the Elasticsearch Home Directory.

Step 2: Configuring a Single-Node Cluster (Development Default)

By default, when you run Elasticsearch for the first time, it attempts to start as a single-node cluster. However, modern versions often require explicit configuration, especially regarding security and memory settings, even for local development.

Configuration files are located in the config/ directory within your Elasticsearch Home.

Essential Configuration (`config/elasticsearch.yml`)

The main configuration file is elasticsearch.yml. For a local, single-node setup, you must configure at least the cluster name and node name.

# Cluster configuration
cluster.name: dev-cluster

# Node configuration
node.name: node-1

# Network settings (Use localhost for development)
network.host: 127.0.0.1

# HTTP Port (Default is 9200)
http.port: 9200

# Important for development: Disable security initially (Use with caution in production!)
xpack.security.enabled: false

Warning on Security: Disabling xpack.security.enabled is common for initial development setup to simplify testing. Never run this configuration in a publicly accessible environment. For modern Elasticsearch versions (7.x+), you will often need to run setup commands first to generate initial passwords if security remains enabled.

Starting the Single Node

Navigate to the root of your extracted directory and run the appropriate startup script. On Linux/macOS:

./bin/elasticsearch

On Windows using PowerShell:

.\bin\elasticsearch.bat

Wait for the logs to indicate that the node has started successfully, usually showing a message like started.

Step 3: Verifying the Cluster Status

Once the process is running, you can verify the cluster status using curl against the default HTTP port (9200).

Checking Cluster Health

This command checks the overall health of the cluster:

curl -X GET "http://localhost:9200/_cat/health?v"

Expected Output Snippet:

epoch      timestamp cluster     status node.total node.data shards prio initialized
1678886400 10:00:00  dev-cluster green    1          1        0   0           1

The status field should be green, indicating a healthy, single-node cluster.

Checking Node Information

You can also verify the node information:

curl -X GET "http://localhost:9200/_cat/nodes?v"

Step 4: Setting Up a Multi-Node Cluster (For Realistic Testing)

For more realistic development or testing of shard allocation, you need at least two nodes. This requires running multiple separate instances of Elasticsearch, each with distinct configurations.

Directory Structure

Create separate directories for each node within your main working area (e.g., es_data/node1, es_data/node2). Copy the base Elasticsearch distribution into each of these directories, or link them to the main installation.

Node Configuration Differences

For each node, create a unique config/elasticsearch.yml file:

Node 1 Configuration (`es_data/node1/config/elasticsearch.yml`)

cluster.name: dev-cluster
node.name: node-1
network.host: 127.0.0.1
http.port: 9200
transport.port: 9300
path.data: node1_data  # Unique data path
xpack.security.enabled: false
discovery.seed_hosts: ["127.0.0.1:9300", "127.0.0.1:9301"]
# This line tells node 1 how to find other members
cluster.initial_master_nodes: ["node-1", "node-2"]

Node 2 Configuration (`es_data/node2/config/elasticsearch.yml`)

cluster.name: dev-cluster
node.name: node-2
network.host: 127.0.0.1
http.port: 9201  # Must use a different HTTP port
transport.port: 9301  # Must use a different transport port
path.data: node2_data  # Unique data path
xpack.security.enabled: false
discovery.seed_hosts: ["127.0.0.1:9300", "127.0.0.1:9301"]
# This line tells node 2 how to find other members
cluster.initial_master_nodes: ["node-1", "node-2"]

Starting the Multi-Node Cluster

Start Node 1: Navigate to es_data/node1 and execute ./bin/elasticsearch.
Start Node 2: Navigate to es_data/node2 and execute ./bin/elasticsearch.

Verifying the Multi-Node Cluster

Check the node count using the API against any running HTTP port (e.g., 9200):

curl -X GET "http://localhost:9200/_cat/nodes?v"