OpenSearch, an open-source search, analytics, and visualization suite, is a powerful tool for businesses looking to ingest, secure, search, aggregate, view, and analyze data. In this comprehensive guide, we will walk you through the process of installing OpenSearch on a Debian 11 server. We will cover prerequisites, system setup, downloading OpenSearch, configuring it, generating TLS certificates, creating users, running OpenSearch as a systemd service, installing OpenSearch Dashboards, and accessing OpenSearch Dashboards.

Prerequisites

Before we dive into the installation process, let’s ensure we have all the necessary prerequisites in place:

1. A Debian 11 server with at least 8GB of RAM. For this guide, we will refer to the server as ‘node1’ with the local IP address ‘192.168.5.50’.
2. A non-root user with sudo/root administrator privileges.

If you have these requirements ready, we can proceed with the installation of OpenSearch.

Setup System

To optimize your Debian server for OpenSearch deployment, follow these steps:

1. Set up the system hostname by running the following command:

sudo hostnamectl set-hostname node1

2. Add the fully qualified domain name (FQDN) configuration to the ‘/etc/hosts’ file:

echo "192.168.5.50 node1.example.lan node1" >> /etc/hosts

3. Verify the FQDN of your server with the following command:

hostname -f

4. Disable swap on your system by running the following commands:

sudo sed -i '/ swap / s/^(.*)$/#1/g' /etc/fstab
sudo swapoff -a

5. Verify the swap status with the following command:

free -m

6. Increase the max memory maps on your system by adding the parameter ‘vm.maxmapcount=262144′ to the ‘/etc/sysctl.conf’ file:

sudo echo "vm.max_map_count=262144" >> /etc/sysctl.conf
sudo sysctl -p

7. Verify the max memory maps with the following command:

cat /proc/sys/vm/max_map_count

With the system set up, we can now proceed to download and install OpenSearch.

Downloading OpenSearch

In this step, we will download the OpenSearch package and set up the installation directory. Follow these steps:

1. Create a new system user ‘opensearch’ by running the following command:

sudo adduser --system --shell /bin/bash -U 10001 --no-create-home opensearch

2. Create a new group ‘opensearch’ by running the following command:

sudo groupadd opensearch

3. Add the user ‘opensearch’ to the group ‘opensearch’ with the following command:

sudo usermod -aG opensearch opensearch

4. Create a new home directory ‘/home/opensearch’ and change its ownership to the ‘opensearch’ user:

mkdir -p /home/opensearch
sudo chown -R opensearch /home/opensearch

5. Download the OpenSearch package by running the following command:

wget https://artifacts.opensearch.org/releases/bundle/opensearch/2.4.1/opensearch-2.4.1-linux-x64.tar.gz

6. Extract the OpenSearch package to ‘/opt/opensearch’ and change its ownership to the ‘opensearch’ user:

tar xf opensearch-2.4.1-linux-x64.tar.gz
mv opensearch-2.4.1 /opt/opensearch
sudo chown -R opensearch /opt/opensearch

Now that we have downloaded and set up the OpenSearch package, we can proceed to configure OpenSearch.

Configuring OpenSearch

In this step, we will configure OpenSearch to run on a specific IP address, enable security plugins, and set up the maximum heap memory. Follow these steps:

1. Open the OpenSearch config file ‘config/opensearch.yml’ with a text editor:

sudo nano /opt/opensearch/config/opensearch.yml

2. Add the following lines to the file to bind OpenSearch to a specific IP address, configure it for single-node deployment, and enable the security plugins:

# Bind OpenSearch to interface or IP address
network.host: 192.168.5.50

# OpenSearch deployment type
discovery.type: single-node

# Re-enable security plugins
plugins.security.disabled: false

3. Open the OpenSearch JVM options file ‘config/jvm.options’:

sudo nano /opt/opensearch/config/jvm.options

4. Adjust the max heap memory for the OpenSearch process by modifying the ‘-Xms’ and ‘-Xmx’ parameters. For example, to allocate 2GB of memory, use the following values:

-Xms2g
-Xmx2g

5. Set up the environment variable ‘OPENSEARCHJAVAHOME’ for the current session:

export OPENSEARCH_JAVA_HOME=/opt/opensearch/jdk
echo $OPENSEARCH_JAVA_HOME

With OpenSearch now configured, we can generate TLS certificates to secure the deployment.

Generating TLS Certificates

TLS certificates are essential for securing your OpenSearch deployment. In this step, we will generate the necessary certificates. Follow these steps:

1. Create a new directory ‘/opt/opensearch/config/certs’ to store the TLS certificates:

mkdir -p /opt/opensearch/config/certs
cd /opt/opensearch/config/certs

2. Generate a private key for the root CA certificates:

openssl genrsa -out root-ca-key.pem 2048

3. Generate a self-signed root CA certificate:

openssl req -new -x509 -sha256 -key root-ca-key.pem -subj "/C=CA/ST=ONTARIO/L=TORONTO/O=ORG/OU=UNIT/CN=ROOT" -out root-ca.pem -days 730

4. Generate the admin certificate private key:

openssl genrsa -out admin-key-temp.pem 2048

5. Convert the admin private key to PKCS#8 format:

openssl pkcs8 -inform PEM -outform PEM -in node1-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out node1-key.pem

6. Generate the admin CSR from the private key:

openssl req -new -key node1-key.pem -subj "/C=CA/ST=ONTARIO/L=TORONTO/O=ORG/OU=UNIT/CN=node1.exampla.lan" -out node1.csr

7. Sign the admin CSR with the root CA certificate and private key:

openssl x509 -req -in admin.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out admin.pem -days 730

8. Generate the node private key:

openssl genrsa -out node1-key-temp.pem 2048

9. Convert the node private key to PKCS#8 format:

openssl pkcs8 -inform PEM -outform PEM -in node1-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out node1-key.pem

10. Create a new CSR for the node certificate:

openssl req -new -key node1-key.pem -subj "/C=CA/ST=ONTARIO/L=TORONTO/O=ORG/OU=UNIT/CN=node1.exampla.lan" -out node1.csr

11. Create a SAN extension file ‘node1.ext’ for the node certificate:

echo 'subjectAltName=DNS:node1.example.lan' > node1.ext

12. Sign the node certificate CSR with the root CA certificate and private key:

openssl x509 -req -in node1.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out node1.pem -days 730 -extfile node1.ext

13. Remove the temporary certificate, CSR, and SAN extension files:

rm *temp.pem *csr *ext

14. Convert the root CA certificate to .crt format:

openssl x509 -outform der -in root-ca.pem -out root-ca.crt

15. Add the root CA certificate to your Debian system:

sudo cp root-ca.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates

16. Set the proper permission and ownership for the certificates:

sudo chown -R opensearch /opt/opensearch/config/certs
sudo chmod 0700 /opt/opensearch/config/certs
sudo chmod 0600 /opt/opensearch/config/certs/*.pem
sudo chmod 0600 /opt/opensearch/config/certs/*.crt

With the TLS certificates generated, we can now add them to the OpenSearch configuration.

Adding TLS Certificates to OpenSearch

To add the TLS certificates to OpenSearch, follow these steps:

1. Create a new bash script ‘add.sh’ to add the certificates and TLS security plugin settings to the OpenSearch config file:

nano add.sh

2. Add the following lines to the ‘add.sh’ script, replacing the file paths and certificate details with your own:

#! /bin/bash

echo "plugins.security.ssl.transport.pemcert_filepath: /opt/opensearch/config/certs/node1.pem" | sudo tee -a /opt/opensearch/config/opensearch.yml
echo "plugins.security.ssl.transport.pemkey_filepath: /opt/opensearch/config/certs/node1-key.pem" | sudo tee -a /opt/opensearch/config/opensearch.yml
echo "plugins.security.ssl.transport.pemtrustedcas_filepath: /opt/opensearch/config/certs/root-ca.pem" | sudo tee -a /opt/opensearch/config/opensearch.yml
echo "plugins.security.ssl.http.enabled: true" | sudo tee -a /opt/opensearch/config/opensearch.yml
echo "plugins.security.ssl.http.pemcert_filepath: /opt/opensearch/config/certs/node1.pem" | sudo tee -a /opt/opensearch/config/opensearch.yml
echo "plugins.security.ssl.http.pemkey_filepath: /opt/opensearch/config/certs/node1-key.pem" | sudo tee -a /opt/opensearch/config/opensearch.yml
echo "plugins.security.ssl.http.pemtrustedcas_filepath: /opt/opensearch/config/certs/root-ca.pem" | sudo tee -a /opt/opensearch/config/opensearch.yml
echo "plugins.security.allow_default_init_securityindex: true" | sudo tee -a /opt/opensearch/config/opensearch.yml
echo "plugins.security.authcz.admin_dn:" | sudo tee -a /opt/opensearch/config/opensearch.yml
echo " - 'CN=A,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'" | sudo tee -a /opt/opensearch/config/opensearch.yml
echo "plugins.security.nodes_dn:" | sudo tee -a /opt/opensearch/config/opensearch.yml
echo " - 'CN=node1.example.lan,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'" | sudo tee -a /opt/opensearch/config/opensearch.yml
echo "plugins.security.audit.type: internal_opensearch" | sudo tee -a /opt/opensearch/config/opensearch.yml
echo "plugins.security.enable_snapshot_restore_privilege: true" | sudo tee -a /opt/opensearch/config/opensearch.yml
echo "plugins.security.check_snapshot_restore_write_privileges: true" | sudo tee -a /opt/opensearch/config/opensearch.yml
echo "plugins.security.restapi.roles_enabled: [\"all_access\", \"security_rest_api_access\"]" | sudo tee -a /opt/opensearch/config/opensearch.yml

3. Save and exit the ‘add.sh’ script.

Make the ‘add.sh’ script executable and execute it to add the TLS certificates and security plugin settings to the OpenSearch config file:

chmod +x add.sh
./add.sh

With the certificates added, we can now create a user on OpenSearch.

Creating User on OpenSearch

To create a user on OpenSearch, follow these steps:

1. Set up the environment variable for ‘OPENSEARCHJAVAHOME’ and make the OpenSearch security tools executable:

export OPENSEARCH_JAVA_HOME=/opt/opensearch/jdk
chmod 755 /opt/opensearch/plugins/opensearch-security/tools/*.sh

2. Move to the directory ‘/opt/opensearch/plugins/opensearch-security/tools’ and generate a hashed password for the OpenSearch user ‘admin’:

cd /opt/opensearch/plugins/opensearch-security/tools
./hash.sh

3. Generate another hashed password for the OpenSearch user ‘kibanaserver’:

./hash.sh

4. Open the ‘internal_users.yml’ file in the ‘/opt/opensearch/config/opensearch-security/’ directory:

sudo nano /opt/opensearch/config/opensearch-security/internal_users.yml

5. Remove the default user settings and replace them with the following lines, using the hashed passwords generated in step 2 and 3:

admin:
hash: "$2y$12$ChrsBPaDAJsuel.HXFi2Ie2Jn1MpdzXA4Nd1jeyXf65N97RDJc3Ky"
reserved: true
backend_roles:
- "admin"
description: "Admin user"

kibanaserver:
hash: "$2y$12$wIeuRDp5txoJ3d6.lyybJOPwoRaizuuBvlKKzAGdAiu.I/qaX8hXu"
reserved: true
description: "Demo OpenSearch Dashboards user"

6. Save and exit the ‘internal_users.yml’ file.

Ensure the ownership of the OpenSearch installation directory is set to the ‘opensearch’ user and log in as the ‘opensearch’ user:

sudo chown -R opensearch /opt/opensearch
su - opensearch

Move to the ‘/opt/opensearch/bin’ directory and run the OpenSearch installation:

cd /opt/opensearch/bin
./opensearch

Open another terminal, connect to your OpenSearch server, and log in as the ‘opensearch’ user:

su - opensearch

Move to the ‘/opt/opensearch/plugins/opensearch-security/tools’ directory and run the ‘securityadmin.sh’ script to apply the new changes to OpenSearch users:

cd /opt/opensearch/plugins/opensearch-security/tools
OPENSEARCH_JAVA_HOME=/opt/opensearch/jdk ./securityadmin.sh -h 192.168.5.50 -p 9200 -cd /opt/opensearch/config/opensearch-security/ -cacert /opt/opensearch/config/certs/root-ca.pem -cert /opt/opensearch/config/certs/admin.pem -key /opt/opensearch/config/certs/admin-key.pem -icl -nhnv

Terminate the OpenSearch process by pressing ‘Ctrl+C’ in the first terminal session, then rerun the OpenSearch server:

./opensearch

In the second terminal session, verify the authentication by running the following commands with the respective passwords:

curl https://192.168.5.50:9200 -u admin:password -k
curl https://192.168.5.50:9200 -u kibanaserver:kibanapass -k

If the authentication is successful, you should see detailed information about your OpenSearch server.

With the users created and OpenSearch running as a systemd service, we can now proceed to install OpenSearch Dashboards.

Installing OpenSearch Dashboards

To install OpenSearch Dashboards, follow these steps:

1. Download the OpenSearch Dashboards package:

wget https://artifacts.opensearch.org/releases/bundle/opensearch-dashboards/2.4.1/opensearch-dashboards-2.4.1-linux-x64.tar.gz

2. Extract the OpenSearch Dashboards package to ‘/opt/opensearch-dashboards’ and change its ownership to the ‘opensearch’ user:

tar xf opensearch-dashboards-2.4.1-linux-x64.tar.gz
mv opensearch-dashboards-* /opt/opensearch-dashboards
sudo chown -R opensearch /opt/opensearch-dashboards

3. Move to the ‘/opt/opensearch-dashboards’ directory and open the OpenSearch Dashboards config file ‘config/opensearch_dashboards.yml’:

cd /opt/opensearch-dashboards
sudo nano config/opensearch_dashboards.yml

4. Uncomment the following lines and replace the values with your own:

# OpenSearch Dashboards is served by a back end server. This setting specifies the port to use.
server.port: 5601

# Specifies the address to which the OpenSearch Dashboards server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "192.168.5.50"

# OpenSearch connection details
opensearch.hosts: [https://192.168.5.50:9200]
opensearch.ssl.verificationMode: none
opensearch.username: kibanaserver
opensearch.password: kibanapass

5. Save and exit the ‘config/opensearch_dashboards.yml’ file.

With OpenSearch Dashboards installed, we can now set it up as a systemd service.

Running OpenSearch Dashboards as a Systemd Service

To run OpenSearch Dashboards as a systemd service, follow these steps:

1. Create a new systemd service file ‘/etc/systemd/system/opensearch-dashboards.service’:

sudo nano /etc/systemd/system/opensearch-dashboards.service

2. Add the following lines to the file:

[Unit]
Description=OpenSearch-Dashboards
Wants=network-online.target
After=network-online.target opensearch.service

[Service]
Type=simple
User=opensearch
Environment=NODE_ENV=production
Environment=CONFIG_PATH=/opt/opensearch-dashboards/config/opensearch_dashboards.yml
WorkingDirectory=/opt/opensearch-dashboards
ExecStart=/opt/opensearch-dashboards/bin/opensearch-dashboards
StandardOutput=journal
StandardError=inherit
Restart=on-failure

[Install]
WantedBy=multi-user.target

3. Save and exit the file.

Reload the systemd manager to apply the changes:

sudo systemctl daemon-reload

Start and enable the OpenSearch Dashboards service:

sudo systemctl start opensearch-dashboards
sudo systemctl enable opensearch-dashboards

Verify the status of the OpenSearch Dashboards service:

sudo systemctl status opensearch-dashboards

If the service is active and running, you have successfully installed and configured OpenSearch Dashboards.

Accessing OpenSearch Dashboards

To access OpenSearch Dashboards, follow these steps:

1. Open your web browser and enter the IP address of your OpenSearch Dashboards server with port 5601 (e.g., http://192.168.5.50:5601).
2. You will be greeted with the OpenSearch Dashboards login page. Enter your username and password (e.g., ‘kibanaserver’) and click ‘Log in’.
3. Once logged in, you will see the OpenSearch Dashboards homepage with the message ‘Welcome to OpenSearch Dashboards’.

Congratulations! You have successfully installed and configured OpenSearch and OpenSearch Dashboards on your Debian 11 server. You can now explore the various features and functionalities offered by OpenSearch to analyze and visualize your data.

In conclusion, OpenSearch provides a powerful search, analytics, and visualization suite that enables businesses to harness the full potential of their data. By following this comprehensive guide, you have successfully set up OpenSearch on your Debian 11 server, secured it with TLS certificates, created users, and installed OpenSearch Dashboards for data visualization. With OpenSearch and OpenSearch Dashboards, you can unlock actionable insights and make data-driven decisions for your business.

For reliable and scalable cloud hosting solutions, consider Shape.host’s Cloud VPS services. Shape.host offers efficient and secure cloud hosting solutions, empowering businesses with the infrastructure they need to succeed.

Start your OpenSearch journey today with Shape.host’s Cloud VPS services.