Great Graphite Monitoring on Ubuntu 22.04: Install & Configure

Graphite is an open-source tool that allows you to track and graph the performance of computer systems. Whether you want to monitor websites, applications, business services, or networked servers, Graphite provides you with the flexibility to gain both detailed representation and broad overviews of the performance and health of the metrics you are tracking. In this tutorial, we will guide you through the installation and configuration process of Graphite on a Ubuntu 22.04 server.

Prerequisites

Before we begin, make sure you have the following:

A server running Ubuntu 22.04.
A Fully Qualified Domain Name (FQDN) pointing to the server. For this tutorial, we will use the domain graphite.example.com.
A non-root user with sudo privileges.
The Uncomplicated Firewall (UFW) enabled and running.

Start by updating your system to ensure everything is up to date:

sudo apt update && sudo apt upgrade

Next, install the basic utility packages if they are not already installed:

sudo apt install wget curl nano unzip -y

Step 1 – Configure Firewall

Before installing any packages, we need to configure the firewall to allow HTTP and HTTPS connections. Check the status of the firewall:

sudo ufw status

You should see the following output:

Status: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere
OpenSSH(v6)                ALLOW       Anywhere(v6)

To allow HTTP and HTTPS ports, run the following commands:

sudo ufw allow http
sudo ufw allow https

Confirm the changes by checking the status again:

sudo ufw status

You should see the following output:

Status: active

To                            Action        From
--                            ------        ----
OpenSSH                       ALLOW       Anywhere
80/tcp                        ALLOW       Anywhere
443                           ALLOW       Anywhere
OpenSSH(v6)                   ALLOW       Anywhere(v6)
80/tcp(v6)                    ALLOW       Anywhere(v6)
443(v6)                       ALLOW       Anywhere(v6)

Step 2 – Install Required Packages

We will install Graphite using the PIP Python package manager. To begin, install the necessary packages:

sudo apt install vim python3-dev python3-pip libcairo2-dev libffi-dev build-essential -y

Step 3 – Install Graphite and Graphite Web

Once the required packages are installed, we can proceed with installing Graphite and Graphite Web. We will install Graphite in the /opt/graphite directory. Run the following commands to install Graphite:

export PYTHONPATH="/opt/graphite/lib/:/opt/graphite/webapp/"
sudo pip install --no-binary=:all: https://github.com/graphite-project/whisper/tarball/master
sudo pip install --no-binary=:all: https://github.com/graphite-project/carbon/tarball/master
sudo pip install --no-binary=:all: https://github.com/graphite-project/graphite-web/tarball/master

Step 4 – Install and Configure PostgreSQL

Next, we will install PostgreSQL, which is the database backend for Graphite. We will use the official APT repository for PostgreSQL. Run the following commands to add the PostgreSQL GPG key and repository:

curl https://www.postgresql.org/media/keys/ACCC4CF8.asc | gpg --dearmor | sudo tee /usr/share/keyrings/postgresql-key.gpg >/dev/null
sudo sh -c 'echo "deb [signed-by=/usr/share/keyrings/postgresql-key.gpg arch=amd64] http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
sudo apt update

Once the repository is added, install PostgreSQL and the necessary helper packages:

sudo apt install postgresql postgresql-contrib libpq-dev -y

Check the status of the PostgreSQL service to ensure it is running:

sudo systemctl status postgresql

If everything is working fine, you should see the following output:

? postgresql.service - PostgreSQL RDBMS
Loaded: loaded (/lib/systemd/system/postgresql.service; enabled; vendor preset: enabled)
Active: active (exited) since Tue 2022-09-27 10:09:35 UTC; 4s ago
Main PID: 4456 (code=exited, status=0/SUCCESS)
Tasks: 1 (limit: 1074)
Memory: 4.2M
CGroup: /system.slice/postgresql.service

Now, log in to the PostgreSQL shell:

sudo -u postgres psql

Create a database user for Graphite:

CREATE USER graphite WITH PASSWORD 'your_password';

Create a database for Graphite and give ownership to the graphite user:

CREATE DATABASE graphitedb WITH OWNER graphite;

Exit the PostgreSQL shell:

Step 5 – Configure Graphite Carbon and Web

The next step is to configure Graphite Carbon and Graphite Web.

Configure Carbon

Carbon is the storage backend for Graphite. It consists of three services: carbon-cache,carbon-relay, and carbon-aggregator. In this tutorial, we will focus on configuring carbon-cache.

Start by creating the carbon.conf file:

sudo cp /opt/graphite/conf/carbon.conf.example /opt/graphite/conf/carbon.conf

Next, create the storage schemas configuration:

sudo cp /opt/graphite/conf/storage-schemas.conf.example /opt/graphite/conf/storage-schemas.conf

Open the storage schema configuration file for editing:

sudo nano /opt/graphite/conf/storage-schemas.conf

Inside the file, you will find entries like:

[carbon]
pattern = ^carbon.
retentions = 60:90d

This entry specifies a pattern that matches a regular expression ^carbon. and retains the data with a retention policy of 60:90d. This means the data will be recorded every 60 seconds and stored for 90 days.

You can add your own entry for monitoring data points. Let’s take an example where our data point entries will start with the string test. Add the following entry before the default entry at the bottom of the file:

[test]
pattern = ^test.
retentions = 10s:10m,1m:1h

This entry will match any metrics beginning with test. It will store the data it collects two times, with varying detail. The first definition (10s:10m) will create a data point every ten seconds and store the data only for ten minutes. The second definition (1m:1h) will create a data point every minute and aggregate the data from the past minute to create the point. It will store the data at this level of detail for one hour.

Save the file by pressing Ctrl + X and entering Y when prompted.

Start the carbon-cache service:

sudo /opt/graphite/bin/carbon-cache.py start

Configure the Graphite Web

Next, we need to configure the Graphite web application.

Generate a secret key for the Graphite application. Copy the displayed key for later use:

python3 -c 'from django.core.management.utils import get_random_secret_key; print(get_random_secret_key())'

Create the web app settings file:

sudo cp /opt/graphite/webapp/graphite/local_settings.py.example /opt/graphite/webapp/graphite/local_settings.py

Open the local_settings.py file for editing:

sudo nano /opt/graphite/webapp/graphite/local_settings.py

Uncomment the SECRET_KEY variable and enter a random value for it:

SECRET_KEY = 'your-secret-key'

Uncomment the ALLOWED_HOSTS variable:

ALLOWED_HOSTS = ['*']

Uncomment the TIME_ZONE variable and set it to the appropriate value:

TIME_ZONE = 'Asia/Kolkata'

Uncomment the USE_REMOTE_USER_AUTHENTICATION variable and set it to True:

USE_REMOTE_USER_AUTHENTICATION = True

Change the database settings to match the details you provided earlier:

e the database settings to match the details you provided earlier:

DATABASES = {
    'default': {
        'NAME': 'graphitedb',
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'USER': 'graphite',
        'PASSWORD': 'your_password',
        'HOST': '127.0.0.1',
        'PORT': '',
    }
}

Save the file by pressing Ctrl + X and entering Y when prompted.

Install some prerequisites for Python’s PostgreSQL wrapper:

sudo pip install psycopg2-binary

Import the database schema:

sudo PYTHONPATH=/opt/graphite/webapp/ django-admin.py migrate --settings=graphite.settings

Collect the static files:

sudo PYTHONPATH=/opt/graphite/webapp/ django-admin.py collectstatic --settings=graphite.settings

Set the correct ownership settings:

sudo chown -R www-data:www-data /opt/graphite/storage/
sudo chown -R www-data:www-data /opt/graphite/static/
sudo chown -R www-data:www-data /opt/graphite/webapp/

Create a root user for login:

sudo PYTHONPATH=/opt/graphite/webapp/ django-admin.py createsuperuser --settings=graphite.settings

Follow the prompts to create a superuser. This user will be used later to log in to the Graphite application.

Step 6 – Configure Apache

Graphite ships with Apache configuration files by default. Install Apache server:

sudo apt install apache2 libapache2-mod-wsgi-py3 -y

Create the mod_wsgi file:

sudo cp /opt/graphite/conf/graphite.wsgi.example /opt/graphite/conf/graphite.wsgi

Copy the graphite example configuration file to the Apache location:

sudo cp /opt/graphite/examples/example-graphite-vhost.conf /etc/apache2/sites-available/graphite.conf

Open the Graphite configuration file for editing:

sudo nano /etc/apache2/sites-available/graphite.conf

Change the port number in the first line from 80 to 127.0.0.1:8080. This restricts access to the Graphite application from the local machine. Replace graphite.example.com with your own domain name:

<VirtualHost 127.0.0.1:8080>
    ServerName graphite.example.com

Add the following lines below the Alias /static/ /opt/graphite/static/ line:

<Directory /opt/graphite/static/>
    Require all granted
</Directory>

Save the file by pressing Ctrl + X and entering Y when prompted.

Disable the default virtual host and enable the Graphite virtual host file:

sudo a2dissite 000-default
sudo a2ensite graphite

Next, we need to tell Apache to listen on port 8080 and stop listening on port 80 because we will be using Nginx as a proxy server.

Open the file /etc/apache2/ports.conf for editing:

sudo nano /etc/apache2/ports.conf

Find the line Listen 80 and replace it with the following:

Listen 127.0.0.1:8080

Save the file by pressing Ctrl + X and entering Y when prompted.

Restart the Apache server:

sudo systemctl restart apache2

To verify that Graphite is working properly and is accessible, run the following command:

curl 127.0.0.1:8080

You should see the Graphite browser page HTML output.

Step 7 – Install Nginx

We will use Nginx as a proxy server for Apache. This allows us to take advantage of Nginx’s security features while using the existing Graphite configuration. Ubuntu 22.04 ships with an older version of Nginx, so we need to download the official Nginx repository to install the latest version.

Import Nginx’s signing key:

curl https://nginx.org/keys/nginx_signing.key | gpg --dearmor | sudo tee /usr/share/keyrings/nginx-archive-keyring.gpg > /dev/null

Add the repository for the Nginx stable version:

echo "deb [signed-by=/usr/share/keyrings/nginx-archive-keyring.gpg arch=amd64] http://nginx.org/packages/ubuntu $(lsb_release -cs) nginx" | sudo tee /etc/apt/sources.list.d/nginx.list

Update the system repositories:

sudo apt update

Install Nginx:

sudo apt install nginx -y

Verify the installation:

nginx -v

You should see the Nginx version number.

Start the Nginx server:

sudo systemctl start nginx

Step 8 – Install SSL

To secure our Graphite installation, we will use Certbot to generate an SSL certificate. Certbot is a free and open-source tool that simplifies the process of obtaining and renewing SSL certificates. We will use the Snapd version of Certbot, which is available by default on Ubuntu 22.04.

Ensure that your version of Snapd is up to date:

sudo snap install core
sudo snap refresh core

Install Certbot:

sudo snap install --classic certbot

Ensure that the Certbot command can be run by creating a symbolic link to the /usr/bin directory:

sudo ln -s /snap/bin/certbot/usr/bin/certbot

Generate an SSL certificate:

sudo certbot certonly --nginx --agree-tosc --no-eff-email --staple-ocsp --preferred-challenges http -m test@example.com -d graphite.example.com

Replace graphite.example.com with your own domain name. The command will download the certificate to the /etc/letsencrypt/live/graphite.example.com directory on your server.

Next, generate a Diffie-Hellman group certificate:

sudo openssl dhparam-dsaparam -out /etc/ssl/certs/dhparam.pem 4096

Check the Certbot renewal scheduler service:

sudo systemctl list-timers

You should see snap.certbot.renew.service as one of the services scheduled to run.

To check if the SSL renewal process is working correctly, perform a dry run:

sudo certbot renew --dry-run

If you see no errors, your certificate will renew automatically.

Step 9 – Configure Nginx

Open the Nginx configuration file for editing:

sudo nano /etc/nginx/nginx.conf

Add the following line before the line include /etc/nginx/conf.d/*.conf;:

server_names_hash_bucket_size 64;

Save the file by pressing Ctrl + X and entering Y when prompted.

Create and open the file /etc/nginx/conf.d/graphite.conf for editing:

sudo nano /etc/nginx/conf.d/graphite.conf

Paste the following code into the file:

server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name graphite.example.com;
access_log /var/log/nginx/graphite.access.log;
error_log /var/log/nginx/graphite.error.log;

# SSL
ssl_certificate /etc/letsencrypt/live/graphite.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/graphite.example.com/privkey.pem;
ssl_trusted_certificate /etc/letsencrypt/live/graphite.example.com/chain.pem;
ssl_session_timeout 5m;
ssl_session_cache shared:MozSSL:10m;
ssl_session_tickets off;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
ssl_ecdh_curve X25519:prime256v1:secp384r1:secp521r1;
ssl_stapling on;
ssl_stapling_verify on;
ssl_dhparam /etc/ssl/certs/dhparam.pem;
resolver 8.8.8.8;

location / {
proxy_set_header Connection "upgrade";
proxy_set_header Upgrade $http_upgrade;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://127.0.0.1:8080;
proxy_redirect off;
}
}

# Enforce HTTPS
server {
listen 80;
listen [::]:80;
server_name graphite.example.com;
return 301 https://$host$request_uri;
}

Replace graphite.example.com with your own domain name.

Save the file by pressing Ctrl + X and entering Y when prompted.

Verify the Nginx configuration file syntax:

sudo nginx -t

If the syntax is correct, restart the Nginx service:

sudo systemctl restart nginx

Step 10 – Access and Use Graphite

Visit the URL https://graphite.example.com in your browser to access the Graphite web interface.

Click the “Login” link on the top right to open the login page. Enter the superuser credentials you created earlier and press the login button to proceed.

You can now start using Graphite to monitor your systems. To add data, you can use the Carbon service we configured earlier. For example, you can add a data point with the following command:

echo "test.count 9 $(date +%s)" | nc -q 0127.0.0.12003

This command adds a data metric with a value of 9 to the system. You can add more data points by looping through values:

for i in 4 6 8 1 6 2; do
    echo "test.count $i $(date +%s)" | nc -q 0127.0.0.12003
    sleep6
done

Go back to the Graphite dashboard and navigate to Metrics > test > count from the left sidebar. You should see the data points you added.

Congratulations! You have successfully installed and configured Graphite on your Ubuntu 22.04 server.

Conclusion

Graphite is a powerful monitoring tool that allows you to track and graph the performance of your systems. By following this tutorial, you have learned how to install and configure Graphite on Ubuntu 22.04, along with Nginx as a proxy server.

Graphite provides you with the flexibility to monitor various metrics and gain insights into the health and performance of your systems. By combining Graphite with other tools like Grafana, you can create customized dashboards and visualize your data in a way that suits your specific needs.

If you have any questions or need further assistance, feel free to reach out to Shape.host for reliable and scalable Linux SSD VPS solutions. Our team of experts is always ready to help you optimize and secure your cloud hosting environment.