26.6 C
Jaipur
Tuesday, September 21, 2021

How to Install Graylog Server on Ubuntu 20.04

Must read

All systems, applications, and software generate information that is stored in files called logs. These logs must be constantly monitored to ensure the proper functioning of the system in question and prevent errors or security risks. They are often scattered over several servers and, as the volume of data increases, their management becomes more and more complex.

Graylog is a free and open-source enterprise-grade log management system that comprises Elasticsearch, MongoDB, and a Graylog server. It consists of the main server, which receives data from its clients installed on different servers, as well as a web interface, which is used to view the data collected by the server. Graylog is a similar tool like Splunk and LogStash.

In this tutorial, we learn how to install Graylog 4 on Ubuntu 20.04. We will also secure the Graylog server with an SSL certificate using Let’s Encrypt.

Requirements:

  • Ubuntu 20.04 machine or later
  • Hardware – 4 CPU Cores, 8 GB RAM, SSD Hard Disk with High IOPS for Elasticsearch Log Storage
  • Elasticsearch
  • MongoDB
  • Oracle Java SE 8 (OpenJDK 8 or above)

Step1: Install MongoDB on Ubuntu 20.04

Graylog uses MongoDB to store the configuration data such as streams, alerts, users, settings, etc. Only the metadata is stored and doesn’t store the logs data. Let’s first install MongoDB on Ubuntu 20.04.

First, install all the prerequisites packages:

$ sudo apt update
$ sudo apt install apt-transport-https openjdk-11-jre-headless uuid-runtime pwgen dirmngr gnupg wget

Now import the public GPG key:

$ wget -qO - https://www.mongodb.org/static/pgp/server-5.0.asc | sudo apt-key add -

To create a source list file, type:

$ sudo touch /etc/apt/sources.list.d/mongodb-org-5.0.list

Now, add the repository source for Ubuntu 20.04:

$ echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/5.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-5.0.list

Now, update the apt index:

$ sudo apt update 

To install MongoDB on Ubuntu, type:

$ sudo apt install mongodb-org

MongoDB service will not start automatically after the installation process is complete. To start the service and enable the service, type:

$ sudo systemctl enable --now mongod.service

Verify the status of the MongoDB service:

$ sudo systemctl status mongod.service

Output:

● mongod.service - MongoDB Database Server
      Loaded: loaded (/lib/systemd/system/mongod.service; disabled; vendor preset: enabled)
      Active: active (running) since Sun 2021-08-22 04:47:46 UTC; 3s ago
        Docs: https://docs.mongodb.org/manual
    Main PID: 17965 (mongod)
      Memory: 66.1M
      CGroup: /system.slice/mongod.service
              └─17965 /usr/bin/mongod --config /etc/mongod.conf
 Aug 22 04:47:46 li175-223 systemd[1]: Started MongoDB Database Server.

As the output shows MongoDB started and the service is up and running.

You can check the version of MongoDB that is installed

$ sudo mongod --version
db version v5.0.2
 Build Info: {
     "version": "5.0.2",
     "gitVersion": "6d9ec525e78465dcecadcff99cce953d380fedc8",
     "openSSLVersion": "OpenSSL 1.1.1f  31 Mar 2020",
     "modules": [],
     "allocator": "tcmalloc",
     "environment": {
         "distmod": "ubuntu2004",
         "distarch": "x86_64",
         "target_arch": "x86_64"
     }
 }

The output shows we have installed MongoDB version 5.0.2.

Step 2: Install Elasticsearch on Ubuntu 20.04

Graylog stores all the log data in Elasticsearch. Refer to Graylog’s official website for the supported version of elasticseach.

Elasticseach need Java, it comes with Java bundled version of OpenJDK

Import the repository’s GPG key:

$ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

Next, add the Elasticsearch repository:

# echo "deb https://artifacts.elastic.co/packages/oss-7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list

Now update the repository

$ sudo apt update

Finally, install the opensource version of elasticsearch by typing:

$ sudo apt install elasticsearch-oss

Modify the elasticsearch configuration file to set some information as the cluster name, the IP address on which the elasticsearch is listening, and the port number

$ sudo vi /etc/elasticsearch/elasticsearch.yml
cluster.name: graylog-server
network.host: 0.0.0.0
http.port: 9200
discovery.zen.ping.unicast.hosts: ["127.0.0.1"]
action.auto_create_index: false

Elasticsearch main configuration files and directory are :

Data is stored – /var/lib/elasticsearch directory.
Configuration files – /etc/elasticsearch directory
Java start-up options – /etc/default/elasticsearch file

Elasticsearch comes with a bundled version of OpenJDK. To use your own version of Java set the ES_JAVA_HOME environment variable.

You can enable and start the Elasticsearch service using one command:

$ sudo systemctl enable --now elasticsearch.service

Now verify service is running properly using the following command:

$ sudo systemctl status elasticsearch.service

Output:

● elasticsearch.service - Elasticsearch
      Loaded: loaded (/lib/systemd/system/elasticsearch.service; disabled; vendor preset: enabled)
      Active: active (running) since Sun 2021-08-22 12:38:24 UTC; 11min ago
        Docs: http://www.elastic.co
    Main PID: 19502 (java)
       Tasks: 41 (limit: 1071)
      Memory: 833.2M
      CGroup: /system.slice/elasticsearch.service
              ├─19502 /bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negat>
              └─19565 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller
 Aug 22 12:38:24 li663-124 systemd[1]: Started Elasticsearch.

Press q to exit, back to command prompt.

Elasticsearch by default works on the port 9200, you should open it on the firewall.

$ sudo ufw allow 9200

You can check if it’s working

$ sudo curl -XGET 'http://localhost:9200'
"name" : "Ubuntunode",
   "cluster_name" : "graylog-server",
   "cluster_uuid" : "sz3jP3rKTPWZlasWwD-rBg",
   "version" : {
     "number" : "7.10.2",
     "build_flavor" : "oss",
     "build_type" : "deb",
     "build_hash" : "747e1cc71def077253878a59143c1f785afa92b9",
     "build_date" : "2021-01-13T00:42:12.435326Z",
     "build_snapshot" : false,
     "lucene_version" : "8.7.0",
     "minimum_wire_compatibility_version" : "6.8.0",
     "minimum_index_compatibility_version" : "6.0.0-beta1"
   },
   "tagline" : "You Know, for Search"
 }

With the output of the command, you can see the line "number": "7.10.2" show the elasticsearch version.

Step 3: Install Graylog on Ubuntu 20.04

Graylog collects the logs from the various inputs and provides the output to the web interface that it offers to manage the logs.

First, we will need to download the Graylog Repo file using the wget command:

$ wget https://packages.graylog2.org/repo/packages/graylog-4.1-repository_1-3_all.deb

To configure the Graylog repository, type:

$ sudo dpkg -i graylog-4.1-repository_1-3_all.deb 

Now we can install the Graylog server

$ apt update
$ sudo apt install graylog-server

Enable and start the graylog server service

$ sudo systemctl enable --now graylog-server.service

Edit the Configuration File

Now we need to set the password_secret and the hast value of your root graylog root_password_sha2.

To set a secret to secure the user passwords, you can use the pwgen command:

$ pwgen -N 1 -s 96
8KOikhjxetTdcJ7qhtPlgtLgBqCX5y4vL36Ig423pqUQn32QatiecCcYol5UEw3XDzwgWDvcFOmMBJzkBZKC52aEpBrItwke

Now, edit the configuration file to set the password

$ sudo vi /etc/graylog/server/server.conf
password_secret = RlTRqWSBENLKeg89iAWlxSaf1zfqLvBW7VX5SH1d2ji3RMKyeXb8bmfOXLl3GaWkxp9oDRfvbjXiEr36AFd6T9CMmnjdG7dn

We must now define the hash value of your Graylog root password account

$ echo -n GraylogRootPassword | shasum -a 256
4b09467e174a03d5ebd720d514f57783ad1e03b4877fff5e0dc45356340ab215  -

Now you can copy and paste it

sudo vi /etc/graylog/server/server.conf
root_password_sha2 = 4b09467e174a03d5ebd720d514f57783ad1e03b4877fff5e0dc45356340ab215

You can add additional information as the Graylog root email address and the network interface used by the Graylog HTTP interface

root_email = "[email protected]"
root_timezone = UTC
http_bind_address = 0.0.0.0:9000

You should open the ports on your firewall

$ sudo ufw allow 9000

Enable and start the graylog server service

$ sudo systemctl enable --now graylog-server.service

You can find the log data for Graylog /var/log/graylog-server/server.log and useful for debugging or when the server won’t start.

You can see the homepage by entering the public ip of your server and the port number from your browser ie http://<server-ip>:9000/. The default root login name of graylog is admin and the password is the one used for the hash value

Step 4: Configuring Nginx as SSL Termination Proxy (Optional)

Its recommended to secure your Graylog web interface with HTTPS. Here will use Nginx as the reverse proxy and generate a free SSL certificate for the domain using Let’s Encrypt.

First, we will need to install Nginx, then cerbot for the let’s encrypt certificate.

Now let’s install Nginx

$ sudo apt install nginx

Now let’s allow Nginx on the firewall

$ sudo ufw allow 'Nginx Full'

Now let’s edit the content of our domain name configuration file. Don’t forget to replace the file name with your domain name

$ sudo vim /etc/nginx/sites-available/websitefortesting.com.conf
server {
         listen 80;
         server_name websitefortesting.com;
         add_header Strict-Transport-Security max-age=2592000;
         rewrite ^ https://$server_name$request_uri? permanent;
         access_log /var/log/nginx/websitefortesting.com.access.log combined;     
         error_log  /var/log/nginx/websitefortesting.com.error.log;
 }

Now let’s activate it

$ sudo ln -s /etc/nginx/sites-available/websitefortesting.com.conf /etc/nginx/sites-enabled/websitefortesting.com.conf

Then verify if the configuration is fine

$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Now we need to install cerbot with the package necessary for nginx

$ sudo apt install certbot python3-certbot-nginx

Now we need to run cerbot for Nginx

$ sudo certbot --nginx
 Saving debug log to /var/log/letsencrypt/letsencrypt.log
 Plugins selected: Authenticator nginx, Installer nginx
 Enter email address (used for urgent renewal and security notices) (Enter 'c' to
 cancel): [email protected]
 
 Please read the Terms of Service at
 https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
 agree in order to register with the ACME server at
 https://acme-v02.api.letsencrypt.org/directory
 
 (A)gree/(C)ancel: A
 
 Would you be willing to share your email address with the Electronic Frontier
 Foundation, a founding partner of the Let's Encrypt project and the non-profit
 organization that develops Certbot? We'd like to send you email about our work
 encrypting the web, EFF news, campaigns, and ways to support digital freedom.
 
 (Y)es/(N)o: N
 Which names would you like to activate HTTPS for?
 
 1: websitefortesting.com
 
 Select the appropriate numbers separated by commas and/or spaces, or leave input
 blank to select all options shown (Enter 'c' to cancel): 1
 Obtaining a new certificate
 Performing the following challenges:
 http-01 challenge for websitefortesting.com
Waiting for verification…
 Cleaning up challenges
 Deploying Certificate to VirtualHost /etc/nginx/sites-enabled/websitefortesting.com.conf
 Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
 
 1: No redirect - Make no further changes to the webserver configuration.
 2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
 new sites, or if you're confident your site works on HTTPS. You can undo this
 change by editing your web server's configuration.
 
 Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
 Redirecting all traffic on port 80 to ssl in /etc/nginx/sites-enabled/websitefortesting.com.conf
 
 Congratulations! You have successfully enabled https://websitefortesting.com
 You should test your configuration at:
 https://www.ssllabs.com/ssltest/analyze.html?d=websitefortesting.com
 
 IMPORTANT NOTES:
 Congratulations! Your certificate and chain have been saved at:
 /etc/letsencrypt/live/websitefortesting.com/fullchain.pem
 Your key file has been saved at:
 /etc/letsencrypt/live/websitefortesting.com/privkey.pem
 Your cert will expire on 2021-11-18. To obtain a new or tweaked
 version of this certificate in the future, simply run certbot again
 with the "certonly" option. To non-interactively renew all of
 your certificates, run "certbot renew"
 If you like Certbot, please consider supporting our work by:
 Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 Donating to EFF:                    https://eff.org/donate-le 

From the output, you can see the location of SSL certificates and the private key. You can use this information to configure SSL for Nginx.

$ sudo vi /etc/nginx/sites-available/websitefortesting.com.conf
server {
         listen 80;
         server_name websitefortesting.com;
         add_header Strict-Transport-Security max-age=2592000;
         rewrite ^ https://$server_name$request_uri? permanent;
         access_log /var/log/nginx/websitefortesting.com.access.log combined;
         error_log  /var/log/nginx/websitefortesting.com.error.log;
 }
 server {
         listen 443 ssl; # managed by Certbot
         server_name websitefortesting.com;
         ssl_certificate /etc/letsencrypt/live/websitefortesting.com/fullchain.pem; # managed by Certbot
         ssl_certificate_key /etc/letsencrypt/live/websitefortesting.com/privkey.pem; # managed by Certbot
         include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
         ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
     location / {             proxy_set_header Host $http_host;             proxy_set_header X-Forwarded-Host $host; 
proxy_set_header X-Forwarded-Server $host;  
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;             proxy_pass http://127.0.0.1:9000;   
  }
 }

Now let’s verify again with

$ sudo nginx -t

And restart the Nginx service

$ sudo systemctl restart nginx.service 

Now access your Graylog URL using the domain name with HTTPS.

After login, you can see your home page

Conclusion

In this tutorial, we learned how to install the Graylog server on Ubuntu 20.04, configured SSL using Nginx as a reverse proxy.

If you face any challenges during the setup process feel free to comment or ask any questions in the comments section.

Source link

- Advertisement -

More articles

LEAVE A REPLY

Please enter your comment!
Please enter your name here

- Advertisement -

Latest article