Analyse Jenkins build logs with Elasticsearch and Kibana

Nowadays Jenkins has been used in many companies to automate their build tasks and Elasticsearch to store documented data to visualization the data via Kibana.

Today we will see how to configure and analyse Jenkins build jobs with Elasticsearch and Kibana.

Step 1 :

Install Jenkins, Elasticsearch and Kibana,

Follow below links to install Jenkins, Elasticsearch and Kibana.

https://www.devopsart.com/2019/09/jenkins-installation-and-configuration.html

https://www.devopsart.com/2019/12/step-by-step-installation-of-latest-elk.html

eg :
Jenkins version : 2.190.1 and IP is, 192.168.56.133
Elasticsearch and Kibana version : 7.5.1 and IP is, 192.168.56.136

Step 2 :

Install  Logstash plugin in Jenkins.

https://plugins.jenkins.io/logstash

Login to Jenkins > Manage Jenkins > Plugin Manager > "Click Available Tab" > search  "Logstash" in filter box, then install it.



Step  3 :

Create Index in Elasticsearch,

Jenkins send the build job logs to this Elasticsearch Index.

curl -X PUT "Elasticsearch_IP:9200/Index_name?pretty"

eg :

curl -X PUT "192.168.56.136:9200/logstash?pretty"








Step 4 :

Configure logstash plugin in Jenkins,

Login to Jenkins > Manage Jenkins >  Configure System > find "Logstash" and do the following,

1.enable "Enable sending logs to an Indexer"
2.Indexer Type : Elastic Search
3.URI : http://Elastisearch_IP:9200/Index_name/type

eg : http://192.168.56.136:9200/logstash/jenkins

 If you have username and password, fill it and save the configuration.











Step 5 :

Create and Configure Jenkins Jobs to send build jobs to Elasticsearch,

Create a Job,

Login to Jenkins > New Item > Enter Job name > Choose the project type (I used Freestyle project)

Go to Build Environment part, click "Add build step" > choose "Execute Shell" > give any basic shell command. (Eg: free -m)

Configure Job to send logs to Elasticsearch,

In the same job, go to Post-build Actions > Choose "Send Console log to Logstash" Save it.

Now click "Build now" and check the logs in Elasticsearch by Chrome browser plugin "Elasticsearch Head"

example data,



Step 6 : 

Add Index Pattern in Kibana to Analyse the logs,

Login to Kibana console > Management >  Index Patterns >  Create Index Patterns, here we can see the Elasticsearch Index then create it.











Thats all, Successfully exported Jenkins build jobs to Elastisearch and visualized by Kibana to analyse the logs.