Setup EKS Cluster using eksctl and Deploy Springboot Microservices into EKS using Jenkins Pipeline

Deploy Springboot Microservices App into Amazon EKS Cluster using Jenkins Pipeline and Kubectl CLI Plug-in | Containerize Springboot App and Deploy into EKS Cluster using Jenkins Pipeline

We will learn how to automate springboot microservices builds using Jenkins pipeline and Deploy into AWS EKS Cluster with help of Kubernetes CLI plug-in.

We will use Springboot Microservices based Java application. I have already created a repo with source code + Dockerfile. The repo also have Jenkinsfile for automating the following:

- Automating builds using Jenkins
- Automating Docker image creation
- Automating Docker image upload into AWS ECR
- Automating Docker Containers Deployments to Kubernetes Cluster

Source  Code for this project is here:

Make sure you fork my repo  https://github.com/imHarry404/springboot-app.git

Pre-requisites:
1. Amazon EKS Cluster is setup and running. Click here to learn how to create Amazon EKS cluster.

2. Create ECR repo in AWS

3. Jenkins Master is up and running

4. Docker installed on Jenkins instance 

5. Docker, Docker pipeline and Kubernetes CLI plug-ins are installed in Jenkins

6. Install kubectl on your instance

Step # 1 - Create Maven3 variable under Global tool configuration in Jenkins

Make sure you create Maven3 variable under Global tool configuration. 

Step #2 - Create Credentials for connecting to Kubernetes Cluster using kubeconfig

Click on Add Credentials, use Kubernetes configuration from drop down.

use secret file from drop down.

execute the below command to login as jenkins user.

sudo su - jenkins

you should see the nodes running in EKS cluster.

kubectl get nodes

Execute the below command to get kubeconfig info, copy the entire content of the file:

cat /var/lib/jenkins/.kube/config

Open your text editor or notepad, copy and paste the entire content and save in a file.

We will upload this file.

Enter ID as K8S and choose File and upload the file and save.

Enter ID as K8S and choose enter directly and paste the above file content and save.

Step # 3 - Create a pipeline in Jenkins

Create a new pipeline job.

Step # 4 - Copy the pipeline code from below
Make sure you change red highlighted values below as per your settings:
Your docker user id should be updated.
your registry credentials ID from Jenkins from step # 1 should be copied

pipeline {

    agent any

    tools {

        maven "Maven3"

    }

    

    environment {

     repository = "456137991393.dkr.ecr.ap-south-1.amazonaws.com/harry"   

     

        

        

    }

    stages {

        stage('Git Checkout') {

            steps {

                script {

                    // Check out code from the specified Git repository

                    git branch: 'main', url: 'https://github.com/imHarry404/springboot-app.git'

                }

            }

        }

        stage('Build Jar') {

            steps {

                script {

                    

                    sh "mvn clean install"

                }

            }

        }    

        stage('Build Docker Image') {

            steps {

                script {

                    docker.build repository

                }

            }

        }

        stage('Push into ECR') {

            steps {

                script {

                    sh "aws ecr get-login-password --region ap-south-1 | docker login --username AWS --password-stdin 456137991393.dkr.ecr.ap-south-1.amazonaws.com"

                    sh "docker push 456137991393.dkr.ecr.ap-south-1.amazonaws.com/harry:latest"

                }

            }

        }

        

        stage('Deploy to k8s') {

            steps {

                script {

                    withKubeConfig(caCertificate: '', clusterName: '', contextName: '', credentialsId: 'k8s', namespace: '', restrictKubeConfigAccess: false, serverUrl: '') {

                        sh "kubectl apply -f eks-deploy-from-ecr.yaml"

                    }

                }

            }

        }

    }

}

Step # 5 - Build the pipeline

Once you create the pipeline and changes values per your configuration, click on Build now:

Step # 6 - Verify deployments to K8S

kubectl get pods

kubectl get deployments

kubectl get services

If you see any errors after deploying the pods, you can check the pod logs.

kubectl logs <pod_name>

Steps # 7 - Access SpringBoot App in K8S cluster

Once build is successful, go to browser and enter master or worker node public ip address along with port number mentioned above

http://loadbalancer_ip_address

You should see page like below:

Note:

Make sure you fork my repo my-git-repo

and make changes in eks-deploy-k8s.yaml to pull Docker image from your AWS ECR repo.

Connect with me 😊

https://www.linkedin.com/in/imharry404/

Connect Jenkins with Microsoft teams

 Connect Jenkins with Microsoft teams.

Create a team from scratch (public/ private  depends on you)

 

Give appropriate team name & description and create

 

Add member in your team using name/email.

 

Now add a channel for your team

 

Give name, description and channel type 

 

if you don't find add Connectors option, go to ... (view more apps) in teams and search for jenkins

 

{Select Add  (for first time it will ask to install & configure)}

Select Add then Add to a team 

 

you will find your team name there, select and click on set up a connector

 

Give name for your Jenkins connection,  it will generate a web-hook url and click on done.

Copy the web-hook url, it will be used later in pipeline setup.

 

Then you can see this message of setup in teams

 

Now create an EC2 instance and install setup jenkins on it and configure jenkins as well.

We need one plugin to be installed

Dashboard >> Manage Jenkins >> Plugins

Install this Office 365 plugin

 

Now create a pipeline.

 

You will see  office 365 connector

 

click on Add Webhook

Paste the url that you copied which configuration.

 

in pipeline select Hello world -> Apply -> Save

 

Build the first job

 

Log of the triggered build

 

in MS-Teams you can see a new message of success.

 

what if the build fails.

go to configure -> in pipeline script make some mistake and save.

now trigger the build again

 

you can see new message of failed, and you can check the build as well.

 

you can customize the notifications in office 365 connector section 

 

Thanks.

Connect with me 😊

https://www.linkedin.com/in/imharry404/

Spring Boot Shopping Cart Web App: Project with git-repo, all files and steps

 Building a Spring Boot Shopping Cart Web App: A Practical Guide

1. Creating VM Machines on AWS

In the initial phase of our journey, we set up a robust development environment on Amazon Web Services (AWS). This involved creating virtual machines to serve as the foundation for our project. The detailed steps and snapshots of required vm machines on AWS can be found below.

 2. Kubernetes (K8s) Setup

With our VMs in place, the next step was to configure Kubernetes for efficient container orchestration. The steps for setting up K8s and joining nodes to the cluster are outlined.

https://github.com/imHarry404/Ekart/blob/main/k8s-setup.txt

 3. Configuring Security Groups with Ports

Ensuring secure communication within our environment is paramount. We established security groups to control incoming and outgoing traffic, implementing port restrictions for enhanced safety.

4. Building Jenkins Pipelines for Success

Our Jenkins setup included defining pipelines to automate the build, test, and deployment phases. A successful pipeline is outlined in detail.

🛠️ 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲 𝗦𝘁𝗮𝗴𝗲𝘀:

𝗗𝗲𝗳𝗶𝗻𝗲 𝗧𝗼𝗼𝗹𝘀:

JDK, Maven, SonarQube - essential tools defined within our pipelines.

𝗖𝗼𝗺𝗽𝗶𝗹𝗲 𝘁𝗵𝗲 𝗖𝗼𝗱𝗲:

Transforming source code into executable binaries with precision.

𝗥𝘂𝗻 𝗨𝗻𝗶𝘁 𝗧𝗲𝘀𝘁𝘀:

Ensuring the integrity of the codebase with thorough unit testing.

𝗦𝘁𝗮𝘁𝗶𝗰 𝗖𝗼𝗱𝗲 𝗔𝗻𝗮𝗹𝘆𝘀𝗶𝘀:

Leveraging SonarQube for in-depth code quality assessments. 🔍

𝗩𝘂𝗹𝗻𝗲𝗿𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗦𝗰𝗮𝗻:

A critical step! Scanning dependencies for potential vulnerabilities to fortify our code.

𝗕𝘂𝗶𝗹𝗱/𝗣𝗮𝗰𝗸𝗮𝗴𝗲 𝘁𝗵𝗲 𝗔𝗽𝗽𝗹𝗶𝗰𝗮𝘁𝗶𝗼𝗻:

Bringing our application to life with a robust build process.

𝗣𝘂𝗯𝗹𝗶𝘀𝗵 𝗔𝗿𝘁𝗶𝗳𝗮𝗰𝘁𝘀:

Storing our precious artifacts securely in Nexus for future use.

𝗕𝘂𝗶𝗹𝗱 & 𝗧𝗮𝗴 𝗗𝗼𝗰𝗸𝗲𝗿 𝗜𝗺𝗮𝗴𝗲:

Crafting Docker images and assigning meaningful tags.

𝗦𝗰𝗮𝗻 𝗖𝗼𝗻𝘁𝗮𝗶𝗻𝗲𝗿 𝗜𝗺𝗮𝗴𝗲𝘀:

Ensuring Docker images are free from vulnerabilities for a secure deployment.

𝗣𝘂𝘀𝗵 𝘁𝗼 𝗜𝗺𝗮𝗴𝗲 𝗥𝗲𝗽𝗼𝘀𝗶𝘁𝗼𝗿𝘆:

Safely storing our Docker images in a repository for accessibility.

𝗬𝗔𝗠𝗟 𝗠𝗮𝗻𝗶𝗳𝗲𝘀𝘁𝘀:

Crafting YAML manifest files for Kubernetes deployment - the heart of our scalable architecture.

𝗗𝗲𝗽𝗹𝗼𝘆 𝘁𝗼 𝗞𝘂𝗯𝗲𝗿𝗻𝗲𝘁𝗲𝘀:

The grand finale! Deploying our application to Kubernetes for efficient orchestration.

5. Exploring the Working Project Homepage

The heart of our endeavor is the fully functional Spring Boot Shopping Cart Web App. This section provides insights into the features, user experience, and the seamless integration of technologies. Explore the working project homepage

access it using node-ip:port (given in pipeline log) 

then login using admin & admin

6. Nexus Repository Snapshots for Dependency Management

Efficient dependency management is crucial for a stable and reproducible build environment. We utilized Nexus Repository to store snapshots of project dependencies. Learn more about our Nexus Repository setup.

7. All Files Present on Repository

For your convenience, all the files associated with this project, including configurations, scripts, and code, are available in the project repository. You can access the repository and its contents 

https://github.com/imHarry404/Ekart/tree/main

8. Image present on DockerHub 

 Conclusion

This blog post serves as a comprehensive guide through the process of building a Spring Boot Shopping Cart Web App. Feel free to explore the provided links and snapshots to gain a deeper understanding of each step in this exciting development journey. Happy coding!

Connect with me 😊

https://www.linkedin.com/in/imharry404/

Kubernetes common failures issues.

Kubernetes is a container orchestration tool ,and while working with it errors are part of it , but it would be time saving if we knew where to debug & fix the issue, as for a beginner things might get overwhelming.

Here are 5 common k8s failures and how to fix them 🛠️

𝟭) 𝗜𝗺𝗮𝗴𝗲-𝗽𝘂𝗹𝗹 𝗯𝗮𝗰𝗸 𝗼𝗳𝗳 ~ Check for 𝙄𝙢𝙖𝙜𝙚 𝙥𝙪𝙡𝙡 𝙥𝙤𝙡𝙞𝙘𝙮 , 𝙥𝙚𝙧𝙢𝙞𝙨𝙨𝙞𝙤𝙣 𝙩𝙤 𝙥𝙪𝙡𝙡 𝙛𝙧𝙤𝙢 𝙧𝙚𝙥𝙤𝙨𝙞𝙩𝙤𝙧𝙮,𝙘𝙤𝙧𝙧𝙚𝙘𝙩 𝙞𝙢𝙖𝙜𝙚 𝙣𝙖𝙢𝙚 𝙖𝙡𝙤𝙣𝙜 𝙬𝙞𝙩𝙝 𝙩𝙖𝙜.

𝙐𝙨𝙚𝙛𝙪𝙡 𝙘𝙤𝙢𝙢𝙖𝙣𝙙𝙨 🔮 

 Kubectl describe po <podname>

 Kubectl get po <podname>

 Kubectl apply -f <deployment file name>

2) 𝘾𝙧𝙖𝙨𝙝-𝙇𝙤𝙤𝙥 𝙗𝙖𝙘𝙠 𝙤𝙛𝙛~ Check for 𝘾𝙤𝙧𝙧𝙚𝙘𝙩 𝙞𝙢𝙖𝙜𝙚 𝙣𝙖𝙢𝙚 𝙖𝙡𝙤𝙣𝙜 𝙬𝙞𝙩𝙝 𝙩𝙖𝙜 , 𝙚𝙣𝙤𝙪𝙜𝙝 𝙧𝙚𝙨𝙤𝙪𝙧𝙘𝙚 𝙘𝙤𝙣𝙨𝙩𝙧𝙖𝙞𝙣𝙩𝙨,𝙢𝙞𝙨𝙘𝙤𝙣𝙛𝙞𝙜𝙪𝙧𝙖𝙩𝙞𝙤𝙣 𝙤𝙛 𝙚𝙣𝙫𝙞𝙧𝙤𝙣𝙢𝙚𝙣𝙩 𝙫𝙖𝙧𝙞𝙖𝙗𝙡𝙚𝙨 , 𝙖𝙥𝙥𝙡𝙞𝙘𝙖𝙩𝙞𝙤𝙣 𝙛𝙖𝙞𝙡𝙪𝙧𝙚 𝙗𝙚𝙘𝙖𝙪𝙨𝙚 𝙤𝙛 ( 𝙛𝙖𝙞𝙡 𝙩𝙤 𝙗𝙪𝙞𝙡𝙙 𝙟𝙖𝙧 𝙛𝙞𝙡𝙚𝙨 , 𝙞𝙨𝙨𝙪𝙚𝙨 𝙬𝙝𝙞𝙡𝙚 𝙗𝙪𝙞𝙡𝙙𝙞𝙣𝙜 𝙙𝙤𝙘𝙠𝙚𝙧 𝙞𝙢𝙖𝙜𝙚.

𝙐𝙨𝙚𝙛𝙪𝙡 𝙘𝙤𝙢𝙢𝙖𝙣𝙙𝙨 🔮 

 Kubectl describe po <podname>

 Kubectl logs <podname>

 Also to check if enough resources are allocated (memory) 

3) 𝙁𝙖𝙞𝙡𝙪𝙧𝙚 𝙬𝙞𝙩𝙝 𝙀𝙭𝙞𝙩 𝙘𝙤𝙙𝙚 1~ Check for 𝘼𝙥𝙥𝙡𝙞𝙘𝙖𝙩𝙞𝙤𝙣 𝙘𝙤𝙙𝙚 𝙘𝙧𝙖𝙨𝙝𝙚𝙨 , 𝙞𝙣𝙘𝙤𝙧𝙧𝙚𝙘𝙩 𝙚𝙣𝙫𝙞𝙧𝙤𝙣𝙢𝙚𝙣𝙩 𝙫𝙖𝙧𝙞𝙖𝙗𝙡𝙚𝙨, 𝙞𝙣𝙨𝙪𝙛𝙛𝙞𝙘𝙞𝙚𝙣𝙩 𝙛𝙞𝙡𝙚 𝙥𝙚𝙧𝙢𝙞𝙨𝙨𝙞𝙤𝙣𝙨.

𝙐𝙨𝙚𝙛𝙪𝙡 𝙘𝙤𝙢𝙢𝙖𝙣𝙙𝙨 🔮 

 Kubectl logs <podname>

 Kubectl get po <podname>

 Kubectl apply -f <deployment file name>

Lookout for any exceptions in logs /missing variables at code level as well .

4) 𝙁𝙖𝙞𝙡𝙪𝙧𝙚 𝙬𝙞𝙩𝙝 𝙀𝙭𝙞𝙩 𝙘𝙤𝙙𝙚125~ Check for 𝙞𝙣𝙘𝙤𝙧𝙧𝙚𝙘𝙩 𝙛𝙞𝙡𝙚 𝙥𝙚𝙧𝙢𝙞𝙨𝙨𝙞𝙤𝙣𝙨 , 𝙚𝙭𝙘𝙚𝙥𝙩𝙞𝙤𝙣𝙨 𝙙𝙪𝙧𝙞𝙣𝙜 𝙗𝙤𝙤𝙩𝙞𝙣𝙜 𝙪𝙥 𝙤𝙛 𝙥𝙤𝙙

𝙐𝙨𝙚𝙛𝙪𝙡 𝙘𝙤𝙢𝙢𝙖𝙣𝙙𝙨 🔮 

 Kubectl logs <podname>

 Kubectl describe po <podname>

5) 𝙋𝙤𝙙/𝙉𝙤𝙙𝙚 𝙉𝙤𝙩 𝙍𝙚𝙖𝙙𝙮 ~ Check for 𝙉𝙚𝙩𝙬𝙤𝙧𝙠 𝘾𝙤𝙣𝙣𝙚𝙘𝙩𝙞𝙫𝙞𝙩𝙮 , 𝙚𝙣𝙤𝙪𝙜𝙝 𝙧𝙚𝙨𝙤𝙪𝙧𝙘𝙚 𝙖𝙡𝙡𝙤𝙘𝙖𝙩𝙞𝙤𝙣 ,𝙪𝙣𝙝𝙚𝙖𝙡𝙩𝙝𝙮 𝙥𝙧𝙤𝙘𝙚𝙨𝙨𝙚𝙨

𝙐𝙨𝙚𝙛𝙪𝙡 𝙘𝙤𝙢𝙢𝙖𝙣𝙙𝙨 🔮 

 Kubectl logs <podname>

 Kubectl get po <podname> and check for its state 

Increase system resource usage

Connect with me 😊

https://www.linkedin.com/in/imharry404/

📘 Top 12 Git Commands Cheatsheet

git init - Initialize a new Git repository.

git clone - Clone a remote repository to your local machine.

git status - Check the current state of your working directory.

git add - Stage changes for the next commit.

git commit - Record staged changes and create a snapshot.

git push - Upload local changes to a remote repository.

git pull - Fetch and merge changes from a remote repository.

git branch - List, create, or delete branches.

git checkout / git switch - Switch between branches or commits.

git merge - Integrate changes from one branch into another.

git diff - View differences between working directory and staging area.

git log - Display a chronological list of commits.

### Basic Commands:

1. **Initialize a Repository:**

   ```bash

   git init

   ```

2. **Clone a Repository:**

   ```bash

   git clone <repository_url>

   ```

3. **Add Changes:**

   ```bash

   git add <file(s)>

   ```

4. **Commit Changes:**

   ```bash

   git commit -m "Commit message"

   ```

### Branching:

5. **Create a New Branch:**

   ```bash

   git branch <branch_name>

   ```

6. **Switch to a Branch:**

   ```bash

   git checkout <branch_name>

   ```

   *(or use `git switch <branch_name>` in Git 2.23 and later)*

7. **Create and Switch to a New Branch:**

   ```bash

   git checkout -b <new_branch_name>

   ```

   *(or use `git switch -c <new_branch_name>` in Git 2.23 and later)*

8. **List Branches:**

   ```bash

   git branch

   ```

### Merging:

9. **Merge Branch into Current Branch:**

   ```bash

   git merge <branch_name>

   ```

### Remote Repositories:

10. **Add a Remote Repository:**

    ```bash

    git remote add <remote_name> <repository_url>

    ```

11. **Fetch Changes from a Remote Repository:**

    ```bash

    git fetch <remote_name>

    ```

12. **Pull Changes from a Remote Repository:**

    ```bash

    git pull <remote_name> <branch_name>

    ```

13. **Push Changes to a Remote Repository:**

    ```bash

    git push <remote_name> <branch_name>

    ```

### Undoing Changes:

14. **Discard Changes in Working Directory:**

    ```bash

    git checkout -- <file(s)>

    ```

15. **Undo Last Commit (Keep Changes in Working Directory):**

    ```bash

    git reset HEAD^

    ```

16. **Undo Last Commit (Discard Changes):**

    ```bash

    git reset --hard HEAD^

    ```

### Logging and Status:

17. **View Changes:**

    ```bash

    git status

    ```

18. **View Commit History:**

    ```bash

    git log

    ```

### Miscellaneous:

19. **Configure Git:**

    ```bash

    git config --global user.name "Your Name"

    git config --global user.email "your.email@example.com"

    ```

20. **Ignore Files:**

    Create a `.gitignore` file and list files/directories to be ignored.

Remember to replace placeholders like `<repository_url>`, `<branch_name>`, `<remote_name>`, etc., with your actual values. This cheat sheet covers basic Git commands, and Git offers more advanced features that you may explore as needed.

Connect with me 😊

https://www.linkedin.com/in/imharry404/

 

2048 game in few steps on AWS-EKS

 

Create an EKS cluster and deploy 2048 game into that cluster

==================================================

Task 1: Create an EKS cluster

=============================

Name: <yourname>-eks-cluster-1

Use K8S version 1.25

Create an IAM role 'eks-cluster-role' with 1 policy attached: AmazonEKSClusterPolicy

Create another IAM role 'eks-node-grp-role' with 3 policies attached: 

(Allows EC2 instances to call AWS services on your behalf.)

    - AmazonEKSWorkerNodePolicy

    - AmazonEC2ContainerRegistryReadOnly

    - AmazonEKS_CNI_Policy

Choose default VPC, Choose 2 or 3 subnets

Choose a security group which open the ports 22, 80, 8080

cluster endpoint access: public

# For VPC CNI, CoreDNS and kube-proxy, choose the default versions, For CNI, latest and default are 

# different. But go with default.

Click 'Create'. This process will take 10-12 minutes. Wait till your cluster shows up as Active. 

Task 2: Add Node Groups to our cluster

======================================

Now, lets add the worker nodes where the pods can run

Open the cluster > Compute > Add NodeGrp

Name: <yourname>-eks-nodegrp-1 

Select the role you already created

Leave default values for everything else

AMI - choose the default 1 (Amazon Linux 2)

change desired/minimum/maximum to 1 (from 2)

Enable SSH access. Choose a security group which allwos 22, 80, 8080

Choose default values for other fields 

Node group creation may take 2-3 minutes

Task 3: Authenticate to this cluster

===================================

Reference:

https://docs.aws.amazon.com/eks/latest/userguide/create-kubeconfig.html

Open cloudshell

# Type on your AWS CLI window 

aws sts get-caller-identity

# observe your account and user id details

# Create a  kubeconfig file where it stores the credentials for EKS:

# kubeconfig configuration allows you to connect to your cluster using the kubectl command line.

aws eks update-kubeconfig --region region-code --name my-cluster

ex: aws eks update-kubeconfig --region us-east-1 --name unus-eks-cluster-1 # Use the cluster name you just 

created

# see if you can get the nodes you created

kubectl get nodes

# Install nano editor in cloudshell. We will need this in the next task

sudo yum install nano -y

Task 4: Create a new POD in EKS for the 2048 game

================================================

# clean up the files in cloudshell (Optional)

rm *.* 

# create the config file in YAML to deploy 2048 game pod into the cluster

nano 2048-pod.yaml

### code starts ###

apiVersion: v1

kind: Pod

metadata:

   name: 2048-pod

   labels:

      app: 2048-ws

spec:

   containers:

   - name: 2048-container

     image: blackicebird/2048

     ports:

       - containerPort: 80

### code ends ###

# apply the config file to create the pod

kubectl apply -f 2048-pod.yaml

#pod/2048-pod created

# view the newly created pod

kubectl get pods

Task 5: Setup Load Balancer Service

===================================

nano mygame-svc.yaml  

### code starts ###

apiVersion: v1

kind: Service

metadata:

   name: mygame-svc

spec:

   selector:

      app: 2048-ws

   ports:

   - protocol: TCP

     port: 80

     targetPort: 80

   type: LoadBalancer

### code ends ###

# apply the config file

kubectl apply -f mygame-svc.yaml

# view details of the modified service

kubectl describe svc mygame-svc

# Access the LoadBalancer Ingress on the kops instance

curl <LoadBalancer_Ingress>:<Port_number>

or

curl a06aa56b81f5741268daca84dca6b4f8-694631959.us-east-1.elb.amazonaws.com:80

(try this from your laptop, not from your cloudshell)

# Go to EC2 console. get the DNS name of ELB and paste the DNS into address bar of the browser

# It will show the 2048 game. You can play. (need to wait for 2-3 minutes for the 

# setup to be complete)

Task 3: Cleanup

---------------

# Clean up all the resources created in the task

kubectl get pods

kubectl delete -f 2048-pod.yaml

kubectl get services

kubectl delete -f mygame-svc.yaml

####################################################################

Adding Screenshots of process

Connect with me 😊

https://www.linkedin.com/in/imharry404/