How did we set up the staging environment in our company?
OBJECTIVE: In this blog, I am going to explain how did we set up the STG environment in our current company. The above image is the architectural diagram of the staging environment set up with END to END automation using terraform. whatever we create in the AWS, it is from the terraform and nothing is manual.
COMPONENTS USED :
- Bitbucket
- Code commit
- Terraform
- Code pipeline
- Code build
IN DETAIL: In STG env we are having 3 types of pipelines, 1) Infrastructure pipeline, 2) backend pipeline, and 3) frontend pipeline. We have integrated bitbucket with code commit to getting all the code into the AWS. The source for all the pipelines is code commit. Overall we have 1 infrastructure pipeline to create any infrastructure in the AWS and we will be having many backends and front-end pipelines for deploying microservices.
- Infrastructure Pipeline: In the infrastructure pipeline, we are having 3 stages, A) source, B) build, and C) deploy.
A) source: is used to fetch the latest code from the AWS CodeCommit whenever there is a new commit and push it to the build stage.
B) build: is used to init, validate, and plan the terraform files in the repo. The terraform plan is saved in the .tfplan file and pushed to the s3 bucket.
C) deploy: is used to create or modify the infrastructure by downloading the .tfplan file from the s3 bucket which is planned in the earlier stage.
2. Backend Pipeline: In the backend pipeline, we are having 4 stages, A) source, B) create-infra, C) create-image and D) deploy-app.
A) source: is used to fetch the latest code from the AWS CodeCommit whenever there is a new commit and push it to the create-infra stage.
B) create-infra: is used to create all the resources like target group, CW-log group, task definition, ECR repo, ECS service, DNS record and etc using terraform.
C) create-image: is used to create a docker image from the latest available code in the code commit repo and push it to the ECR repo.
D) deploy-app: is used to update the task definition with the latest image available in the ECR repo and update the ECS service with the latest task definition.
3. Frontend Pipeline: In the frontend pipeline, we are having 3 stages A) source, B) create-infra, and C) deploy.
A) source: is used to fetch the latest code from the AWS CodeCommit whenever there is a new commit and push it to the create-infra stage.
B) create-infra: is used to create resources like s3 bucket, cloud front and DNS record in the Route53 using terraform.
C) deploy: is used to build and push artifacts to the s3 bucket and invalidate the CloudFront to pick the latest code from the s3 bucket.
ACHIEVEMENTS :
1) We don't need to hardcode the access keys in the code to access the AWS services.
2) We can come out from Jenkins as it is not PCI DSS compliance.
3) This is very secure compared to Jenkins deployment because it is managed by AWS.
4) END to END automation.
5) We can reuse these templates to spin up another environment.
6) Even if everything got deleted, with one single click we can create everything as it is earlier.
7) We don't need to administrate EC2 instances that host Jenkins.
8) Management of the access will become very easy as everything is created via pipelines and nothing is created manually.
9) We can limit access to users for read-only for specific services and revoke the rest of the access.
10) we can create a new microservice within minutes of time.
CONCLUSION: We did this staging environment set-up in the medibuddy company. MediBuddy is one of the best (cashless) healthcare providers in India. At MediBuddy you can book Health check packages, online lab tests, online medicines and etc.
REFERENCES:
1) https://registry.terraform.io/providers/hashicorp/aws/latest/docs
3) https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html
