Building a Scalable MERN Stack Application with GCP, Docker, and GitHub CI/CD
Introduction
In the digital age, creating scalable applications is crucial to meet the demands of growing user bases and data volumes. The MERN stack, an acronym for MongoDB, Express, React, and Node.js, offers a robust framework for building full-stack web applications. This blog will guide you through the process of building a scalable MERN stack application using Google Cloud Platform (GCP), Docker, and GitHub Actions for continuous integration and delivery (CI/CD).
What is the MERN Stack?
The MERN stack is a popular choice among developers for creating dynamic web applications. It consists of:
- MongoDB: A NoSQL database that stores data in JSON-like documents, offering flexibility and scalability.
- Express: A web application framework for Node.js that simplifies the process of building web applications and APIs.
- React: A JavaScript library for building user interfaces, particularly single-page applications, by creating reusable UI components.
- Node.js: A JavaScript runtime that allows developers to execute JavaScript on the server side.
Importance of Scalability and Cloud-Native Development
Scalability ensures that your application can handle increased loads efficiently, without compromising performance. Cloud-native development, which involves building applications specifically designed for cloud environments, offers several benefits:
- Elasticity: Applications can scale in response to demand.
- Resilience: Cloud-native applications are designed to recover from failures automatically.
- Cost Efficiency: Pay for only what you use, optimizing resource utilization.
Why Use GCP + Docker + GitHub Actions?
- GCP: Offers a variety of services like Cloud Run and Kubernetes Engine for deploying and managing containerized applications.
- Docker: Enables consistent application deployment across different environments through containerization.
- GitHub Actions: Provides a powerful CI/CD pipeline to automate testing, building, and deployment processes.
Project Overview
We’ll use a sample project—a Task Manager application—to demonstrate the process of building, containerizing, and deploying a MERN stack application.
Sample Project: Task Manager
This Task Manager application allows users to create, update, delete, and manage tasks collaboratively. It features user authentication and real-time updates.
High-Level Architecture
- Frontend: Developed using React, responsible for the user interface.
- Backend: Built with Node.js and Express, handling logic, authentication, and API endpoints.
- Database: MongoDB for data persistence.
- Deployment: Docker containers managed on GCP.
Setting Up the MERN Application
Initialize Backend (Node.js + Express)
- Set up Node.js and Express:
- Initialize a new Node.js project: npm init -y
- Install Express: npm install express
- Create a basic server file (server.js) with the following content:
- const express = require(‘express’);
const app = express();
app.get(’/’, (req, res) => {
res.send('Hello World!’);
});
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
console.log(`Server is running on port ${PORT}`);
}); - Setup API endpoints: Define routes for tasks, authentication, and other features.
Setup Frontend (React)
- Create a React app:
- Use Create React App: npx create-react-app task-manager
- Develop UI components:
- Create components for task lists, task details, and user authentication.
- Manage state with React hooks or context API.
Connect MongoDB
- Local MongoDB:
- Install MongoDB locally and start the service.
- Connect using Mongoose: npm install mongoose
- MongoDB Atlas (Cloud):
- Sign up and create a cluster.
- Connect your application with the connection string provided by MongoDB Atlas.
Folder Structure Best Practices
Organize your project to enhance maintainability:
- Backend:
- /server
├── /models
├── /routes
├── /controllers
├── /middleware
└── server.js - Frontend:
- /client
├── /src
├── /components
├── /hooks
├── /services
├── /styles
└── App.js
Dockerizing the Application
Create Dockerfile for Backend
- Dockerfile (server/Dockerfile):
- FROM node:14
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 3000
CMD [“node”, “server.js”]
Create Dockerfile for Frontend
- Dockerfile (client/Dockerfile):
- FROM node:14
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build
EXPOSE 3000
CMD [“serve”, “-s”, “build”]
Use Docker-Compose for Local Development
- docker-compose.yml:
- version: '3.8’
services:
server:
build: ./server
ports:
- “5000:5000”
volumes:
- ./server:/app
environment:
- NODE_ENV=development
client:
build: ./client
ports:
- “3000:3000”
volumes:
- ./client:/app
environment:
- NODE_ENV=development
mongo:
image: “mongo”
ports:
- “27017:27017”
Explain Volumes, Ports, and Environment Variables
- Volumes: Persist data and allow for live code editing without rebuilding images.
- Ports: Map container ports to host ports for accessibility.
- Environment Variables: Configure application settings without hardcoding them.
Google Cloud Setup
Create GCP Project
- Create a new project in the GCP Console.
Enable Services
- Enable necessary services:
- Cloud Run: For deploying containers directly.
- Google Kubernetes Engine (GKE): For orchestrating containerized applications.
- Artifact Registry: For storing Docker images.
Configure Authentication
- Install gcloud CLI and authenticate:
- gcloud init
- gcloud auth login
CI/CD with GitHub Actions
Setup GitHub Repository
- Create a new repository on GitHub and push your project code.
Create Workflow YAML File
- .github/workflows/main.yml:
- name: CI/CD Pipeline
on:
push:
branches:
- main
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Set up Node.js
uses: actions/setup-node@v2
with:
node-version: '14’
- name: Install dependencies
run: npm install
- name: Run tests
run: npm test
- name: Build Docker images
run: docker-compose build
- name: Push to Artifact Registry
run: |
echo ${{ secrets.GCP_SA_KEY }} | docker login -u _json_key –password-stdin https://LOCATION-docker.pkg.dev
docker push LOCATION-docker.pkg.dev/PROJECT-ID/REPO-NAME/IMAGE
- name: Deploy to Cloud Run
run: |
gcloud run deploy SERVICE-NAME –image LOCATION-docker.pkg.dev/PROJECT-ID/REPO-NAME/IMAGE –region REGION
Secrets Management
- GCP Credentials: Store GCP service account key JSON in GitHub secrets as GCP_SA_KEY.
Deployment on GCP
Deploy Using Cloud Run or Kubernetes
- Cloud Run:
- Deploy your container using gcloud run deploy.
- Kubernetes:
- Use GKE for more complex deployments, managing pods, and services.
Configure Scaling and Load Balancing
- Cloud Run: Set concurrency settings and auto-scaling.
- GKE: Configure horizontal pod autoscaling.
Monitor Logs and Performance
- Use Stackdriver for logging and monitoring application performance.
Best Practices
Environment Separation
- Dev/Staging/Prod: Use different environments for development, testing, and production with separate configurations.
Security
- Secrets Management: Use secrets management tools to handle sensitive information.
- API Keys: Securely store and access API keys.
Logging and Monitoring
- Implement comprehensive logging for tracking issues.
- Monitor application performance and set alerts for anomalies.
Conclusion
Building a scalable MERN stack application with GCP, Docker, and GitHub CI/CD offers numerous benefits, including streamlined deployment, effective resource management, and enhanced reliability. By following the steps outlined in this guide, you can efficiently develop, deploy, and manage your applications in a cloud-native environment.
Future Enhancements
- Microservices: Consider breaking your application into smaller, independent services.
- Serverless: Explore serverless options to further optimize costs and scalability.
