Docker Compose + Microservices + CI/CD for Beginners

So far you have learned:

• How to write a Dockerfile for one .NET API
• How to use GitHub Actions to build and deploy that image

When you move from a single API to multiple services, things change. You now need a way to run several containers together during development, and a clear process to build and deploy them.

This is where Docker Compose comes in.

In this post you will learn:

• What Docker Compose is
• Why it is useful for microservices
• How to write a basic docker-compose.yml
• How it fits into local development
• How it connects with GitHub Actions and production on Azure

1. What is Docker Compose?

Docker Compose is a tool that lets you define and run multiple containers using a single file called docker-compose.yml.

Quick comparison:

Simple analogy:

Dockerfile       = recipe for one dish
Docker Compose   = full menu that starts several dishes at once

2. Why Do We Need Docker Compose?

In a microservices style project you usually have more than one thing running:

• API service
• Background worker
• Database
• Cache (Redis)
• Message broker

Starting each container manually with separate docker run commands is slow and error-prone.

With Docker Compose you can start everything using one command:

docker compose up

And stop everything with:

docker compose down

This makes local development much easier.

3. Basic docker-compose.yml Example

Here is a practical example with an API, a worker, and a SQL Server database:

version: '3.8'

services:
  api:
    build: ./src/Api
    ports:
      - "5000:8080"
    environment:
      - ConnectionStrings__Default=Server=db;Database=MyDb;User=sa;Password=Your_password123
    depends_on:
      - db

  worker:
    build: ./src/Worker
    environment:
      - ConnectionStrings__Default=Server=db;Database=MyDb;User=sa;Password=Your_password123
    depends_on:
      - db

  db:
    image: mcr.microsoft.com/mssql/server:2022-latest
    environment:
      - ACCEPT_EULA=Y
      - MSSQL_SA_PASSWORD=Your_password123
    ports:
      - "1433:1433"
    volumes:
      - sqldata:/var/opt/mssql

volumes:
  sqldata:

What this does:

• api is built from the Dockerfile inside ./src/Api
• worker is another service
• db runs SQL Server
• depends_on makes sure the database starts first
• volumes keeps the database data even after containers are removed

4. Local Development Flow

A typical day for a developer looks like this:

1. Write code for different services
2. Keep one Dockerfile per service
3. Maintain a docker-compose.yml
4. Run: docker compose up
5. Test everything locally
6. Push code to GitHub

Docker Compose gives you a local environment that behaves similarly to production.

5. Important Reality Check: Compose vs Production

This is a point many beginners miss:

Docker Compose is excellent for local development and testing.
It is usually not the main tool used to run large production systems.

So the realistic big picture is:

Local (Docker Compose)
        ↓
CI/CD (GitHub Actions)
        ↓
Production (Azure Container Apps or AKS)

6. How Docker Compose Connects with CI/CD

Docker Compose is mainly used on the developer machine.

The CI/CD pipeline still works with individual Docker images.

Typical flow:

Push code to GitHub
        ↓
GitHub Actions starts
        ↓
Build image for Api
Build image for Worker
        ↓
Push both images to Azure Container Registry
        ↓
Deploy the images to Azure Container Apps (or similar)

You can keep docker-compose.yml in the repository so every developer can run the full system locally. Production does not need to use Compose.

7. Recommended Production-Ready Process

Here is a practical process used by many teams:

Step 1: Local Development

• One Dockerfile per service
• One docker-compose.yml to run all services together
• Developers use docker compose up

Step 2: Source Control

• Code, Dockerfiles, and docker-compose.yml all live in GitHub

Step 3: Continuous Integration (CI)

On every push or pull request:

• Build each Docker image
• Run tests
• Optionally use Docker Compose for integration tests

Step 4: Continuous Deployment (CD)

• Push successful images to Azure Container Registry
• Deploy to Azure Container Apps (recommended starting point) or AKS

Step 5: Configuration and Secrets

• Never store real passwords in docker-compose.yml or Dockerfiles
• Use GitHub Secrets for the pipeline
• Use Azure App Settings or Key Vault in production

8. Clean Project Structure

A good structure for a microservices project looks like this:

/MyMicroservices
├── src
│   ├── Api
│   │   ├── Dockerfile
│   │   └── ...
│   ├── Worker
│   │   ├── Dockerfile
│   │   └── ...
│   └── Shared
├── docker-compose.yml
├── docker-compose.override.yml
└── .github
    └── workflows
        └── deploy.yml

• Each service has its own Dockerfile
• docker-compose.yml is for local use
• GitHub Actions builds and deploys each service

9. Single API vs Microservices

10. Production-Ready Best Practices

1. One Dockerfile per service
   Do not put multiple services inside one image.

2. Keep Docker Compose for local development
   Production should not depend on it.

3. Use multi-stage builds
   Keep final images small and secure.

4. Tag images properly
   Prefer commit SHA or version numbers instead of only latest.

5. Externalize configuration
   Use environment variables and secret stores.

6. Add health checks
   Help the platform know when a service is ready.

7. Separate environments
   Dev, Staging, and Production should have different settings.

8. Start small
   Begin with 2 or 3 services before growing further.

11. Full Picture - Connecting Everything

Developer Machine
─────────────────
Dockerfile(s)
docker-compose.yml
      ↓
docker compose up          ← local testing

GitHub
─────────────────
Push code
      ↓
GitHub Actions
      ↓
Build each Docker image
Push to Azure Container Registry

Azure (Production)
─────────────────
Azure Container Registry
      ↓
Azure Container Apps / App Service / AKS
      ↓
Running microservices

12. Final Takeaway

Docker Compose makes it easy to run multiple services together while you are developing.

The overall flow becomes:

Local development   → Docker Compose
Build & Deploy      → GitHub Actions + Docker
Production runtime  → Azure Container Apps or AKS

You still write Dockerfiles for each service.
You still use GitHub Actions for CI/CD.
Docker Compose simply makes the local experience much better when you have more than one service.

Once you are comfortable with this setup, you can later explore service-to-service communication, API gateways, and more advanced orchestration.

Happy building!