Communication between multiple docker compose projects
February 24, 2020 • 3 min read
By default, two separate docker compose projects won't be able to see each other.
This is because they are part of two different networks.
Docker compose will set up up a single default network for your app when you run it. Containers for each service will join the default network so they could be discovered by other containers of the same project and could reach other containers in the same project.
Containers are not able to communicate with each other across different networks by default. They need to be configured to do so.
Example with two docker compose projects
Let's base our example on Nginx proxy_pass. Let's say that we have two separate docker compose projects in two different git repositories. First repository holds dockerized Nginx and the second contains some dockerized application written in Express.js.
We want our server (Nginx) to be able to serve more than one app in the future so we want to configure Nxing as a proxy. It will accept all incoming requests and then pass them to different apps based on the configuration.
1. Basic docker compose configuration for Nginx project
Server configuration with nginx.conf
The configuration of proxy_pass is pretty straightforward.
In file: nginx/nginx.conf
1events {
2 worker_connections 1024;
3}
4
5http {
6 upstream express_app_staging {
7 server express:5000;
8 }
9 server {
10 listen 80;
11 location / {
12 proxy_pass http://express_app_staging;
13 proxy_redirect off;
14 proxy_set_header Host $host;
15 proxy_set_header X-Real-IP $remote_addr;
16 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
17 proxy_set_header X-Forwarded-Host $server_name;
18 }
19 }
20}
This configuration assumes that our Express.js app runs on port 5000 and the docker-compose service name for this application is express. Then we configure Nginx to proxy all requests from "/" to the express app.
Docker compose config
Nginx will accept an incoming request and then pass it to express service.
In the first paragraph, I mentioned that every docker compose project gets its own default network, so these two services (nginx and express) will be in two separate networks and they won't see each other. To deal with this problem we will create a new network and call it load-balancer.
In file: docker-compose.yml
1version: "3.5"
2services:
3 nginx:
4 image: nginx:alpine
5 container_name: nginx
6 restart: unless-stopped
7 tty: true
8 ports:
9 - "80:80"
10 - "443:443"
11 volumes:
12 - ./nginx/nginx.conf:/etc/nginx/nginx.conf
13 networks:
14 - load-balancer
15networks:
16 load-balancer:
17 driver: bridge
18 name: server_load_balancer
Here we create a custom network called load-balancer. This name is arbitrary, you can pick whatever you want.
Then we add load-balancer to nginx service and we also do a basic configuration for our network:
- We set the driver to bridge. Bridge networks allow containers connected to the same bridge to talk to each other. Docker provides isolation from containers that are no connected to that bridge network. Bridge networks apply only to containers running on thesame Docker daemon host.
- We specify a name for this network: server_load_balancer. This network will be visible from outside the project as server_load_balancer.
Disclaimer: I'm using config version 3.5 which allows me to use the name attribute in the network config. If you don't provide the name then the network would just be visible externally as load-balancer.
With this setup after we start the nginx project, docker compose wil create two networks: default network and our newly created load-balancernetwork.
The next step is to accept load-balancer network in Express project configuration.
Basic docker-compose configuration for Express.js project
In docker compose configuration for the second project we just need to reference the external server_load_balancer network and assign it to express app service.
In file: docker-compose.yml
1version: "3"
2services:
3 app:
4 container_name: express
5 restart: always
6 image: someuser/my_express_app:latest
7 ports:
8 - "5000:5000"
9 env_file:
10 - env-variables.env
11 command: ["./run.sh"]
12 networks:
13 - server_load_balancer
14networks:
15 server_load_balancer:
16 external: true
And thats it! Now this two services should be able to see each other.
Conclusion
Now you can go ahead and run docker-compose up on both projects. Nginx should properly pass requests from nginx to app.
Reference
https://docs.docker.com/compose/networking/
https://docs.docker.com/network/bridge/
Tags
docker-compose
docker
nginx
proxy_pass
load balancer
multple docker-compose projects
networks
bridge
All posts
Personal blog by Bartek Józwowiak. I write mostly about new frontend tools and web development related stuff.