Netns Docker For Mac

Return to site

Netns Docker For Mac

Update (2018-03-22) Since I wrote this document back in 2014,Docker has developed the macvlan networkdriver. That gives you asupported mechanism for direct connectivity to a local layer 2network. I've written an article about working with the macvlandriver.
Docker 会尝试寻找没有被主机使用的 ip 段，尽管它适用于大多数情况下，但是它不是万能的，有时候我们还是需要对ip进一步规划。 Docker允许你管理docker0桥接或者通过-b选项自定义桥接网卡，需要安装bridge-utils软件包。 基本步骤如下： 1、确保docker的进程是停止的。.
This article discusses four ways to make a Docker container appear on a local network. Apr 01, 2020 sudo docker exec -d leaf12 ip netns exec swnet sysctl net.ipv6.conf.swport5.disableipv6=1 sudo docker exec -d leaf12 ip netns exec swnet ip link set swport5 up sudo docker exec -d leaf12 ip link set dev swport6 netns swnet.
$ cd /var/run $ sudo ln -s /var/run/docker/netns netns $ sudo ip netns be663feced43 6e9de12ede80 2-8vty8k3pej 1-f1nvcluv1x 72df0265d4af Comparing the namespace names with the Docker swarm network IDs, we can guess that namespace ‘ 2-8vty8k3pej ‘ is used for ‘net1' network, whose ID is 8vty8k3pejm5.
This article discusses four ways to make a Docker container appear ona local network. These are not suggested as practical solutions, butare meant to illustrate some of the underlying network technologyavailable in Linux.
If you were actually going to use one of these solutions as anythingother than a technology demonstration, you might look to the pipework script, which can automate many of these configurations.
Goals and AssumptionsIn the following examples, we have a host with address 10.12.0.76 onthe 10.12.0.0/21 network. We are creating a Docker container that wewant to expose as 10.12.0.117.
I am running Fedora 20 with Docker 1.1.2. This means, in particular,that my utils-linux package is recent enough to include thensenter command. If you don't have that handy, there is aconvenient Docker recipe to build it for you at jpetazzo/nsenteron GitHub.
A little help along the wayIn this article we will often refer to the PID of a docker container.In order to make this convenient, drop the following into a scriptcalled docker-pid, place it somewhere on your PATH, and make itexecutable:
Nets Docker For Mac DownloadThis allows us to conveniently get the PID of a docker container byname or ID:
In a script called docker-ip, place the following:
And now we can get the ip address of a container like this:
Using NATThis uses the standard Docker network model combined with NAT rules onyour host to redirect inbound traffic to/outbound traffic from theappropriate IP address.
Assign our target address to your host interface:
Start your docker container, using the -p option to bind exposedports to an ip address and port on the host:
With this command, Docker will set up the standard network model:
It will create a veth interface pair.
Connect one end to the docker0 bridge.
Place the other inside the container namespace as eth0.
Assign an ip address from the network used by the docker0 bridge.
Because we added -p 10.12.0.117:80:80 to our command line, Dockerwill also create the following rule in the nat table DOCKERchain (which is run from the PREROUTING chain):
This matches traffic TO our target address (-d 10.12.0.117/32) notoriginating on the docker0 bridge (! -i docker0) destined fortcp port 80 (-p tcp -m tcp --dport 80). Matching traffic hasit's destination set to the address of our docker container (-j DNAT --to-destination 172.17.0.4:80).
From a host elsewhere on the network, we can now access the web serverat our selected ip address:
If our container were to initiate a network connection with anothersystem, that connection would appear to originate with ip address ofour host. We can fix that my adding a SNAT rule to thePOSTROUTING chain to modify the source address:
Note here the use of -I POSTROUTING, which places the rule at thetop of the POSTROUTING chain. This is necessary because, bydefault, Docker has already added the following rule to the top of thePOSTROUTING chain:
Because this MASQUERADE rule matches traffic from any container, weneed to place our rule earlier in the POSTROUTING chain for it tohave any affect.
With these rules in place, traffic to 10.12.0.117 (port 80) isdirected to our web container, and traffic originating in the webcontainer will appear to come from 10.12.0.117.

With Linux Bridge devicesThe previous example was relatively easy to configure, but has a fewshortcomings. If you need to configure an interface using DHCP, or ifyou have an application that needs to be on the same layer 2 broadcastdomain as other devices on your network, NAT rules aren't going towork out.
This solution uses a Linux bridge device, created using brctl, toconnect your containers directly to a physical network.
Start by creating a new bridge device. In this example, we'll createone called br-em1:
We're going to add em1 to this bridge, and move the ip address fromem1 onto the bridge.
WARNING: This is not something you should do remotely, especiallyfor the first time, and making this persistent varies fromdistribution to distribution, so this will not be a persistentconfiguration.
Look at the configuration of interface em1 and note the existing ipaddress:
Look at your current routes and note the default route:
Now, add this device to your bridge:
Configure the bridge with the address that used to belong toem1:
And move the default route to the bridge:
If you were doing this remotely; you would do this all in one linelike this:
Linux developers are great at creating functionality that is similar to Mac, streamlining the programs and making them easy to use and tweak. RECOMMEND try Ubuntu as an alternative to Mac! Brodsolutions. Sep 2018. 10 agrees and 2 disagrees Not for mac folks. Best alternative os for mac. The Best Alternatives to Apple's Mac and MacBook If you're tired of Apple's desktops or laptops, consider these Windows and Linux-powered options. Another WWDC has come and gone, leaving in its.
At this point, verify that you still have network connectivity:
Start up the web container:
This will give us the normal eth0 interface inside the container,but we're going to ignore that and add a new one.
Create a veth interface pair:
Add the web-ext link to the br-eth0 bridge:
And add the web-int interface to the namespace of the container:
Next, we'll use the nsenter command (part of the util-linux package) to run some commands inside the web container. Start by bringing up the link inside the container:
Assign our target ip address to the interface:
And set a new default route inside the container:
Again, we can verify from another host that the web server isavailable at 10.12.0.117:
Note that in this example we have assigned a static ip address, but wecould just have easily acquired an address using DHCP. After running:
We can run:
With Open vSwitch Bridge devicesThis process is largely the same as in the previous example, but weuse Open vSwitch instead of the legacy Linux bridge devices.These instructions assume that you have already installed and startedOpen vSwitch on your system.
Create an OVS bridge using the ovs-vsctl command:
And add your external interface:
And then proceed as in the previous set of instructions.
The equivalent all-in-one command is:
Once that completes, your openvswitch configuration should look likethis:
To add the web-ext interface to the bridge, run:
Instead of:
WARNING: The Open vSwitch configuration persists between reboots.This means that when your system comes back up, em1 will still be amember of br-em, which will probably result in no networkconnectivity for your host.
Before rebooting your system, make sure to ovs-vsctl del-port br-em1 em1.
With macvlan devicesNets Docker For Mac OsThis process is similar to the previous two, but instead of using abridge device we will create a macvlan, which is a virtual networkinterface associated with a physical interface. Unlike the previoustwo solutions, this does not require any interruption to your primarynetwork interface.
Start by creating a docker container as in the previous examples:
Create a macvlan interface associated with your physical interface:
This creates a new macvlan interface named em1p0 (but you canname it anything you want) associated with interface em1. We aresetting it up in bridge mode, which permits all macvlan interfacesto communicate with eachother.
Add this interface to the container's network namespace:
Bring up the link:
And configure the ip address and routing:
And demonstrate that from another host the web server is availableat 10.12.0.117:
But note that if you were to try the same thing on the host, you wouldget:
The host is unable to communicate with macvlan devices via theprimary interface. You can create anothermacvlan interface onthe host, give it an address on the appropriate network, and then setup routes to your containers via that interface:
Chapter 1 - Vagrantfile, Virtual Box, and Docker for Mac Installation Download Vagrant Up for Machttps://www.vagrantup.com/downloads.html
Download Virtual Box for Machttps://www.virtualbox.org/wiki/Downloads
Download Docker Toolbox for Machttps://www.docker.com/products/docker-toolbox
Download Docker for Machttps://www.docker.com/products/docker#/mac
Forum for Docker https://forums.docker.com/c/docker-for-mac
Guide for Docker for Machttps://docs.docker.com/docker-for-mac/
Examples for Docker for Machttps://docs.docker.com/docker-for-mac/examples/
Setup Docker Compose and MySQLhttps://docs.docker.com/compose/rails/
Check Docker ClientCreate CoreOS Docker host using Vagrant VM by:Note: Use of Vagrant seed for docker development is an alternative approach to using docker-machine.
Cloning coreos-vagrant Git Repo into host machine directory docker-host
Making the following changes to the Vagrantfile
Expose Docker TCP connection port in config.rb of coreos-vagrant directory Note that Vagrantfile creates a CoreOS VM using VirtualBox software hypervisor
Note that Vagrantfile parses config.rb containing a set of options configuring CoreOS cluster
Generate user-data file containing #cloud-config to simplify provisioning of CoreOS VM using built-in coreos-cloudinit
Startup and SSH using VirtualBox Provider (default Vagrant provider) Note vagrant up triggers vagrant to download CoreOS image (if necessary) and relaunch instance
Note vagrant ssh connects to VM from directory with Vagrantfile
Use Vagrant VM:Setup shell environment so local Docker client may communicate with Docker daemon on VM
Show Vagrant environments on host machine
Check Docker environment variables
Important: Restart Bash terminal to avoid error connecting to Docker Daemon 
Connect to Docker Daemon (running on port 2375). Note use unencrypted only in Development.
Important Note: Docker Daemon of Docker for Mac or Docker Toolbox already runs in a Linux VM, so you do not need to (and cannot) run it manually with dockerd. It is already running in the whale icon is in the top bar. https://github.com/docker/docker/issues/27102
Connect to Shell on Vagrant VM
Share Folders between Vagrant VM (CoreOS) and Host (OSX). IDE may now be used to edit files on Vagrant VM.Enable synced folders by editing Vagrantfile as follows (uncommenting the config.vm.synced_folder line of code)
Restart Vagrant VM
Note: If an exports invalid error occurs then open and validate the contents of /etc/exports on the Host (OSX)
Check that files synchronise between these folders
Interactive Docker Containers within Vagrant VM (Docker host):From the shell of Vagrant VM, test download a Docker Image from Docker Hub for the base distribution specified using Docker Daemon(i.e. Ubuntu, Fedora, and CentOS). A Docker Container inside Vagrant VM filesystem is created based on the downloaded Docker Image.The new Docker Container has a network, IP addresss, and bridge to communicate with localhost.The command and flags run by Docker on the new container launch an interactive Bash shell instance of the host (i.e. Ubuntu) in it that we are logged into as root user.
List all available Linux commands, aliases, built-ins, keywords, and functions
Show Docker Container Host Entries
Show Docker Container IP address
Show Docker Container Running Processes
Install software package on Docker Container
Exit Docker Container to return to Vagrant VM
List Docker Containers (both stopped and running) from within Vagrant VM. Note: Those that were run with --rm flag are not shown.
List Docker Containers (only last 2 that were stopped or running)
Create Container (but not Run), Start, Restart, Re-Attach (to process using original Options, i.e. command, flags), and Delete Docker Container
Important Note: Press Enter after Re-Attach to Container (otherwise it hangs)

Daemonized (non-Interactive) Docker Containers within Vagrant VM (Docker host) for running apps and services:Create and run a Daemonized Docker Container within Vagrant. Flags detach the container to the background and performs a command until process stopped.
View under the hood of the Docker Container by debugging the last few log entries
Monitor (follow) the Docker Container log entries using the -f flag (similar to the tail -f binary) with timestamps and without reading previous entries to the log file
Nets Docker For Mac CatalinaNote: Transport Layer Security (TLS) is a cryptographic protocol that provides communications security over a network (predecessor SSL)
Inspect processes (and PIDs) of Daemonized Docker Container
Statistics (CPU, memory, network, storage I/O, metrics) of Daemonized Docker Container
Background Processes (using detach -d flag) started on already running containers are executed (management, monitoring, or maintenance) with Process Owner in Daemonized Docker Container
Note: docker exec works on already running containers, otherwise error returned:
Inspect Docker Container (config info, names, commands)
Selectively Inspect Docker Container (using GoLang template with -f flag https://golang.org/pkg/text/template/)
View Docker Container storage location on Vagrant VM (i.e. CoreOS). Start shell as superuser first
Log CollectionChapter 2 - Dockerfile, Docker Images Docker Container based on a Docker Image (based on a public image or custom) stored in Repository of a Registry
Docker Image has filesystem layers with mapping to each build step
Docker Image Management storage backend communicates with underlying Linux filesystem to build layers into usable image
Docker Image Management storage backends include fast Copy-on-Write (CoW)
Docker Image Management storage backends supported include AUFS, BTRS, Device-mapper, and overlayfs
Docker Hub is the default Registry
Top-Level Repositories are managed by Docker and vendors (i.e. ubuntu)
Top-Level Docker Images offer bare runtime to run the distribution
User Repositories are by individual developers (i.e. ltfschoen/ruby)
Tag comprises a series of image layers in single repository (representative of version control)
Docker Image is built upon the Union Mount of its parent image layers above the base image (each layer is a filesystem)
Show Docker Images downloaded from repositories that contain layers and metadata
Go to local storage location of Docker Images
Pull specific Tag (i.e. 3.4) for Top-Level Docker Image
Search Docker Hub for publicly available Docker Images that may already be pre-installed with applications
Login/Logout of Docker Hub
Docker Build Command (using Dockerfile)Docker FileDockerfile and docker build is repeatable, transparent, and idempotent, and so is preferred over docker commit
Dockerfile instructs how Docker Image is to be assembled using application code.
Dockerfile uses DSL with instructions paired with arguments on each line of code
Dockerfile lines of code are processed from top to bottom
Each line of code creates a new Docker Image Layer.
Only layers deviating from previous Docker Image Layer must be built
Dockerfile docker build process is:1) Docker runs an initial new Docker Container from the base Docker Image (FROM)2) Docker executes instruction (first line of code) inside initial Docker Container3) Docker runs a docker commit to commit initial Docker Image Layer4) Docker runs a second new Docker Container based on the initial Docker Image Layer5) Docker executes instruction (next line of code) inside second Docker Container6) Docker runs a docker commit to commit second Docker Image Layer7) Docker runs a third new Docker Container based on the second Docker Image Layer8) Docker repeats steps 5) to 7) for subsequent lines of code to build up a Docker Container
Docker Container that is built houses the application (i.e. Rails-based app) with all dependencies
Debug errors when building from Dockerfile by running Docker Container from the last successfully created Docker Image
Dockerfile FROMDefault instance is base Linux image
Custom Docker Registry offers official framework Docker Images and Tags (i.e. Node.js). Docker Image base inherits from Docker Container defined by FROM (i.e. node:0.10.33 specifically locks Docker Image base to specific Ubuntu Linux image running Node 0.10.33)
Dockerfile MAINTAINER - provides author field metadata in Docker Image
Dockerfile LABEL - provides KV pairs of metadata in Docker Image for search purposes. Find with docker inspect
Dockerfile USERImportant: Docker Containers run processes on the host machine kernel. Do not be run as root user in Production for security
Dockerfile ENV - set shell environment variables used in build process on the image
Dockerfile RUNexecute instruction/command to append new Docker Image Layer on the current Docker Image
start and create file structure, and install dependencies
commit the new Docker Image Layer if command successful
Important: Reduce build time by abstracting common RUN processes (i.e. updates) into code lines (Docker Image layers) of Base Image
Important: Reduce rebuild time by ordering commands with code imports at the end (as all layers after introduced change are rebuilt)
Important: Combine logically grouped commands into single Dockerfile code line since each instruction creates a new Docker Image layer (i.e. use ADD to import scripts and required supporting files from local filesystem into image)
Dockerfile WORKDIR - changes working directory in image for remaining build instructions
Dockerfile CMD - final instruction launches process to run in Docker Container
Dockerfile EXPOSE - ports Docker Container listens to. docker run flag (--expose) opens these ports at runtime
Important: Single function in CMD to easy horizontal scaling of individual functions in architecture
Nets Docker For Mac InstallerBuild Custom ContainerBuild new Custom Container based on Top-Level Container that we build from a Top-Level Image
Example - Static Web ServerCreate build environment (build context) for Docker to upload to Docker Daemon when build is run.
Note: Previously we configred the Vagantfile to synchronise the following folders between Vagrant VM (CoreOS) and Host (OSX):
~/code/docker-host-coreos-share (Host)
/home/core/share (Vagrant VM)
Create Dockerfile that builds a Docker Image where the Docker Container is a static web server
Note: Perform these on Host machine since folders are synchronised
Build Dockerfile into Docker Image.Tagging with repository/image_name:version.Use -f flag to specify directory within build context containing Dockerfile.
Note: docker build must be run on the Vagrant VM (CoreOS), otherwise the following error message appears Cannot connect to the Docker daemon. Is the docker daemon running on this host?
Check Docker Images updated
Check Docker Image history (Docker Image Layers are shown)
Docker Image Templates using Build CacheDocker uses cache and re-builds from earliest image layer change.Bypass using --no-cache flag.
Dockerfile should have template at top of file with package repositories and updates so cache is hit.
Launch Container using Docker Image-d flag detaches in background for long processes (i.e. Nginx daemon)
-p flag manages network ports published by Docker at container runtime to the host machine, with options including:
Mapping of randomly assigned Docker Host port(i.e. 32768 to 61000) to Port 80 on Docker Container
Mapping of specific Docker Host port (i.e. 8080) direct binding to Port 80 on Docker Container(i.e. sudo docker run -d -p 8080:80 --name static_web ltfschoen/static_web_8080 nginx -g 'daemon off;')
Mapping of specific Interface (i.e. 127.0.0.1) and specific Docker Host port (i.e. 8080) direct binding to Port 80 on Docker Container(i.e. sudo docker run -d -p 127.0.0.1:8080:80 --name static_web ltfschoen/static_web_8080_i127_0_0_1:v1 nginx -g 'daemon off;')
Mapping of specific Interface (i.e. 127.0.0.1) and random Docker Host port to Port 80 on Docker Container(i.e. sudo docker run -d -p 127.0.0.1::80 --name static_web ltfschoen/static_web_random_i127_0_0_1 nginx -g 'daemon off;')
Mapping of random Docker Host port to Port 80 on Docker Container and publish additional ports specified with EXPOSE instructions(i.e. sudo docker run -d -P --name static_web_all ltfschoen/static_web_all nginx -g 'daemon off;')
Note: Random port is assigned when not specified
Note: Only one Docker Container may be bound to a specific port on host machine
Note: Bind UDP ports by appending to port binding /udp
Command to run (i.e. nginx -g 'daemon off;') launches Nginx in foreground to run web server

With Linux Bridge devicesThe previous example was relatively easy to configure, but has a fewshortcomings. If you need to configure an interface using DHCP, or ifyou have an application that needs to be on the same layer 2 broadcastdomain as other devices on your network, NAT rules aren't going towork out.
This solution uses a Linux bridge device, created using brctl, toconnect your containers directly to a physical network.
Start by creating a new bridge device. In this example, we'll createone called br-em1:
We're going to add em1 to this bridge, and move the ip address fromem1 onto the bridge.
WARNING: This is not something you should do remotely, especiallyfor the first time, and making this persistent varies fromdistribution to distribution, so this will not be a persistentconfiguration.
Look at the configuration of interface em1 and note the existing ipaddress:
Look at your current routes and note the default route:
Now, add this device to your bridge:
Configure the bridge with the address that used to belong toem1:
And move the default route to the bridge:
If you were doing this remotely; you would do this all in one linelike this:
Linux developers are great at creating functionality that is similar to Mac, streamlining the programs and making them easy to use and tweak. RECOMMEND try Ubuntu as an alternative to Mac! Brodsolutions. Sep 2018. 10 agrees and 2 disagrees Not for mac folks. Best alternative os for mac. The Best Alternatives to Apple's Mac and MacBook If you're tired of Apple's desktops or laptops, consider these Windows and Linux-powered options. Another WWDC has come and gone, leaving in its.
At this point, verify that you still have network connectivity:
Start up the web container:
This will give us the normal eth0 interface inside the container,but we're going to ignore that and add a new one.
Create a veth interface pair:
Add the web-ext link to the br-eth0 bridge:
And add the web-int interface to the namespace of the container:
Next, we'll use the nsenter command (part of the util-linux package) to run some commands inside the web container. Start by bringing up the link inside the container:
Assign our target ip address to the interface:
And set a new default route inside the container:
Again, we can verify from another host that the web server isavailable at 10.12.0.117:
Note that in this example we have assigned a static ip address, but wecould just have easily acquired an address using DHCP. After running:
We can run:
With Open vSwitch Bridge devicesThis process is largely the same as in the previous example, but weuse Open vSwitch instead of the legacy Linux bridge devices.These instructions assume that you have already installed and startedOpen vSwitch on your system.
Create an OVS bridge using the ovs-vsctl command:
And add your external interface:
And then proceed as in the previous set of instructions.
The equivalent all-in-one command is:
Once that completes, your openvswitch configuration should look likethis:
To add the web-ext interface to the bridge, run:
Instead of:
WARNING: The Open vSwitch configuration persists between reboots.This means that when your system comes back up, em1 will still be amember of br-em, which will probably result in no networkconnectivity for your host.
Before rebooting your system, make sure to ovs-vsctl del-port br-em1 em1.
With macvlan devicesNets Docker For Mac OsThis process is similar to the previous two, but instead of using abridge device we will create a macvlan, which is a virtual networkinterface associated with a physical interface. Unlike the previoustwo solutions, this does not require any interruption to your primarynetwork interface.
Start by creating a docker container as in the previous examples:
Create a macvlan interface associated with your physical interface:
This creates a new macvlan interface named em1p0 (but you canname it anything you want) associated with interface em1. We aresetting it up in bridge mode, which permits all macvlan interfacesto communicate with eachother.
Add this interface to the container's network namespace:
Bring up the link:
And configure the ip address and routing:
And demonstrate that from another host the web server is availableat 10.12.0.117:
But note that if you were to try the same thing on the host, you wouldget:
The host is unable to communicate with macvlan devices via theprimary interface. You can create anothermacvlan interface onthe host, give it an address on the appropriate network, and then setup routes to your containers via that interface:
Chapter 1 - Vagrantfile, Virtual Box, and Docker for Mac Installation Download Vagrant Up for Machttps://www.vagrantup.com/downloads.html
Download Virtual Box for Machttps://www.virtualbox.org/wiki/Downloads
Download Docker Toolbox for Machttps://www.docker.com/products/docker-toolbox
Download Docker for Machttps://www.docker.com/products/docker#/mac
Forum for Docker https://forums.docker.com/c/docker-for-mac
Guide for Docker for Machttps://docs.docker.com/docker-for-mac/
Examples for Docker for Machttps://docs.docker.com/docker-for-mac/examples/
Setup Docker Compose and MySQLhttps://docs.docker.com/compose/rails/
Check Docker ClientCreate CoreOS Docker host using Vagrant VM by:Note: Use of Vagrant seed for docker development is an alternative approach to using docker-machine.
Cloning coreos-vagrant Git Repo into host machine directory docker-host
Making the following changes to the Vagrantfile
Expose Docker TCP connection port in config.rb of coreos-vagrant directory Note that Vagrantfile creates a CoreOS VM using VirtualBox software hypervisor
Note that Vagrantfile parses config.rb containing a set of options configuring CoreOS cluster
Generate user-data file containing #cloud-config to simplify provisioning of CoreOS VM using built-in coreos-cloudinit
Startup and SSH using VirtualBox Provider (default Vagrant provider) Note vagrant up triggers vagrant to download CoreOS image (if necessary) and relaunch instance
Note vagrant ssh connects to VM from directory with Vagrantfile
Use Vagrant VM:Setup shell environment so local Docker client may communicate with Docker daemon on VM
Show Vagrant environments on host machine
Check Docker environment variables
Important: Restart Bash terminal to avoid error connecting to Docker Daemon 
Connect to Docker Daemon (running on port 2375). Note use unencrypted only in Development.
Important Note: Docker Daemon of Docker for Mac or Docker Toolbox already runs in a Linux VM, so you do not need to (and cannot) run it manually with dockerd. It is already running in the whale icon is in the top bar. https://github.com/docker/docker/issues/27102
Connect to Shell on Vagrant VM
Share Folders between Vagrant VM (CoreOS) and Host (OSX). IDE may now be used to edit files on Vagrant VM.Enable synced folders by editing Vagrantfile as follows (uncommenting the config.vm.synced_folder line of code)
Restart Vagrant VM
Note: If an exports invalid error occurs then open and validate the contents of /etc/exports on the Host (OSX)
Check that files synchronise between these folders
Interactive Docker Containers within Vagrant VM (Docker host):From the shell of Vagrant VM, test download a Docker Image from Docker Hub for the base distribution specified using Docker Daemon(i.e. Ubuntu, Fedora, and CentOS). A Docker Container inside Vagrant VM filesystem is created based on the downloaded Docker Image.The new Docker Container has a network, IP addresss, and bridge to communicate with localhost.The command and flags run by Docker on the new container launch an interactive Bash shell instance of the host (i.e. Ubuntu) in it that we are logged into as root user.
List all available Linux commands, aliases, built-ins, keywords, and functions
Show Docker Container Host Entries
Show Docker Container IP address
Show Docker Container Running Processes
Install software package on Docker Container
Exit Docker Container to return to Vagrant VM
List Docker Containers (both stopped and running) from within Vagrant VM. Note: Those that were run with --rm flag are not shown.
List Docker Containers (only last 2 that were stopped or running)
Create Container (but not Run), Start, Restart, Re-Attach (to process using original Options, i.e. command, flags), and Delete Docker Container
Important Note: Press Enter after Re-Attach to Container (otherwise it hangs)
Daemonized (non-Interactive) Docker Containers within Vagrant VM (Docker host) for running apps and services:Create and run a Daemonized Docker Container within Vagrant. Flags detach the container to the background and performs a command until process stopped.
View under the hood of the Docker Container by debugging the last few log entries
Monitor (follow) the Docker Container log entries using the -f flag (similar to the tail -f binary) with timestamps and without reading previous entries to the log file
Nets Docker For Mac CatalinaNote: Transport Layer Security (TLS) is a cryptographic protocol that provides communications security over a network (predecessor SSL)
Inspect processes (and PIDs) of Daemonized Docker Container
Statistics (CPU, memory, network, storage I/O, metrics) of Daemonized Docker Container
Background Processes (using detach -d flag) started on already running containers are executed (management, monitoring, or maintenance) with Process Owner in Daemonized Docker Container
Note: docker exec works on already running containers, otherwise error returned:
Inspect Docker Container (config info, names, commands)
Selectively Inspect Docker Container (using GoLang template with -f flag https://golang.org/pkg/text/template/)
View Docker Container storage location on Vagrant VM (i.e. CoreOS). Start shell as superuser first
Log CollectionChapter 2 - Dockerfile, Docker Images Docker Container based on a Docker Image (based on a public image or custom) stored in Repository of a Registry
Docker Image has filesystem layers with mapping to each build step
Docker Image Management storage backend communicates with underlying Linux filesystem to build layers into usable image
Docker Image Management storage backends include fast Copy-on-Write (CoW)
Docker Image Management storage backends supported include AUFS, BTRS, Device-mapper, and overlayfs
Docker Hub is the default Registry
Top-Level Repositories are managed by Docker and vendors (i.e. ubuntu)
Top-Level Docker Images offer bare runtime to run the distribution
User Repositories are by individual developers (i.e. ltfschoen/ruby)
Tag comprises a series of image layers in single repository (representative of version control)
Docker Image is built upon the Union Mount of its parent image layers above the base image (each layer is a filesystem)
Show Docker Images downloaded from repositories that contain layers and metadata
Go to local storage location of Docker Images
Pull specific Tag (i.e. 3.4) for Top-Level Docker Image
Search Docker Hub for publicly available Docker Images that may already be pre-installed with applications
Login/Logout of Docker Hub
Docker Build Command (using Dockerfile)Docker FileDockerfile and docker build is repeatable, transparent, and idempotent, and so is preferred over docker commit
Dockerfile instructs how Docker Image is to be assembled using application code.
Dockerfile uses DSL with instructions paired with arguments on each line of code
Dockerfile lines of code are processed from top to bottom
Each line of code creates a new Docker Image Layer.
Only layers deviating from previous Docker Image Layer must be built
Dockerfile docker build process is:1) Docker runs an initial new Docker Container from the base Docker Image (FROM)2) Docker executes instruction (first line of code) inside initial Docker Container3) Docker runs a docker commit to commit initial Docker Image Layer4) Docker runs a second new Docker Container based on the initial Docker Image Layer5) Docker executes instruction (next line of code) inside second Docker Container6) Docker runs a docker commit to commit second Docker Image Layer7) Docker runs a third new Docker Container based on the second Docker Image Layer8) Docker repeats steps 5) to 7) for subsequent lines of code to build up a Docker Container
Docker Container that is built houses the application (i.e. Rails-based app) with all dependencies
Debug errors when building from Dockerfile by running Docker Container from the last successfully created Docker Image
Dockerfile FROMDefault instance is base Linux image
Custom Docker Registry offers official framework Docker Images and Tags (i.e. Node.js). Docker Image base inherits from Docker Container defined by FROM (i.e. node:0.10.33 specifically locks Docker Image base to specific Ubuntu Linux image running Node 0.10.33)
Dockerfile MAINTAINER - provides author field metadata in Docker Image
Dockerfile LABEL - provides KV pairs of metadata in Docker Image for search purposes. Find with docker inspect
Dockerfile USERImportant: Docker Containers run processes on the host machine kernel. Do not be run as root user in Production for security
Dockerfile ENV - set shell environment variables used in build process on the image
Dockerfile RUNexecute instruction/command to append new Docker Image Layer on the current Docker Image
start and create file structure, and install dependencies
commit the new Docker Image Layer if command successful
Important: Reduce build time by abstracting common RUN processes (i.e. updates) into code lines (Docker Image layers) of Base Image
Important: Reduce rebuild time by ordering commands with code imports at the end (as all layers after introduced change are rebuilt)
Important: Combine logically grouped commands into single Dockerfile code line since each instruction creates a new Docker Image layer (i.e. use ADD to import scripts and required supporting files from local filesystem into image)
Dockerfile WORKDIR - changes working directory in image for remaining build instructions
Dockerfile CMD - final instruction launches process to run in Docker Container
Dockerfile EXPOSE - ports Docker Container listens to. docker run flag (--expose) opens these ports at runtime
Important: Single function in CMD to easy horizontal scaling of individual functions in architecture
Nets Docker For Mac InstallerBuild Custom ContainerBuild new Custom Container based on Top-Level Container that we build from a Top-Level Image
Example - Static Web ServerCreate build environment (build context) for Docker to upload to Docker Daemon when build is run.
Note: Previously we configred the Vagantfile to synchronise the following folders between Vagrant VM (CoreOS) and Host (OSX):
~/code/docker-host-coreos-share (Host)
/home/core/share (Vagrant VM)
Create Dockerfile that builds a Docker Image where the Docker Container is a static web server
Note: Perform these on Host machine since folders are synchronised
Build Dockerfile into Docker Image.Tagging with repository/image_name:version.Use -f flag to specify directory within build context containing Dockerfile.
Note: docker build must be run on the Vagrant VM (CoreOS), otherwise the following error message appears Cannot connect to the Docker daemon. Is the docker daemon running on this host?
Check Docker Images updated
Check Docker Image history (Docker Image Layers are shown)
Docker Image Templates using Build CacheDocker uses cache and re-builds from earliest image layer change.Bypass using --no-cache flag.
Dockerfile should have template at top of file with package repositories and updates so cache is hit.
Launch Container using Docker Image-d flag detaches in background for long processes (i.e. Nginx daemon)
-p flag manages network ports published by Docker at container runtime to the host machine, with options including:
Mapping of randomly assigned Docker Host port(i.e. 32768 to 61000) to Port 80 on Docker Container
Mapping of specific Docker Host port (i.e. 8080) direct binding to Port 80 on Docker Container(i.e. sudo docker run -d -p 8080:80 --name static_web ltfschoen/static_web_8080 nginx -g 'daemon off;')
Mapping of specific Interface (i.e. 127.0.0.1) and specific Docker Host port (i.e. 8080) direct binding to Port 80 on Docker Container(i.e. sudo docker run -d -p 127.0.0.1:8080:80 --name static_web ltfschoen/static_web_8080_i127_0_0_1:v1 nginx -g 'daemon off;')
Mapping of specific Interface (i.e. 127.0.0.1) and random Docker Host port to Port 80 on Docker Container(i.e. sudo docker run -d -p 127.0.0.1::80 --name static_web ltfschoen/static_web_random_i127_0_0_1 nginx -g 'daemon off;')
Mapping of random Docker Host port to Port 80 on Docker Container and publish additional ports specified with EXPOSE instructions(i.e. sudo docker run -d -P --name static_web_all ltfschoen/static_web_all nginx -g 'daemon off;')
Note: Random port is assigned when not specified
Note: Only one Docker Container may be bound to a specific port on host machine
Note: Bind UDP ports by appending to port binding /udp
Command to run (i.e. nginx -g 'daemon off;') launches Nginx in foreground to run web server
Check Port mapping on host machine assigned to a given port 80 of a Docker Container ID/name
Selectively Inspect Docker Container
Connect to the Docker Container housing the Nginx web server using cURL
Attempt with the following:
Destroy Docker Containers and re-build Docker Image with different Dockerfilewith change to EXPOSE 80 (not necessary to delete the Docker Container with docker rmi ltfschoen/static_web:v1 though due to caching)
Check port 80 is correctly binded by Nginx
Check IP Table rules that are in place
Attempt Bridged Adapter option
Error below when running vagrant up after changing from NAT to Bridge Adapter in VirtualBox GUI > Settings > Network > Adapter 1 because Adapter 3 is already set to Bridge Adapter, and making following changes to Vagrantfile:
Solved with vagrant halt, started VM from Virtual Box GUI, then goto bar menu and chose quit VM. https://github.com/mitchellh/vagrant/issues/1809Note: This solution causes another cascading problem:
Error encountered when run vagrant up again
Solved by having both Public and Private in Vagrantfilehttps://github.com/mitchellh/vagrant/issues/2748http://stackoverflow.com/questions/24984020/unable-to-connect-to-vagrant-apache-on-one-computer
Note: Still unable to get response from VM when curl from Host to Docker Container IP/port (i.e. curl 172.18.0.2:8080)
Misc Links:http://stackoverflow.com/questions/33814696/how-to-connect-to-a-docker-container-from-outside-the-host-same-network-windohttps://www.virtualbox.org/manual/ch06.htmlhttp://stackoverflow.com/questions/25433488/vagrant-can-not-ping-guest-machine-from-the-hosthttp://serverfault.com/questions/495914/vagrant-slow-internet-connection-in-guesthttps://friendsofvagrant.github.io/v1/docs/bridged_networking.html
TODO Default Router approach:http://superuser.com/questions/752954/need-to-do-bridged-adapter-only-in-vagrant-no-nathttps://www.vagrantup.com/docs/networking/public_network.html
TODO Key Link:Alternative using Docker Compose instead of Vagrant https://hharnisc.github.io/
In progress up to CMD section on page 99
Other LinksOtherNote: Docker Toolbox is for older Macs.
Uninstall Docker Toolbox. Download shell script and run with sudo bash uninstall.shhttps://github.com/docker/toolbox/blob/master/osx/uninstall.sh
TODORelevant folders:
/Users/Ls/code/docker-host-coreos-share/ltfschoen/static_web
/Users/Ls/code/docker-host/coreos-vagrant
Learn Chef with Docker
Control Docker Daemon with: dockerd
Apache Kafka (broker) and Apache Zookeeper (run using Docker)
https://github.com/HackerHappyHour/docker-toolbox-vagrant
https://docs.docker.com/engine/tutorials/dockerimages/#building-an-image-from-a-dockerfile
https://docs.docker.com/engine/reference/builder/
https://www.percona.com/blog/2016/05/11/quick-start-mysql-testing-using-docker-mac/