容器集群管理之Swarm

作者: 风 哥 分类: Docker 发布时间: 2019-01-11 15:03

Swarm介绍

Swarm是什么?

Swarm是Docker公司自研发的容器集群管理系统,Swarm在早期是作为一个独立服务存在,在Docker Engine v1.12中集成了 Swarm的集群管理和编排功能。可以通过初始化Swarm或加入现有Swarm来启用Docker引擎的Swarm模式。 Docker Engine CLI和API包括了管理Swarm节点命令,比如添加、删除节点,以及在Swarm中部署和编排服务。 也增加了服务栈(Stack)、服务(Service)、任务(Task)概念。

 

Swarm两种角色:

Manager:接收客户端服务定义,将任务发送到worker节点;维护集群期望状态和集群管理功能及Leader选举。默认情况下 manager节点也会运行任务,也可以配置只做管理任务。 Worker:接收并执行从管理节点分配的任务,并报告任务当前状态,以便管理节点维护每个服务期望状态

 

Swarm特点:

1. Docker Engine集成集群管理

使用Docker Engine CLI 创建一个Docker Engine的Swarm模式,在集群中部署应用程序服务。

2. 去中心化设计

Swarm角色分为Manager和Worker节点,Manager节点故障不影响应用使用。

3. 扩容缩容

可以声明每个服务运行的容器数量,通过添加或删除容器数自动调整期望的状态。

4. 期望状态协调

Swarm Manager节点不断监视集群状态,并调整当前状态与期望状态之间的差异。例如,设置一个服务运行10个副本容器,如果两个副本的服务器节点崩溃,Manager将创建两个新的副本替代崩溃的副本。并将新的副本分配到可用的worker节点。

5. 多主机网络

可以为服务指定overlay网络。当初始化或更新应用程序时,Swarm manager会自动为overlay网络上的容器分配IP地址。

6. 服务发现

Swarm manager节点为集群中的每个服务分配唯一的DNS记录和负载均衡VIP。可以通过Swarm内置的DNS服务器查询集群中每个运行的容器。

7. 负载均衡

实现服务副本负载均衡,提供入口访问。也可以将服务入口暴露给外部负载均衡器再次负载均衡。

8. 安全传输

Swarm中的每个节点使用TLS相互验证和加密,确保安全的其他节点通信。

9. 滚动更新

升级时,逐步将应用服务更新到节点,如果出现问题,可以将任务回滚到先前版本。

  • 集群部署及节点管理

使用swarm前提:

Docker版本1.12+

集群节点之间保证TCP 2377、TCP/UDP 7946和UDP 4789端口通信

节点规划:

操作系统:CentOS Linux release 7.4.1708 (Core)

管理节点:docker-manager 172.17.16.4

工作节点:docker-worker01 172.17.16.17

工作节点:docker-worker02 172.17.16.15

 

我们在管理节点初始化swarm,把生成的token复制到另外两个worker节点去执行

< 8  docker-manager - [root]: ~ > # docker swarm init --advertise-addr 172.17.16.4
Swarm initialized: current node (cuvo8o1dbqhrxf02vmdjsd1f9) is now a manager.

To add a worker to this swarm, run the following command:

    docker swarm join --token SWMTKN-1-0dezidfvydgcat6xfjxv7j6h8j6132yr00sbkjxujizyb2xx6m-4kg8jkg2e8qh7u918ojoctnwq 172.17.16.4:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
两台工作节点加入swarm:docker swarm join –token SWMTKN-1-0dezidfvydgcat6xfjxv7j6h8j6132yr00sbkjxujizyb2xx6m-4kg8jkg2e8qh7u918ojoctnwq 172.17.16.4:2377
[root@docker-worker01 ~]# docker swarm join --token SWMTKN-1-0dezidfvydgcat6xfjxv7j6h8j6132yr00sbkjxujizyb2xx6m-4kg8jkg2e8qh7u918ojoctnwq 172.17.16.4:2377
This node joined a swarm as a worker.
[root@docker-worker02 ~]# docker swarm join --token SWMTKN-1-0dezidfvydgcat6xfjxv7j6h8j6132yr00sbkjxujizyb2xx6m-4kg8jkg2e8qh7u918ojoctnwq 172.17.16.4:2377
This node joined a swarm as a worker.

在管理节点docker-manager,通过执行命令查看加入的节点信息

< 15  docker-manager - [root]: ~ > # docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
cuvo8o1dbqhrxf02vmdjsd1f9 *   docker-manager      Ready               Active              Leader              18.09.0
q7vjyerffgvs8hkfwoom4dr5r     docker-worker01     Ready               Active                                  18.09.0
nw7lcg0bgxlmqbkmy4898fc1o     docker-worker02     Ready               Active                                  18.09.0

通过 inspect –pretty可以更友好的看到worker工作节点的信息, json格式返回,这里面显示了主机名、ip、加入的时间等等…

< 2  docker-manager - [root]: ~ > # docker node inspect --pretty docker-worker01
ID:			q7vjyerffgvs8hkfwoom4dr5r
Hostname:              	docker-worker01
Joined at:             	2019-01-10 10:30:46.577565321 +0000 utc
Status:
 State:			Ready
 Availability:         	Active
 Address:		172.17.16.17
Platform:
 Operating System:	linux
 Architecture:		x86_64
Resources:
 CPUs:			2
 Memory:		15.51GiB
Plugins:
 Log:		awslogs, fluentd, gcplogs, gelf, journald, json-file, local, logentries, splunk, syslog
 Network:		bridge, host, macvlan, null, overlay, weavemesh
 Volume:		local
Engine Version:		18.09.0
TLS Info:
 TrustRoot:
-----BEGIN CERTIFICATE-----
MIIBaTCCARCgAwIBAgIULgmTxe/WCzKdKlh27dq+zMdp2NMwCgYIKoZIzj0EAwIw
EzERMA8GA1UEAxMIc3dhcm0tY2EwHhcNMTkwMTEwMTAyMTAwWhcNMzkwMTA1MTAy
MTAwWjATMREwDwYDVQQDEwhzd2FybS1jYTBZMBMGByqGSM49AgEGCCqGSM49AwEH
A0IABNiv2KnwJdQPLPEiPnc7gMI+teV+45ehv63OGCGRvrTDiMzTWlgWgLD69S9O
+NKWhVOI2UIODIgFNELsT3RPFf6jQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMB
Af8EBTADAQH/MB0GA1UdDgQWBBStRniDW6LNtOplrXGjcmq2UXOsqjAKBggqhkjO
PQQDAgNHADBEAiAaAPXGLJICLP196rYh5hPoK8cxMlOJyp890x/iqYV9NwIgSCm4
VXY8Y6FrCd+FUoMBvoyRx9NxiLoTOHbQRrU+Qyg=
-----END CERTIFICATE-----

 Issuer Subject:	MBMxETAPBgNVBAMTCHN3YXJtLWNh
 Issuer Public Key:	MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE2K/YqfAl1A8s8SI+dzuAwj615X7jl6G/rc4YIZG+tMOIzNNaWBaAsPr1L0740paFU4jZQg4MiAU0QuxPdE8V/g==
  • 服务管理

  • 服务创建

接下来使用help帮助查看操作SERVICE的有哪些命令可以使用

< 3  docker-manager - [root]: ~ > # docker service --help

Usage:	docker service COMMAND

Manage services

Commands:
  create      Create a new service
  inspect     Display detailed information on one or more services
  logs        Fetch the logs of a service or task
  ls          List services
  ps          List the tasks of one or more services
  rm          Remove one or more services
  rollback    Revert changes to a service's configuration
  scale       Scale one or multiple replicated services
  update      Update a service

Run 'docker service COMMAND --help' for more information on a command.此

创建nginx服务,设置了2个副本数,三台主机应该有两台主机是有web nginx容器的,使用docker ps 可以看到会随机调度到任意中的机器创建的容器

< 5  docker-manager - [root]: ~ > # docker service create --replicas  2 --name web nginx
9ebvf56obpdtzywrvauaqqy12
overall progress: 2 out of 2 tasks
1/2: running   [==================================================>]
2/2: running   [==================================================>]
< 6  docker-manager - [root]: ~ > # docker service  ls
ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
9ebvf56obpdt        web                 replicated          2/2                 nginx:latest



< 7  docker-manager - [root]: ~ > # docker ps
CONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS               NAMES
53d56ac66642        nginx:latest             "nginx -g 'daemon of…"   3 minutes ago       Up 3 minutes        80/tcp              web.1.fpzsi9b4vex78sez7u1rbcjb1

查看节点容器,此时刚刚创建的容器被分配到docker-manager和docker-worker02这两台主机了。

[root@docker-worker02 ~]# docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS              PORTS               NAMES
7818e23a9737        nginx:latest        "nginx -g 'daemon of…"   About a minute ago   Up About a minute   80/tcp              web.2.fdgioqww8afvp7xbhiz569zyd

接下来我们对服务进行更新,设置6个副本数,然后修改nginx的默认页面内容,三台主机各创建2个容器

< 62  docker-manager - [root]: ~ > # docker service update  --replicas 6  web
web
overall progress: 6 out of 6 tasks
1/6: running   [==================================================>]
2/6: running   [==================================================>]
3/6: running   [==================================================>]
4/6: running   [==================================================>]
5/6: running   [==================================================>]
6/6: running   [==================================================>]
verify: Service converged
< 63  docker-manager - [root]: ~ > # docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS              PORTS               NAMES
2d923b06af75        nginx:latest        "nginx -g 'daemon of…"   32 seconds ago       Up 31 seconds       80/tcp              web.6.nly6fnlroqh6enax6r5ovpc63
c2b86cb70086        nginx:latest        "nginx -g 'daemon of…"   About a minute ago   Up About a minute   80/tcp              web.2.mqzbx2cn0t00ql1vs46w5xad0

查看详细的信息,ps -f “desired-state=running” 过滤正在运行的容器

< 64  docker-manager - [root]: ~ > # docker service ps web
ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE            ERROR               PORTS
smp1g3479n9p        web.1               nginx:latest        docker-worker02     Running             Running 21 minutes ago
mqzbx2cn0t00        web.2               nginx:latest        docker-manager      Running             Running 21 minutes ago
th83ifp3y97v        web.3               nginx:latest        docker-worker01     Running             Running 21 minutes ago
9jzbvs2d6dci        web.4               nginx:latest        docker-worker01     Running             Running 20 minutes ago
pzcykxzc8b1d        web.5               nginx:latest        docker-worker02     Running             Running 20 minutes ago
nly6fnlroqh6        web.6               nginx:latest        docker-manager      Running             Running 20 minutes ago

扩容、缩容服务实例数

< 67  docker-manager - [root]: ~ > # docker service scale web=3

滚动更新服务,创建redis:3.0.6版本,然后将其更新到3.0.7

< 72  docker-manager - [root]: ~ > # docker service create --replicas 3 --name redis --update-delay 10s redis:3.0.6
k9f0fxjai4oca1gcyah9llfnl
overall progress: 3 out of 3 tasks
1/3: running   [==================================================>]
2/3: running   [==================================================>]
3/3: running   [==================================================>]
verify: Service converged
< 73  docker-manager - [root]: ~ > #
< 73  docker-manager - [root]: ~ > # docker service ps redis
ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE           ERROR               PORTS
lxluks7zeteo        redis.1             redis:3.0.6         docker-worker02     Running             Running 3 minutes ago
sdyltojn8ylj        redis.2             redis:3.0.6         docker-worker01     Running             Running 3 minutes ago
l6yrufkih9b2        redis.3             redis:3.0.6         docker-manager      Running             Running 3 minutes ago
< 74  docker-manager - [root]: ~ > # docker service update --image redis:3.0.7 redis
redis
overall progress: 3 out of 3 tasks
1/3: running   [==================================================>]
2/3: running   [==================================================>]
3/3: running   [==================================================>]
verify: Service converged

使用docker service ps 查看刚才滚动更新的redis信息,redis:3.0.6处于Shutdown的状态,而redis:3.0.7已经处于Running的状态了。

< 76  docker-manager - [root]: ~ > # docker service ps redis
ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE                 ERROR               PORTS
jz7rgblwhu3u        redis.1             redis:3.0.7         docker-worker02     Running             Running 49 seconds ago
lxluks7zeteo         \_ redis.1         redis:3.0.6         docker-worker02     Shutdown            Shutdown 56 seconds ago
dbfa4oewsx6o        redis.2             redis:3.0.7         docker-worker01     Running             Running 16 seconds ago
sdyltojn8ylj         \_ redis.2         redis:3.0.6         docker-worker01     Shutdown            Shutdown 37 seconds ago
7jogokleuvgi        redis.3             redis:3.0.7         docker-manager      Running             Running about a minute ago
l6yrufkih9b2         \_ redis.3         redis:3.0.6         docker-manager      Shutdown            Shutdown about a minute ago

创建服务时设定更新策略

< 77  docker-manager - [root]: ~ > # docker service create --name my_web  --replicas 10  --update-delay 10s  --update-parallelism 2  --update-failure-action continue  nginx:1.12
ss8y95qykz0hwv2ovyyhz9s1a
overall progress: 10 out of 10 tasks
1/10: running   [==================================================>]
2/10: running   [==================================================>]
3/10: running   [==================================================>]
4/10: running   [==================================================>]
5/10: running   [==================================================>]
6/10: running   [==================================================>]
7/10: running   [==================================================>]
8/10: running   [==================================================>]
9/10: running   [==================================================>]
10/10: running   [==================================================>]
verify: Service converged
< 79  docker-manager - [root]: ~ > # docker service ps my_web
ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE            ERROR               PORTS
qcoeap5033fq        my_web.1            nginx:1.12          docker-manager      Running             Running 50 seconds ago
31wrqcgy4idp        my_web.2            nginx:1.12          docker-worker01     Running             Running 50 seconds ago
ks7te604lrjy        my_web.3            nginx:1.12          docker-worker02     Running             Running 49 seconds ago
4bp5og72iox7        my_web.4            nginx:1.12          docker-manager      Running             Running 50 seconds ago
et014m1wx268        my_web.5            nginx:1.12          docker-worker02     Running             Running 48 seconds ago
6xvt54u6jcef        my_web.6            nginx:1.12          docker-worker01     Running             Running 50 seconds ago
kv3vca0aznp7        my_web.7            nginx:1.12          docker-worker02     Running             Running 48 seconds ago
hgezfadql3qh        my_web.8            nginx:1.12          docker-worker01     Running             Running 50 seconds ago
r018mxqdphua        my_web.9            nginx:1.12          docker-manager      Running             Running 50 seconds ago
bwia124bxu33        my_web.10           nginx:1.12          docker-worker01     Running             Running 50 seconds ago

创建服务时设定回滚策略,把之前所有的服务全部删除掉 docker service rm {服务名称}

< 84  docker-manager - [root]: ~ > # docker service create  --name my_web  --replicas 10  --rollback-parallelism 2  --rollback-monitor 20s  --rollback-max-failure-ratio .2  nginx:1.12
unuve7qkcnltz0uxxsir2wbsv
overall progress: 10 out of 10 tasks
1/10: running   [==================================================>]
2/10: running   [==================================================>]
3/10: running   [==================================================>]
4/10: running   [==================================================>]
5/10: running   [==================================================>]
6/10: running   [==================================================>]
7/10: running   [==================================================>]
8/10: running   [==================================================>]
9/10: running   [==================================================>]
10/10: running   [==================================================>]
verify: Service converged
服务更新

把nginx1.12版本升级到1.13

< 86  docker-manager - [root]: ~ > # docker service update --image nginx:1.13 my_web
my_web
overall progress: 10 out of 10 tasks
1/10: running   [==================================================>]
2/10: running   [==================================================>]
3/10: running   [==================================================>]
4/10: running   [==================================================>]
5/10: running   [==================================================>]
6/10: running   [==================================================>]
7/10: running   [==================================================>]
8/10: running   [==================================================>]
9/10: running   [==================================================>]
10/10: running   [==================================================>]
verify: Service converged
手动回滚

把nginx1.13版本回滚到1.12版本


< 90  docker-manager - [root]: /var/log > # docker service update --rollback my_web
my_web
rollback: manually requested rollback
overall progress: rolling back update: 10 out of 10 tasks
1/10: running   [>                                                  ]
2/10: running   [>                                                  ]
3/10: running   [>                                                  ]
4/10: running   [>                                                  ]
5/10: running   [>                                                  ]
6/10: running   [>                                                  ]
7/10: running   [>                                                  ]
8/10: running   [>                                                  ]
9/10: running   [>                                                  ]
10/10: running   [>                                                  ]
verify: Service converged
< 91  docker-manager - [root]: /var/log > # docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                      PORTS               NAMES
f6a6f5a3fd15        nginx:1.12          "nginx -g 'daemon of…"   54 seconds ago      Up 52 seconds               80/tcp              my_web.9.bscak7oxxp4aresyjz7ui8xl4
e282e9c8b20a        nginx:1.12          "nginx -g 'daemon of…"   54 seconds ago      Up 52 seconds               80/tcp              my_web.8.gi5m641lpqkavtadmnr6ua82x
36b1102115a4        nginx:1.12          "nginx -g 'daemon of…"   56 seconds ago      Up 54 seconds               80/tcp              my_web.3.ltvpyb4pr85cfeyae1xn2qodw
e1e7731ab205        nginx:1.13          "nginx -g 'daemon of…"   3 hours ago         Exited (0) 53 seconds ago                       my_web.8.zb19e2jom4wl0k2mwsrmi5446
105cef6ad830        nginx:1.13          "nginx -g 'daemon of…"   3 hours ago         Exited (0) 50 seconds ago                       my_web.1.pciu1ou9p2t76idy7n2br2yk8
542e7f279ab4        nginx:1.13          "nginx -g 'daemon of…"   3 hours ago         Exited (0) 55 seconds ago                       my_web.10.nwvosslj49iwbtehaxf5v47jc
fdd77b2e2d31        nginx:1.13          "nginx -g 'daemon of…"   3 hours ago         Exited (0) 52 seconds ago                       my_web.9.p6m6mum1pl4ym2rsr259vt1yq
45769ae06d32        nginx:1.12          "nginx -g 'daemon of…"   3 hours ago         Exited (0) 3 hours ago                          my_web.10.9zo91gsh5pls0v8vambt1o00f
a84f62139937        nginx:1.12          "nginx -g 'daemon of…"   3 hours ago         Exited (0) 3 hours ago                          my_web.6.6s09f3rs0vhjinhiwmux9tjlo
85620a89c2f7        nginx:1.12          "nginx -g 'daemon of…"   3 hours ago         Exited (0) 3 hours ago                          my_web.1.o0l1i8svkoikdo4rgw076jz6e
  • 使用原生Overlay网络

在管理节点docker-manager创建overlay网络

< 95  docker-manager - [root]: ~ > # docker network create --driver overlay my-network
8fhh9d0bzccza5somzx8w8luq
< 96  docker-manager - [root]: ~ > # docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
5100d0900df6        bridge              bridge              local
d11876760876        docker_gwbridge     bridge              local
e257522a4748        host                host                local
u8mt10siiykl        ingress             overlay             swarm
a57ddb359d82        macvlan_net         macvlan             local
8fhh9d0bzccz        my-network          overlay             swarm
78a4942e9715        none                null                local
50dee1256a9e        weave               weavemesh           local
< 101  docker-manager - [root]: ~ > # docker service create  --replicas 3  --network my-network  --name my-overlay  busybox
xhvskk2i3srkxf0lgvpmpnxmz
overall progress: 0 out of 3 tasks
< 106  docker-manager - [root]: ~ > # docker service update --args "ping baidu.com" my-overlay
my-overlay
overall progress: 3 out of 3 tasks
1/3: running   [==================================================>]
2/3: running   [==================================================>]
3/3: running   [==================================================>]
verify: Service converged
< 107  docker-manager - [root]: ~ > # docker service ps my-overlay
ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE             ERROR               PORTS
p2n5virrurei        my-overlay.1        busybox:latest      docker-manager      Running             Running 23 seconds ago
spco6rzdm1au         \_ my-overlay.1    busybox:latest      docker-manager      Shutdown            Shutdown 24 seconds ago
o2u5txkh7lgz         \_ my-overlay.1    busybox:latest      docker-manager      Shutdown            Complete 33 seconds ago
bsij242nxuan         \_ my-overlay.1    busybox:latest      docker-manager      Shutdown            Complete 39 seconds ago
jlccbsx1yxco         \_ my-overlay.1    busybox:latest      docker-manager      Shutdown            Complete 46 seconds ago
muwlna5np2jy        my-overlay.2        busybox:latest      docker-worker01     Running             Running 17 seconds ago
onrd0j9ves2n         \_ my-overlay.2    busybox:latest      docker-worker01     Shutdown            Complete 23 seconds ago
p36ggt7c1pok         \_ my-overlay.2    busybox:latest      docker-worker01     Shutdown            Complete 30 seconds ago
7obqzwsljo64         \_ my-overlay.2    busybox:latest      docker-worker01     Shutdown            Complete 36 seconds ago
fchhvgh5un2x         \_ my-overlay.2    busybox:latest      docker-worker01     Shutdown            Complete 42 seconds ago
u0flsw5tew3w        my-overlay.3        busybox:latest      docker-worker02     Running             Running 22 seconds ago
xt88afdgcobh         \_ my-overlay.3    busybox:latest      docker-worker02     Shutdown            Complete 22 seconds ago
tj49ug3awqng         \_ my-overlay.3    busybox:latest      docker-worker02     Shutdown            Complete 29 seconds ago
awh6mnsjv3db         \_ my-overlay.3    busybox:latest      docker-worker02     Shutdown            Complete 35 seconds ago
mjqa6krk5ac2         \_ my-overlay.3    busybox:latest      docker-worker02     Shutdown            Complete 42 seconds ago

在两个worker节点运行busybox 相互ping测试,此时网络互通

[root@docker-worker01 ~]# docker exec -it 8e8b9b4a6d6d sh
/ # ifconfig
eth0      Link encap:Ethernet  HWaddr 02:42:0A:00:00:BC
          inet addr:10.0.0.188  Bcast:10.0.0.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1450  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth1      Link encap:Ethernet  HWaddr 02:42:AC:13:00:03
          inet addr:172.19.0.3  Bcast:172.19.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:121 errors:0 dropped:0 overruns:0 frame:0
          TX packets:121 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:11315 (11.0 KiB)  TX bytes:11240 (10.9 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:4 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:295 (295.0 B)  TX bytes:295 (295.0 B)

/ #
/ # ping 10.0.0.190
PING 10.0.0.190 (10.0.0.190): 56 data bytes
64 bytes from 10.0.0.190: seq=0 ttl=64 time=0.656 ms
64 bytes from 10.0.0.190: seq=1 ttl=64 time=0.547 ms

将现有服务连接到overlay网络

docker service update –network-add {my-network} my-web

删除正在运行的服务网络连接

docker service update –network-rm {my-network} my-web

  • 数据持久化

  • volume

docker-manager创建数据卷,worket01和worker02同步创建的数据卷信息

< 7  docker-manager - [root]: ~ > # docker service create --mount 'type=volume,src=test,dst=/apps' --name v3-hello nginx
ogtzb3rof4tgwtols0p2c0vtw
overall progress: 1 out of 1 tasks
1/1: running   [==================================================>]
verify: Service converged
[root@docker-worker02 ~]# docker volume ls
DRIVER              VOLUME NAME
local               test

服务启动到docker-manager,进入到容器里面查看数据是否存在

< 12  docker-manager - [root]: ~ > # docker service ps v3-hello
ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE           ERROR               PORTS
e9p44zhjxn31        v3-hello.1          nginx:latest        docker-manager      Running             Running 3 minutes ago
< 2  docker-manager - [root]: ~ > # cd /var/lib/docker/volumes/test/_data/
< 3  docker-manager - [root]: /var/lib/docker/volumes/test/_data > # ll
total 0
< 4  docker-manager - [root]: /var/lib/docker/volumes/test/_data > # echo "volume test" >test.txt
< 5  docker-manager - [root]: /var/lib/docker/volumes/test/_data > # ll
total 4
-rw-r--r-- 1 root root 12 2019-01-12 11:19 test.txt
< 6  docker-manager - [root]: /var/lib/docker/volumes/test/_data > #
< 6  docker-manager - [root]: /var/lib/docker/volumes/test/_data > # docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                    PORTS               NAMES
7a0ec355fca5        nginx:latest        "nginx -g 'daemon of…"   26 minutes ago      Up 26 minutes             80/tcp              v3-hello.1.e9p44zhjxn315v5kunakt5jx4
674f8723bc08        busybox:latest      "ping baidu.com"         20 hours ago        Up 20 hours                                   my-overlay.1.p2n5virrureiubtgyjynbze7a
141cabb3b066        busybox:latest      "sh"                     20 hours ago        Exited (0) 20 hours ago                       my-overlay.1.spco6rzdm1au1k003e4hq1wli
ae903eafedfa        busybox:latest      "sh"                     20 hours ago        Exited (0) 20 hours ago                       my-overlay.1.o2u5txkh7lgzog5esogmn5hq1
5f60e7fcbddf        busybox:latest      "sh"                     20 hours ago        Exited (0) 20 hours ago                       my-overlay.1.bsij242nxuanvdqey1f74npxm
0c8321597763        busybox:latest      "sh"                     20 hours ago        Exited (0) 20 hours ago                       my-overlay.1.jlccbsx1yxcolpgrrc7dti3kd
< 7  docker-manager - [root]: /var/lib/docker/volumes/test/_data > # docker exec -it 7a0ec355fca5 bash
root@7a0ec355fca5:/# cat /apps/test.txt
volume test
root@7a0ec355fca5:/#

查看数据卷详细信息

docker volume inspect <VOLUME-NAME>
使用NFS共享存储作为数据卷 

三台机器都需要安装nfs软件

< 1  docker-manager - [root]: ~ > # yum install -y nfs-utils

#配置nfs共享目录
< 3  docker-manager - [root]: ~ > # cat /etc/exports
/apps/docker/ 172.17.16.0/24(rw,sync,all_squash,no_root_squash,anonuid=501,anongid=501)



< 12 docker-manager - [root]: ~ > # groupadd nfs -g501
< 13 docker-manager - [root]: ~ > # useradd -s /sbin/noligin nfs -u501 -g501 -M
< 14 docker-manager - [root]: ~ > # id nfs
uid=501(nfs) gid=501(nfs) groups=501(nfs)





< 4  docker-manager - [root]: ~ > # exportfs -rv
exporting 172.17.16.0/24:/apps/docke

#启动服务

< 5  docker-manager - [root]: ~ > # systemctl restart nfs



配置客户端

worker01和worker02两个节点都要安装nfs-utils

[root@docker-worker01 ~]# yum install -y nfs-utils
[root@docker-worker02 ~]# yum install -y nfs-utils

测试连接性

[root@docker-worker01 ~]#  showmount -e  172.17.16.4
Export list for 172.17.16.4:
/apps/docker 172.17.16.0/24

在docker-manager管理节点创建服务

docker service create --mount 'type=volume,src=nfs-test,dst=/usr/share/nginx/html,volume-driver=local,volume-opt=type=nfs,volume-opt=device=172.17.16.4:/apps/docker,"volume-opt=o=addr=172.17.16.4,vers=4,soft,timeo=180,bg,tcp,rw"'  --name my-nfs-nginx nginx
qxl9gfntig0avsgf4hmdvzlzi
overall progress: 1 out of 1 tasks
1/1: running   [==================================================>]
verify: Service converged

docker-worker01和docker-worker02节点上,进入容器后touch一个文件,在docker-manager主机会同步

bind挂载、读写
注意:在使用bind方式挂载时,一定要确保挂载的目录为空,否则的话挂载目录的数据将会丢失。

< 9  docker-manager - [root]: ~ > # docker service create --mount type=bind,src=/var/log,dst=/data --name test3-bind busybox ping baidu.com
vfwqh9etazrth0szc4w232ahw
overall progress: 1 out of 1 tasks
1/1: running   [==================================================>]
verify: Service converged

通过docker inspect test3-bind*查看容器的详细信息

在docker-worker节点进入容器查看挂载目录是否成功

[root@docker-worker02 ~]# docker exec -it ef3a1198e6dd sh
/ # ls /data/
anaconda                        boot.log-20190112               cron                            messages                        secure-201806211529522941.gz    secure-201809011535771642.gz    secure-201810201540006082.gz    tuned
audit                           boot.log-20190113               dmesg                           messages-201812101544413502.gz  secure-201806271530040981.gz    secure-201809101536551223.gz    secure-201810241540351201.gz    wtmp
boot.log                        btmp                            dmesg.old                       ntpstats                        secure-201807021530472261.gz    secure-201809181537242241.gz    secure-201810291540782361.gz    yum.log
boot.log-20190107               btmp-20190101                   firewalld                       qcloud_action.log               secure-201807071530905821.gz    secure-201809221537585622.gz    secure-201811091541733182.gz    yum.log-20190101
boot.log-20190108               chrony                          grubby_prune_debug              qemu-ga                         secure-201807191531942381.gz    secure-201809271538018881.gz    secure-201811141542166382.gz    zabbix
boot.log-20190109               cloud-init-output.log           journal                         rhsm                            secure-201807291532805481.gz    secure-201810051538709301.gz    secure-201811241543030381.gz
boot.log-20190110               cloud-init.log                  lastlog                         samba                           secure-201808111533958141.gz    secure-201810081538968082.gz    spooler
boot.log-20190111               cluster                         maillog                         secure                          secure-201808191534649461.gz    secure-201810161539661202.gz    tallylog
/ #

bind挂载,只读方式

docker service create \
--mount type=bind,src=<HOST-PATH>,dst=<CONTAINER-PATH>,readonly \
--name myservice \
<IMAGE>
  • 服务发现与负载均衡

Swarm模式内置DNS组件,可以自动为集群中的每个服务分配DNS记录。Swarm manager使用内部负载均衡,根据服务的DNS名称在集群内的服务 之间分发请求。

Swarm manager使用 ingress load blancing暴露你想从外部访问集群提供的服务。Swarm manager自动为服务分配一个范围30000-32767端口 的Published Port,也可以为该服务指定一个Published Port。

ingress network是一个特殊的overlay网络,便于服务的节点直接负载均衡。当任何swarm节点在已发布的端口上接收到请求时,它将该请求 转发给调用的IPVS模块,IPVS跟踪参与该服务的所有容器IP地址,选择其中一个,并通过ingress network将请求路由给它。

创建web服务

docker service create --replicas 3 --network my-network --name my-web nginx
voev6u0lm9aw0xzwcvxvjl7aa
overall progress: 3 out of 3 tasks
1/3: running   [==================================================>]
2/3: running   [==================================================>]
3/3: running   [==================================================>]
verify: Service converged

在创建一个busybox网络测试工具,用来测试nsloockup解析记录

< 18  docker-manager - [root]: ~ > # docker service create --replicas 3 --network my-network --name my-hello busybox ping baidu.com
xeeox8kxwcn9quj2pzo1do4ly
overall progress: 3 out of 3 tasks
1/3: running   [==================================================>]
2/3: running   [==================================================>]
3/3: running   [==================================================>]
verify: Service converged

进入容器查看DNS记录

[root@docker-worker01 ~]# docker exec -it d1e6758cc7b3 sh
/ # nslookup  my-web
Server:		127.0.0.11
Address:	127.0.0.11:53

获取虚拟Ip

< 47  docker-manager - [root]: ~ > # docker service inspect -f '{{json .Endpoint.VirtualIPs}}' my-hello
[{"NetworkID":"8fhh9d0bzccza5somzx8w8luq","Addr":"10.0.0.227/24"}]

设置DNS轮询模式–endpoint-mode ,访问任意一个主机IP都会请求到集群中的内容

< 29  docker-manager - [root]: ~ > # docker service create --replicas 3 --network my-network --name my-web --endpoint-mode dnsrr   nginx
w6bvdsqzr38i4h2u80zsyagyw
overall progress: 3 out of 3 tasks
1/3: running   [==================================================>]
2/3: running   [==================================================>]
3/3: running   [==================================================>]
verify: Service converged

 

为了保证服务的可用性可以使用 HAProxy 来添加外部负载均衡

  • 高可用性

在swarm设计中如果manager节点挂掉之后,并不会影响服务的访问性,此时可以通过worker节点依旧可以提供服务

Manager节点任务:

维护集群状态
调度服务
提供swarm模式的HTTP API

为了利用swarm模式的容错功能,应保持集群中奇数管理员来支持manager节点故障。当leader故障时,会选举新的leader。

故障恢复:

如果swarm失去法定人数,swarm不能自动恢复,工作节点上的任务继续运行,不受影响,但无法执行管理任务,包括扩展或 更新服务,加入或删除节点。恢复的最佳方式是将丢失的leader节点重新联机。如果不可能,唯一方法是使用—force-newcluster管理节点的操作,这将去除本机之外的所有管理器身份。

< 1  docker-manager - [root]: ~ > # docker swarm join-token --help

Usage:	docker swarm join-token [OPTIONS] (worker|manager)

Manage join tokens

Options:
  -q, --quiet    Only display token
      --rotate   Rotate join token
< 2  docker-manager - [root]: ~ > # docker swarm join-token manager
To add a manager to this swarm, run the following command:

    docker swarm join --token SWMTKN-1-0dezidfvydgcat6xfjxv7j6h8j6132yr00sbkjxujizyb2xx6m-9fg85nmnrmyxi66b85iqjh1wy 172.17.16.4:2377

手动提升node的角色为manager, 提升之后状态为Reachable备用,当停止manager之后,会自动选举其中一个node为manager管理

< 5  docker-manager - [root]: ~ > # docker node promote docker-worker01
Node docker-worker01 promoted to a manager in the swarm.
< 6  docker-manager - [root]: ~ > # docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
cuvo8o1dbqhrxf02vmdjsd1f9 *   docker-manager      Ready               Active              Leader              18.09.0
q7vjyerffgvs8hkfwoom4dr5r     docker-worker01     Ready               Active              Reachable           18.09.0
nw7lcg0bgxlmqbkmy4898fc1o     docker-worker02     Ready               Active                                  18.09.0

加入一个新节点

docker swarm init –force-new-cluster –advertise-addr 192.168.1.111:2377

  • 配置文件存储

生成一个基本的Nginx配置文件

< 18  docker-manager - [root]: ~ > # cat site.conf
server {
  listen 80;
  server_name localhost;
  location / {
      root /usr/share/nginx/html;
      index index.html index.htm;
    }
}

将site.conf保存到docker配置中

< 1  docker-manager - [root]: ~ > # docker config create site.conf site.conf
34uhgjcfcddb43kwl459wwhjk
< 2  docker-manager - [root]: ~ > # docker config ls
ID                          NAME                CREATED             UPDATED
34uhgjcfcddb43kwl459wwhjk   site.conf           6 seconds ago       6 seconds ago

创建一个Nginx并应用这个配置

< 4  docker-manager - [root]: ~ > # docker service create --name nginx --config source=site.conf,target=/etc/nginx/conf.d/site.conf --publish 8080:80 nginx:latest
w8x7ph9c0nlrub5fmh52xzs6m
overall progress: 1 out of 1 tasks
1/1: running   [==================================================>]
verify: Service converged

进入到nginx容器中可以看到共享存储的文件

[root@docker-worker01 ~]# docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
196df9d4bd3b        nginx:latest        "nginx -g 'daemon of…"   4 minutes ago       Up 4 minutes        80/tcp              nginx.1.dkshawp9e7qlmu2wdukyfng8v
[root@docker-worker01 ~]# docker exec -it 196df9d4bd3b bash
root@196df9d4bd3b:/# ls /etc/nginx/conf.d/
default.conf  site.conf
root@196df9d4bd3b:/#

 

 

发表评论

电子邮件地址不会被公开。 必填项已用*标注