lxc(linux containers)是一种基于容器的操作系统层级的虚拟化技术,一个容器就是一个虚拟的执行环境,容器有效地将由单个操作系统管理的资源划分到孤立的组中,以更好地在孤立的组之间平衡有冲突的资源使用需求。
一、如何实现lxc
lxc在资源管理方面依赖于linux内核的cgroups(control groups)子系统,cgroups子系统是linux内核提供的一种可以限制、记录、隔离进程组(process groups)所使用的物理资源(如:CPU, Memory, IO等)的机制。LXC在资源隔离控制上依赖于Linux内核的Namespaces机制,PID,IPC,Network等系统资源不再是全局性的,而是属于特定的Namespace。要创建新的Namespace,只需要在调用clone时指定相应的flag。不同container内的进程属于不同的Namespace,彼此透明,互不干扰。
二、相对于硬件层面的虚拟化技术,lxc有如下优点:
⑴更小的性能开销
⑵布署简单快速
三、lxc安装配置
1、先安装cgroup
yum -y install libcgroup
service cgconfig start #cgroup对应的服务名称为cgconfig
[root@node3 ~]# yum -y install libcgroup
...
[root@node3 ~]# service cgconfig start
Starting cgconfig service: [ OK ]
2、提供虚拟网桥接口
在/etc/sysconfig/network-scripts目录中新建名为ifcfg-br0的配置文件,其内容如下:
DEVICE=br0
TYPE=Bridge
BOOTPROTO=static
IPADDR=172.16.100.7
NETMASK=255.255.0.0
GATEWAY=172.16.0.1
ONBOOT=yes
DELAY=0
NM_CONTROLLED=no
接下将桥接的网卡(假设为eth0)关联至前面定义的桥接设备,编辑/etc/sysconfig/network-script/ifcfg-eth0为类似如下内容:
DEVICE=eth0
BOOTPROTO=static
NM_CONTROLLED=no
ONBOOT=yes
TYPE=Ethernet
BRIDGE=br0
上述步骤无误后重启network服务即可。另外,还有其它简单的方式来实现桥接设备的创建,例如使用brctl或virsh等。
[root@node3 ~]# cd /etc/sysconfig/network-scripts
[root@node3 network-scripts]# cp ifcfg-eth0 ifcfg-eth0:0
[root@node3 network-scripts]# vim ifcfg-eth0:0
DEVICE=eth0:0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=static
IPADDR=172.16.100.1
NETMASK=255.255.0.0
[root@node3 network-scripts]# cp ifcfg-eth0:0 ifcfg-br0
[root@node3 network-scripts]# vim ifcfg-br0 #定义虚拟网桥
DEVICE=br0
TYPE=Bridge
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=static
IPADDR=172.16.100.1
NETMASK=255.255.0.0
DELAY=0
[root@node3 network-scripts]# vim ifcfg-eth0:0 #将eth0:0关联至虚拟网桥
DEVICE=eth0:0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=static
BRIDGE=br0
[root@node3 network-scripts]# service network restart
...
[root@node3 network-scripts]# ifconfig
br0 Link encap:Ethernet HWaddr 00:00:00:00:00:00
inet addr:172.16.100.1 Bcast:172.16.255.255 Mask:255.255.0.0
inet6 addr: fe80::4c16:22ff:febb:9462/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:746 (746.0 b)
eth0 Link encap:Ethernet HWaddr 00:0C:29:CB:26:9B
inet addr:192.168.30.13 Bcast:192.168.30.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fecb:269b/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1482 errors:0 dropped:0 overruns:0 frame:0
TX packets:882 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:545127 (532.3 KiB) TX bytes:125219 (122.2 KiB)
...
3、安装lxc
epel源中提供的lxc版本未带centos系统模板。可到官网下载源码编译安装,以下安装的是根据lxc-1.0.5源码包制作成的适用于centos 6 x86_64平台的rpm包
[root@node3 ~]# yum -y install lxc-libs-1.0.5-1.el6.x86_64.rpm lxc-1.0.5-1.el6.x86_64.rpm
...
[root@node3 ~]# rpm -ql lxc
/etc/bash_completion.d
/etc/bash_completion.d/lxc
/etc/lxc/default.conf
/etc/rc.d/init.d/lxc
/usr/bin/lxc-attach
/usr/bin/lxc-autostart
/usr/bin/lxc-cgroup
/usr/bin/lxc-checkconfig
/usr/bin/lxc-clone
/usr/bin/lxc-config
/usr/bin/lxc-console
/usr/bin/lxc-create
/usr/bin/lxc-destroy
/usr/bin/lxc-execute
...
/usr/share/lxc/templates
/usr/share/lxc/templates/lxc-alpine
/usr/share/lxc/templates/lxc-altlinux
/usr/share/lxc/templates/lxc-archlinux
/usr/share/lxc/templates/lxc-busybox
/usr/share/lxc/templates/lxc-centos
....
4、检查lxc运行环境:lxc-checkconfig
[root@node3 ~]# lxc-checkconfig
Kernel configuration not found at /proc/config.gz; searching...
Kernel configuration found at /boot/config-2.6.32-431.el6.x86_64
--- Namespaces ---
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Network namespace: enabled
Multiple /dev/pts instances: enabled
--- Control groups ---
Cgroup: enabled
Cgroup namespace: enabled
Cgroup device: enabled
...
5、创建容器
模板文件目录:/usr/local/lxc/share/lxc/templates。其中的lxc-centos即为创建lxc centos系统的模板。
默认配置文件:/etc/lxc/default.conf,其中使用的桥接接口名称为virbr0,与前面的创建的接口名称不一致,因此需要作出修改。当然,也可以将此文件复制之后进行修改,并以为作为接下来的要创建的容器的专用配置文件。修改后的default.conf如下所示。
lxc.network.type = veth
lxc.network.link = br0
lxc.network.flags = up
创建容器:
lxc-create -n NAME -t TEMPLATE [-f CONFIGFILE][options]
例:lxc-create -n centos -t /usr/share/lxc/templates/lxc-centos
容器默认存放目录:/var/lib/lxc
目标容器的root用户的初始密码在/var/lib/lxc/NAME/tmp_root_pass文件中;要注意的是,root用户的初始密码已经被设置为过期,因此第一次登录时需要修改密码后方可使用。
[root@node3 ~]# vim /etc/lxc/default.conf
lxc.network.type = veth
lxc.network.link = br0 #修改成定义的虚拟网桥名称
lxc.network.flags = up
[root@node3 ~]# lxc-create -n centos2 -t /usr/share/lxc/templates/lxc-centos #创建一个名为centos2的容器
...
Complete!
Download complete.
Copy /var/cache/lxc/centos/x86_64/6/rootfs to /var/lib/lxc/centos2/rootfs ...
Copying rootfs to /var/lib/lxc/centos2/rootfs ...
Storing root password in '/var/lib/lxc/centos2/tmp_root_pass'
Expiring password for user root.
passwd: Success
Container rootfs and config have been created.
Edit the config file to check/enable networking setup.
The temporary root password is stored in:
'/var/lib/lxc/centos2/tmp_root_pass'
The root password is set up as expired and will require it to be changed
at first login, which you should do as soon as possible. If you lose the
root password or wish to change it without starting the container, you
can change it from the host by running the following command (which will
also reset the expired flag):
chroot /var/lib/lxc/centos2/rootfs passwd
[root@node3 ~]# cd /var/lib/lxc
[root@node3 lxc]# ls
centos2
[root@node3 lxc]# ls centos2
config rootfs tmp_root_pass
[root@node3 lxc]# cat centos2/tmp_root_pass
Root-centos2-pBZoiZ #初始密码
[root@node3 lxc]# ls centos2/rootfs
bin boot dev etc home lib lib64 media mnt opt proc root sbin selinux srv sys tmp usr var
[root@node3 lxc]# chroot /var/lib/lxc/centos2/rootfs passwd #更改密码
Changing password for user root.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.
6、列出容器:lxc-ls
7、启动目标容器:
启动lxc容器需要使用lxc-start命令。此命令的常用选项有:
-n NAME:要启动的容器的名称
-d:在后台运行此容器
-o /path/to/somefile:日志文件的保存位置
-l:日志级别
例:lxc-start -n centos
启动后可直接连接至目标系统的控制台,并在显示登录提示符后登录系统即可。
[root@node3 ~]# lxc-ls #显示所有容器
centos2
[root@node3 ~]# lxc-start -n centos2 #启动目标容器
CentOS release 6.7 (Final)
Kernel 2.6.32-431.el6.x86_64 on an x86_64
centos2 login: init: rcS main process (8) killed by TERM signal
Entering non-interactive startup
iptables: No config file. [WARNING]
Bringing up loopback interface: [ OK ]
Bringing up interface eth0:
Determining IP information for eth0... failed.
[FAILED]
Starting system logger: [ OK ]
Mounting filesystems: [ OK ]
Starting sshd: [ OK ]
CentOS release 6.7 (Final)
Kernel 2.6.32-431.el6.x86_64 on an x86_64
centos2 login: root
Password:
Last login: Wed Apr 6 09:24:29 on lxc/console
[root@centos2 network-scripts]# vi ifcfg-eth0 #给接口配置ip地址、网关等
DEVICE=eth0
BOOTPROTO=static
ONBOOT=yes
NM_CONTROLLED=no
TYPE=Ethernet
IPADDR=172.16.100.2
NETMASK=255.255.0.0
GATEWAY=172.16.100.1
[root@centos2 network-scripts]# ifup eth0
Determining if ip address 172.16.100.2 is already in use for device eth0...
[root@centos2 network-scripts]# ifconfig
eth0 Link encap:Ethernet HWaddr FE:2B:06:93:7C:45
inet addr:172.16.100.2 Bcast:172.16.255.255 Mask:255.255.0.0
inet6 addr: fe80::fc2b:6ff:fe93:7c45/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:12 errors:0 dropped:0 overruns:0 frame:0
TX packets:30 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:944 (944.0 b) TX bytes:1916 (1.8 KiB)
...
[root@centos2 network-scripts]# ping 172.16.100.1
PING 172.16.100.1 (172.16.100.1) 56(84) bytes of data.
64 bytes from 172.16.100.1: icmp_seq=1 ttl=64 time=0.920 ms
64 bytes from 172.16.100.1: icmp_seq=2 ttl=64 time=0.048 ms
...
[root@centos2 network-scripts]# ping 192.168.30.13
PING 192.168.30.13 (192.168.30.13) 56(84) bytes of data.
64 bytes from 192.168.30.13: icmp_seq=1 ttl=64 time=2.70 ms
64 bytes from 192.168.30.13: icmp_seq=2 ttl=64 time=0.083 ms
...
8、登录容器:lxc-console -n NAME #也可通过ssh的方式登录
9、停止容器:lxc-stop -n NAME
[root@node3 ~]# lxc-stop -n centos2 #停止容器
[root@node3 ~]# lxc-start -n centos2 -d #后台运行容器centos2
[root@node3 ~]# ssh root@172.16.100.2
The authenticity of host '172.16.100.2 (172.16.100.2)' can't be established.
RSA key fingerprint is 21:f4:88:54:49:eb:4e:83:16:ab:78:7e:aa:fe:ff:9b.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.16.100.2' (RSA) to the list of known hosts.
root@172.16.100.2's password:
Last login: Wed Apr 6 12:52:34 2016
[root@centos2 ~]# hostname
centos2
[root@centos2 ~]# logout
Connection to 172.16.100.2 closed.
#此时的容器还无法联网,先做SNAT让其能联网下载软件包
[root@node3 ~]# iptables -t nat -A POSTROUTING -s 172.16.0.0/16 -j SNAT --to-source 192.168.30.13
[root@node3 ~]# echo 1 > /proc/sys/net/ipv4/ip_forward
[root@node3 ~]# lxc-console -n centos2
Connected to tty 1
Type <Ctrl+a q> to exit the console, <Ctrl+a Ctrl+a> to enter Ctrl+a itself
CentOS release 6.7 (Final)
Kernel 2.6.32-431.el6.x86_64 on an x86_64
centos2 login: root
Password:
Last login: Wed Apr 6 13:01:46 on lxc/console
[root@centos2 ~]# ping 192.168.30.2 #可以连接外网了
PING 192.168.30.2 (192.168.30.2) 56(84) bytes of data.
64 bytes from 192.168.30.2: icmp_seq=1 ttl=127 time=4.87 ms
64 bytes from 192.168.30.2: icmp_seq=2 ttl=127 time=0.150 ms
64 bytes from 192.168.30.2: icmp_seq=3 ttl=127 time=0.246 ms
...
[root@centos2 ~]# yum -y install httpd
...
Complete!
[root@centos2 ~]# vi /var/www/html/index.html
hello
[root@centos2 ~]# service httpd start #启动httpd服务
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
[ OK ]
[root@centos2 ~]# iptables -F
[root@node3 ~]# curl 172.16.100.2
hello
10、克隆容器:lxc-clone [options] ORIG NEW
[root@node3 ~]# lxc-clone centos2 centos3
lxc_container: error: Original container (centos2) is running
clone failed
[root@node3 ~]# lxc-stop -n centos2
[root@node3 ~]# lxc-clone centos2 centos3
Created container centos3 as copy of centos2
[root@node3 ~]# lxc-start -n centos3
CentOS release 6.7 (Final)
Kernel 2.6.32-431.el6.x86_64 on an x86_64
centos3 login: init: rcS main process (8) killed by TERM signal
Entering non-interactive startup
iptables: No config file. [WARNING]
Bringing up loopback interface: [ OK ]
Bringing up interface eth0: Determining if ip address 172.16.100.2 is already in use for device eth0...
[ OK ]
Starting system logger: [ OK ]
Mounting filesystems: [ OK ]
Starting sshd: [ OK ]
CentOS release 6.7 (Final)
Kernel 2.6.32-431.el6.x86_64 on an x86_64
centos3 login: root
Password:
Last login: Wed Apr 6 13:01:46 on lxc/console
[root@centos3 ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=static
ONBOOT=yes
NM_CONTROLLED=no
TYPE=Ethernet
IPADDR=172.16.100.3
NETMASK=255.255.0.0
GATEWAY=172.16.100.1
[root@centos3 ~]# service network restart
Shutting down interface eth0: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface eth0: Determining if ip address 172.16.100.3 is already in use for device eth0...
[ OK ]
[root@centos3 ~]# vi /var/www/html/index.html
how are you
[root@centos3 ~]# service httpd start
...
[root@centos3 ~]# ss -tnl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 0 *:22 *:*
LISTEN 0 0 :::80 :::*
LISTEN 0 0 :::22 :::*
[root@centos3 ~]# iptables -F
[root@node3 ~]# curl 172.16.100.3
how are you
11、挂起/恢复容器:lxc-freeze/lxc-unfreeze
12、销毁容器:lxc-destory -n NAME
在lxc的容器中目测无法修改内核参数,但在docker中,让容器以特权模式启动是可以的
[root@centos2 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
-bash: /proc/sys/net/ipv4/conf/lo/arp_ignore: Read-only file system