ZFS可以进行快照与异步复制功能，测试验证一下这个功能
- 参考文档
环境准备
- 操作系统采用
debian 12.4 (bookworm)，内核6.1.0-17-amd64。 - 本次搭建采用两台Linux虚拟机。配置如下
| 主机名 | IP | CPU/内存 | 数据硬盘 |
|---|---|---|---|
| node1 | 192.168.230.170 | 2C4G | 1 * 100G |
| node2 | 192.168.230.171 | 2C4G | 1 * 100G |
安装配置
-
以下步骤需在两个节点执行。
- 添加contrib镜像源到apt
root@node1:~# apt-add-repository contrib Adding component(s) 'contrib' to all repositories. Press [ENTER] to continue or Ctrl-c to cancel. Added contrib to: deb http://mirrors.huaweicloud.com/debian/ bookworm main contrib non-free-firmware Added contrib to: deb-src http://mirrors.huaweicloud.com/debian/ bookworm main contrib non-free-firmware Added contrib to: deb http://security.debian.org/debian-security bookworm-security main contrib non-free-firmware Added contrib to: deb-src http://security.debian.org/debian-security bookworm-security main contrib non-free-firmware Added contrib to: deb http://mirrors.huaweicloud.com/debian/ bookworm-updates main contrib non-free-firmware Added contrib to: deb-src http://mirrors.huaweicloud.com/debian/ bookworm-updates main contrib non-free-firmware Get:1 http://mirrors.huaweicloud.com/debian bookworm InRelease [151 kB] Get:2 http://security.debian.org/debian-security bookworm-security InRelease [48.0 kB] Get:3 http://mirrors.huaweicloud.com/debian bookworm-updates InRelease [55.4 kB] Get:4 http://mirrors.huaweicloud.com/debian bookworm/main Sources [9,489 kB] Get:5 http://security.debian.org/debian-security bookworm-security/main Sources [83.0 kB] Get:6 http://security.debian.org/debian-security bookworm-security/contrib Sources [856 B] Get:7 http://security.debian.org/debian-security bookworm-security/main amd64 Packages [144 kB] Get:8 http://security.debian.org/debian-security bookworm-security/main Translation-en [86.0 kB] Get:9 http://security.debian.org/debian-security bookworm-security/contrib amd64 Packages [644 B] Get:10 http://security.debian.org/debian-security bookworm-security/contrib Translation-en [372 B] Get:11 http://mirrors.huaweicloud.com/debian bookworm/contrib Sources [51.3 kB] Get:12 http://mirrors.huaweicloud.com/debian bookworm/main amd64 Packages [8,786 kB] Get:13 http://mirrors.huaweicloud.com/debian bookworm/main Translation-en [6,109 kB] Get:14 http://mirrors.huaweicloud.com/debian bookworm/contrib amd64 Packages [54.1 kB] Get:15 http://mirrors.huaweicloud.com/debian bookworm/contrib Translation-en [48.7 kB] Get:16 http://mirrors.huaweicloud.com/debian bookworm/contrib amd64 DEP-11 Metadata [16.5 kB] Get:17 http://mirrors.huaweicloud.com/debian bookworm/contrib DEP-11 48x48 Icons [52.7 kB] Get:18 http://mirrors.huaweicloud.com/debian bookworm/contrib DEP-11 64x64 Icons [106 kB] Get:19 http://mirrors.huaweicloud.com/debian bookworm-updates/contrib Sources [776 B] Get:20 http://mirrors.huaweicloud.com/debian bookworm-updates/non-free-firmware Sources [2,076 B] Get:21 http://mirrors.huaweicloud.com/debian bookworm-updates/contrib amd64 Packages [768 B] Get:22 http://mirrors.huaweicloud.com/debian bookworm-updates/contrib Translation-en [408 B] Get:23 http://mirrors.huaweicloud.com/debian bookworm-updates/non-free-firmware amd64 Packages [616 B] Get:24 http://mirrors.huaweicloud.com/debian bookworm-updates/non-free-firmware Translation-en [384 B] Fetched 25.3 MB in 12s (2,071 kB/s) Reading package lists... Done N: Repository 'http://mirrors.huaweicloud.com/debian bookworm InRelease' changed its 'Version' value from '12.4' to '12.5'- 安装zfsutils-linux。（在这一步中，apt安装输出中可以看出，对内核进行了更新。6.1.0-17-amd64 > 6.1.0-18-amd64）
root@node1:~# apt list | grep zfs WARNING: apt does not have a stable CLI interface. Use with caution in scripts. golang-go-zfs-dev/stable 3.0.0-1 all libguestfs-zfs/stable 1:1.48.6-2 amd64 libpam-zfs/stable 2.1.11-1 amd64 libvirt-daemon-driver-storage-zfs/stable 9.0.0-4 amd64 libzfs4linux/stable 2.1.11-1 amd64 libzfsbootenv1linux/stable 2.1.11-1 amd64 libzfslinux-dev/stable 2.1.11-1 amd64 python3-libzfs/stable 0.0+git20230207.c1bd4a0-1 amd64 python3-pyzfs/stable 2.1.11-1 amd64 pyzfs-doc/stable 2.1.11-1 all zfs-auto-snapshot/stable 1.2.4-2 all zfs-dkms/stable 2.1.11-1 all zfs-dracut/stable 2.1.11-1 all zfs-fuse/stable 0.7.0-25+b1 amd64 zfs-initramfs/stable 2.1.11-1 all zfs-test/stable 2.1.11-1 amd64 zfs-zed/stable 2.1.11-1 amd64 zfsnap/stable 1.11.1-8.1 all zfsutils-linux/stable 2.1.11-1 amd64 root@node1:~# apt install zfsutils-linux -y Reading package lists... Done Building dependency tree... Done Reading state information... Done The following additional packages will be installed: binutils binutils-common binutils-x86-64-linux-gnu build-essential dkms dpkg-dev fakeroot g++ g++-12 gcc gcc-12 libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan8 libbinutils libc-bin libc-dev-bin libc-devtools libc-l10n libc6 libc6-dev libcc1-0 libcrypt-dev libctf-nobfd0 libctf0 libdpkg-perl libfakeroot libfile-fcntllock-perl libgcc-12-dev libgprofng0 libitm1 liblsan0 libnsl-dev libnvpair3linux libstdc++-12-dev libtirpc-dev libtsan2 libubsan1 libuutil3linux libzfs4linux libzpool5linux linux-compiler-gcc-12-x86 linux-headers-6.1.0-18-amd64 linux-headers-6.1.0-18-common linux-headers-amd64 linux-image-6.1.0-18-amd64 linux-image-amd64 linux-kbuild-6.1 linux-libc-dev locales make manpages-dev patch python3-distutils python3-lib2to3 rpcsvc-proto zfs-dkms zfs-zed Suggested packages: binutils-doc menu debian-keyring g++-multilib g++-12-multilib gcc-12-doc gcc-multilib autoconf automake libtool flex bison gdb gcc-doc gcc-12-multilib gcc-12-locales glibc-doc libnss-nis libnss-nisplus git bzr libstdc++-12-doc linux-doc-6.1 debian-kernel-handbook make-doc ed diffutils-doc debhelper nfs-kernel-server samba-common-bin zfs-initramfs | zfs-dracut The following NEW packages will be installed: binutils binutils-common binutils-x86-64-linux-gnu build-essential dkms dpkg-dev fakeroot g++ g++-12 gcc gcc-12 libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan8 libbinutils libc-dev-bin libc-devtools libc6-dev libcc1-0 libcrypt-dev libctf-nobfd0 libctf0 libdpkg-perl libfakeroot libfile-fcntllock-perl libgcc-12-dev libgprofng0 libitm1 liblsan0 libnsl-dev libnvpair3linux libstdc++-12-dev libtirpc-dev libtsan2 libubsan1 libuutil3linux libzfs4linux libzpool5linux linux-compiler-gcc-12-x86 linux-headers-6.1.0-18-amd64 linux-headers-6.1.0-18-common linux-headers-amd64 linux-image-6.1.0-18-amd64 linux-kbuild-6.1 linux-libc-dev make manpages-dev patch python3-distutils python3-lib2to3 rpcsvc-proto zfs-dkms zfs-zed zfsutils-linux The following packages will be upgraded: libc-bin libc-l10n libc6 linux-image-amd64 locales 5 upgraded, 55 newly installed, 0 to remove and 64 not upgraded. Need to get 151 MB of archives. After this operation, 728 MB of additional disk space will be used. Get:1 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libc6 amd64 2.36-9+deb12u4 [2,748 kB] Get:2 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libc-bin amd64 2.36-9+deb12u4 [606 kB] Get:3 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libcc1-0 amd64 12.2.0-14 [41.7 kB] Get:4 http://mirrors.huaweicloud.com/debian bookworm/main amd64 binutils-common amd64 2.40-2 [2,487 kB] Get:5 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libbinutils amd64 2.40-2 [572 kB] Get:6 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libctf-nobfd0 amd64 2.40-2 [153 kB] Get:7 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libctf0 amd64 2.40-2 [89.8 kB] Get:8 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libgprofng0 amd64 2.40-2 [812 kB] Get:9 http://mirrors.huaweicloud.com/debian bookworm/main amd64 binutils-x86-64-linux-gnu amd64 2.40-2 [2,246 kB] Get:10 http://mirrors.huaweicloud.com/debian bookworm/main amd64 binutils amd64 2.40-2 [65.0 kB] Get:11 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libitm1 amd64 12.2.0-14 [26.1 kB] Get:12 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libasan8 amd64 12.2.0-14 [2,195 kB] Get:13 http://mirrors.huaweicloud.com/debian bookworm/main amd64 liblsan0 amd64 12.2.0-14 [969 kB] Get:14 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libtsan2 amd64 12.2.0-14 [2,196 kB] Get:15 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libubsan1 amd64 12.2.0-14 [883 kB] Get:16 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libgcc-12-dev amd64 12.2.0-14 [2,437 kB] Get:17 http://mirrors.huaweicloud.com/debian bookworm/main amd64 gcc-12 amd64 12.2.0-14 [19.3 MB] Get:18 http://mirrors.huaweicloud.com/debian bookworm/main amd64 gcc amd64 4:12.2.0-3 [5,216 B] Get:19 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libdpkg-perl all 1.21.22 [603 kB] Get:20 http://mirrors.huaweicloud.com/debian bookworm/main amd64 patch amd64 2.7.6-7 [128 kB] Get:21 http://mirrors.huaweicloud.com/debian bookworm/main amd64 make amd64 4.3-4.1 [396 kB] Get:22 http://mirrors.huaweicloud.com/debian bookworm/main amd64 dpkg-dev all 1.21.22 [1,353 kB] Get:23 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libc-dev-bin amd64 2.36-9+deb12u4 [45.4 kB] Get:24 http://mirrors.huaweicloud.com/debian bookworm/main amd64 linux-libc-dev amd64 6.1.76-1 [1,930 kB] Get:25 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libcrypt-dev amd64 1:4.4.33-2 [118 kB] Get:26 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libtirpc-dev amd64 1.3.3+ds-1 [191 kB] Get:27 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libnsl-dev amd64 1.3.0-2 [66.4 kB] Get:28 http://mirrors.huaweicloud.com/debian bookworm/main amd64 rpcsvc-proto amd64 1.4.3-1 [63.3 kB] Get:29 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libc6-dev amd64 2.36-9+deb12u4 [1,897 kB] Get:30 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libstdc++-12-dev amd64 12.2.0-14 [2,046 kB] Get:31 http://mirrors.huaweicloud.com/debian bookworm/main amd64 g++-12 amd64 12.2.0-14 [10.7 MB] Get:32 http://mirrors.huaweicloud.com/debian bookworm/main amd64 g++ amd64 4:12.2.0-3 [1,356 B] Get:33 http://mirrors.huaweicloud.com/debian bookworm/main amd64 build-essential amd64 12.9 [7,704 B] Get:34 http://mirrors.huaweicloud.com/debian bookworm/main amd64 dkms all 3.0.10-8+deb12u1 [48.7 kB] Get:35 http://mirrors.huaweicloud.com/debian bookworm/main amd64 python3-lib2to3 all 3.11.2-3 [76.3 kB] Get:36 http://mirrors.huaweicloud.com/debian bookworm/main amd64 python3-distutils all 3.11.2-3 [131 kB] Get:37 http://mirrors.huaweicloud.com/debian bookworm/contrib amd64 zfs-dkms all 2.1.11-1 [2,304 kB] Get:38 http://mirrors.huaweicloud.com/debian bookworm/contrib amd64 libnvpair3linux amd64 2.1.11-1 [58.4 kB] Get:39 http://mirrors.huaweicloud.com/debian bookworm/contrib amd64 libuutil3linux amd64 2.1.11-1 [51.3 kB] Get:40 http://mirrors.huaweicloud.com/debian bookworm/contrib amd64 libzfs4linux amd64 2.1.11-1 [232 kB] Get:41 http://mirrors.huaweicloud.com/debian bookworm/contrib amd64 libzpool5linux amd64 2.1.11-1 [1,285 kB] Get:42 http://mirrors.huaweicloud.com/debian bookworm/contrib amd64 zfsutils-linux amd64 2.1.11-1 [502 kB] Get:43 http://mirrors.huaweicloud.com/debian bookworm/contrib amd64 zfs-zed amd64 2.1.11-1 [75.6 kB] Get:44 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libc-l10n all 2.36-9+deb12u4 [674 kB] Get:45 http://mirrors.huaweicloud.com/debian bookworm/main amd64 locales all 2.36-9+deb12u4 [3,902 kB] Get:46 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libfakeroot amd64 1.31-1.2 [28.3 kB] Get:47 http://mirrors.huaweicloud.com/debian bookworm/main amd64 fakeroot amd64 1.31-1.2 [66.9 kB] Get:48 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libalgorithm-diff-perl all 1.201-1 [43.3 kB] Get:49 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libalgorithm-diff-xs-perl amd64 0.04-8+b1 [11.4 kB] Get:50 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libalgorithm-merge-perl all 0.08-5 [11.8 kB] Get:51 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libc-devtools amd64 2.36-9+deb12u4 [53.0 kB] Get:52 http://mirrors.huaweicloud.com/debian bookworm/main amd64 libfile-fcntllock-perl amd64 0.22-4+b1 [34.8 kB] Get:53 http://mirrors.huaweicloud.com/debian bookworm/main amd64 linux-compiler-gcc-12-x86 amd64 6.1.76-1 [785 kB] Get:54 http://mirrors.huaweicloud.com/debian bookworm/main amd64 linux-headers-6.1.0-18-common all 6.1.76-1 [9,958 kB] Get:55 http://mirrors.huaweicloud.com/debian bookworm/main amd64 linux-kbuild-6.1 amd64 6.1.76-1 [1,042 kB] Get:56 http://mirrors.huaweicloud.com/debian bookworm/main amd64 linux-headers-6.1.0-18-amd64 amd64 6.1.76-1 [1,315 kB] Get:57 http://mirrors.huaweicloud.com/debian bookworm/main amd64 linux-headers-amd64 amd64 6.1.76-1 [1,420 B] Get:58 http://mirrors.huaweicloud.com/debian bookworm/main amd64 linux-image-6.1.0-18-amd64 amd64 6.1.76-1 [68.8 MB] Get:59 http://mirrors.huaweicloud.com/debian bookworm/main amd64 linux-image-amd64 amd64 6.1.76-1 [1,476 B] Get:60 http://mirrors.huaweicloud.com/debian bookworm/main amd64 manpages-dev all 6.03-2 [2,030 kB] Fetched 151 MB in 1min 15s (2,002 kB/s) apt-listchanges: Reading changelogs... Extracting templates from packages: 100% Preconfiguring packages ... (Reading database ... 154260 files and directories currently installed.) Preparing to unpack .../libc6_2.36-9+deb12u4_amd64.deb ... Unpacking libc6:amd64 (2.36-9+deb12u4) over (2.36-9+deb12u3) ... Setting up libc6:amd64 (2.36-9+deb12u4) ... (Reading database ... 154260 files and directories currently installed.) Preparing to unpack .../libc-bin_2.36-9+deb12u4_amd64.deb ... Unpacking libc-bin (2.36-9+deb12u4) over (2.36-9+deb12u3) ... Setting up libc-bin (2.36-9+deb12u4) ... Selecting previously unselected package libcc1-0:amd64. (Reading database ... 154260 files and directories currently installed.) Preparing to unpack .../00-libcc1-0_12.2.0-14_amd64.deb ... Unpacking libcc1-0:amd64 (12.2.0-14) ... Selecting previously unselected package binutils-common:amd64. Preparing to unpack .../01-binutils-common_2.40-2_amd64.deb ... Unpacking binutils-common:amd64 (2.40-2) ... Selecting previously unselected package libbinutils:amd64. Preparing to unpack .../02-libbinutils_2.40-2_amd64.deb ... Unpacking libbinutils:amd64 (2.40-2) ... Selecting previously unselected package libctf-nobfd0:amd64. Preparing to unpack .../03-libctf-nobfd0_2.40-2_amd64.deb ... Unpacking libctf-nobfd0:amd64 (2.40-2) ... Selecting previously unselected package libctf0:amd64. Preparing to unpack .../04-libctf0_2.40-2_amd64.deb ... Unpacking libctf0:amd64 (2.40-2) ... Selecting previously unselected package libgprofng0:amd64. Preparing to unpack .../05-libgprofng0_2.40-2_amd64.deb ... Unpacking libgprofng0:amd64 (2.40-2) ... Selecting previously unselected package binutils-x86-64-linux-gnu. Preparing to unpack .../06-binutils-x86-64-linux-gnu_2.40-2_amd64.deb ... Unpacking binutils-x86-64-linux-gnu (2.40-2) ... Selecting previously unselected package binutils. Preparing to unpack .../07-binutils_2.40-2_amd64.deb ... Unpacking binutils (2.40-2) ... Selecting previously unselected package libitm1:amd64. Preparing to unpack .../08-libitm1_12.2.0-14_amd64.deb ... Unpacking libitm1:amd64 (12.2.0-14) ... Selecting previously unselected package libasan8:amd64. Preparing to unpack .../09-libasan8_12.2.0-14_amd64.deb ... Unpacking libasan8:amd64 (12.2.0-14) ... Selecting previously unselected package liblsan0:amd64. Preparing to unpack .../10-liblsan0_12.2.0-14_amd64.deb ... Unpacking liblsan0:amd64 (12.2.0-14) ... Selecting previously unselected package libtsan2:amd64. Preparing to unpack .../11-libtsan2_12.2.0-14_amd64.deb ... Unpacking libtsan2:amd64 (12.2.0-14) ... Selecting previously unselected package libubsan1:amd64. Preparing to unpack .../12-libubsan1_12.2.0-14_amd64.deb ... Unpacking libubsan1:amd64 (12.2.0-14) ... Selecting previously unselected package libgcc-12-dev:amd64. Preparing to unpack .../13-libgcc-12-dev_12.2.0-14_amd64.deb ... Unpacking libgcc-12-dev:amd64 (12.2.0-14) ... Selecting previously unselected package gcc-12. Preparing to unpack .../14-gcc-12_12.2.0-14_amd64.deb ... Unpacking gcc-12 (12.2.0-14) ... Selecting previously unselected package gcc. Preparing to unpack .../15-gcc_4%3a12.2.0-3_amd64.deb ... Unpacking gcc (4:12.2.0-3) ... Selecting previously unselected package libdpkg-perl. Preparing to unpack .../16-libdpkg-perl_1.21.22_all.deb ... Unpacking libdpkg-perl (1.21.22) ... Selecting previously unselected package patch. Preparing to unpack .../17-patch_2.7.6-7_amd64.deb ... Unpacking patch (2.7.6-7) ... Selecting previously unselected package make. Preparing to unpack .../18-make_4.3-4.1_amd64.deb ... Unpacking make (4.3-4.1) ... Selecting previously unselected package dpkg-dev. Preparing to unpack .../19-dpkg-dev_1.21.22_all.deb ... Unpacking dpkg-dev (1.21.22) ... Selecting previously unselected package libc-dev-bin. Preparing to unpack .../20-libc-dev-bin_2.36-9+deb12u4_amd64.deb ... Unpacking libc-dev-bin (2.36-9+deb12u4) ... Selecting previously unselected package linux-libc-dev:amd64. Preparing to unpack .../21-linux-libc-dev_6.1.76-1_amd64.deb ... Unpacking linux-libc-dev:amd64 (6.1.76-1) ... Selecting previously unselected package libcrypt-dev:amd64. Preparing to unpack .../22-libcrypt-dev_1%3a4.4.33-2_amd64.deb ... Unpacking libcrypt-dev:amd64 (1:4.4.33-2) ... Selecting previously unselected package libtirpc-dev:amd64. Preparing to unpack .../23-libtirpc-dev_1.3.3+ds-1_amd64.deb ... Unpacking libtirpc-dev:amd64 (1.3.3+ds-1) ... Selecting previously unselected package libnsl-dev:amd64. Preparing to unpack .../24-libnsl-dev_1.3.0-2_amd64.deb ... Unpacking libnsl-dev:amd64 (1.3.0-2) ... Selecting previously unselected package rpcsvc-proto. Preparing to unpack .../25-rpcsvc-proto_1.4.3-1_amd64.deb ... Unpacking rpcsvc-proto (1.4.3-1) ... Selecting previously unselected package libc6-dev:amd64. Preparing to unpack .../26-libc6-dev_2.36-9+deb12u4_amd64.deb ... Unpacking libc6-dev:amd64 (2.36-9+deb12u4) ... Selecting previously unselected package libstdc++-12-dev:amd64. Preparing to unpack .../27-libstdc++-12-dev_12.2.0-14_amd64.deb ... Unpacking libstdc++-12-dev:amd64 (12.2.0-14) ... Selecting previously unselected package g++-12. Preparing to unpack .../28-g++-12_12.2.0-14_amd64.deb ... Unpacking g++-12 (12.2.0-14) ... Selecting previously unselected package g++. Preparing to unpack .../29-g++_4%3a12.2.0-3_amd64.deb ... Unpacking g++ (4:12.2.0-3) ... Selecting previously unselected package build-essential. Preparing to unpack .../30-build-essential_12.9_amd64.deb ... Unpacking build-essential (12.9) ... Selecting previously unselected package dkms. Preparing to unpack .../31-dkms_3.0.10-8+deb12u1_all.deb ... Unpacking dkms (3.0.10-8+deb12u1) ... Selecting previously unselected package python3-lib2to3. Preparing to unpack .../32-python3-lib2to3_3.11.2-3_all.deb ... Unpacking python3-lib2to3 (3.11.2-3) ... Selecting previously unselected package python3-distutils. Preparing to unpack .../33-python3-distutils_3.11.2-3_all.deb ... Unpacking python3-distutils (3.11.2-3) ... Selecting previously unselected package zfs-dkms. Preparing to unpack .../34-zfs-dkms_2.1.11-1_all.deb ... Unpacking zfs-dkms (2.1.11-1) ... Selecting previously unselected package libnvpair3linux. Preparing to unpack .../35-libnvpair3linux_2.1.11-1_amd64.deb ... Unpacking libnvpair3linux (2.1.11-1) ... Selecting previously unselected package libuutil3linux. Preparing to unpack .../36-libuutil3linux_2.1.11-1_amd64.deb ... Unpacking libuutil3linux (2.1.11-1) ... Selecting previously unselected package libzfs4linux. Preparing to unpack .../37-libzfs4linux_2.1.11-1_amd64.deb ... Unpacking libzfs4linux (2.1.11-1) ... Selecting previously unselected package libzpool5linux. Preparing to unpack .../38-libzpool5linux_2.1.11-1_amd64.deb ... Unpacking libzpool5linux (2.1.11-1) ... Selecting previously unselected package zfsutils-linux. Preparing to unpack .../39-zfsutils-linux_2.1.11-1_amd64.deb ... Unpacking zfsutils-linux (2.1.11-1) ... Selecting previously unselected package zfs-zed. Preparing to unpack .../40-zfs-zed_2.1.11-1_amd64.deb ... Unpacking zfs-zed (2.1.11-1) ... Preparing to unpack .../41-libc-l10n_2.36-9+deb12u4_all.deb ... Unpacking libc-l10n (2.36-9+deb12u4) over (2.36-9+deb12u3) ... Preparing to unpack .../42-locales_2.36-9+deb12u4_all.deb ... Unpacking locales (2.36-9+deb12u4) over (2.36-9+deb12u3) ... Selecting previously unselected package libfakeroot:amd64. Preparing to unpack .../43-libfakeroot_1.31-1.2_amd64.deb ... Unpacking libfakeroot:amd64 (1.31-1.2) ... Selecting previously unselected package fakeroot. Preparing to unpack .../44-fakeroot_1.31-1.2_amd64.deb ... Unpacking fakeroot (1.31-1.2) ... Selecting previously unselected package libalgorithm-diff-perl. Preparing to unpack .../45-libalgorithm-diff-perl_1.201-1_all.deb ... Unpacking libalgorithm-diff-perl (1.201-1) ... Selecting previously unselected package libalgorithm-diff-xs-perl:amd64. Preparing to unpack .../46-libalgorithm-diff-xs-perl_0.04-8+b1_amd64.deb ... Unpacking libalgorithm-diff-xs-perl:amd64 (0.04-8+b1) ... Selecting previously unselected package libalgorithm-merge-perl. Preparing to unpack .../47-libalgorithm-merge-perl_0.08-5_all.deb ... Unpacking libalgorithm-merge-perl (0.08-5) ... Selecting previously unselected package libc-devtools. Preparing to unpack .../48-libc-devtools_2.36-9+deb12u4_amd64.deb ... Unpacking libc-devtools (2.36-9+deb12u4) ... Selecting previously unselected package libfile-fcntllock-perl. Preparing to unpack .../49-libfile-fcntllock-perl_0.22-4+b1_amd64.deb ... Unpacking libfile-fcntllock-perl (0.22-4+b1) ... Selecting previously unselected package linux-compiler-gcc-12-x86. Preparing to unpack .../50-linux-compiler-gcc-12-x86_6.1.76-1_amd64.deb ... Unpacking linux-compiler-gcc-12-x86 (6.1.76-1) ... Selecting previously unselected package linux-headers-6.1.0-18-common. Preparing to unpack .../51-linux-headers-6.1.0-18-common_6.1.76-1_all.deb ... Unpacking linux-headers-6.1.0-18-common (6.1.76-1) ... Selecting previously unselected package linux-kbuild-6.1. Preparing to unpack .../52-linux-kbuild-6.1_6.1.76-1_amd64.deb ... Unpacking linux-kbuild-6.1 (6.1.76-1) ... Selecting previously unselected package linux-headers-6.1.0-18-amd64. Preparing to unpack .../53-linux-headers-6.1.0-18-amd64_6.1.76-1_amd64.deb ... Unpacking linux-headers-6.1.0-18-amd64 (6.1.76-1) ... Selecting previously unselected package linux-headers-amd64. Preparing to unpack .../54-linux-headers-amd64_6.1.76-1_amd64.deb ... Unpacking linux-headers-amd64 (6.1.76-1) ... Selecting previously unselected package linux-image-6.1.0-18-amd64. Preparing to unpack .../55-linux-image-6.1.0-18-amd64_6.1.76-1_amd64.deb ... Unpacking linux-image-6.1.0-18-amd64 (6.1.76-1) ... Preparing to unpack .../56-linux-image-amd64_6.1.76-1_amd64.deb ... Unpacking linux-image-amd64 (6.1.76-1) over (6.1.69-1) ... Selecting previously unselected package manpages-dev. Preparing to unpack .../57-manpages-dev_6.03-2_all.deb ... Unpacking manpages-dev (6.03-2) ... Setting up linux-image-6.1.0-18-amd64 (6.1.76-1) ... I: /vmlinuz.old is now a symlink to boot/vmlinuz-6.1.0-17-amd64 I: /initrd.img.old is now a symlink to boot/initrd.img-6.1.0-17-amd64 I: /vmlinuz is now a symlink to boot/vmlinuz-6.1.0-18-amd64 I: /initrd.img is now a symlink to boot/initrd.img-6.1.0-18-amd64 /etc/kernel/postinst.d/initramfs-tools: update-initramfs: Generating /boot/initrd.img-6.1.0-18-amd64 /etc/kernel/postinst.d/zz-update-grub: Generating grub configuration file ... Found background image: /usr/share/images/desktop-base/desktop-grub.png Found linux image: /boot/vmlinuz-6.1.0-18-amd64 Found initrd image: /boot/initrd.img-6.1.0-18-amd64 Found linux image: /boot/vmlinuz-6.1.0-17-amd64 Found initrd image: /boot/initrd.img-6.1.0-17-amd64 Found linux image: /boot/vmlinuz-6.1.0-15-amd64 Found initrd image: /boot/initrd.img-6.1.0-15-amd64 Warning: os-prober will not be executed to detect other bootable partitions. Systems on them will not be added to the GRUB boot configuration. Check GRUB_DISABLE_OS_PROBER documentation entry. done Setting up manpages-dev (6.03-2) ... Setting up libnvpair3linux (2.1.11-1) ... Setting up libc-l10n (2.36-9+deb12u4) ... Setting up libfile-fcntllock-perl (0.22-4+b1) ... Setting up libalgorithm-diff-perl (1.201-1) ... Setting up binutils-common:amd64 (2.40-2) ... Setting up linux-libc-dev:amd64 (6.1.76-1) ... Setting up libctf-nobfd0:amd64 (2.40-2) ... Setting up linux-image-amd64 (6.1.76-1) ... Setting up locales (2.36-9+deb12u4) ... Generating locales (this might take a while)... en_US.UTF-8... done Generation complete. Setting up libfakeroot:amd64 (1.31-1.2) ... Setting up fakeroot (1.31-1.2) ... update-alternatives: using /usr/bin/fakeroot-sysv to provide /usr/bin/fakeroot (fakeroot) in auto mode Setting up libtirpc-dev:amd64 (1.3.3+ds-1) ... Setting up rpcsvc-proto (1.4.3-1) ... Setting up make (4.3-4.1) ... Setting up patch (2.7.6-7) ... Setting up libdpkg-perl (1.21.22) ... Setting up libubsan1:amd64 (12.2.0-14) ... Setting up libnsl-dev:amd64 (1.3.0-2) ... Setting up libcrypt-dev:amd64 (1:4.4.33-2) ... Setting up libasan8:amd64 (12.2.0-14) ... Setting up linux-headers-6.1.0-18-common (6.1.76-1) ... Setting up libtsan2:amd64 (12.2.0-14) ... Setting up libbinutils:amd64 (2.40-2) ... Setting up libuutil3linux (2.1.11-1) ... Setting up linux-kbuild-6.1 (6.1.76-1) ... Setting up libc-dev-bin (2.36-9+deb12u4) ... Setting up python3-lib2to3 (3.11.2-3) ... Setting up libalgorithm-diff-xs-perl:amd64 (0.04-8+b1) ... Setting up libcc1-0:amd64 (12.2.0-14) ... Setting up liblsan0:amd64 (12.2.0-14) ... Setting up libitm1:amd64 (12.2.0-14) ... Setting up libc-devtools (2.36-9+deb12u4) ... Setting up libalgorithm-merge-perl (0.08-5) ... Setting up libctf0:amd64 (2.40-2) ... Setting up python3-distutils (3.11.2-3) ... Setting up libzfs4linux (2.1.11-1) ... Setting up libgprofng0:amd64 (2.40-2) ... Setting up libgcc-12-dev:amd64 (12.2.0-14) ... Setting up libc6-dev:amd64 (2.36-9+deb12u4) ... Setting up libzpool5linux (2.1.11-1) ... Setting up binutils-x86-64-linux-gnu (2.40-2) ... Setting up libstdc++-12-dev:amd64 (12.2.0-14) ... Setting up binutils (2.40-2) ... Setting up dpkg-dev (1.21.22) ... Setting up zfsutils-linux (2.1.11-1) ... modprobe: FATAL: Module zfs not found in directory /lib/modules/6.1.0-17-amd64 Created symlink /etc/systemd/system/zfs-import.target.wants/zfs-import-cache.service → /lib/systemd/system/zfs-import-cache.service. Created symlink /etc/systemd/system/zfs.target.wants/zfs-import.target → /lib/systemd/system/zfs-import.target. Created symlink /etc/systemd/system/zfs-mount.service.wants/zfs-load-module.service → /lib/systemd/system/zfs-load-module.service. Created symlink /etc/systemd/system/zfs.target.wants/zfs-load-module.service → /lib/systemd/system/zfs-load-module.service. Created symlink /etc/systemd/system/zfs.target.wants/zfs-mount.service → /lib/systemd/system/zfs-mount.service. Created symlink /etc/systemd/system/zfs.target.wants/zfs-share.service → /lib/systemd/system/zfs-share.service. Created symlink /etc/systemd/system/zfs-volumes.target.wants/zfs-volume-wait.service → /lib/systemd/system/zfs-volume-wait.service. Created symlink /etc/systemd/system/zfs.target.wants/zfs-volumes.target → /lib/systemd/system/zfs-volumes.target. Created symlink /etc/systemd/system/multi-user.target.wants/zfs.target → /lib/systemd/system/zfs.target. zfs-import-scan.service is a disabled or a static unit, not starting it. Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 145. Setting up gcc-12 (12.2.0-14) ... Setting up g++-12 (12.2.0-14) ... Setting up gcc (4:12.2.0-3) ... Setting up dkms (3.0.10-8+deb12u1) ... Setting up linux-compiler-gcc-12-x86 (6.1.76-1) ... Setting up zfs-dkms (2.1.11-1) ... Loading new zfs-2.1.11 DKMS files... Building for 6.1.0-17-amd64 6.1.0-18-amd64 Module build for kernel 6.1.0-17-amd64 was skipped since the kernel headers for this kernel do not seem to be installed. Building initial module for 6.1.0-18-amd64 Done. zavl.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/6.1.0-18-amd64/updates/dkms/ znvpair.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/6.1.0-18-amd64/updates/dkms/ zunicode.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/6.1.0-18-amd64/updates/dkms/ zcommon.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/6.1.0-18-amd64/updates/dkms/ zfs.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/6.1.0-18-amd64/updates/dkms/ icp.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/6.1.0-18-amd64/updates/dkms/ zlua.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/6.1.0-18-amd64/updates/dkms/ spl.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/6.1.0-18-amd64/updates/dkms/ zzstd.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/6.1.0-18-amd64/updates/dkms/ depmod... Setting up g++ (4:12.2.0-3) ... update-alternatives: using /usr/bin/g++ to provide /usr/bin/c++ (c++) in auto mode Setting up build-essential (12.9) ... Setting up linux-headers-6.1.0-18-amd64 (6.1.76-1) ... /etc/kernel/header_postinst.d/dkms: dkms: running auto installation service for kernel 6.1.0-18-amd64. dkms: autoinstall for kernel: 6.1.0-18-amd64. Setting up linux-headers-amd64 (6.1.76-1) ... Processing triggers for initramfs-tools (0.142) ... update-initramfs: Generating /boot/initrd.img-6.1.0-18-amd64 Processing triggers for libc-bin (2.36-9+deb12u4) ... Processing triggers for man-db (2.11.2-2) ... Setting up zfs-zed (2.1.11-1) ... Created symlink /etc/systemd/system/zed.service → /lib/systemd/system/zfs-zed.service. Created symlink /etc/systemd/system/zfs.target.wants/zfs-zed.service → /lib/systemd/system/zfs-zed.service. root@node1:~# lsmod | grep zfs- 确保zfs-dkms已经安装
root@node1:~# modprobe zfs modprobe: FATAL: Module zfs not found in directory /lib/modules/6.1.0-17-amd64 root@node1:~# apt install zfs-dkms Reading package lists... Done Building dependency tree... Done Reading state information... Done zfs-dkms is already the newest version (2.1.11-1). zfs-dkms set to manually installed. The following package was automatically installed and is no longer required: linux-image-6.1.0-15-amd64 Use 'apt autoremove' to remove it. 0 upgraded, 0 newly installed, 0 to remove and 64 not upgraded.- 重启节点，使用新内核启动。
root@node1:~# modprobe zfs modprobe: FATAL: Module zfs not found in directory /lib/modules/6.1.0-17-amd64 root@node1:~# reboot
创建单节点ZFS 文件系统并进行快照克隆测试
-
在node1 节点执行
- 检查相关内核模块已经加载
root@node1:~# lsmod | grep zfs zfs 4018176 6 zunicode 335872 1 zfs zzstd 589824 1 zfs zlua 192512 1 zfs zavl 20480 1 zfs icp 327680 1 zfs zcommon 110592 2 zfs,icp znvpair 118784 2 zfs,zcommon spl 122880 6 zfs,icp,zzstd,znvpair,zcommon,zavl root@node1:~# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 50G 0 disk ├─sda1 8:1 0 487M 0 part /boot ├─sda2 8:2 0 1K 0 part └─sda5 8:5 0 49.5G 0 part ├─debian12--vg-root 254:0 0 16.8G 0 lvm / ├─debian12--vg-swap_1 254:1 0 976M 0 lvm [SWAP] └─debian12--vg-home 254:2 0 31.8G 0 lvm /home sdb 8:16 0 100G 0 disk sr0 11:0 1 1024M 0 rom- 创建zpool(这里有个小插曲，最后再附上清理之前drbd残留配置的过程)
root@node1:~# zpool create datapool /dev/sdb invalid vdev specification use '-f' to override the following errors: /dev/sdb contains a filesystem of type 'drbd' root@node1:~# sgdisk -Z /dev/sdb Creating new GPT entries in memory. GPT data structures destroyed! You may now partition the disk using fdisk or other utilities. root@node1:~# zpool create datapool /dev/sdb root@node1:~# zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT datapool 99.5G 126K 99.5G - - 0% 0% 1.00x ONLINE -- 创建zfs文件系统并挂载
root@node1:~# zfs create datapool/docs -o mountpoint=/mnt/zfs_test root@node1:~# df -h Filesystem Size Used Avail Use% Mounted on udev 1.9G 0 1.9G 0% /dev tmpfs 389M 1.4M 387M 1% /run /dev/mapper/debian12--vg-root 17G 5.7G 9.9G 37% / tmpfs 1.9G 0 1.9G 0% /dev/shm tmpfs 5.0M 8.0K 5.0M 1% /run/lock /dev/sda1 455M 181M 249M 43% /boot /dev/mapper/debian12--vg-home 32G 23M 30G 1% /home tmpfs 389M 64K 388M 1% /run/user/112 tmpfs 389M 48K 389M 1% /run/user/0 datapool 97G 128K 97G 1% /datapool datapool/docs 97G 128K 97G 1% /mnt/zfs_test- 写入数据，创建第一个快照，覆盖数据
root@node1:~# echo `date` > /mnt/zfs_test/data.file1 root@node1:~# cat /mnt/zfs_test/data.file1 Tue Mar 5 05:04:28 PM CST 2024 root@node1:~# zfs snapshot datapool/docs@first root@node1:~# zfs list NAME USED AVAIL REFER MOUNTPOINT datapool 190K 96.4G 24K /datapool datapool/docs 24.5K 96.4G 24.5K /mnt/zfs_test root@node1:~# zfs list -t snapshot NAME USED AVAIL REFER MOUNTPOINT datapool/docs@first 0B - 24.5K - root@node1:~# echo `date` > /mnt/zfs_test/data.file1 root@node1:~# cat /mnt/zfs_test/data.file1 Tue Mar 5 05:07:32 PM CST 2024- 回滚快照
root@node1:~# zfs rollback datapool/docs@first root@node1:~# cat /mnt/zfs_test/data.file1 Tue Mar 5 05:04:28 PM CST 2024- 克隆文件系统
root@node1:~# mkdir /mnt/zfs_clone root@node1:~# zfs clone datapool/docs@first datapool/clone_1 root@node1:~# zfs list NAME USED AVAIL REFER MOUNTPOINT datapool 212K 96.4G 24K /datapool datapool/clone_1 0B 96.4G 24.5K /datapool/clone_1 datapool/docs 39K 96.4G 24.5K /mnt/zfs_test root@node1:~# ls -l /datapool/clone_1/data.file1 ^C root@node1:~# echo `date` > /datapool/clone_1/data.file1 root@node1:~# cat /datapool/clone_1/data.file1 Tue Mar 5 05:17:45 PM CST 2024- 清理全部配置
root@node1:~# zfs destroy datapool/clone_1 root@node1:~# zfs list NAME USED AVAIL REFER MOUNTPOINT datapool 218K 96.4G 24K /datapool datapool/docs 39K 96.4G 24.5K /mnt/zfs_test root@node1:~# zfs destroy datapool/docs@first root@node1:~# zfs list NAME USED AVAIL REFER MOUNTPOINT datapool 198K 96.4G 24K /datapool datapool/docs 24.5K 96.4G 24.5K /mnt/zfs_test root@node1:~# zfs list -t snapshot no datasets available root@node1:~# zfs list NAME USED AVAIL REFER MOUNTPOINT datapool 200K 96.4G 24K /datapool datapool/docs 24.5K 96.4G 24.5K /mnt/zfs_test root@node1:~# zfs destroy datapool/docs root@node1:~# zfs list NAME USED AVAIL REFER MOUNTPOINT datapool 164K 96.4G 24K /datapool
ZFS VOL snapshot(快照)/bookmark(书签)同步功能测试
- node1 创建对应的 volume 并写入测试数据
- 创建测试volume
root@node1:~# zfs create -V 10G datapool/vol_1 root@node1:~# zfs list NAME USED AVAIL REFER MOUNTPOINT datapool 10.3G 86.1G 24K /datapool datapool/vol_1 10.3G 96.4G 12K - root@node1:~# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 50G 0 disk ├─sda1 8:1 0 487M 0 part /boot ├─sda2 8:2 0 1K 0 part └─sda5 8:5 0 49.5G 0 part ├─debian12--vg-root 254:0 0 16.8G 0 lvm / ├─debian12--vg-swap_1 254:1 0 976M 0 lvm [SWAP] └─debian12--vg-home 254:2 0 31.8G 0 lvm /home sdb 8:16 0 100G 0 disk ├─sdb1 8:17 0 100G 0 part └─sdb9 8:25 0 8M 0 part sr0 11:0 1 1024M 0 rom zd0 230:0 0 10G 0 disk- 创建pv - vg - lv
root@node1:~# pvcreate /dev/zd0 Physical volume "/dev/zd0" successfully created. root@node1:~# pvs PV VG Fmt Attr PSize PFree /dev/sda5 debian12-vg lvm2 a-- <49.52g 0 /dev/zd0 lvm2 --- 10.00g 10.00g root@node1:~# vgcreate vg_1 /dev/zd0 Volume group "vg_1" successfully created root@node1:~# lvcreate -n lv_1 -l 100%FREE vg_1 Logical volume "lv_1" created. root@node1:~# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 50G 0 disk ├─sda1 8:1 0 487M 0 part /boot ├─sda2 8:2 0 1K 0 part └─sda5 8:5 0 49.5G 0 part ├─debian12--vg-root 254:0 0 16.8G 0 lvm / ├─debian12--vg-swap_1 254:1 0 976M 0 lvm [SWAP] └─debian12--vg-home 254:2 0 31.8G 0 lvm /home sdb 8:16 0 100G 0 disk ├─sdb1 8:17 0 100G 0 part └─sdb9 8:25 0 8M 0 part sr0 11:0 1 1024M 0 rom zd0 230:0 0 10G 0 disk └─vg_1-lv_1 254:3 0 10G 0 lvm- 创建文件系统挂载，并写入测试数据
root@node1:~# mkfs.ext4 /dev/vg_1/lv_1 mke2fs 1.47.0 (5-Feb-2023) Discarding device blocks: done Creating filesystem with 2620416 4k blocks and 655360 inodes Filesystem UUID: ba898715-7ef5-472b-98e4-e94723d2ff0c Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 Allocating group tables: done Writing inode tables: done Creating journal (16384 blocks): done Writing superblocks and filesystem accounting information: done root@node1:~# mount /dev/vg_1/lv_1 /mnt/zfs_test/ root@node1:~# df -h Filesystem Size Used Avail Use% Mounted on udev 1.9G 0 1.9G 0% /dev tmpfs 389M 1.4M 387M 1% /run /dev/mapper/debian12--vg-root 17G 5.7G 9.9G 37% / tmpfs 1.9G 0 1.9G 0% /dev/shm tmpfs 5.0M 8.0K 5.0M 1% /run/lock /dev/sda1 455M 181M 249M 43% /boot /dev/mapper/debian12--vg-home 32G 23M 30G 1% /home tmpfs 389M 64K 388M 1% /run/user/112 tmpfs 389M 48K 389M 1% /run/user/0 datapool 87G 128K 87G 1% /datapool /dev/mapper/vg_1-lv_1 9.8G 24K 9.3G 1% /mnt/zfs_test root@node1:~# echo `date` > /mnt/zfs_test/test.file root@node1:~# cat /mnt/zfs_test/test.file Thu Mar 7 06:06:08 PM CST 2024 - node2 初始化zpool
- 创建zpool
root@node2:/home/allen# zpool create backpool /dev/sdb root@node2:/home/allen# zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT backpool 99.5G 130K 99.5G - - 0% 0% 1.00x ONLINE - - node1 -> node2 snapshot增量同步测试
- 取消文件系统挂载，创建snapshot
root@node1:~# umount /mnt/zfs_test root@node1:~# zfs snapshot datapool/vol_1@snap1 root@node1:~# zfs list -t snapshot NAME USED AVAIL REFER MOUNTPOINT datapool/vol_1@snap1 0B - 229M -- 初始化全量同步，挂载目标端文件系统
root@node1:~# zfs send datapool/vol_1@snap1 | ssh 192.168.230.171 zfs receive backpool/vol_1 root@192.168.230.171's password: root@node2:/home/allen# zfs list NAME USED AVAIL REFER MOUNTPOINT backpool 229M 96.2G 24K /backpool backpool/vol_1 229M 96.2G 229M - root@node2:/home/allen# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 50G 0 disk ├─sda1 8:1 0 487M 0 part /boot ├─sda2 8:2 0 1K 0 part └─sda5 8:5 0 49.5G 0 part ├─debian12--vg-root 254:0 0 16.8G 0 lvm / ├─debian12--vg-swap_1 254:1 0 976M 0 lvm [SWAP] └─debian12--vg-home 254:2 0 31.8G 0 lvm /home sdb 8:16 0 100G 0 disk ├─sdb1 8:17 0 100G 0 part └─sdb9 8:25 0 8M 0 part sr0 11:0 1 1024M 0 rom zd0 230:0 0 10G 0 disk └─vg_1-lv_1 254:3 0 10G 0 lvm root@node2:/home/allen# mkdir /mnt/zfs_test root@node2:/home/allen# mount /dev/vg_1/lv_1 /mnt/zfs_test/ root@node2:/home/allen# df -h Filesystem Size Used Avail Use% Mounted on udev 1.9G 0 1.9G 0% /dev tmpfs 389M 1.4M 387M 1% /run /dev/mapper/debian12--vg-root 17G 5.7G 9.9G 37% / tmpfs 1.9G 0 1.9G 0% /dev/shm tmpfs 5.0M 8.0K 5.0M 1% /run/lock /dev/sda1 455M 181M 249M 43% /boot /dev/mapper/debian12--vg-home 32G 1.1G 29G 4% /home tmpfs 389M 64K 388M 1% /run/user/112 tmpfs 389M 48K 389M 1% /run/user/0 backpool 97G 128K 97G 1% /backpool /dev/mapper/vg_1-lv_1 9.8G 28K 9.3G 1% /mnt/zfs_test root@node2:/home/allen# cat /mnt/zfs_test/test.file Thu Mar 7 06:06:08 PM CST 2024- 重新挂载文件系统，创建snapshot
root@node1:~# mount /dev/vg_1/lv_1 /mnt/zfs_test/ root@node1:~# echo `date` > /mnt/zfs_test/test.file root@node1:~# cat /mnt/zfs_test/test.file Thu Mar 7 06:14:46 PM CST 2024 root@node1:~# sync;echo 3 > /proc/sys/vm/drop_caches root@node1:~# zfs snapshot datapool/vol_1@snap3 root@node1:~# zfs list -t snapshot NAME USED AVAIL REFER MOUNTPOINT datapool/vol_1@snap1 129K - 229M - datapool/vol_1@snap2 0B - 229M -- 增量同步，重新挂载目标端文件系统
root@node1:~# zfs send -i snap1 datapool/vol_1@snap2 | ssh 192.168.230.171 zfs receive backpool/vol_1 -F Warning: incremental source didn't specify type, assuming snapshot. Use '@' or '#' prefix to avoid ambiguity. root@node2:~# zfs list -t snapshot NAME USED AVAIL REFER MOUNTPOINT backpool/vol_1@snap1 128K - 229M - backpool/vol_1@snap2 0B - 229M - root@node2:~# mount /dev/vg_1/lv_1 /mnt/zfs_test/ root@node2:~# cat /mnt/zfs_test/test.file Thu Mar 7 06:14:46 PM CST 2024 - node1 -> node2 bookmark增量同步测试（测试不通过重新挂载文件系统，更新数据）
- 增加snap2对应bookmark书签，删除snap1 snap2 快照。
root@node1:~# zfs list -t snapshot,bookmark NAME USED AVAIL REFER MOUNTPOINT datapool/vol_1@snap1 129K - 229M - datapool/vol_1@snap2 0B - 229M - root@node1:~# zfs bookmark datapool/vol_1@snap2 datapool/vol_1#bookmark2 root@node1:~# zfs destroy datapool/vol_1@snap1 root@node1:~# zfs destroy datapool/vol_1@snap2- 更新数据，创建snap3。使用“sync和drop_cache2”刷新buffer/cache
root@node1:~# echo `date` > /mnt/zfs_test/test.file root@node1:~# cat /mnt/zfs_test/test.file Thu Mar 7 06:36:47 PM CST 2024 root@node1:~# sync;echo 3 > /proc/sys/vm/drop_caches root@node1:~# zfs snapshot datapool/vol_1@snap3- 同步数据，在远端检查是否更细成功。(同步增加-F 参数，强制更新)
root@node1:~# zfs send -i datapool/vol_1#bookmark2 datapool/vol_1@snap3 | ssh 192.168.230.171 zfs receive backpool/vol_1 cannot receive incremental stream: destination backpool/vol_1 has been modified since most recent snapshot root@node1:~# zfs send -i datapool/vol_1#bookmark2 datapool/vol_1@snap3 | ssh 192.168.230.171 zfs receive backpool/vol_1 -F root@node2:~# zfs list -t snapshot NAME USED AVAIL REFER MOUNTPOINT backpool/vol_1@snap1 128K - 229M - backpool/vol_1@snap2 95K - 229M - backpool/vol_1@snap3 0B - 229M - root@node2:~# cat /mnt/zfs_test/test.file Thu Mar 7 06:14:46 PM CST 2024 root@node2:~# sync;echo 3 > /proc/sys/vm/drop_caches root@node2:~# cat /mnt/zfs_test/test.file Thu Mar 7 06:36:47 PM CST 2024
附：dd清理drbd残留分区信息与LVM激活
-
dd清理drbd残留分区信息遇见的小问题
- dd 写0 覆盖起始分区后，blkid依然可见分区信息.
root@node2:~# dd if=/dev/zero of=/dev/sdb bs=4M count=16 oflag=direct,nonblock 16+0 records in 16+0 records out 67108864 bytes (67 MB, 64 MiB) copied, 0.0101462 s, 6.6 GB/s root@node2:~# blkid /dev/mapper/debian12--vg-swap_1: UUID="fc3a9fe8-f2ae-4156-8e57-2e308e30a983" TYPE="swap" /dev/sdb: UUID="bc8cdb84ca4c572f" TYPE="drbd" /dev/mapper/debian12--vg-home: UUID="f0e98ab7-bbff-4ff7-a8cb-0ee7853213b4" BLOCK_SIZE="4096" TYPE="ext4" /dev/mapper/debian12--vg-root: UUID="9f00d05d-164c-4342-b547-d400d2be1c96" BLOCK_SIZE="4096" TYPE="ext4" /dev/sda5: UUID="qygNBy-0q7c-k5bB-cCmG-WTDU-Dmbb-V5CoBA" TYPE="LVM2_member" PARTUUID="62e06a1c-05" /dev/sda1: UUID="1108a04e-ec4f-4206-8caa-fcac63ba98f2" BLOCK_SIZE="1024" TYPE="ext2" PARTUUID="62e06a1c-01" root@node2:~# dd if=/dev/sdb of=mbr.txt bs=1 count=2048 2048+0 records in 2048+0 records out 2048 bytes (2.0 kB, 2.0 KiB) copied, 0.0023028 s, 889 kB/s root@node2:~# hexdump -C mbr.txt 00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 00000800- wipfs 检查信息，测试清理与恢复效果。具体看man wipefs帮助文档
root@node2:/home/allen# wipefs /dev/sdb DEVICE OFFSET TYPE UUID LABEL sdb 0x18fffff03c drbd bc8cdb84ca4c572f root@node2:/home/allen# wipefs /dev/sdb DEVICE OFFSET TYPE UUID LABEL sdb 0x18fffff03c drbd bc8cdb84ca4c572f root@node2:/home/allen# wipefs --all --backup /dev/sdb /dev/sdb: 4 bytes were erased at offset 0x18fffff03c (drbd): 83 74 02 6c root@node2:/home/allen# ls -l ~/wipefs-sdb-0x18fffff03c.bak -rw------- 1 root root 4 Mar 5 16:37 /root/wipefs-sdb-0x18fffff03c.bak root@node2:/home/allen# hexdump -C ~/wipefs-sdb-0x18fffff03c.bak 00000000 83 74 02 6c |.t.l| 00000004 root@node2:/home/allen# blkid /dev/sdb root@node2:/home/allen# dd if=~/wipefs-sdb-0x18fffff03c.bak of=/dev/sdb seek=$((0x18fffff03c)) bs=1 conv=notrunc 4+0 records in 4+0 records out 4 bytes copied, 0.000769421 s, 5.2 kB/s root@node2:/home/allen# blkid /dev/sdb /dev/sdb: UUID="bc8cdb84ca4c572f" TYPE="drbd"- 计算说明
参考链接
# 16进制转换为10进制后，从Bytes转换到GB，正好为磁盘大小100GB，由此可知drbd的分区信息存储在磁盘最后4 Bytes位置。 root@node2:/home/allen# echo $((0x18fffff03c)) 107374178364In this case what the utility shows is the signature of the dos partition table found on the device. As you can see, the offset of the signature is expressed in hexadecimal (base 16) form. With seek we specify the offset which should be used: the data should be provided in bytes, so to convert the hexadecimal value, we use the shell arithmetic expansion ($(())) -
lvs遇见远端节点未激活
- zfs目标端可正常识别
root@node2:~# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 100G 0 disk ├─sda1 8:1 0 100G 0 part └─sda9 8:9 0 8M 0 part sdb 8:16 0 50G 0 disk ├─sdb1 8:17 0 487M 0 part /boot ├─sdb2 8:18 0 1K 0 part └─sdb5 8:21 0 49.5G 0 part ├─debian12--vg-root 254:0 0 16.8G 0 lvm / ├─debian12--vg-swap_1 254:1 0 976M 0 lvm [SWAP] └─debian12--vg-home 254:2 0 31.8G 0 lvm /home sr0 11:0 1 1024M 0 rom zd0 230:0 0 10G 0 disk- 执行scan命令后可扫描到
root@node2:~# pvscan PV /dev/sdb5 VG debian12-vg lvm2 [<49.52 GiB / 0 free] PV /dev/zd0 VG vg_1 lvm2 [<10.00 GiB / 0 free] Total: 2 [<59.52 GiB] / in use: 2 [<59.52 GiB] / in no VG: 0 [0 ] root@node2:~# vgscan Found volume group "debian12-vg" using metadata type lvm2 Found volume group "vg_1" using metadata type lvm2 root@node2:~# lvscan ACTIVE '/dev/debian12-vg/root' [<16.81 GiB] inherit ACTIVE '/dev/debian12-vg/swap_1' [976.00 MiB] inherit ACTIVE '/dev/debian12-vg/home' [<31.76 GiB] inherit inactive '/dev/vg_1/lv_1' [<10.00 GiB] inherit- vgchange命令，激活vg
root@node2:~# lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert home debian12-vg -wi-ao---- <31.76g root debian12-vg -wi-ao---- <16.81g swap_1 debian12-vg -wi-ao---- 976.00m lv_1 vg_1 -wi------- <10.00g root@node2:~# vgchange -a y vg_1 1 logical volume(s) in volume group "vg_1" now active root@node2:~# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 100G 0 disk ├─sda1 8:1 0 100G 0 part └─sda9 8:9 0 8M 0 part sdb 8:16 0 50G 0 disk ├─sdb1 8:17 0 487M 0 part /boot ├─sdb2 8:18 0 1K 0 part └─sdb5 8:21 0 49.5G 0 part ├─debian12--vg-root 254:0 0 16.8G 0 lvm / ├─debian12--vg-swap_1 254:1 0 976M 0 lvm [SWAP] └─debian12--vg-home 254:2 0 31.8G 0 lvm /home sr0 11:0 1 1024M 0 rom zd0 230:0 0 10G 0 disk └─vg_1-lv_1 254:3 0 10G 0 lvm
粤公网安备案：