iscsi服务简介与实战

admin 2024年01月13日 20次浏览

1、存储技术简介

1.1、存储技术分类

  • DAS(直连式存储):DAS 将存储设备通过 SCSI 接口或者光纤通道直接连接到一台服务器。这种方式的优点是简单、可靠,适用于小型系统或者对数据量要求不高的环境。
  • NAS(网络接入存储):NAS 存储设备和服务器彻底分离,存储设备通过标准的以太网连接到需要存储服务的计算机。NAS 基于 TCP/IP 协议的网络文件存储服务,体现 NFS、SMB 等网络文件共享服务
  • SAN(存储区域网络)是一种基于块存储的存储方式,通过专用高速网络将一个或多个网络存储设备和服务器连接起来,形成一个专用的存储系统。SAN 采用光纤通道技术,通过光纤通道交换机连接存储阵列和服务器主机,建立专用于数据存储的区域网络。SAN 结构有两种,IPSAN 与 FCSAN。IPSAN 是在 SAN 后产生的,以 IP 网络构建存储网络,而 FCSAN 则以光纤通道构建存储网络

1.2、iscsi介绍

iSCSI(Internet Small Computer System Interface)是一种基于 TCP/IP 网络的存储协议,它允许服务器通过网络连接到存储设备,并将其作为本地磁盘使用。其工作原理的核心是将存储设备虚拟化为逻辑卷,并将其通过网络传输到服务器

iSCSI 存储系统通常由以下几个组件组成:iSCSI 发起者(Initiator)、iSCSI 目标(Target)、存储设备和网络。发起者是指运行 iSCSI 协议的服务器(客户端),负责将存储设备连接到本地系统。目标则是存储设备(服务器)上的逻辑卷,被发起者访问和使用。存储设备是实际存储数据的物理设备,可以是硬盘阵列、磁带库或闪存设备等。网络则提供了发起者和目标之间的通信通道,可以是局域网(LAN)或广域网(WAN)

当发起者启动时,它会发送一个 iSCSI 请求到目标,请求连接到一个逻辑卷。目标接收到请求后,会验证发起者的身份和权限,然后建立一个 iSCSI 会话。会话建立后,发起者和目标之间可以进行数据传输。在数据传输过程中,发起者将数据块分割为小的数据包,并通过网络发送给目标。目标接收到数据包后,将其缓存并写入存储设备

为了确保数据的完整性和可靠性,iSCSI 协议使用了一些技术,如数据校验和、流量控制和错误恢复机制。此外,iSCSI 还支持一些高级功能,如快照、镜像和复制。这些功能可以在存储设备上进行,而无需对发起者进行任何修改。这使得 iSCSI 存储系统非常灵活和可扩展

2、iSCSI服务器部署

2.1、创建共享设备

  • 创建共享设备

    root@debian12:~# fdisk /dev/vdb
    ...
    Command (m for help): n
    Partition type
       p   primary (0 primary, 0 extended, 4 free)
       e   extended (container for logical partitions)
    Select (default p): p
    Partition number (1-4, default 1):
    First sector (2048-104857599, default 2048):
    Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-104857599, default 104857599):
    
    Created a new partition 1 of type 'Linux' and of size 50 GiB.
    
    Command (m for help): w
    The partition table has been altered.
    Calling ioctl() to re-read partition table.
    Syncing disks.
    
  • 创建文件系统

    root@debian12:~# mkfs.ext4 /dev/vdb1
    mke2fs 1.47.0 (5-Feb-2023)
    Discarding device blocks: done
    Creating filesystem with 13106944 4k blocks and 3276800 inodes
    Filesystem UUID: 916d1d5f-7b37-4abd-a9ec-f6d5d25dfe61
    Superblock backups stored on blocks:
            32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
            4096000, 7962624, 11239424
    
    Allocating group tables: done
    Writing inode tables: done
    Creating journal (65536 blocks): done
    Writing superblocks and filesystem accounting information: done
    
  • 创建共享存储文件

    # 此处创建1个10GB大小的稀疏文件
    root@debian12:~# dd if=/dev/zero of=/opt/vdb1 bs=1 seek=10G count=0
    输入了 0+0 块记录
    输出了 0+0 块记录
    0 字节已复制,0.000178639 s,0.0 kB/s
    
    # 确认创建文件
    root@debian12:~# ls -lh /opt
    总计 0
    -rw-r--r-- 1 root root 10G  1月12日 20:12 vdb1
    
    root@debian12:~# du -h /opt/*
    0       /opt/vdb1
    

2.2、安装iSCSI服务

2.2.1、centos

# 安装target服务
[root@centos7 ~]# yum install targetcli -y

# 启动target服务
[root@centos7 ~]# systemctl enable --now targetclid
Created symlink from /etc/systemd/system/multi-user.target.wants/targetclid.service to /usr/lib/systemd/system/targetclid.service.
Created symlink from /etc/systemd/system/sockets.target.wants/targetclid.socket to /usr/lib/systemd/system/targetclid.socket.

# 查看target服务状态
[root@centos7 ~]# systemctl status targetclid
● targetclid.service - Targetcli daemon
   Loaded: loaded (/usr/lib/systemd/system/targetclid.service; enabled; vendor preset: disabled)
   Active: active (running) since 六 2024-01-13 16:03:04 CST; 38s ago
     Docs: man:targetclid(8)
 Main PID: 1407 (targetclid)
   CGroup: /system.slice/targetclid.service
           └─1407 /usr/bin/python /usr/bin/targetclid

1月 13 16:03:04 centos7 systemd[1]: Started Targetcli daemon.
1月 13 16:03:05 centos7 targetclid[1407]: Warning: Could not load preferences file /root/.targetcli/prefs.bin.

2.2.2、Debian

# 安装target服务
root@debian12:~# apt install targetcli-fb -y

# 启动target服务
root@debian12:~# systemctl start targetclid
root@debian12:~# systemctl enable targetclid
Created symlink /etc/systemd/system/multi-user.target.wants/targetclid.service → /lib/systemd/system/targetclid.service.
Created symlink /etc/systemd/system/sockets.target.wants/targetclid.socket → /lib/systemd/system/targetclid.socket.

# 查看服务状态
root@debian12:~# systemctl status targetclid
● targetclid.service - Targetcli daemon
     Loaded: loaded (/lib/systemd/system/targetclid.service; enabled; preset: e>
     Active: active (running) since Fri 2024-01-12 16:49:29 CST; 3min 38s ago
TriggeredBy: ○ targetclid.socket
       Docs: man:targetclid(8)
   Main PID: 1170 (targetclid)
      Tasks: 3 (limit: 2307)
     Memory: 18.1M
        CPU: 145ms
     CGroup: /system.slice/targetclid.service
             └─1170 /usr/bin/python3 /usr/bin/targetclid

1月 12 16:49:29 debian12 systemd[1]: Started targetclid.service - Targetcli dae>
1月 12 16:49:29 debian12 targetclid[1170]: Warning: Could not load preferences >
lines 1-14/14 (END)

2.3、共享库操作

targetcli 命令提供了用于管理 iSCSI 服务的工具套件,所有的操作都在该工具下执行

共享库的操作包括:

  • backstores:后端存储库,其支持的类型包括:
    • block:共享本地磁盘块设备
    • fileio:允许将文件视为磁盘映像。当创建这种类型的存储对象时,它们可以支持回写或直写操作
    • pscsi:共享本地 SCSI 设备
    • ramdisk:将内存作为块SCSI设备共享
  • iscsi:共享目标(target)
  • vhost:专门为虚拟化共享的目标,

2.3.1、创建存储库

# 创建块存储库
/> cd backstores/
/backstores> cd block
/backstores/block> create store-block /dev/vdb1
Created block storage object store-block using /dev/vdb1.

# 创建文件类型存储库
/backstores> fileio/ create store-file /opt/vdb1
Created fileio store-file with size 10737418240

2.3.2、查看存储库

/backstores> ls
o- backstores ............................................................ [...]
  o- block ................................................ [Storage Objects: 1]
  | o- store-block ................ [/dev/vdb1 (50.0GiB) write-thru deactivated]
  |   o- alua ................................................. [ALUA Groups: 1]
  |     o- default_tg_pt_gp ..................... [ALUA state: Active/optimized]
  o- fileio ............................................... [Storage Objects: 1]
  | o- store-file ................. [/opt/vdb1 (10.0GiB) write-back deactivated]
  |   o- alua ................................................. [ALUA Groups: 1]
  |     o- default_tg_pt_gp ..................... [ALUA state: Active/optimized]
  o- pscsi ................................................ [Storage Objects: 0]
  o- ramdisk .............................................. [Storage Objects: 0]

2.3.3、删除存储库

/backstores/block> delete store-block
Deleted storage object store-block.

2.4、创建共享设备

2.4.1、创建共享名

共享名称必须符合 IQN 规范,即:iqn.xxx.yyy.zzz:共享名

/> iscsi/ create iqn.2024-01.local.debian12:storage
Created target iqn.2024-01.local.debian12:storage.
Created TPG 1.
Global pref auto_add_default_portal=true
Created default portal listening on all IPs (0.0.0.0), port 3260.

/> iscsi/ create iqn.2024-01.local.debian12:storage-10gb
Created target iqn.2024-01.local.debian12:storage-10gb.
Created TPG 1.
Global pref auto_add_default_portal=true
Created default portal listening on all IPs (0.0.0.0), port 3260.

2.4.2、查看共享名信息

/> iscsi/ ls
o- iscsi .......................................................... [Targets: 2]
  o- iqn.2024-01.local.debian12:storage .................................... [TPGs: 1]
  | o- tpg1 ............................................. [no-gen-acls, no-auth]
  |   o- acls ........................................................ [ACLs: 1]
  |   | o- iqn.2024-01.local.debian12:client1 ........................ [Mapped LUNs: 0]
  |   o- luns ........................................................ [LUNs: 0]
  |   o- portals .................................................. [Portals: 1]
  |     o- 0.0.0.0:3260 ................................................... [OK]
  o- iqn.2024-01.local.debian12:storage-10gb ............................... [TPGs: 1]
    o- tpg1 ............................................. [no-gen-acls, no-auth]
      o- acls ........................................................ [ACLs: 0]
      o- luns ........................................................ [LUNs: 0]
      o- portals .................................................. [Portals: 1]
        o- 0.0.0.0:3260 ................................................... [OK]

tpg1:用户组

acls:允许访问的客户端

luns:允许访问的LUN

portals:监听端口

2.4.3、创建访问策略

# 允许名称为client1的客户端访问
/iscsi> cd iqn.2024-01.local.debian12:storage/tpg1/acls
/iscsi/iqn.20...age/tpg1/acls> create 
Created Node ACL for iqn.2024-01.local.debian12:client1

/> cd iscsi/iqn.2024-01.local.debian12:storage-10gb/tpg1/acls
/iscsi/iqn.20...0gb/tpg1/acls> create iqn.2024-01.local.debian12
Created Node ACL for iqn.2024-01.local.debian12

# 查看访问权限
/> iscsi/iqn.2024-01.local.debian12:storage/tpg1/acls/ ls
o- acls .............................................................. [ACLs: 1]
  o- iqn.2024-01.local.debian12:client1 .............................. [Mapped LUNs: 0]
/>
/> iscsi/iqn.2024-01.local.debian12:storage-10gb/tpg1/acls/ ls
o- acls .............................................................. [ACLs: 1]
  o- iqn.2024-01.local.debian12 ..................................... [Mapped LUNs: 0]

client1:只允许名称为 client1 的客户端访问

2.4.4、创建共享卷

# 创建共享卷
/> iscsi/iqn.2024-01.local.debian12:storage/tpg1/luns create /backstores/block/store-block
Created LUN 0.
Created LUN 0->0 mapping in node ACL iqn.2024-01.local.debian12:client1

/> iscsi/iqn.2024-01.local.debian12:storage-10gb/tpg1/luns create /backstores/fileio/store-file
Created LUN 0.
Created LUN 0->0 mapping in node ACL iqn.2024-01.local.debian12

2.4.5、查看共享卷

/> iscsi/iqn.2024-01.local.debian12:storage/tpg1/luns ls
o- luns .............................................................. [LUNs: 1]
  o- lun0 ................... [block/store-block (/dev/vdb1) (default_tg_pt_gp)]
  
/> iscsi/iqn.2024-01.local.debian12:storage-10gb//tpg1/luns ls
o- luns .............................................................. [LUNs: 1]
  o- lun0 ................... [fileio/store-file (/opt/vdb1) (default_tg_pt_gp)]

2.4.6、修改监听端口

# 删除监听端口
/iscsi/iqn.20.../tpg1/portals> delete 0.0.0.0 3260
Deleted network portal 0.0.0.0:3260

# 创建监听端口
/iscsi/iqn.20.../tpg1/portals> create  0.0.0.0 3260
Using default IP port 3260
Binding to INADDR_ANY (0.0.0.0)
Created network portal 0.0.0.0:3260.

# 查看监听端口
o- portals ........................................................ [Portals: 1]
  o- 0.0.0.0:3260 ......................................................... [OK]

注意:

​ 在修改端口时,如果有多个 IQN ,则需要先将所有监听端口全部删除,然后再全部新建监听端口,否则会出现 “Could not create NetworkPortal in configFS” 错误

2.4.7、查看整体配置

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- block .............................................. [Storage Objects: 1]
  | | o- store-block ................ [/dev/vdb1 (50.0GiB) write-thru activated]
  | |   o- alua ............................................... [ALUA Groups: 1]
  | |     o- default_tg_pt_gp ................... [ALUA state: Active/optimized]
  | o- fileio ............................................. [Storage Objects: 1]
  | | o- store-file ................. [/opt/vdb1 (10.0GiB) write-back activated]
  | |   o- alua ............................................... [ALUA Groups: 1]
  | |     o- default_tg_pt_gp ................... [ALUA state: Active/optimized]
  | o- pscsi .............................................. [Storage Objects: 0]
  | o- ramdisk ............................................ [Storage Objects: 0]
  o- iscsi ........................................................ [Targets: 2]
  | o- iqn.2024-01.local.debian12:storage .................................. [TPGs: 1]
  | | o- tpg1 ........................................... [no-gen-acls, no-auth]
  | |   o- acls ...................................................... [ACLs: 1]
  | |   | o- iqn.2024-01.local.debian12:client1 ...................... [Mapped LUNs: 1]
  | |   |   o- mapped_lun0 ....................... [lun0 block/store-block (rw)]
  | |   o- luns ...................................................... [LUNs: 1]
  | |   | o- lun0 ........... [block/store-block (/dev/vdb1) (default_tg_pt_gp)]
  | |   o- portals ................................................ [Portals: 1]
  | |     o- 0.0.0.0:3260 ................................................. [OK]
  | o- iqn.2024-01.local.debian12:storage-10gb ............................. [TPGs: 1]
  |   o- tpg1 ........................................... [no-gen-acls, no-auth]
  |     o- acls ...................................................... [ACLs: 1]
  |     | o- iqn.2024-01.local.debian12 ............................. [Mapped LUNs: 1]
  |     |   o- mapped_lun0 ....................... [lun0 fileio/store-file (rw)]
  |     o- luns ...................................................... [LUNs: 1]
  |     | o- lun0 ........... [fileio/store-file (/opt/vdb1) (default_tg_pt_gp)]
  |     o- portals ................................................ [Portals: 1]
  |       o- 0.0.0.0:3260 ................................................. [OK]
  o- loopback ..................................................... [Targets: 0]
  o- vhost ........................................................ [Targets: 0]
  o- xen-pvscsi ................................................... [Targets: 0]

2.5、保存配置

直接使用 exit 命令即可退出并保存配置

/> exit
Global pref auto_save_on_exit=true
Last 10 configs saved in /etc/target/backup/.
Configuration saved to /etc/target/saveconfig.json

2.6、使共享生效

重启 targetclid 服务即可生效

root@debian12:~# systemctl restart targetclid

2.7、删除共享设备

删除共享设备可以直接将 /iscsi 下的 iqn 共享删除即可,但是为了安全,建议按以下步骤执行删除操作:

  • 在客户端卸载已挂载的 iSCSI 卷

  • 删除 portals (如果使用非默认配置)

  • 删除 LUN

  • 删除 ACL

  • 删除 IQN

  • 删除 backstores 内容

3、iSCSI客户端配置

  • 安装 iSCSI 工具包

    在客户端中,需要安装 iscsi 客户端工具才能配置、使用 iSCSI 服务

    # centos
    [root@centos7 ~]# yum install iscsi-initiator-utils -y
    
    # Debian
    root@debian12:~# apt install open-iscsi -y
    
  • 修改iSCSI名称

    需要将客户端名称设置为 iscsi/iqn.xxx.yyy.zzz:NAME/tpg1/acls/ 中相同的名称

    • Linux系统中修改

      [root@centos7 ~]# vim /etc/iscsi/initiatorname.iscsi
      InitiatorName=iqn.2024-01.local.debian12:client1
      
    • Windows中修改

  • 发现iSCSI目标

    root@debian12:~# iscsiadm --mode discoverydb --type sendtargets --portal 192.168.10.122 --discover
    192.168.10.122:3260,1 iqn.2024-01.local.debian12:storage
    192.168.10.122:3260,1 iqn.2024-01.local.debian12:storage-10gb
    
  • 登录目标

    root@debian12:~# iscsiadm --mode node --targetname iqn.2024-01.local.debian12:storage-10gb --portal 192.168.10.122:3260 --login
    Logging in to [iface: default, target: iqn.2024-01.local.debian12:storage-10gb, portal: 192.168.10.122,3260]
    Login to [iface: default, target: iqn.2024-01.local.debian12:storage-10gb, portal: 192.168.10.122,3260] successful.
    
  • 查看挂载情况

    root@kvm-debian12:~# lsblk
    NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
    sda      8:0    0   10G  0 disk
    
  • 客户端名称不符的情况

    root@debian12:~# iscsiadm --mode node --targetname iqn.2024-01.local.debian12:storage --portal 192.168.10.122:3260 --login
    Logging in to [iface: default, target: iqn.2024-01.local.debian12:storage, portal: 192.168.10.122,3260]
    iscsiadm: Could not login to [iface: default, target: iqn.2024-01.local.debian12:storage, portal: 192.168.10.122,3260].
    iscsiadm: initiator reported error (24 - iSCSI login failed due to authorization failure)
    iscsiadm: Could not log into all portals
    

更多客户端配置与使用信息,请参见:Centos挂载iSCSI