NetApp FAS3210 NAS存储在线更换硬盘

存储更换硬盘，大致可分为三个步骤。一是检查状态，确定硬盘位置，二是更换硬盘，在系统中进行自动或者手动确认；三是检查存储状态。对于前两步，一般来说，坏盘指示灯会有所标识，而基本上所有存储类产品的硬盘都支持热插拨。所以，对于一个想应付客户、草草了事的人来说，最为简便的方法就是看面板指示灯，哪个灯色异常换哪个。如果凑巧该存储可以自动识别硬盘并做相应操作，那便万事大吉。 但是，存储无小事，不论该设备盘多盘少，都是生产中最基本的环节，也是最重要的单元。所以，本着对自己对别人负责的态度，操作存储，还是要小心为妙，再多的细心也是必要的。 一、检查系统状态 Fas3210A> sysconfig -r Aggregate aggr0 (online, raid_dp) (block checksums) Plex /aggr0/plex0 (online, normal, active) RAID group /aggr0/plex0/rg0 (normal) RAID Disk Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks) --------- ------          ------------- ---- ---- ---- ----- --------------    -------------- dparity   0a.01.0         0a    1   0   SA:A   -  BSAS  7200 1695466/3472315904 1695759/3472914816 parity    0a.01.1         0a    1   1   SA:A   -  BSAS  7200 1695466/3472315904 1695759/3472914816 data      0a.01.2         0a    1   2   SA:A   -  BSAS  7200 1695466/3472315904 1695759/3472914816 Aggregate aggr1 (online, raid_dp, reconstruct) (block checksums) Plex /aggr1/plex0 (online, normal, active) RAID group /aggr1/plex0/rg0 (reconstruction 5% completed) RAID Disk Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks) --------- ------          ------------- ---- ---- ---- ----- --------------    -------------- dparity   0a.02.0         0a    2   0   SA:A   -  BSAS  7200 1695466/3472315904 1695759/3472914816 parity    0a.03.0         0a    3   0   SA:A   -  BSAS  7200 1695466/3472315904 1695759/3472914816 data      0a.01.3         0a    1   3   SA:A   -  BSAS  7200 1695466/3472315904 1695759/3472914816 FAILED      0a.02.6         0a    2   6   SA:A   -  BSAS  7200 1695466/3472315904 1695759/3472914816 data      0a.03.1         0a    3   1   SA:A   -  BSAS  7200 1695466/3472315904 1695759/3472914816 data      0a.02.7         0a    2   7   SA:A   -  BSAS  7200 1695466/3472315904 1695759/3472914816 Spare disks (empty) Broken disks RAID Disk       Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks) ---------       ------          ------------- ---- ---- ---- ----- --------------    -------------- failed          0a.03.9         0a    3   9   SA:A   -  BSAS  7200 1695466/3472315904 1695759/3472914816 Partner disks RAID Disk       Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks) ---------       ------          ------------- ---- ---- ---- ----- --------------    -------------- partner         0a.01.7         0a    1   7   SA:A   -  BSAS  7200 0/0               1695759/3472914816 partner         0a.01.6         0a    1   6   SA:A   -  BSAS  7200 0/0               1695759/3472914816 partner         0a.01.20        0a    1   20  SA:A   -  BSAS  7200 0/0               1695759/3472914816 partner         0a.01.13        0a    1   13  SA:A   -  BSAS  7200 0/0               1695759/3472914816 如上所示，坏盘有两块，在命令结果中显示为“failed”，并且有相应的位置。 也可以通过查看所有硬盘状态来获取坏盘信息 Fas3210A> disk show -v DISK       OWNER                      POOL   SERIAL NUMBER         HOME ------------ -------------              -----  -------------         ------------- 0a.02.2      Fas3210B  (1574419336)    Pool0  WD-WMAY02394580       Fas3210B  (1574419336) 0a.01.4      Fas3210B  (1574419336)    Pool0  B9KKWBZF              Fas3210B  (1574419336) 0a.03.7      Fas3210B  (1574419336)    Pool0  WD-WMAY04134224       Fas3210B  (1574419336) 0a.01.1      Fas3210A  (1574419769)    Pool0  BFGUKM5F              Fas3210A  (1574419769) 0a.01.3      Fas3210A  (1574419769)    Pool0  BFGUKKBF              Fas3210A  (1574419769) 0a.02.6      Fas3210A  (1574419769)   FAILED  WD-WMAY02391728       Fas3210A  (1574419769) 0a.03.9      Fas3210A  (1574419769)    FAILED WD-WMAY02504662       Fas3210A  (1574419769) 在确定坏盘槽位后就可以更换硬盘，但是在这之前，理应查看硬盘PN，作相关的采购准备 Fas3210A> storage show disk DISK                  SHELF BAY SERIAL           VENDOR   MODEL      REV --------------------- --------- ---------------- -------- ---------- ---- 0a.02.6                 2    6  WD-WMAY02391728  NETAPP   X306_WMANT NA04 0a.02.7                 2    7  WD-WMAY02377652  NETAPP   X306_WMANT NA04 0a.02.8                 2    8  WD-WMAY02501788  NETAPP   X306_WMANT NA04 0a.03.8                 3    8  WD-WMAY02481326  NETAPP   X306_WMANT NA04 0a.03.9                 3    9  WD-WMAY02504662  NETAPP   X306_WMANT NA04 0a.03.10                3   10  WD-WMAY02459742  NETAPP   X306_WMANT NA04 如上所示，坏盘的PN为“X306_WMANT NA04”，或可通过以下命令查看 Fas3210A> sysconfig -v NetApp Release 8.0.2 7-Mode: Mon Jun 13 14:13:45 PDT 2011 System ID: 1574419769 (Fas3210A); partner ID: 1574419336 (Fas3210B) System Serial Number: 850000103829 (Fas3210A) System Rev: F4 System Storage Configuration: Single-Path HA System ACP Connectivity: NA slot 0: System Board 2.3 GHz (System Board XVI F4) Model Name:         FAS3210 Part Number:        111-00585 Revision:           F4 Serial Number:      5006779457 BIOS version:       5.1.1 Loader version:     3.2 Processors:         2 Processor type:     Intel(R) Xeon(R) CPU           E5220  @ 2.33GHz Memory Size:        5120 MB Memory Attributes:  Bank Interleaving Hoisting Rank Interleaving Normal ECC NVMEM Size:         640 MB of Main Memory Used CMOS RAM Status:    OK Service Processor           Status: Online Firmware Version:   1.2.2 Mgmt MAC Address:   00:A0:98:18:77:C8 Ethernet Link:      down Using DHCP:         no IPv4 configuration: IP Address:         unknown Netmask:            unknown Gateway:            unknown slot 0: Internal BGE 10/100 Ethernet Controller e0M MAC Address:    00:a0:98:18:77:c6 (auto-100tx-fd-up) e0P MAC Address:    00:a0:98:18:77:c7 (auto-100tx-fd-up) Device Type:        BCM5721 slot 0: Dual 10G Ethernet Controller T320E-SFP/KR Device Type:        CT-FE-3 Version Number:     T3-SRAM1.1.0-BR1016-02-01-FW7.7.192-DR04 Serial Number:      jb04050693 c0a MAC Address:    00:a0:98:18:77:c4 (auto-unknown-enabling) c0b MAC Address:    00:a0:98:18:77:c5 (auto-10g_kr-fd-up) slot 0: Dual 10/100/1000 Ethernet Controller G20 e0a MAC Address:    00:a0:98:18:77:c2 (auto-100tx-fd-up) e0b MAC Address:    00:a0:98:18:77:c3 (auto-100tx-fd-up) Device Type:        Rev 6 slot 0: SAS Host Adapter 0a (PMC-Sierra PM8001 rev. C, SAS, ) Firmware rev:       01.10.14.00 Base WWN:           5:00a098:000808d:e0 Phy State:          [0] Enabled, 3.0 Gb/s [1] Enabled, 3.0 Gb/s [2] Enabled, 3.0 Gb/s [3] Enabled, 3.0 Gb/s QSFP Vendor:        Molex Inc. QSFP Part Number:   112-00177+A0 QSFP Type:          Passive Copper 2m ID:01 QSFP Serial Number: 116620073 ID     Vendor   Model            FW    Size 01.23: NETAPP   X306_HJUPI02TSSA NA01 1695.4GB (3907029168 512B/sect) 02.0 : NETAPP   X306_WMANT02TSSM NA04 1695.4GB (3907029168 512B/sect) 02.1 : NETAPP   X306_WMANT02TSSM NA04 1695.4GB (3907029168 512B/sect) 02.2 : NETAPP   X306_WMANT02TSSM NA04 1695.4GB (3907029168 512B/sect) 02.3 : NETAPP   X306_WMANT02TSSM NA04 1695.4GB (3907029168 512B/sect) 02.4 : NETAPP   X306_WMANT02TSSM NA04 1695.4GB (3907029168 512B/sect) 02.5 : NETAPP   X306_HMARK02TSSM NA00 1695.4GB (3907029168 512B/sect) 02.6 : NETAPP   X306_WMANT02TSSM NA04 1695.4GB (3907029168 512B/sect) (Failed) 02.7 : NETAPP   X306_WMANT02TSSM NA04 1695.4GB (3907029168 512B/sect) 03.6 : NETAPP   X306_WMANT02TSSM NA04 1695.4GB (3907029168 512B/sect) 03.7 : NETAPP   X306_WMANT02TSSM 4321 1695.4GB (3907029168 512B/sect) 03.8 : NETAPP   X306_WMANT02TSSM NA04 1695.4GB (3907029168 512B/sect) 03.9 : NETAPP   X306_WMANT02TSSM NA04 1695.4GB (3907029168 512B/sect) (Failed) 03.10: NETAPP   X306_WMANT02TSSM NA04 1695.4GB (3907029168 512B/sect) Shelf   1: IOM3  Firmware rev. IOM3 A: 0131 IOM3 B: 0131 Shelf   2: IOM3  Firmware rev. IOM3 A: 0131 IOM3 B: 0131 Shelf   3: IOM3  Firmware rev. IOM3 A: 0131 IOM3 B: 0131 slot 0: SAS Host Adapter 0b (PMC-Sierra PM8001 rev. C, SAS, <OFFLINE (hard)>) Firmware rev:       01.10.14.00 Base WWN:           5:00a098:000808d:e4 Phy State:          [4] Disabled [5] Disabled [6] Disabled [7] Disabled QSFP Vendor:        not available QSFP Part Number:   not available QSFP Type:          not available QSFP Serial Number: not available slot 0: Intel ICH USB EHCI Adapter u0a (0xdf901400) boot0   Micron Technology Real SSD eUSB 2GB, rev 2.00/11.10, addr 2 1936MB 512B/sect (0FF0022700058229) slot 0: FC Host Adapter 0c (QLogic 2432 rev. 2, L-port, <OFFLINE (hard)>) Firmware rev:      5.4.0 Host Loop Id:      0 FC Node Name:      5:00a:098000:808dd8 FC Port Name:      5:00a:098000:808dd8 SFP Vendor:        FINISAR CORP. SFP Part Number:   FTLF8524P2BNV SFP Serial Number: PL82Q4S SFP Capabilities:  1, 2 or 4 Gbit Link Data Rate:    N/A slot 0: FC Host Adapter 0d (QLogic 2432 rev. 2, L-port, <OFFLINE (hard)>) Firmware rev:      5.4.0 Host Loop Id:      0 FC Node Name:      5:00a:098100:808dd8 FC Port Name:      5:00a:098100:808dd8 SFP Vendor:        FINISAR CORP. SFP Part Number:   FTLF8524P2BNV SFP Serial Number: PL82RFY SFP Capabilities:  1, 2 or 4 Gbit Link Data Rate:    N/A slot 1: Quad Gigabit Ethernet Controller 82580 e1a MAC Address:    00:1b:21:c4:36:88 (auto-unknown-down) e1b MAC Address:    00:1b:21:c4:36:89 (auto-unknown-down) e1c MAC Address:    00:1b:21:c4:36:8a (auto-unknown-down) e1d MAC Address:    00:1b:21:c4:36:8b (auto-unknown-down) Device Type:    150E, PBA E68891-015 slot 2: Quad Gigabit Ethernet Controller 82580 e2a MAC Address:    00:1b:21:c4:35:c4 (auto-unknown-down) e2b MAC Address:    00:1b:21:c4:35:c5 (auto-unknown-down) e2c MAC Address:    00:1b:21:c4:35:c6 (auto-unknown-down) e2d MAC Address:    00:1b:21:c4:35:c7 (auto-unknown-down) Device Type:    150E, PBA E68891-015 二、更换硬盘 标识硬盘 Fas3210A> priv set advanced（进入高级模式） Fas3210A*> led_on 0a.2.6（使硬盘指示灯闪烁） Fas3210A*> led_off 0a.2.6（取消硬盘指示灯闪烁） Fas3210A*> priv set admin（退出高级模式） 打开硬盘扣，取出坏盘，换上新盘，关在系统中查看新盘状态 Fas3210A*> disk show -v DISK       OWNER                      POOL   SERIAL NUMBER         HOME ------------ -------------              -----  -------------         ------------- 0a.02.3      Fas3210B  (1574419336)    Pool0  WD-WMAY02393819       Fas3210B  (1574419336) 0a.02.4      Fas3210B  (1574419336)    Pool0  WD-WMAY02501482       Fas3210B  (1574419336) 0a.01.14     Fas3210B  (1574419336)    Pool0  BFGULBNF              Fas3210B  (1574419336) 0a.01.23     Fas3210B  (1574419336)    Pool0  BFGUKM4F              Fas3210B  (1574419336) 0a.01.5      Fas3210B  (1574419336)    Pool0  BFGS1EDF              Fas3210B  (1574419336) 0a.01.1      Fas3210A  (1574419769)    Pool0  BFGUKM5F              Fas3210A  (1574419769) 0a.01.3      Fas3210A  (1574419769)    Pool0  BFGUKKBF              Fas3210A  (1574419769) 0a.02.6      Not Owned      NONE            YFG95P3A 0a.02.8      Fas3210A  (1574419769)    Pool0  WD-WMAY02501788       Fas3210A  (1574419769) 0a.02.0      Fas3210A  (1574419769)    Pool0  WD-WMAY02481293       Fas3210A  (1574419769) 0a.03.10     Fas3210A  (1574419769)    Pool0  WD-WMAY02459742       Fas3210A  (1574419769) 0a.03.3      Fas3210A  (1574419769)    Pool0  WD-WMAY02482034       Fas3210A  (1574419769) 0a.02.12     Fas3210A  (1574419769)    Pool0  BFGB2R9F              Fas3210A  (1574419769) 0a.03.2      Fas3210A  (1574419769)    Pool0  WD-WMAY02481343       Fas3210A  (1574419769) 0a.02.10     Fas3210A  (1574419769)    Pool0  WD-WMAY02389974       Fas3210A  (1574419769) 0a.02.5      Fas3210A  (1574419769)    Pool0  YFH7R0BA              Fas3210A  (1574419769) 如果如上所示，新盘状态为“Not Owned”，则需执行以下命令 Fas3210A*> disk assign 0a.03.9（将该盘分配给当前控制器） Fas3210B*> disk assign 0a.03.9 -o Fas3210A(将该盘分配给特定的控制器） 三、检查存储状态 再用命令“disk show -v”查看硬盘状态、“sysconfig -r”查看系统状态 如果换上去的是新盘，在做完“disk assign”后，一般不会有后续问题。如果是拆机备件，就要留意在“sysconfig -r”输出里面该盘是否是“not zeroed”或者没有自动被分配为热备盘而是多余新建了一个卷簇。 对于新更换盘被添加到热备盘组并且“not zeroed”的情况，热行命令“disk zero spares”即可解决。对于新建卷簇的情况，则需要执行以下命令删除该卷簇。 查看卷簇状态，确定哪些是固有，哪个是新建（假设新建的卷簇为"agg1(1)"） Fas3210A*> aggr status 改变该卷簇状态为离线 Fas3210A*> aggr offline aggr1(1) 删除该卷簇 Fas3210A*> aggr destroy aggr1(1) 再次查看系统状态，确保新更换硬盘已被分配到热备盘组并且状态正常 Fas3210A*>sysconfig -r 附： 查看日志命令 Fas3210A> rdfile /etc/messages 开启系统磁盘自动分配 Fas3210A*>options disk.auto_assign off（关闭） Fas3210A*>options disk.auto_assign（查看） 部分拆机备件本无问题但更换后出现"failed"状态，可执行以下命令强制改变硬盘状态，如果改变后仍为"failed"状态，则该备件故障 Fas3210A*> disk unfail 0a.2.6 切换回普通模式 Fas3210A*>priv set