1、2010年5月DS8000存儲容災技術11111DS8000存儲介紹13東四容災方案介紹DS8000存儲容災方案介紹2目錄4DS8000存儲PPRC介紹DS8000 硬件圖示2DS8000 組件FramesFrames3DS8000 體系架構4將物理磁盤組成邏輯卷,提供給效勞器使用,使得這些邏輯卷看起來像是物理磁盤一樣DS8000 邏輯配置磁盤虛擬化5DS8000 邏輯配置層次One array site used to construct one RAID Array (8 DDMs) RAID 5, 10, 61 Array per Rank Rank can be CKD or FB1-

2、N ranks in Extent PoolAssociated with Server0 or Server1 Divided into ExtentsVolumes are made in Extent Pool6DS8000 邏輯配置層次（續(xù)）Volumes or LUNsMade up of extents from one extent poolMin size 0.1 GB or 1 cylinderMax size 2 TB (FB) 56 GB(CKD)Associated with LSS during configurationVolume GroupContains LU

3、Nsand host attachments -FB LUN maskingOne host attachment (one port or port group) can be member of only one volume groupOne volume can be member of multiple volume groupsMultiple hosts (even different open systems server types) can be member of the same volume groupHostMultiple server ports can be

4、specified (in one or more groups) in one host definition7DS8000 邏輯配置ArrayAn array is created from one array site.Forming an array means defining it for a specific RAID typeRAID 5, 10 ,6In the DS8000 current implementation, one array is defined using one array siteAccording to the DS8000 sparing algo

5、rithm, from zero to two spares may be taken from the array siteAn array is formed using one array site and while the array could be accessed by each adapter of the device adapter pair, it is managed by one device adapterWhich adapter and which server manages this array are defined later in the confi

6、guration pathD1, D2, D3, and so on, stand for the set of data contained on one disk within a stripe on the arrayIf, for example 1 GB of data is written, it is distributed across all the disks of the arrayArray Site, Array 配置例子DS8000 邏輯配置RankWhen a new rank is defined, its name is chosen by the DS St

7、orage ManagerFor example: R1, R2, or R3, and so onYou have to add an array to a rankProcess of forming a rank:The array is formatted for either FB (open systems) or CKD (zSeries) data. This determines the size of the set of data contained on one disk within a stripe on the array.The capacity of the

8、array is subdivided into equal sized partitions, called extents. The extent size depends on the extent type, FB or CKDOne extent is the minimum physical allocation unit when a LUN or CKD volume is createdDS8000 邏輯配置Extent PoolAn extent pool is a logical construct to aggregate the extents from a set

9、of ranks to form a domain for extent allocation to a logical volumeSame RAID type and the same disk RPM characteristicsSo that the extents in the extent pool have homogeneous characteristicsNo predefined affinity of ranks or arrays to a storage serverThe affinity of the rank (and its associated arra

10、y) to a given server is determined at the point it is assigned to an extent poolWhen assigned to extent pools ranks are organized in two rank groups:Rank group 0 is controlled by server 0Rank group 1 is controlled by server 1One or more ranks with the same extent type can be assigned to an extent po

11、olOne rank can be assigned to only one extent pool.There can be as many extent pools as there are ranks.At least two extent pools (one per server to balance the activity) of each typeCKD or FBAdditional extent pools may be used to segregate ranks with different DDM types11DS8000 邏輯配置邏輯卷A logical vol

12、ume is composed of a set of extents from one extent pool.On a DS8000, up to 65280 volumes can be created12DS8000 邏輯配置LSSA logical subsystem (LSS) is another logical construct. It groups logical volumes, LUNs, in groups of up to 256 logical volumes.LSSs also have an affinity to the servers. All even-

13、numbered LSSs(X00, X02, X04, up to XFE) belong to server 0All odd-numbered LSSs(X01, X03, X05, up to XFD) belong to server 1.13DS8000 邏輯配置LSS 例子14DS8000 邏輯配置Host AttachmentMultiple server ports can be specified (in one or more groups) in one host definitionEach host attachment can be associated with

14、 a volume group to define which LUNsthat HBA is allowed to access. Multiple host attachments can share the same volume group.15DS8000 邏輯配置Volume GroupMaps LUNsto server ports or port groupsFB LUN maskingLUN masking at VG level, not LUN levelSpecific host attachment can be in only one volume groupOpt

15、ions for shared accessPlace hosts in separate volume groups and shared volumes in multiple volume groupsPlace shared volumes and multiple hosts in single volume groupOne Host port/port group can be member of one volume groupOne Volume can be member of more than one volume groupMore than one host (ev

16、en different open systems server types) can be member of one volume group16DS8000 邏輯配置總結171818181818DS8000存儲容災方案介紹2目錄1919191919建立存儲容災重要性無法預知的 IT 硬件設備的損壞、斷電、火災、自然災害、恐怖襲擊等,造成數(shù)據(jù)喪失或業(yè)務的突然中斷；系統(tǒng)人員誤操作造成意外宕機或關鍵數(shù)據(jù)喪失,無法防止；手段頻多的黑客攻擊、病毒入侵、垃圾郵件、網(wǎng)絡與系統(tǒng)的漏洞,造成網(wǎng)絡癱瘓、系統(tǒng)崩潰。一旦發(fā)生災難(洪水、地震、火災等),或者人為災難(用戶失誤、磁盤失效等)導致數(shù)據(jù)喪失或者業(yè)務中斷

17、時,能夠快速、及時地恢復數(shù)據(jù),保證業(yè)務的連續(xù)運行。提高系統(tǒng)可用性,增加業(yè)務的可持續(xù)性,減少關鍵業(yè)務應用程序受故障影響在災難和不可預知事件發(fā)生后,最快恢復關鍵業(yè)務數(shù)據(jù),保障業(yè)務運營持續(xù)性DS8000 容災技術介紹 IBM 企業(yè)級存儲容災解決方案通過IBM System Storage DS8000 企業(yè)級高端磁盤存儲系統(tǒng),結合IBM 特有的數(shù)據(jù)復制技術Metro Mirror(同步的數(shù)據(jù)復制)和 Global Mirror (異步的數(shù)據(jù)復制),在兩套或多套DS8000 磁盤存儲設備間建立數(shù)據(jù)復制關系從而實現(xiàn)高可用性,在數(shù)據(jù)存儲方面提高IT 根底架構的整體可用性。方案配合Power 效勞器AIX

18、操作系統(tǒng)的特色軟件PowerHA/XD 及IBM 全面的業(yè)務連續(xù)性方案配套效勞,建立完善的災難備份中心,從而保證關鍵數(shù)據(jù)的可恢復性與業(yè)務應用的可持續(xù)性20遠程數(shù)據(jù)同步復制技術-Metro MirrorPPRC 的同步實現(xiàn)機制1）在生產(chǎn)系統(tǒng)中的應用程序將數(shù)據(jù)寫到生產(chǎn)系統(tǒng)的磁盤。2）生產(chǎn)系統(tǒng)中心的磁盤數(shù)據(jù)傳輸?shù)絺浞葜行牡拇疟P3）當生產(chǎn)數(shù)據(jù)都寫入備份中心存儲設備后,備份磁盤將寫完操作信息返給生產(chǎn)磁盤4）生產(chǎn)系統(tǒng)收到災備系統(tǒng)傳回的已寫信息之后,生產(chǎn)機的磁盤系統(tǒng)通知主機該寫操作已完畢。# 現(xiàn)在同步復制技術 PPRC 已經(jīng)正式改名為Metro Mirror同步數(shù)據(jù)復制零數(shù)據(jù)喪失支持最遠303公里數(shù)據(jù)寫入后

19、備存儲返回寫操作完成信息返回數(shù)據(jù)寫入成功信息數(shù)據(jù)寫入存儲21遠程數(shù)據(jù)異步復制技術（保證一致性） -Global MirrorPPRC-XD 的異步機制1)在生產(chǎn)系統(tǒng)中的應用程序將數(shù)據(jù)寫到生產(chǎn)系統(tǒng)的磁盤。2)當數(shù)據(jù)完全寫入生產(chǎn)中心存儲設備后,I/O完成的信息會返給生產(chǎn)主機,當生產(chǎn)系統(tǒng)收到磁盤系統(tǒng)傳回的已寫信息之后,生產(chǎn)系統(tǒng)應用繼續(xù)執(zhí)行,即可以進行下一步操作。3)備份機磁盤數(shù)據(jù)復制,備份磁盤復制生產(chǎn)系統(tǒng)數(shù)據(jù)。生產(chǎn)磁盤會定期把已經(jīng)改變了的數(shù)據(jù)傳輸?shù)絺浞葜行牡拇疟P。4)當數(shù)據(jù)已經(jīng)寫入備份中心存儲設備后,寫完操作信息會返回給生產(chǎn)磁盤,生產(chǎn)中心的磁盤系統(tǒng)認為此次操作完成異地 RPO 最短 3-5 秒,無距

20、離限制,保證數(shù)據(jù)一致性。數(shù)據(jù)寫入存儲返回數(shù)據(jù)寫入成功信息22本地數(shù)據(jù)復制技術 FlashCopy FlashCopy 提供一個“時間點的拷貝效勞功能,從源卷到目標卷快速地復制數(shù)據(jù)。邏輯拷貝通常可以在數(shù)秒時間內(nèi)完成,然后就釋放源卷,進行正常工作。而物理拷貝操作在后臺進行當物理拷貝進行過程中,拷貝和被拷貝數(shù)據(jù)都能被使用23PPRC、flashcopy技術的結合A -B做PPRC,B是不能被效勞器訪問的,為充分利用災備中心的設備,DS8000 FlashCopy 功能可以對B進行一份或多分快照,用于系統(tǒng)查詢、批處理、測試。24DS8000 容災解決方案介紹-兩地三中心方案25DS8000 容災解決方

21、案介紹 -存儲HA+異地災備(HA+GM)2627272727273東四容災方案介紹目錄東四現(xiàn)有存儲序號型號序列號IP地址物理位置1DS8100IBM.2107-75APML119東四6層機房2DS8300IBM.2107-75BYYC120東四6層機房3DS8100IBM.4東四6層機房4DS6800IBM.79東四6層機房5DS8300TIBM.2107-75DMLN130金盞4層機房28東四容災系統(tǒng)序號系統(tǒng)生產(chǎn)端存儲容災端存儲容災類型1IBS帳務系統(tǒng)DS6_DS8100JZ_DS8300T應用級2IBS收費系統(tǒng)DS6_DS8100JZ_DS8300T應用級3IBS統(tǒng)計庫系統(tǒng)DS6_DS8

22、100JZ_DS8300T數(shù)據(jù)級4綜合結算系統(tǒng)DS6_DS8100JZ_DS8300T數(shù)據(jù)級5ERP系統(tǒng)DS6_DS8100JZ_DS8300T數(shù)據(jù)級6IBS綜合接口機系統(tǒng)DS6_DS8100JZ_DS8300T數(shù)據(jù)級7IBSOD系統(tǒng)DS6_DS8300JZ_DS8300T應用級8IBS匯集系統(tǒng)DS6_DS8300JZ_DS8300T應用級9IBS計費系統(tǒng)DS6_DS8300JZ_DS8300T應用級10一卡充數(shù)據(jù)庫系統(tǒng)DS6_DS8300JZ_DS8300T數(shù)據(jù)級11一卡充物理節(jié)點系統(tǒng)DS6_DS8300JZ_DS8300T數(shù)據(jù)級12稅控系統(tǒng)DS6_DS8300JZ_DS8300T數(shù)據(jù)級13

23、綜合積分系統(tǒng)DS6_DS8300JZ_DS8300T數(shù)據(jù)級14IOM系統(tǒng)DS8_DS6800JZ_DS8300T應用級15CSMDS8_SUN9990JZ_SUN9990T應用級29北京聯(lián)通容災拓撲圖303131313131目錄4DS8000存儲PPRC介紹DS8000邏輯配置步驟1.安裝激活許可2.配置IO卡,定義IO的類型.3.創(chuàng)立arrays. 配置arrays的raid類型（raid5.raid10,radi6)4.創(chuàng)立 ranks.5.創(chuàng)立 extent pools. 配置extent pool 的server(Server 0 or Server 1), 并指定rank到exten

24、t pool.6.創(chuàng)立LUN,定義lun類型 (FB,CDK)7.創(chuàng)立volgrp8.創(chuàng)立主機連接關系32安裝許可1.通過DSCLI下的lskey命令,查看許可證狀態(tài)2.通過applykey命令,安裝許可證3.通過lskey驗證33配置I/O端口1.通過lsioport命令,檢查端口的狀態(tài)2.通過setioport命令,設置端口狀態(tài) dscli setioporttopology scsi-fcpI000034創(chuàng)立Array1.通過lsarraysite命令檢查DS8000上安裝的Arraysite2.通過mkarray命令創(chuàng)立Array35創(chuàng)立Array3.通過lsarray命令驗證創(chuàng)立的A

25、rray36創(chuàng)立Rank1.在創(chuàng)立Array的根底上,通過mkrank命令創(chuàng)立Rank2.通過lsrank命令驗證已創(chuàng)立的Rank37創(chuàng)立ExtpoolExtpool,并指定,RankRank到Extpool1.通過mkextpool命令驗證創(chuàng)立的Extpool2.通過lsexptool命令查看Extpool,確認其ID38創(chuàng)立ExtpoolExtpool,并指定,RankRank到Extpool3.通過chrank命令,將Rank指定到Extpool4.通過lsexptool命令查看Extpool,可以看到Extpool的容量39創(chuàng)立標準的FB卷1.適用于單個Rank的Extpool情況2.

26、通過mkfbvol命令,創(chuàng)立FB卷3.通過lsfbvol命令,檢查創(chuàng)立好的卷40創(chuàng)立在存儲池中條帶化的FB卷1.適用于多個Rank的Extpool情況2.通過mkfbvol命令,創(chuàng)立FB卷,注意eamrotateexts參數(shù)3.通過lsfbvol命令,檢查創(chuàng)立好的卷41創(chuàng)立 Volume Group1.確定需要連接的主機類型scsimap256 或是scsimask42創(chuàng)立 Volume Group2.通過mkvolgrp命令,創(chuàng)立VG3.通過showvolgrp命令,查看VG中的卷43創(chuàng)立 Volume Group4.通過chvolgrp命令,修改VG中的卷44創(chuàng)立主機連接,并指定VG到相應

27、的主機連接1.通過mkhostconnect命令,創(chuàng)立主機連接 建立hostconnect連接時,選擇存儲Ioport時盡量選擇不同IO卡的口,防止IO卡故障造成連接不可用。mkhotconnect -wwname 10000000C9846593 -profile IBM pSeries - AIX -volgrp V9 -ioport I0202 p570_ZHJFDB1_FCS1mkhotconnect -wwname 10000000C9846593 -profile IBM pSeries - AIX -volgrp V9 -ioport I0203 p570_ZHJFDB1_FCS

28、1mkhotconnect -wwname 10000000C984507D -profile IBM pSeries - AIX -volgrp V9 -ioport I0332 p570_ZHJFDB1_FCS2mkhotconnect -wwname 10000000C984507D -profile IBM pSeries - AIX -volgrp V9 -ioport I0333 p570_ZHJFDB1_FCS2mkhotconnect -wwname 10000000C984659E -profile IBM pSeries - AIX -volgrp V9 -ioport I

29、0202 p570_ZHJFDB2_FCS1mkhotconnect -wwname 10000000C984659E -profile IBM pSeries - AIX -volgrp V9 -ioport I0203 p570_ZHJFDB2_FCS1mkhotconnect -wwname 10000000C984644D -profile IBM pSeries - AIX -volgrp V9 -ioport I0332 p570_ZHJFDB2_FCS2mkhotconnect -wwname 10000000C984644D -profile IBM pSeries - AIX

30、 -volgrp V9 -ioport I0333 p570_ZHJFDB2_FCS22.通過lshostconnect命令,Name ID WWPN HostType Profile portgrp volgrpID ESSIOport=p570_ZHJFDB1_FCS1 0C9846593 - IBM pSeries - AIX 0 V9 I0202p570_ZHJFDB1_FCS1 0C9846593 - IBM pSeries - AIX 0 V9 I0203p570_ZHJFDB1_FCS2 0C984507D - IBM pSeries - AIX 0 V9 I0332p570_Z

31、HJFDB1_FCS2 0C984507D - IBM pSeries - AIX 0 V9 I0333p570_ZHJFDB2_FCS1 0C984659E - IBM pSeries - AIX 0 V9 I0202p570_ZHJFDB2_FCS1 0C984659E - IBM pSeries - AIX 0 V9 I0203p570_ZHJFDB2_FCS2 0C984644D - IBM pSeries - AIX 0 V9 I0332p570_ZHJFDB2_FCS2 0C984644D - IBM pSeries - AIX 0 V9 I03333.通過chhostconnec

32、t命令,可以修改指定的VG45多路徑管理軟件多路徑管理軟件 Subsystem Device Driver (SDD)SUN StorEdgeTraffic Manager for SUN Solaris.HP PVLinksfor HP-UX.IBM AIX native multipathingI/O support (MPIO).IBM i5/OS V5R3 multipathsupport.In addition, there are third-party multipathingsolutions, such as VeritasDMP, which is part of Veri

33、tasVolume Manager46建立PPRC步驟1、確定生產(chǎn)端、容災端lun已建立 lun對應關系原則,兩端lun的大小、id最好一樣。 lss分配原則,按系統(tǒng)分類,給每個系統(tǒng)分配特定的LSS,根據(jù)存儲整體規(guī)劃2、確定生產(chǎn)端與容災端的物理鏈路存在3、建立PPRCpath,并修改lss屬性。pprcpath是建立在物理鏈路的根底上的邏輯鏈路。每條物理鏈路最多能建256條邏輯鏈路。pprcpath的通信是基于LSS每個LSS最多對應4個LSS東四lss屬性timeout 為30秒4、建立pprc關系47確定容災LUN對應關系以綜合結算系統(tǒng)為例生產(chǎn)端存儲：東四數(shù)據(jù)保護DS8100 容災端存儲：

34、金盞DS8300T生產(chǎn)端LUN:7800-7805,7810-7815,7820-7826,7830-7832, 7900-790,7910-7915,7920-7926,7930-793容災端LUN:7800-7805,7810-7815,7820-7826,7830-7832, 7900-790,7910-7915,7920-7926,7930-79348確定可用的物理鏈路可以通過 lsavailpprcport 命令查詢1、首先要先查詢到兩端的設備id 和設備的 wwwn號,可以用 lssi命令查詢到/*PPRC from shujubaohu_DS8100 to DR_DS8300T*

35、/+shujubaohu:DS8100: IBM.2107-75APMFC044+DR_DS8300T:DS8300: IBM.2107-75DMLFC243+2.輸入lsavailpprcport 命令lsavailpprcport -dev IBM.2107-75APML1 -remotedev IBM.2107-75DMLN1 -remotewwnn 5005076305FFC243 -fullid 78:783、輸出結果Local Port Attached Port Type=IBM.2107-75APML1/I0033 IBM.2

36、107-75DMLN1/I0030 FCPIBM.2107-75APML1/I0233 IBM.2107-75DMLN1/I0130 FCP49建立pprcpath1、用mkpprcpath命令建立pprcpathmkpprcpath -dev IBM.2107-75APML1 -remotedev IBM.2107-75DMLN1 -remotewwnn 5005076305FFC243 -srclss 78 -tgtlss 78 I0033:I0030 I0233:I0130mkpprcpath -dev IBM.2107-75APML1 -remotedev IBM.2107-75DML

37、N1 -remotewwnn 5005076305FFC243 -srclss 79 -tgtlss 79 I0033:I0030 I0233:I0130pprcpath方向是單向的,如果要建立反向的pprcpathf需要在對端存儲上建立pprcpath.2、用lspprcpath命令查詢 50修改LSS屬性1、用chlss命令修改chlss -dev IBM.2107-75APML1 -pprcconsistgrp enable -extlongbusy 30 78chlss -dev IBM.2107-75APML1 -pprcconsistgrp enable -extlongbusy

38、30 792、用showlss 命令查看51建立PPRC關系1、用mkpprc命令建立pprc關系mkpprc -dev IBM.2107-75APML1 -remotedev IBM.2107-75DMLN1 -type mmir 7800-7805:7800-7805 7810-7815:7810-7815 7820-7826:7820-7826 7830-7832:7830-7832 7900-7905:7900-7905 7910-7915:7910-7915 7920-7926:7920-7926 7930-7932:7930-7932-type 參數(shù)是控制pprc的類型,mmir為

39、同步, gcp為異步。-dev 參數(shù)為生產(chǎn)端設備id-remotedev 參數(shù)為容災端設備id2、通過lspprc命令可以查看PPRC的狀態(tài)lspprc -dev IBM.2107-75APML1 -remotedev IBM.2107-75DMLN1 -l 7800-7805:7800-7805 7810-7815:7810-7815 7820-7826:7820-7826 7830-7832:7830-7832 7900-7905:7900-7905 7910-7915:7910-7915 7920-7926:7920-7926 7930-7932:7930-7932剛執(zhí)行完命令時,ppr

40、cr的狀態(tài)為 copy-pending,在后臺進行同步,當所有數(shù)據(jù)同步完成后的狀態(tài)為 full-duplex。52建立PPRC關系53生產(chǎn)端向容災端存儲切換步驟 -數(shù)據(jù)級容災1、停止生產(chǎn)端應用2、停止生產(chǎn)端主機環(huán)境3、failoverpprc ,從生產(chǎn)端切到容災端（在容災端操作）4、建立容災端到生產(chǎn)端pprcpath（在容災端操作）5、failbackpprc,沉著災端同到生產(chǎn)端（在容災端操作）6、容災端存儲對生產(chǎn)端主機授權7、啟動生產(chǎn)端主機環(huán)境8、啟動生產(chǎn)端應用注: 第4、5步為在生產(chǎn)端存可用的情況,如果生產(chǎn)端存儲不可用,此步驟不執(zhí)行。#以上步驟為在zone通道已打通的情況54failove

41、rpprcfailoverpprc在災備端轉換備份卷為主卷建立反向的Metro Mirror關系,此時LUN的對應關系為Suspended狀態(tài),數(shù)據(jù)并未反向同步。1、前提條件,生產(chǎn)端已跟容災端建立PPRC關系2、-dev參數(shù)為容災端的設備ID,-remotedev參數(shù)為生產(chǎn)端設備ID3、LUN對應關系為 容災端LUN對應生產(chǎn)端LUN55failoverpprc生產(chǎn)端LUN關系容災端LUN關系56failoverpprc在災備端執(zhí)行Failover57failbackpprc在災備端執(zhí)行Failbackfailbackpprc將數(shù)據(jù)拷貝回原有的生產(chǎn)卷1、先決條件是在容災端執(zhí)行failoverpp

42、rc命令2、已建立容災端到生產(chǎn)端pprcpath通道3、在容災端執(zhí)行failbackpprc命令4、-dev 參數(shù)為容災端設備ID,-remotedev 為生產(chǎn)端設備ID5、LUN對應關系為容災端對應生產(chǎn)端LUN58容災端到生產(chǎn)端存儲回切步驟 -數(shù)據(jù)級容災1、停止生產(chǎn)端應用2、停止生產(chǎn)端主機環(huán)境3、failoverpprc 沉著災端回切到生產(chǎn)端（在生產(chǎn)端操作）4、建立生產(chǎn)端到容災端pprcpath（在生產(chǎn)端操作）5、failbackpprc 從生產(chǎn)端數(shù)據(jù)同步至容災端（在生產(chǎn)端操作）6、生產(chǎn)端存對生產(chǎn)端主機授權7、啟動生產(chǎn)端主機環(huán)境8、啟動生產(chǎn)端應用注: 此時生產(chǎn)端存儲為可用,pprc的方向為：容災端-生產(chǎn)端。#以上步驟為在zone通道已打通的情況59生產(chǎn)端向容災端存儲切換步驟 -應用級容災1、停止生產(chǎn)端應用2、停止生產(chǎn)端主機環(huán)境3、failov