text:
En savoir plus à propos de l'abonnement Scribd
Découvrez tout ce que Scribd a à offrir, dont les livres et les livres audio des principaux éditeurs.
Harald HoyerNathan StrazMichael ChristieDavid HowellsEric Sandeen
Ro b EversServer Develo pment Kernel Sto firstname.lastname@example.com mOnline Sto rageDavid Ho wellsServer Develo pment Hardware Enablementdho firstname.lastname@example.com mFS-CacheDavid LehmanBase Operating System firstname.lastname@example.com mSto rage co nfiguratio n during installatio nJeff Mo yerServer Develo pment Kernel File Systemjmo email@example.com mSo lid-State DisksEric SandeenServer Develo pment Kernel File firstname.lastname@example.com mext3, ext4 , XFS, Encrypted File SystemsMike SnitzerServer Develo pment Kernel Sto firstname.lastname@example.com mI/O Stack and LimitsRed Hat Subject Matter ExpertsCo ntributo rs
Edited byJacquelynn EastEngineering Co ntent firstname.lastname@example.com mDo n Do mingoEngineering Co ntent Servicesddo mingo @redhat.co m
Legal NoticeCo pyright 20 13 Red Hat Inc. and o thers.This do cument is licensed by Red Hat under the Creative Co mmo ns Attributio n-ShareAlike 3.0Unpo rted License. If yo u distribute this do cument, o r a mo dified versio n o f it, yo u must pro videattributio n to Red Hat, Inc. and pro vide a link to the o riginal. If the do cument is mo dified, all RedHat trademarks must be remo ved.Red Hat, as the licenso r o f this do cument, waives the right to enfo rce, and agrees no t to assert,Sectio n 4 d o f CC-BY-SA to the fullest extent permitted by applicable law.Red Hat, Red Hat Enterprise Linux, the Shado wman lo go , JBo ss, MetaMatrix, Fedo ra, the InfinityLo go , and RHCE are trademarks o f Red Hat, Inc., registered in the United States and o therco untries.Linux is the registered trademark o f Linus To rvalds in the United States and o ther co untries.Java is a registered trademark o f Oracle and/o r its affiliates.XFS is a trademark o f Silico n Graphics Internatio nal Co rp. o r its subsidiaries in the UnitedStates and/o r o ther co untries.MySQL is a registered trademark o f MySQL AB in the United States, the Euro pean Unio n ando ther co untries.No de.js is an o fficial trademark o f Jo yent. Red Hat So ftware Co llectio ns is no t fo rmallyrelated to o r endo rsed by the o fficial Jo yent No de.js o pen so urce o r co mmercial pro ject.The OpenStack Wo rd Mark and OpenStack Lo go are either registered trademarks/servicemarks o r trademarks/service marks o f the OpenStack Fo undatio n, in the United States and o therco untries and are used with the OpenStack Fo undatio n's permissio n. We are no t affiliated with,endo rsed o r spo nso red by the OpenStack Fo undatio n, o r the OpenStack co mmunity.All o ther trademarks are the pro perty o f their respective o wners.
AbstractThis guide pro vides instructio ns o n ho w to effectively manage sto rage devices and file systemso n Red Hat Enterprise Linux 7. It is intended fo r use by system administrato rs with basic tointermediate kno wledge o f Red Hat Enterprise Linux o r Fedo ra.
T able of Contents. .haptC. . . .er. .1. .. O. .verview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7. . . . . . . . . .1.1. What' s New in Red Hat Enterp ris e Linux 77. . . . . . fseCrypt. . not. . . . included. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7. . . . . . . . . .. . . . emSyst. . . St. . orage. . . . . .Manager. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7. . . . . . . . . .. . . . isXFS. . t. he. . .default. . . . . . .File. . . Syst. . . . em. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7. . . . . . . . . .. . . .systFile. . . .em. . .rest. . . ruct. . . .ure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7. . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7. . . . . . . . . .Snapper. . . RFSBT. . . . (T. .echnology. . . . . . . . . .Preview). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7. . . . . . . . . .. . . . . . .noNFSv2. . longer. . . . . . .support. . . . . . .ed. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8. . . . . . . . . .. .artP. . .I.. File. . . .Syst. . . .ems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9. . . . . . . . . .. .haptC. . . .er. .2. .. File. . . .Syst. . . .em. . .St. .ruct. . . ure. . . .and. . . Maint. . . . . enance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 0. . . . . . . . . .2 .1. O verview o f Files ys tem Hierarc hy Stand ard (FHS)2 .2. Sp ec ial Red Hat Enterp ris e Linux File Lo c atio ns
. .haptC. . . .er. .3.. .Bt. . rfs. . .(T. .echnology. . . . . . . . . .Preview). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 9. . . . . . . . . .3 .1. Creating a b trfs File Sys tem3 .2. Mo unting a b trfs file s ys tem3 .3. Res iz ing a b trfs file s ys tem
.........aEnlarging. .bt. .rfs. . .File. . . Syst. . . . em. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2. 0. . . . . . . . . .. . . . . . . . . a. .btShrinking. .rfs. . .File. . . Syst. . . . em. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2. 1. . . . . . . . . ..Set. . .t.he. . File. . . .Syst. . . .em. . .Siz. . .e. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2. 1. . . . . . . . . .3 .4. Integ rated Vo lume Manag ement o f Multip le Devic es213 .5. SSD O p timiz atio n3 .6 . b trfs referenc es
. .haptC. . . .er. .4. .. T. he. . . Ext. . . 3. .File. . . Syst. . . . em. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2. 7. . . . . . . . . .4 .1. Creating an Ext3 File Sys tem284 .2. Co nverting to an Ext3 File Sys tem284 .3. Reverting to an Ext2 File Sys tem29. .haptC. . . .er. .5.. .T. he. . .Ext. . .4. .File. . . Syst. . . . em. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30...........5 .1. Creating an Ext4 File Sys tem5 .2. Mo unting an Ext4 File Sys tem
.Writ. . .e. Barriers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33...........5 .3. Res iz ing an Ext4 File Sys tem335 .4. O ther Ext4 File Sys tem Utilities33. .haptC. . . .er. .6. .. T. he. . . XFS. . . .File. . . Syst. . . . em. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35...........6 .1. Creating an XFS File Sys tem6 .2. Mo unting an XFS File Sys tem
.Writ. . .e. Barriers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37...........6 .3. XFS Q uo ta Manag ement37
Red Hat Ent erprise Linux 7 St orage Administ rat ion G uide6 .3. XFS Q uo ta Manag ement
.Set. . t. ing. . . .Project. . . . . . Limit. . . . .s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39...........6 .4. Inc reas ing the Siz e o f an XFS File Sys tem396 .5. Rep airing an XFS File Sys tem406 .6 . Sus p end ing an XFS File Sys tem6 .7. Bac kup and Res to ratio n o f XFS File Sys tems
. . . . . . .ModeSimple. . . . . for. . . xfsrest. . . . . . ore. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4. 2. . . . . . . . . .. . . . . . . .iveCumulat. . .Mode. . . . . for. . . xfsrest. . . . . . ore. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4. 2. . . . . . . . . ..Int. .eract. . . . ive...O. .perat. . . . .ion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4. 3. . . . . . . . . .6 .8 . O ther XFS File Sys tem Utilities436 .9 . Mig rating fro m ext4 to XFS
. .haptC. . . .er. .7. .. G. .lobal. . . . File. . . .Syst. . . .em. . .2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4. 7. . . . . . . . . .. .haptC. . . .er. .8. .. Net. . . work. . . . .File. . . Syst. . . . em. . . (NFS). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4. 8. . . . . . . . . .8 .1. Ho w NFS Wo rks488 .2. p NFS8 .3. NFS Client Co nfig uratio n
8 .4. auto fs8 .5. Co mmo n NFS Mo unt O p tio ns
. . . .alledInst. . . . .Document. . . . . . . . .at. .ion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7. 0. . . . . . . . . .. . . . . . .WebsitUseful. . . . . .es. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7. 0. . . . . . . . . .. . . . . edRelat. . .Books. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7. 0. . . . . . . . . .. .haptC. . . .er. .9. .. FS. . . Cache. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7. 2. . . . . . . . . .9 .1. Perfo rmanc e G uarantee
. .haptC. . . .er. .1. 0. .. Clust. . . . .er. . File. . . .Syst. . . .ems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7. 8. . . . . . . . . .10 .1. Cep h
. .artP. . .II.. .St. .orage. . . . . Administ. . . . . . . .rat. . ion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8. 1. . . . . . . . . .. .haptC. . . .er. .1. 1. .. St. . orage. . . . . .Considerat. . . . . . . . . ions. . . . .During. . . . . . Inst. . . .allat. . . .ion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8. 2. . . . . . . . . .11.1. Sp ec ial Co ns id eratio ns82. . . . . . . e. .PartSeparat. . . it. .ions. . . . for. . . /home,. . . . . . /opt. . . . ,. /usr/local. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8. 2. . . . . . . . . .. . . . . .andDASD. . . .z.FCP. . . .Devices. . . . . . .on. . .IBM. . . .Syst. . . .em. . .Z. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8. 2. . . . . . . . . .. . . . . . . ingEncrypt. . . .Block. . . . . Devices. . . . . . . Using. . . . . . LUKS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8. 2. . . . . . . . . .. . aleSt. . . BIO. . . .S. RAID. . . . . Met. . . .adat. . . .a. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8. 2. . . . . . . . . .
. .haptC. . . .er. .1. 3.. . Part. . . .it. ions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8. 9. . . . . . . . . .13.1. Viewing the Partitio n Tab le13.2. Creating a Partitio n
. .haptC. . . .er. .1. 4. .. Snapper. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9. 5. . . . . . . . . .14.1. Initial Snap p er Setup
. .haptC. . . .er. .1. 5.. . Swap. . . . . Space. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9. 9. . . . . . . . . .15.1. Ad d ing Swap Sp ac e9915.2. Remo ving Swap Sp ac e
10 1
10 3
. .haptC. . . .er. .1. 6. .. Syst. . . . em. . . St. . orage. . . . . .Manager. . . . . . . .(SSM). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.0. 4. . . . . . . . . .16 .1. SSM Ins tallatio n10 416 .2. SSM Co mmand s
10 4
. .haptC. . . .er. .1. 7. .. Disk....Q. .uot. . . as. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.1. 6. . . . . . . . . .17.1. Co nfig uring Dis k Q uo tas17.2. Manag ing Dis k Q uo tas
. .haptC. . . .er. .1. 8. .. Redundant. . . . . . . . . . .Array. . . . .of. . Independent. . . . . . . . . . . .Disks. . . . . (RAID). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.2. 4. . . . . . . . . .18 .1. RAID Typ es
. . . . . . . . .RAIDFirmware. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.2. 4. . . . . . . . . .. . . . . . . . . RAIDHardware. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.2. 4. . . . . . . . . .. . . . wareSoft. . . . .RAID. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 2. 5. . . . . . . . . .18 .2. RAID Levels and Linear Sup p o rt18 .3. Linux RAID Sub s ys tems
. . . . . .HardwareLinux. . . . . . . . RAID. . . . . cont. . . . roller. . . . . drivers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.2. 7. . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.2. 7. . . . . . . . . .mdraid. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.2. 8. . . . . . . . . .dmraid18 .4. RAID Sup p o rt in the Ins taller
Red Hat Ent erprise Linux 7 St orage Administ rat ion G uide18 .5. Co nfig uring RAID Sets
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.2. 8. . . . . . . . . .mdadm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.2. 8. . . . . . . . . .dmraid18 .6 . Ad vanc ed RAID Devic e Creatio n
. .haptC. . . .er. .1. 9. .. Using. . . . . .t.he. . mount. . . . . . .Command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 31...........19 .1. Lis ting Currently Mo unted File Sys tems
. .haptC. . . .er. .2. 0. .. T. .he. . volume_key. . . . . . . . . . .Funct. . . . . ion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.4. 1. . . . . . . . . .2 0 .1. vo lume_key Co mmand s
. .haptC. . . .er. .2. 1. .. Access. . . . . . .Cont. . . . rol. . . List...s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.4. 6. . . . . . . . . .2 1.1. Mo unting File Sys tems
. .haptC. . . .er. .2. 2. .. Solid. . . . . .St. .at. e. .Disk. . . . Deployment. . . . . . . . . . .G. .uidelines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 51...........2 2.1. Dep lo yment Co ns id eratio ns
. . . SchedulerI/O. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 52............ . . .ualVirt. . .Memory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 52............ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 52Swap............ .haptC. . . .er. .2. 3.. . Writ. . . .e. Barriers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1. 53...........2 3.1. Imp o rtanc e o f Write Barriers
.How. . . .Writ. . . e. .Barriers. . . . . . . Work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1. 53...........2 3.2. Enab ling /Dis ab ling Write Barriers1532 3.3. Write Barrier Co ns id eratio ns
. . . . . . . . . WritDisabling. . . .e. Caches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 54............ . . .t eryBat. . . .Backed. . . . . . .Writ. . . e. .Caches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 54............ . . . . EndHigh. . . . Arrays. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1. 55............ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1. 55NFS............ .haptC. . . .er. .2. 4. .. St. . orage. . . . . .I/O. . .Alignment. . . . . . . . . and. . . . Siz. . .e. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 56...........2 4.1. Parameters fo r Sto rag e Ac c es s
2 4.2. Us ers p ac e Ac c es s
. . . . . .Intsysfs. . erface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 57............ . . . . .DeviceBlock. . . . . . ioct. . . .ls. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 57...........
. . .A. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 58AT............SCSI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 58...........2 4.4. Stac king I/O Parameters1582 4.5. Lo g ic al Vo lume Manag er
. . ilut. . linux. . . . . ng's. . . . .libblkid. . . . . . .and. . . fdisk. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 59............ . . . edpart. . .and. . . .libpart. . . . . ed. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.6. 0. . . . . . . . . .. . . .SystFile. . . .em. . .t.ools. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.6. 0. . . . . . . . . .. .haptC. . . .er. .2. 5.. . Set. . . t.ing. . . Up. . .A. .Remot. . . . . .e. Diskless. . . . . . . .Syst. . . .em. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.6. 1. . . . . . . . . .2 5.1. Co nfig uring a tftp Servic e fo r Dis kles s Clients
16 1
2 5.3. Co nfig uring an Exp o rted File Sys tem fo r Dis kles s Clients
16 2
. .haptC. . . .er. .2. 6. .. .O. nline. . . . . St. . orage. . . . . .Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.6. 4. . . . . . . . . .2 6 .1. Targ et Setup16 42 6 .2. Fib re Channel
2 6 .5. iSCSI2 6 .6 . Pers is tent Naming
18 0
18 1
18 318 4
18 4
18 8
19 1
19 219 5
. . . . . . .IssuesKnown. . . . . . Wit. . .h. .rescan. . . . . . scsi. . . . .bus.sh. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. 9. 5. . . . . . . . . .2 6 .17. Mo d ifying Link Lo s s Behavio r
19 6
. . . . .ErrorSCSI. . . . .Handler. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.9. 7. . . . . . . . . .2 6 .17.2.2. rep lac ement_timeo ut
19 8
2 6 .17.3. iSCSI Ro o t
...........TConfiguring. .imeout. . . . . .s. for...a. .Specific. . . . . . . Session. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.9. 8. . . . . . . . . .2 6 .18 . Co ntro lling the SCSI Co mmand Timer and Devic e Status
19 9
. . . . . . .StDevice. .at. .es. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.9. 9. . . . . . . . . .. . . . . . . . . .T.imerCommand. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.9. 9. . . . . . . . . .2 6 .19 . O nline Sto rag e Co nfig uratio n Tro ub les ho o ting
20 0
. .haptC. . . .er. .2. 7. .. Device. . . . . . .Mapper. . . . . . .Mult. . . .ipat. . . hing. . . . .and. . . Virt. . . .ual. . .St. .orage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.0. 2. . . . . . . . . .2 7.1. Virtual Sto rag e20 22 7.2. DM-Multip ath
20 2
Red Hat Ent erprise Linux 7 St orage Administ rat ion G uide. .haptC. . . .er. .2. 8. .. Ext. . . ernal. . . . . Array. . . . . Management. . . . . . . . . . . .(libSt. . . . .orageMgmt. . . . . . . . . .) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.0. 4. . . . . . . . . .2 8 .1. What is lib Sto rag eMg mt
20 4
20 5
20 620 7
. . . . . . . . .HistRevision. . . ory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2. 1. 3. . . . . . . . . .I.ndex. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.1. 7. . . . . . . . . .
Chapt er 1 . O verview
Chapter 1. OverviewThe Storage Administration Guide contains extensive information on supported file systems and datastorage features in Red Hat Enterprise Linux 7. This book is intended as a quick reference foradministrators managing single-node (that is, non-clustered) storage solutions.The Storage Administration Guide is split into two parts: File Systems, and Storage Administration.The File Systems part details the various file systems Red Hat Enterprise Linux 7 supports. Itdescribes them and explains how best to utilize them.The Storage Administration part details the various tools and storage administration tasks Red HatEnterprise Linux 7 supports. It describes them and explains how best to utilize them.
SnapperRed Hat Enterprise Linux 7 introduces a new tool called snapper that allows for the easy creation andmanagement of snapshots for LVM and BTRFS. For more information refer to Chapter 14, Snapper.
Red Hat Ent erprise Linux 7 St orage Administ rat ion G uide
BTRFS is a local file system that aims to provide better performance and scalability, includingintegrated LVM operations. This file system is not fully supported by Red Hat and as such is atechnology preview. For more information on Btrfs, see Chapter 3, Btrfs (Technology Preview).
NoteWhat directories are available depends on what is installed on any given system. Thefollowing lists are only an example of what may be found.
The d f command reports the system's disk space usage. Its output looks similar to the following:
By default, d f shows the partition size in 1 kilobyte blocks and the amount of used and availabledisk space in kilobytes. To view the information in megabytes and gigabytes, use the command d f h. The -h argument stands for " human-readable" format. The output for d f -h looks similar to thefollowing:
NoteIn the above examples, the mounted partition /d ev/shm represents the system's virtualmemory file system.The d u command displays the estimated amount of space being used by files in a directory,displaying the disk usage of each subdirectory. The last line in the output of d u shows the total diskusage of the directory; to see only the total disk usage of a directory in human-readable format, used u -hs. For more options, refer to man d u.To view the system's partitions and disk space usage in a graphical format, use the Gnome SystemMo ni to r by clicking on Ap p licat io n s Syst em T o o ls Syst em Mo n it o r or using thecommand g no me-system-mo ni to r. Select the Fi l e Systems tab to view the system's partitions.The figure below illustrates the Fi l e Systems tab.
2 .1 .1 .2 . T he /bo o t/ Dire ct o ryThe /bo o t/ directory contains static files required to boot the system, for example, the Linux kernel.These files are essential for the system to boot properly.
WarningD o not remove the /bo o t/ directory. D oing so renders the system unbootable.
D escrip t io n
2 .1 .1 .4 . T he /etc/ Dire ct o ryThe /etc/ directory is reserved for configuration files that are local to the machine. It should containno binaries; any binaries should be moved to /usr/bi n/ or /usr/sbi n/.For example, the /etc/skel / directory stores " skeleton" user files, which are used to populate ahome directory when a user is first created. Applications also store their configuration files in thisdirectory and may reference them when executed. The /etc/expo rts file controls which file systemsexport to remote hosts.
2 .1 .1 .5 . T he /med i a/ Dire ct o ryThe /med i a/ directory contains subdirectories used as mount points for removable media such asUSB storage media, D VD s, CD -ROMs, and Z ip disks.
2 .1 .1 .6 . T he /mnt/ Dire ct o ryThe /mnt/ directory is reserved for temporarily mounted file systems, such as NFS file systemmounts. For all removable storage media, use the /med i a/ directory. Automatically detectedremovable media will be mounted in the /med i a directory.
ImportantThe /mnt directory must not be used by installation programs.
2 .1 .1 .7 . T he /o pt/ Dire ct o ry
The /o pt/ directory is normally reserved for software and add-on packages that are not part of thedefault installation. A package that installs to /o pt/ creates a directory bearing its name, forexample, /o pt/packagename/. In most cases, such packages follow a predictable subdirectorystructure; most store their binaries in /o pt/packagename/bi n/ and their man pages in/o pt/packagename/man/.
2 .1 .1 .8 . T he /pro c/ Dire ct o ryThe /pro c/ directory contains special files that either extract information from the kernel or sendinformation to it. Examples of such information include system memory, CPU information, andhardware configuration. For more information about /pro c/, refer to Section 2.3, The /proc VirtualFile System .
2 .1 .1 .9 . T he /srv/ Dire ct o ryThe /srv/ directory contains site-specific data served by a Red Hat Enterprise Linux system. Thisdirectory gives users the location of data files for a particular service, such as FTP, WWW, or CVS.D ata that only pertains to a specific user should go in the /ho me/ directory.
2 .1 .1 .1 0 . T he /sys/ Dire ct o ryThe /sys/ directory utilizes the new sysfs virtual file system specific to the kernel. With theincreased support for hot plug hardware devices in the kernel, the /sys/ directory containsinformation similar to that held by /pro c/, but displays a hierarchical view of device informationspecific to hot plug devices.
2 .1 .1 .1 1 . T he /usr/ Dire ct o ryThe /usr/ directory is for files that can be shared across multiple machines. The /usr/ directory isoften on its own partition and is mounted read-only. At a minimum, /usr/ should contain thefollowing subdirectories:/usr/bi nThis directory is used for binaries./usr/etcThis directory is used for system-wide configuration files./usr/g amesThis directory stores games./usr/i ncl ud eThis directory is used for C header files./usr/kerbero sThis directory is used for Kerberos-related binaries and files./usr/l i bThis directory is used for object files and libraries that are not designed to be directlyutilized by shell scripts or users.
As of Red Hat Enterprise Linux 7.0, the /l i b/ directory has been merged with /usr/l i b. Itnow also contains libraries needed to execute the binaries in /usr/bi n/ and/usr/sbi n/. These shared library images are used to boot the system or executecommands within the root file system./usr/l i bexecThis directory contains small helper programs called by other programs./usr/sbi nAs of Red Hat Enterprise Linux 7.0, /sbi n has been moved to /usr/sbi n. This means thatit contains all system administration binaries, including those essential for booting,restoring, recovering, or repairing the system. The binaries in /usr/sbi n/ require rootprivileges to use./usr/shareThis directory stores files that are not architecture-specific./usr/srcThis directory stores source code./usr/tmp lin ked t o /var/tmpThis directory stores temporary files.The /usr/ directory should also contain a /l o cal / subdirectory. As per the FHS, this subdirectoryis used by the system administrator when installing software locally, and should be safe from beingoverwritten during system updates. The /usr/l o cal directory has a structure similar to /usr/, andcontains the following subdirectories:/usr/l o cal /bi n/usr/l o cal /etc/usr/l o cal /g ames/usr/l o cal /i ncl ud e/usr/l o cal /l i b/usr/l o cal /l i bexec/usr/l o cal /sbi n/usr/l o cal /share/usr/l o cal /srcRed Hat Enterprise Linux's usage of /usr/l o cal / differs slightly from the FHS. The FHS states that/usr/l o cal / should be used to store softwarethat should remain safe from system softwareupgrades. Since the R PM Packag e Man ag er can perform software upgrades safely, it is notnecessary to protect files by storing them in /usr/l o cal /.Instead, Red Hat Enterprise Linux uses /usr/l o cal / for software local to the machine. Forinstance, if the /usr/ directory is mounted as a read-only NFS share from a remote host, it is stillpossible to install a package or program under the /usr/l o cal / directory.
2 .1 .1 .1 2 . T he /var/ Dire ct o ry
Since the FHS requires Linux to mount /usr/ as read-only, any programs that write log files or needspo o l / or l o ck/ directories should write them to the /var/ directory. The FHS states /var/ is forvariable data, which includes spool directories and files, logging data, transient and temporary files.Below are some of the directories found within the /var/ directory:/var/acco unt//var/arpwatch//var/cache//var/crash//var/d b//var/empty//var/ftp//var/g d m//var/kerbero s//var/l i b//var/l o cal //var/l o ck//var/l o g //var/mai l linked to /var/spo o l /mai l //var/mai l man//var/named //var/ni s//var/o pt//var/preserve//var/run//var/spo o l //var/tmp//var/tux//var/www//var/yp/System log files, such as messag es and l astl o g , go in the /var/l o g / directory. The/var/l i b/rpm/ directory contains RPM system databases. Lock files go in the /var/l o ck/directory, usually in directories for the program using the file. The /var/spo o l / directory hassubdirectories that store data files for some programs. These subdirectories include:/var/spo o l /at/
2.2. Special Red Hat Ent erprise Linux File Locat ionsRed Hat Enterprise Linux extends the FHS structure slightly to accommodate special files.Most files pertaining to RPM are kept in the /var/l i b/rpm/ directory. For more information on RPM,refer to man rpm.The /var/cache/yum/ directory contains files used by the Packag e U p d at er, including RPMheader information for the system. This location may also be used to temporarily store RPMsdownloaded while updating the system. For more information about the Red Hat Network, refer to thedocumentation online at https://rhn.redhat.com/.Another location specific to Red Hat Enterprise Linux is the /etc/sysco nfi g / directory. Thisdirectory stores a variety of configuration information. Many scripts that run at boot time use the filesin this directory.
The /pro c file system is not used for storage. Its main purpose is to provide a file-based interface tohardware, memory, running processes, and other system components. Real-time information can beretrieved on many system components by viewing the corresponding /pro c file. Some of the fileswithin /pro c can also be manipulated (by both users and applications) to configure the kernel.The following /pro c files are relevant in managing and monitoring system storage:/p ro c/d evicesD isplays various character and block devices that are currently configured./p ro c/f ilesyst emsLists all file system types currently supported by the kernel./p ro c/md st atContains current information on multiple-disk or RAID configurations on the system, if theyexist./p ro c/mo u n t sLists all mounts currently used by the system./p ro c/p art it io n sContains partition block allocation information.For more information about the /pro c file system, refer to the Red Hat Enterprise Linux 7 DeploymentGuide.
ImportantBTRFS is a Technology Preview in Red Hat Enterprise Linux 7.
NoteThis does not mean all devices will be added to the file system, it only scans them.max_in lin e= numberUse this option to set the maximum amount of space (in bytes) that can be used to inlinedata within a metadata B-tree leaf. The default is 8192 bytes. For 4k pages it is limited to3900 bytes due to additional headers that need to fit into the leaf.allo c_st art = numberUse this option to set where in the disk allocations start.t h read _p o o l= numberUse this option to assign the number of worker threads allocated.
NoteThe unit size is not case specific; it accepts both G or g for GiB.The command does not accept t for terabytes or p for petabytes. It only accepts k, m, and g .
3.4 .1. File Syst em Creat ion wit h Mult iple DevicesThe mkfs. btrfs command, as detailed in Section 3.1, Creating a btrfs File System , accepts theoptions -d for data, and -m for metadata. Valid specifications are:rai d 0rai d 1rai d 10d upsi ng l eThe -m si ng l e option instructs that no duplication of metadata is done. This may be desired whenusing hardware raid.
NoteRaid10 requires at least four devices to run correctly.
Use the si ng l e option to use the full capacity of each drive when the drives are different sizes.# mkfs.btrfs -d single /dev/device1 /dev/device2 /dev/device3To add a new device to an already created multi-device file system, use the following command:# btrfs device add /dev/device1 /mount-pointAfter rebooting or reloading the btrfs module, use the btrfs d evi ce scan command to discover allmulti-device file systems. See Section 3.4.2, D evice scanning btrfs multiple devices for moreinformation.
The btrfs fi l esystem bal ance command balances (restripes) the allocated extents across allexisting devices.An example of all these commands together to add a new device is as follows:
Balancing a file system will take some time as it reads all of the file system's data and metadata andrewrites it across the new device.
ImportantIf the metadata is not converted from the single-device default, it remains as D UP. This doesnot guarantee that copies of the block are on separate devices. If data is not converted it doesnot have any redundant copies at all.
Use the btrfs d evi ce d el ete command to remove an online device. It redistributes any extents inuse to other devices in the file system in order to be safely removed.
ImportantIt is impossible to go below the minimum number of devices required for the specific raidlayout, even including the missing one. It may be required to add a new device in order toremove the failed one.For example, for a raid1 layout with two devices, if a device fails it is required to:1. mount in degraded mode,2. add a new device,3. and, remove the missing device.
Using UUID 's will also work and be more stable than device paths.
NoteThe ssd mount option only enables the ssd option. Use the no ssd option to disable it.Some SSD s perform best when reusing block numbers often, while others perform much better whenclustering strictly allocates big chunks of unused space. By default, mo unt -o ssd will findgroupings of blocks where there are several free blocks that might have allocated blocks mixed in.The command mo unt -o ssd _spread ensures there are no allocated blocks mixed in. Thisimproves performance on lower end SSD s.
NoteThe ssd _spread option enables both the ssd and the ssd _spread options. Use the no ssdto disable both these options.The ssd _spread option is never automatically set if none of the ssd options are providedand any of the devices are non-rotational.
These options will all need to be tested with your specific build to see if their use improves or reducesperformance, as each combination of SSD firmware and application loads are different.
NoteThe only journaling mode in ext3 supported by Red Hat is d ata= o rd ered (default).D at a In t eg rit yThe ext3 file system prevents loss of data integrity in the event that an unclean systemshutdown occurs. The ext3 file system allows you to choose the type and level of protectionthat your data receives. With regard to the state of the file system, ext3 volumes areconfigured to keep a high level of data consistency by default.Sp eedD espite writing some data more than once, ext3 has a higher throughput in most casesthan ext2 because ext3's journaling optimizes hard drive head motion. You can choosefrom three journaling modes to optimize speed, but doing so means trade-offs in regards todata integrity if the system was to fail.
NoteThe only journaling mode in ext3 supported by Red Hat is d ata= o rd ered (default).
NoteRed Hat Enterprise Linux 7 provides a unified extN driver. It does this by disabling the ext2 andext3 configurations and instead uses ext4 . ko for these on-disk formats. This means thatkernel messages will always refer to ext4 regardless of the ext file system used.The following sections cover creating and tuning ext3 partitions. For ext2 partitions, skip thepartitioning and formatting sections below and go directly to Section 4.2, Converting to an Ext3 FileSystem .
NoteTo convert ext2 to ext3, always use the e2fsck utility to check your file system before and afterusing tune2fs. Before trying to convert ext2 to ext3, back up all file systems in case anyerrors occur.In addition, Red Hat recommends creating a new ext3 file system and migrating data to it,instead of converting from ext2 to ext3 whenever possible.
To convert an ext2 file system to ext3, log in as root and type the following command in a terminal:# tune2fs -j block_deviceblock_device contains the ext2 file system to be converted.Issue the d f command to display mounted file systems.
NoteIf a . jo urnal file exists at the root level of the partition, delete it.To permanently change the partition to ext2, remember to update the /etc/fstab file, otherwise itwill revert back after booting.
NoteAs with ext3, an ext4 volume must be umounted in order to perform an fsck. For moreinformation, see Chapter 4, The Ext3 File System.Main Feat u resExt4 uses extents (as opposed to the traditional block mapping scheme used by ext2 andext3), which improves performance when using large files and reduces metadata overheadfor large files. In addition, ext4 also labels unallocated block groups and inode tablesections accordingly, which allows them to be skipped during a file system check. Thismakes for quicker file system checks, which becomes more beneficial as the file systemgrows in size.Allo cat io n Feat u resThe ext4 file system features the following allocation schemes:Persistent pre-allocationD elayed allocationMulti-block allocationStripe-aware allocationBecause of delayed allocation and other performance optimizations, ext4's behavior ofwriting files to disk is different from ext3. In ext4, when a program writes to the file system, itis not guaranteed to be on-disk unless the program issues an fsync() call afterwards.By default, ext3 automatically forces newly created files to disk almost immediately evenwithout fsync(). This behavior hid bugs in programs that did not use fsync() to ensurethat written data was on-disk. The ext4 file system, on the other hand, often waits severalseconds to write out changes to disk, allowing it to combine and reorder writes for betterdisk performance than ext3.
WarningUnlike ext3, the ext4 file system does not force data to disk on transaction commit. Assuch, it takes longer for buffered writes to be flushed to disk. As with any file system,use data integrity calls such as fsync() to ensure that data is written to permanentstorage.O t h er Ext 4 Feat u res
NoteThe only supported journaling mode in ext4 is d ata= o rd ered (default).Subsecond timestamps This gives timestamps to the subsecond.
For striped block devices (for example, RAID 5 arrays), the stripe geometry can be specified at the timeof file system creation. Using proper stripe geometry greatly enhances the performance of an ext4 filesystem.
When creating file systems on LVM or MD volumes, mkfs. ext4 chooses an optimal geometry. Thismay also be true on some hardware RAID s which export geometry information to the operatingsystem.To specify stripe geometry, use the -E option of mkfs. ext4 (that is, extended file system options)with the following sub-options:st rid e= valueSpecifies the RAID chunk size.st rip e- wid t h = valueSpecifies the number of data disks in a RAID device, or the number of stripe units in thestripe.For both sub-options, value must be specified in file system block units. For example, to create a filesystem with a 64k stride (that is, 16 x 4096) on a 4k-block file system, use the following command:# mkfs. ext4 -E stri d e= 16 ,stri pe-wi d th= 6 4 /d ev/deviceFor more information about creating file systems, refer to man mkfs. ext4 .
ImportantIt is possible to use tune2fs to enable some ext4 features on ext3 file systems, and to use theext4 driver to mount an ext3 file system. These actions, however, are not supported in Red HatEnterprise Linux 7, as they have not been fully tested. Because of this, Red Hat cannotguarantee consistent performance and predictable behavior for ext3 file systems converted ormounted in this way.It is also possible to add a specific UUID to the file system. See Section 4.1, Creating an Ext3 FileSystem for more information.
Writ e Barriers
Write BarriersBy default, ext4 uses write barriers to ensure file system integrity even when power is lost to a devicewith write caches enabled. For devices without write caches, or with battery-backed write caches,disable barriers using the no barri er option, as in:# mo unt -o no barri er /d ev/device /mount/pointFor more information about write barriers, refer to Chapter 23, Write Barriers.
NoteThe size parameter is optional (and often redundant) when expanding. The resi ze2fsautomatically expands to fill all available space of the container, usually a logical volume orpartition.For more information about resizing an ext4 file system, refer to man resi ze2fs.
Changes the label on an ext4 file system. This tool also works on ext2 and ext3 file systems.quotaControls and reports on disk space (blocks) and file (inode) usage by users and groups onan ext4 file system. For more information on using q uo ta, refer to man q uo ta andSection 17.1, Configuring D isk Quotas .f sf reez eTo suspend access to a file system, use the command # fsfreeze -f mount-point tofreeze it and # fsfreeze -u mount-point to unfreeze it. This halts access to the filesystem and creates a stable image on disk.
NoteIt is unnecessary to use fsfreeze for device-mapper drives.For more information see the fsfreeze(8) manpage.As demonstrated in Section 5.2, Mounting an Ext4 File System , the tune2fs utility can also adjustconfigurable file system parameters for ext2, ext3, and ext4 file systems. In addition, the followingtools are also useful in debugging and analyzing ext4 file systems:d eb u g f sD ebugs ext2, ext3, or ext4 file systems.e2imag eSaves critical ext2, ext3, or ext4 file system metadata to a file.For more information about these utilities, refer to their respective man pages.
NoteCreating or expanding files occasionally fails with an unexpected ENOSPC writefailure even though the disk space appears to be sufficient. This is due to XFS'sperformance-oriented design. In practice, it does not become a problem since it onlyoccurs if remaining space is only a few blocks.O t h er XFS Feat u resThe XFS file system also supports the following:Extended attributes ( xattr)This allows the system to associate several additional name/value pairs per file. Itis enabled by default.Quota journalingThis avoids the need for lengthy quota consistency checks after a crash.Project/directory quotasThis allows quota restrictions over a directory tree.Subsecond timestampsThis allows timestamps to go to the subsecond.
agcount=4, agsize=3277258
swidth=0 blksascii-ci=0blocks=6400, version=2sunit=0 blks, lazy-
blocks=0, rtextents=0
NoteAfter an XFS file system is created, its size cannot be reduced. However, it can still be enlargedusing the xfs_g ro wfs command (refer to Section 6.4, Increasing the Size of an XFS FileSystem ).For striped block devices (for example, RAID 5 arrays), the stripe geometry can be specified at the timeof file system creation. Using proper stripe geometry greatly enhances the performance of an XFSfilesystem.When creating filesystems on LVM or MD volumes, mkfs. xfs chooses an optimal geometry. Thismay also be true on some hardware RAID s that export geometry information to the operating system.If the device exports stripe geometry information, mkfs (for ext3, ext4, and xfs) will automatically usethis geometry. If stripe geometry is not detected by mkfs and even though the storage does, in fact,have stripe geometry, it is possible to manually specify it at mkfs time using the following options:su = valueSpecifies a stripe unit or RAID chunk size. The value must be specified in bytes, with anoptional k, m, or g suffix.sw= valueSpecifies the number of data disks in a RAID device, or the number of stripe units in thestripe.The following example specifies a chunk size of 64k on a RAID device containing 4 stripe units:
NoteUnlike mke2fs, mkfs.xfs does not utilize a configuration file; they are all specified on thecommand line.
Write BarriersBy default, XFS uses write barriers to ensure file system integrity even when power is lost to a devicewith write caches enabled. For devices without write caches, or with battery-backed write caches,disable the barriers by using the no barri er option:# mo unt -o no barri er /dev/device /mount/pointFor more information about write barriers, refer to Chapter 23, Write Barriers.
Once quotas are enabled, the xfs_q uo ta tool can be used to set limits and report on disk usage. Bydefault, xfs_q uo ta is run interactively, and in basic mode. Basic mode sub-commands simply reportusage, and are available to all users. Basic xfs_q uo ta sub-commands include:q u o t a username/userIDShow usage and limits for the given username or numeric userIDdfShows free and used counts for blocks and inodes.In contrast, xfs_q uo ta also has an expert mode. The sub-commands of this mode allow actualconfiguration of limits, and are available only to users with elevated privileges. To use expert modesub-commands interactively, run xfs_q uo ta -x. Expert mode sub-commands include:rep o rt /pathReports quota information for a specific file system.limitModify quota limits.For a complete list of sub-commands for either basic or expert mode, use the sub-command hel p.All sub-commands can also be run directly from a command line using the -c option, with -x forexpert sub-commands.
NoteThe commands bso ft and bhard count by the byte.
ImportantWhile real-time blocks (rtbhard /rtbso ft) are described in man xfs_q uo ta as valid unitswhen setting quotas, the real-time sub-volume is not enabled in this release. As such, thertbhard and rtbso ft options are not applicable.
ImportantRed Hat recommends the use of xfs_q uo ta over all other available tools.For more information about setting XFS quotas, refer to man xfs_q uo ta, man pro ji d (5), andman pro jects(5).
NoteWhile XFS file systems can be grown while mounted, their size cannot be reduced at all.For more information about growing a file system, refer to man xfs_g ro wfs.
WarningThe xfs_repai r utility cannot repair an XFS file system with a dirty log. To clear the log,mount and unmount the XFS file system. If the log is corrupt and cannot be replayed, use the L option (" force log zeroing" ) to clear the log, that is, xfs_repai r -L /dev/device. Beaware that this may result in further corruption or data loss.For more information about repairing an XFS file system, refer to man xfs_repai r.
NoteThe xfs_freeze utility is provided by the xfspro g s package, which is only available onx86_64.To suspend (that is, freeze) an XFS file system, use:
# xfs_freeze -f /mount/pointTo unfreeze an XFS file system, use:# xfs_freeze -u /mount/pointWhen taking an LVM snapshot, it is not necessary to use xfs_freeze to suspend the file systemfirst. Rather, the LVM management tools will automatically suspend the XFS file system before takingthe snapshot.For more information about freezing and unfreezing an XFS file system, refer to man xfs_freeze.
6.7. Backup and Rest orat ion of XFS File Syst emsXFS file system backup and restoration involves two utilities: xfsd ump and xfsresto re.To backup or dump an XFS file system, use the xfsd ump utility. Red Hat Enterprise Linux 7 supportsbackups to tape drives or regular file images, and also allows multiple dumps to be written to thesame tape. The xfsd ump utility also allows a dump to span multiple tapes, although only one dumpcan be written to a regular file. In addition, xfsd ump supports incremental backups, and can excludefiles from a backup using size, subtree, or inode flags to filter them.In order to support incremental backups, xfsd ump uses dump levels to determine a base dump towhich a specific dump is relative. The -l option specifies a dump level (0-9). To perform a fullbackup, perform a level 0 dump on the file system (that is, /path/to/filesystem), as in:# xfsd ump -l 0 -f /d ev/device /path/to/filesystem
NoteThe -f option specifies a destination for a backup. For example, the /d ev/st0 destination isnormally used for tape drives. An xfsd ump destination can be a tape drive, regular file, orremote tape device.In contrast, an incremental backup will only dump files that changed since the last level 0 dump. Alevel 1 dump is the first incremental dump after a full dump; the next incremental dump would be level2, and so on, to a maximum of level 9. So, to perform a level 1 dump to a tape drive:# xfsd ump -l 1 -f /d ev/st0 /path/to/filesystemConversely, the xfsresto re utility restores file systems from dumps produced by xfsd ump. Thexfsresto re utility has two modes: a default simple mode, and a cumulative mode. Specific dumps areidentified by session ID or session label. As such, restoring a dump requires its corresponding sessionID or label. To display the session ID and labels of all dumps (both full and incremental), use the -Ioption:# xfsresto re -IThis will provide output similar to the following:
xfsresto re
The simple mode allows users to restore an entire file system from a level 0 dump. After identifying alevel 0 dump's session ID (that is, session-ID), restore it fully to /path/to/destination using:# xfsresto re -f /d ev/st0 -S session-ID /path/to/destination
NoteThe -f option specifies the location of the dump, while the -S or -L option specifies whichspecific dump to restore. The -S option is used to specify a session ID , while the -L option isused for session labels. The -I option displays both session labels and ID s for each dump.
The cumulative mode of xfsresto re allows file system restoration from a specific incrementalbackup, for example, level 1 to level 9. To restore a file system from an incremental backup, simply addthe -r option:# xfsresto re -f /d ev/st0 -S session-ID -r /path/to/destination
Interactive OperationThe xfsresto re utility also allows specific files from a dump to be extracted, added, or deleted. Touse xfsresto re interactively, use the -i option, as in:xfsresto re -f /d ev/st0 -i /destination/directoryThe interactive dialogue will begin after xfsresto re finishes reading the specified device. Availablecommands in this dialogue include cd , l s, ad d , d el ete, and extract; for a complete list ofcommands, use hel p.For more information about dumping and restoring XFS file systems, refer to man xfsd ump and manxfsresto re.
Copies XFS file system metadata to a file. Red Hat only supports using the xfs_metad umputility to copy unmounted file systems or read-only mounted file systems; otherwise,generated dumps could be corrupted or inconsistent.xf s_md rest o reRestores an XFS metadump image (generated using xfs_metad ump) to a file systemimage.xf s_d bD ebugs an XFS file system.For more information about these utilities, refer to their respective man pages.
NoteThe ext4 file system is still fully supported in Red Hat Enterprise Linux 7 and may be selected atinstallation if desired. While it is possible to migrate from ext4 to XFS it is not required.
6.9.1. Ext 3/4 commands and t ools and t heir XFS count erpart sThe following table shows common commands for ext3 and ext4, and their XFS-specific counterparts.T ab le 6 .1. C o mmo n C o mman d s f o r ext 3/4 an d XFST ask
ext 3/4
tune2fsd ump and resto re
mkfs. xfsxfs_repai rxfs_g ro wfsxfs_metad ump andxfs_md resto rexfs_ad mi nxfsd ump and xfsresto re
The following table shows generic tools that function on XFS file systems as well, but the XFSversions have more specific functionality and as such are recommended.T ab le 6 .2. G en eric t o o ls f o r ext 2 an d XFST ask
ext 4
QuotaFile mapping
q uo tafi l efrag
xfs_q uo taxfs_bmap
Every XFS administrative tool has a complete man page and many of these commands are explainedin further detail in Chapter 6, The XFS File System.
6.9.2. Behavioral and Administ rat ive Differences Bet ween Ext 3/4 and XFSFile syst em rep airExt3/4 runs e2fsck in userspace at boot time to recover the journal as needed. XFS, bycomparison, performs journal recovery in kernelspace at mount time. An fsck. xfs shellscript is provided but does not perform any useful action as it is only there to satisfyinitscript requirements.When an XFS file system repair or check is requested, use the the xfs_repai r command.Use the -n option for a read-only check.The xfs_repai r command will not operate on a file system with a dirty log. To repair sucha file system mo unt and unmo unt must first be performed to replay the log. If the log iscorrupt and cannot be replayed, the -L option can be used to zero out of the log.For more information on file system repair of XFS file systems, refer to Section 12.2.2, XFSMet ad at a erro r b eh avio rThe ext3/4 file system has configurable behavior when metadata errors are encountered,with the default being to simply continue. When XFS encounters a metadata error that is notrecoverable it will shut down the file system and return a EFSC O R R UP T ED error. The systemlogs will contain details of the error encounted and will recommend running xfs_repai r ifnecessary.Q u o t asXFS quotas are not a remountable option. The -o q uo ta option must be specified on theinitial mont for quotas to be in effect.While the standard tools in the quota package can perform basic quota administrativetasks (tools such as setquota and repquota), the xfs_q uo ta tool can be used for XFSspecific features, such as Project Quota administration.The q uo tacheck command has no effect on an XFS file system. The first time quotaaccounting is turned on XFS does an automatic q uo tacheck internally. Because XFSquota metadata is a first-class, journaled metadata object, the quota system will always beconsistent until quotas are manually turned off.File syst em resiz eThe XFS file system has no utility to shrink a file system. XFS file systems can be grownonline via the xfs_g ro wfs command.In o d e n u mb ersFor file systems above 1T with 256 byte inodes, or above 2T with 512 byte inodes, XFSinode numbers may exceed 2^32. These large inode numbers will cause 32-bit stat calls tofail with EO VER FLO W. In general, applications should properly handle these larger inodenumbers but, if necessary XFS may be mounted with -o i no d e32 to enforce inode nmbersbelow 2^32.
NoteThis will not affect inodes already allocated with 64-bit numbers.This option changes allocation behavior and may lead to early ENOSPC if no space isavailable to allocate inodes in the lower disk blocks. The inode32 option should not beused unless required by a specific environment.Sp ecu lat ive p reallo cat io nXFS uses speculative preallocation to allocate blocks past EOF as files are written. Thisavoids file fragmentation due to concurrent streaming write workloads on NFS servers. Bydefault, this preallocation increases with the size of the file and will be apparent in " du"output. If a file wit speculative preallocation is not dirtied for five minutes the preallocationwill be discarded. If the inode is cycled out of cache before that time, then the preallocationwill be discarded when the inode is reclaimed.If premature ENOSPC problems are seen due to speculative preallocation, a fixedpreallocation amount may be specified with the -o al l o csi ze= amount mount option.Frag men t at io n - relat ed t o o lsFragmentation is rarel a significant issue on XFS file systems due to heuristics andbehaviors, such as delayed allocation and speculative preallocation. However, tools existfor measuring file system fragmentation as well as defragmenting file systems. Their use isnot encouraged.The xfs_d b frag command attempts to distill all file system allocations into a singlefragmentation number, expressed as a percentage. The output of the comand requiressignificant expertise to understand it's meaning. For example, a fragmentation factor of75% means only an average of 4 extents per file. For this reason the output of xfs_db's fragis not considered useful and more careful analysis of any fragmentation problems isrecommended.
WarningThe xfs_fsr command may be used to defragment individual files, or all files on afile system. The later is especially no recommended as it may destroy locality of filesand may fragment freespace.
NoteNFS version 2 (NFSv2) is no longer supported by Red Hat.All versions of NFS can use Transmission Control Protocol (TCP) running over an IP network, withNFSv4 requiring it. NFSv3 can use the User Datagram Protocol (UD P) running over an IP network toprovide a stateless network connection between the client and server.When using NFSv3 with UD P, the stateless UD P connection (under normal conditions) has lessprotocol overhead than TCP. This can translate into better performance on very clean, noncongested networks. However, because UD P is stateless, if the server goes down unexpectedly, UD Pclients continue to saturate the network with requests for the server. In addition, when a frame is lostwith UD P, the entire RPC request must be retransmitted; with TCP, only the lost frame needs to beresent. For these reasons, TCP is the preferred protocol when connecting to an NFS server.The mounting and locking protocols have been incorporated into the NFSv4 protocol. The serveralso listens on the well-known TCP port 2049. As such, NFSv4 does not need to interact withrpcbi nd [1] , l o ckd , and rpc. statd daemons. The rpc. mo untd daemon is still required on theNFS server to set up the exports, but is not involved in any over-the-wire operations.
NoteTCP is the default transport protocol for NFS version 2 and 3 under Red Hat Enterprise Linux.UD P can be used for compatibility purposes as needed, but is not recommended for wideusage. NFSv4 requires TCP.All the RPC/NFS daemons have a ' -p' command line option that can set the port, makingfirewall configuration easier.
After TCP wrappers grant access to the client, the NFS server refers to the /etc/expo rtsconfiguration file to determine whether the client is allowed to access any exported file systems. Onceverified, all file and directory operations are available to the user.
ImportantIn order for NFS to work with a default installation of Red Hat Enterprise Linux with a firewallenabled, configure IPTables with the default TCP port 2049. Without proper IPTablesconfiguration, NFS will not function properly.The NFS initialization script and rpc. nfsd process now allow binding to any specified portduring system start up. However, this can be error-prone if the port is unavailable, or if itconflicts with another daemon.
NoteThe po rtmap service was used to map RPC program numbers to IP address port numbercombinations in earlier versions of Red Hat Enterprise Linux. This service is now replaced byrpcbi nd in Red Hat Enterprise Linux 7 to enable IPv6 support. For more information aboutthis change, refer to the following links:TI-RPC / rpcbind support: http://nfsv4.bullopensource.org/doc/tirpc_rpcbind.phpIPv6 support in NFS: http://nfsv4.bullopensource.org/doc/nfs_ipv6.php
nfsservi ce nfs start starts the NFS server and the appropriate RPC processes to servicerequests for shared NFS file systems.n f slo ckservi ce nfsl o ck start activates a mandatory service that starts the appropriate RPC
servi ce nfsl o ck start activates a mandatory service that starts the appropriate RPCprocesses allowing NFS clients to lock files on the server.rp cb in drpcbi nd accepts port reservations from local RPC services. These ports are then madeavailable (or advertised) so the corresponding remote RPC services can access them.rpcbi nd responds to requests for RPC services and sets up connections to the requestedRPC service. This is not used with NFSv4.The following RPC processes facilitate NFS services:rp c.mo u n t dThis process is used by an NFS server to process MO UNT requests from NFSv3 clients. Itchecks that the requested NFS share is currently exported by the NFS server, and that theclient is allowed to access it. If the mount request is allowed, the rpc.mountd server replieswith a Success status and provides the Fi l e-Hand l e for this NFS share back to the NFSclient.rp c.n f sdrpc. nfsd allows explicit NFS versions and protocols the server advertises to be defined. Itworks with the Linux kernel to meet the dynamic demands of NFS clients, such as providingserver threads each time an NFS client connects. This process corresponds to the nfsservice.lo ckdl o ckd is a kernel thread which runs on both clients and servers. It implements the NetworkLock Manager (NLM) protocol, which allows NFSv3 clients to lock files on the server. It isstarted automatically whenever the NFS server is run and whenever an NFS file system ismounted.rp c.st at dThis process implements the Network Status Monitor (NSM) RPC protocol, which notifies NFSclients when an NFS server is restarted without being gracefully brought down. rpc. statdis started automatically by the nfsl o ck service, and does not require user configuration.This is not used with NFSv4.rp c.rq u o t adThis process provides user quota information for remote users. rpc. rq uo tad is startedautomatically by the nfs service and does not require user configuration.rp c.id map drpc. i d mapd provides NFSv4 client and server upcalls, which map between on-the-wireNFSv4 names (which are strings in the form of user@ domain) and local UID s and GID s.For i d mapd to function with NFSv4, the /etc/i d mapd . co nf file must be configured. Thisservice is required for use with NFSv4, although not when all hosts share the same D NSdomain name. Refer to the knowledge base articlehttps://access.redhat.com/site/solutions/130783 when using a local domain.
8.2. pNFSSupport for Parallel NFS (pNFS) as part of the NFS v4.1 standard is available as of Red HatEnterprise Linux 6.4. The pNFS architecture improves the scalability of NFS, with possible
improvements to performance. That is, when a server implements pNFS as well, a client is able toaccess data through multiple servers concurrently. It supports three storage protocols or layouts:files, objects, and blocks.
NoteThe protocol allows for three possible pNFS layout types: files, objects, and blocks. While theRed Hat Enterprise Linux 6.4 client only supported the files layout type, Red HatEnterprise Linux 7 supports the files layout type, with objects and blocks layout types beingincluded as a technology preview.To enable this functionality, use the following mount option on mounts from a pNFS-enabled server:-o v4 . 1After the server is pNFS-enabled, the nfs_l ayo ut_nfsv4 1_fi l es kernel is automatically loadedon the first mount. The mount entry in the output should contain mi no rversi o n= 1. Use thefollowing command to verify the module was loaded:$ l smo d | g rep nfs_l ayo ut_nfsv4 1_fi l esFor more information on pNFS, refer to: http://www.pnfs.com.
fail. The file system type nfs4 is also available for legacy reasons; this is equivalent to running mo unt-t nfs -o nfsvers= 4 host: /remote/export /local/directory.Refer to man mo unt for more details.If an NFS share was mounted manually, the share will not be automatically mounted upon reboot.Red Hat Enterprise Linux offers two methods for mounting remote file systems automatically at boottime: the /etc/fstab file and the auto fs service. Refer to Section 8.3.1, Mounting NFS FileSystems using /etc/fstab and Section 8.4, auto fs for more information.
defaults 0 0
The mount point /pub must exist on the client machine before this command can be executed. Afteradding this line to /etc/fstab on the client system, use the command mo unt /pub, and the mountpoint /pub is mounted from the server.The /etc/fstab file is referenced by the netfs service at boot time, so lines referencing NFS shareshave the same effect as manually typing the mo unt command during the boot process.A valid /etc/fstab entry to mount an NFS export should contain the following information:server:/remote/export /local/directory nfs options 0 0The variables server, /remote/export, /local/directory, and options are the same ones used whenmanually mounting an NFS share. Refer to Section 8.3, NFS Client Configuration for a definition ofeach variable.
NoteThe mount point /local/directory must exist on the client before /etc/fstab is read. Otherwise,the mount will fail.For more information about /etc/fstab, refer to man fstab.
8.4 . auto fsOne drawback to using /etc/fstab is that, regardless of how infrequently a user accesses the NFSmounted file system, the system must dedicate resources to keep the mounted file system in place.This is not a problem with one or two mounts, but when the system is maintaining mounts to manysystems at one time, overall system performance can be affected. An alternative to /etc/fstab is to
use the kernel-based auto mo unt utility. An automounter consists of two components:a kernel module that implements a file system, anda user-space daemon that performs all of the other functions.The auto mo unt utility can mount and unmount NFS file systems automatically (on-demandmounting), therefore saving system resources. It can be used to mount other file systems includingAFS, SMBFS, CIFS, and local file systems.
ImportantThe nfs-utils package is now a part of both the 'NFS file server' and the 'Network File SystemClient' groups. As such, it is no longer installed by default with the Base group. Ensure thatnfs-utils is installed on the system first before attempting to automount an NFS share.autofs is also part of the 'Network File System Client' group.
auto fs uses /etc/auto . master (master map) as its default primary configuration file. This can bechanged to use another supported network source and name using the auto fs configuration (in/etc/sysco nfi g /auto fs) in conjunction with the Name Service Switch (NSS) mechanism. Aninstance of the auto fs version 4 daemon was run for each mount point configured in the mastermap and so it could be run manually from the command line for any given mount point. This is notpossible with auto fs version 5, because it uses a single daemon to manage all configured mountpoints; as such, all automounts must be configured in the master map. This is in line with the usualrequirements of other industry standard automounters. Mount point, hostname, exported directory,and options can all be specified in a set of files (or other supported network sources) rather thanconfiguring them manually for each host.
implementations. A new configuration file has been added for this support:/etc/auto fs_l d ap_auth. co nf. The default configuration file is self-documenting, anduses an XML format.Pro p er u se o f t h e N ame Service Swit ch ( nsswi tch) co n f ig u rat io n .The Name Service Switch configuration file exists to provide a means of determining fromwhere specific configuration data comes. The reason for this configuration is to allowadministrators the flexibility of using the back-end database of choice, while maintaining auniform software interface to access the data. While the version 4 automounter is becomingincreasingly better at handling the NSS configuration, it is still not complete. Autofs version5, on the other hand, is a complete implementation.Refer to man nsswi tch. co nf for more information on the supported syntax of this file. Notall NSS databases are valid map sources and the parser will reject ones that are invalid.Valid sources are files, yp, ni s, ni spl us, l d ap, and hesi o d .Mu lt ip le mast er map en t ries p er au t o f s mo u n t p o in tOne thing that is frequently used but not yet mentioned is the handling of multiple mastermap entries for the direct mount point /-. The map keys for each entry are merged andbehave as one map.
optionsIf supplied, these will apply to all entries in the given map provided they don't themselveshave options specified. This behavior is different from auto fs version 4 where options werecumulative. This has been changed to implement mixed environment compatibility.
files nis
Because the automounter only processes the first occurrence of a mount point, /ho me will containthe contents of /etc/auto . ho me instead of the NIS auto . ho me map.Alternatively, to augment the site-wide auto . ho me map with just a few entries, create an/etc/auto . ho me file map, and in it put the new entries. At the end, include the NIS auto . ho memap. Then the /etc/auto . ho me file map will look similar to:
mydir someserver:/export/mydir+auto.homeGiven the NIS auto . ho me map listed above, l s /ho me would now output:beth joe mydirThis last example works as expected because auto fs does not include the contents of a file map ofthe same name as the one it is reading. As such, auto fs moves on to the next map source in thensswi tch configuration.
Ensure that these are the only schema entries not commented in the configuration. Theauto mo untKey replaces the cn attribute in the rfc230 7bi s schema. An LD IF of a sampleconfiguration is described below:
extended LDIFLDAPv3base <> with scope subtreefilter: (& (objectclass=automountMap)(automountMapName=auto.master))requesting: ALL
# auto.master, example.comdn: automountMapName=auto.master,dc=example,dc=comobjectClass: topobjectClass: automountMapautomountMapName: auto.master
# extended LDIF## LDAPv3# base <automountMapName=auto.master,dc=example,dc=com> with scopesubtree# filter: (objectclass=automount)# requesting: ALL## /home, auto.master, example.comdn: automountMapName=auto.master,dc=example,dc=comobjectClass: automountcn: /homeautomountKey: /homeautomountInformation: auto.home#######
extended LDIFLDAPv3base <> with scope subtreefilter: (& (objectclass=automountMap)(automountMapName=auto.home))requesting: ALL
# auto.home, example.comdn: automountMapName=auto.home,dc=example,dc=comobjectClass: automountMapautomountMapName: auto.home# extended LDIF## LDAPv3# base <automountMapName=auto.home,dc=example,dc=com> with scopesubtree# filter: (objectclass=automount)# requesting: ALL## foo, auto.home, example.comdn: automountKey=foo,automountMapName=auto.home,dc=example,dc=comobjectClass: automountautomountKey: fooautomountInformation: filer.example.com:/export/foo# /, auto.home, example.comdn: automountKey=/,automountMapName=auto.home,dc=example,dc=comobjectClass: automountautomountKey: /automountInformation: filer.example.com:/export/&
Beyond mounting a file system with NFS on a remote host, it is also possible to specify other optionsat mount time to make the mounted share easier to use. These options can be used with manualmo unt commands, /etc/fstab settings, and auto fs.The following are options commonly used for NFS mounts:in t rAllows NFS requests to be interrupted if the server goes down or cannot be reached.lo o ku p cach e= modeSpecifies how the kernel should manage its cache of directory entries for a given mountpoint. Valid arguments for mode are al l , no ne, or po s/po si ti ve.n f svers= versionSpecifies which version of the NFS protocol to use, where version is 2, 3, or 4. This is usefulfor hosts that run multiple NFS servers. If no version is specified, NFS uses the highestversion supported by the kernel and mo unt command.The option vers is identical to nfsvers, and is included in this release for compatibilityreasons.n o aclTurns off all ACL processing. This may be needed when interfacing with older versions ofRed Hat Enterprise Linux, Red Hat Linux, or Solaris, since the most recent ACL technologyis not compatible with older systems.n o lo ckD isables file locking. This setting is occasionally required when connecting to older NFSservers.n o execPrevents execution of binaries on mounted file systems. This is useful if the system ismounting a non-Linux file system containing incompatible binaries.n o su idD isables set-user-i d enti fi er or set-g ro up-i d enti fi er bits. This prevents remoteusers from gaining higher privileges by running a setui d program.p o rt = numpo rt= num Specifies the numeric value of the NFS server port. If num is 0 (the default),then mo unt queries the remote host's rpcbi nd service for the port number to use. If theremote host's NFS daemon is not registered with its rpcbi nd service, the standard NFSport number of TCP 2049 is used instead.rsiz e= num an d wsiz e= numThese settings speed up NFS communication for reads (rsi ze) and writes (wsi ze) bysetting a larger data block size (num, in bytes), to be transferred at one time. Be carefulwhen changing these values; some older Linux kernels and network cards do not work wellwith larger block sizes. For NFSv3, the default values for both parameters is set to 8192.For NFSv4, the default values for both parameters is set to 32768.sec= mode
Its default setting is sec= sys, which uses local UNIX UID s and GID s. These use AUT H_SY Sto authenticate NFS operations."sec= krb5 uses Kerberos V5 instead of local UNIX UID s and GID s to authenticate users.sec= krb5i uses Kerberos V5 for user authentication and performs integrity checking ofNFS operations using secure checksums to prevent data tampering.sec= krb5p uses Kerberos V5 for user authentication, integrity checking, and encrypts NFStraffic to prevent traffic sniffing. This is the most secure setting, but it also involves the mostperformance overhead.t cpInstructs the NFS mount to use the TCP protocol.udpInstructs the NFS mount to use the UD P protocol.For a complete list of options and more detailed information on each one, refer to man mo unt andman nfs.
Here, uid and gid are user ID number and group ID number, respectively. The ano nui d andano ng i d options allow you to create a special user and group account for remote NFS users toshare.By default, access control lists (ACLs) are supported by NFS under Red Hat Enterprise Linux. Todisable this feature, specify the no _acl option when exporting the file system.Each default for every exported file system must be explicitly overridden. For example, if the rw optionis not specified, then the exported file system is shared as read-only. The following is a sample linefrom /etc/expo rts which overrides two default options:/ano ther/expo rted /d i recto ry 19 2. 16 8. 0 . 3(rw,async)In this example 19 2. 16 8. 0 . 3 can mount /ano ther/expo rted /d i recto ry/ read/write and allwrites to disk are asynchronous. For more information on exporting options, refer to man expo rtfs.Other options are available where no default value is specified. These include the ability to disablesub-tree checking, allow access from insecure ports, and allow insecure file locks (necessary forcertain early NFS client implementations). Refer to man expo rts for details on these less-usedoptions.
ImportantThe format of the /etc/expo rts file is very precise, particularly in regards to use of the spacecharacter. Remember to always separate exported file systems from hosts and hosts from oneanother with a space character. However, there should be no other space characters in the fileexcept on comment lines.For example, the following two lines do not mean the same thing:/home bob.example.com(rw)/home bob.example.com (rw)The first line allows only users from bo b. exampl e. co m read/write access to the /ho medirectory. The second line allows users from bo b. exampl e. co m to mount the directory asread-only (the default), while the rest of the world can mount it read/write.
Causes all directories listed in /etc/expo rts to be exported by constructing a new exportlist in /etc/l i b/nfs/xtab. This option effectively refreshes the export list with anychanges made to /etc/expo rts.-aCauses all directories to be exported or unexported, depending on what other options arepassed to /usr/sbi n/expo rtfs. If no other options are specified,/usr/sbi n/expo rtfs exports all file systems specified in /etc/expo rts.- o file-systemsSpecifies directories to be exported that are not listed in /etc/expo rts. Replace filesystems with additional file systems to be exported. These file systems must be formatted inthe same way they are specified in /etc/expo rts. This option is often used to test anexported file system before adding it permanently to the list of file systems to be exported.Refer to Section 8.7.1, The /etc/expo rts Configuration File for more information on/etc/expo rts syntax.-iIgnores /etc/expo rts; only options given from the command line are used to defineexported file systems.-uUnexports all shared directories. The command /usr/sbi n/expo rtfs -ua suspendsNFS file sharing while keeping all NFS daemons up. To re-enable NFS sharing, useexpo rtfs -r.-vVerbose operation, where the file systems being exported or unexported are displayed ingreater detail when the expo rtfs command is executed.If no options are passed to the expo rtfs command, it displays a list of currently exported filesystems. For more information about the expo rtfs command, refer to man expo rtfs.
This will add " -p port to be added to rpc.mound's command line (that is, rpc. mo unt -pport).Next, add the following parameters to the /etc/sysctl . co nf file:f s.n f s.n lm_t cp p o rt = portThis is the equivalent of the LOCKD _TCPPORT parameter in Red Hat Enterprise Linux 6.f s.n f s.n lm_u d p p o rt = portThis is the equivalent of the LOCKD _UD PPORT parameter in Red Hat Enterprise Linux 6.If NFS fails to start, check /var/l o g /messag es. Normally, NFS will fail to start if you specify a portnumber that is already in use. After editing /etc/sysco nfi g /nfs and /etc/sysctl . co nf, restartthe NFS service using systemctl restart nfs. Run the rpci nfo -p command to confirm thechanges.
NoteTo allow NFSv4.0 callbacks to pass through firewalls set/pro c/sys/fs/nfs/nfs_cal l back_tcppo rt and allow the server to connect to that porton the client.This process is not needed for NFSv4.1 or higher, and the other ports for mo untd , statd , andl o ckd are not required in a pure NFSv4 environment.
NoteBefore Red Hat Enterprise Linux 6 on older NFS servers, depending on how they areconfigured, it is possible to export filesystems to NFSv4 clients at different paths. Becausethese servers do not enable NFSv4 by default this should not normally be a problem.
NFS is well-suited for sharing entire file systems with a large number of known hosts in a transparentmanner. However, with ease-of-use comes a variety of potential security problems. Consider thefollowing sections when exporting NFS file systems on a server or mounting them on a client. D oingso minimizes NFS security risks and better protects data on the server.
8.8.1. NFS Securit y wit h AUT H_SYS and export cont rolsTraditionally, NFS has given two options in order to control access to exported files.First, the server restricts which hosts are allowed to mount which filesystems either by IP address orby host name.Second, the server enforces file system permissions for users on NFS clients in the same way it doeslocal users. Traditionally it does this using AUT H_SY S (also called AUT H_UNIX) which relies on theclient to state the UID and GID 's of the user. Be aware that this means a malicious or misconfiguredclient can easily get this wrong and allow a user access to files that it should not.To limit the potential risks, administrators often allow read-only access or squash user permissionsto a common user and group ID . Unfortunately, these solutions prevent the NFS share from beingused in the way it was originally intended.Additionally, if an attacker gains control of the D NS server used by the system exporting the NFS filesystem, the system associated with a particular hostname or fully qualified domain name can bepointed to an unauthorized machine. At this point, the unauthorized machine is the system permittedto mount the NFS share, since no username or password information is exchanged to provideadditional security for the NFS mount.Wildcards should be used sparingly when exporting directories through NFS, as it is possible for thescope of the wildcard to encompass more systems than intended.It is also possible to restrict access to the rpcbi nd [1] service with TCP wrappers. Creating rules withi ptabl es can also limit access to ports used by rpcbi nd , rpc. mo untd , and rpc. nfsd .For more information on securing NFS and rpcbi nd , refer to man i ptabl es.
NoteIt is assumed that a Kerberos ticket-granting server (KD C) is installed and configured correctly,prior to configuring an NFSv4 server. Kerberos is a network authentication system whichallows clients and servers to authenticate to each other through use of symmetric encryptionand a trusted third party, the KD C. For more information on Kerberos see Red Hat's IdentityManagement Guide.
The rpcbi nd [1] utility maps RPC services to the ports on which they listen. RPC processes notifyrpcbi nd when they start, registering the ports they are listening on and the RPC program numbersthey expect to serve. The client system then contacts rpcbi nd on the server with a particular RPCprogram number. The rpcbi nd service redirects the client to the proper port number so it cancommunicate with the requested service.Because RPC-based services rely on rpcbi nd to make all connections with incoming clientrequests, rpcbi nd must be available before any of these services start.The rpcbi nd service uses TCP wrappers for access control, and access control rules for rpcbi ndaffect all RPC-based services. Alternatively, it is possible to specify access control rules for each ofthe NFS RPC daemons. The man pages for rpc. mo untd and rpc. statd contain informationregarding the precise syntax for these rules.
service32774 nlockmgr32774 nlockmgr32774 nlockmgr34437 nlockmgr34437 nlockmgr34437 nlockmgr819 rquotad819 rquotad822 rquotad822 rquotad2049 nfs2049 nfs2049 nfs2049 nfs836 mountd839 mountd836 mountd839 mountd836 mountd839 mountd
If one of the NFS services does not start up correctly, rpcbi nd will be unable to map RPC requestsfrom clients for that service to the correct port. In many cases, if NFS is not present in rpci nfooutput, restarting NFS causes the service to correctly register with rpcbi nd and begin working.For more information and a list of options on rpci nfo , refer to its man page.
8.10. ReferencesAdministering an NFS server can be a challenge. Many options, including quite a few not mentionedin this chapter, are available for exporting or mounting NFS shares. Consult the following sources formore information.
Installed Documentationman mo unt Contains a comprehensive look at mount options for both NFS server and clientconfigurations.man fstab Gives details for the format of the /etc/fstab file used to mount file systems atboot-time.man nfs Provides details on NFS-specific file system export and mount options.man expo rts Shows common options used in the /etc/expo rts file when exporting NFS filesystems.
Useful Websiteshttp://linux-nfs.org The current site for developers where project status updates can be viewed.http://nfs.sourceforge.net/ The old home for developers which still contains a lot of usefulinformation.http://www.citi.umich.edu/projects/nfsv4/linux/ An NFSv4 for Linux 2.6 kernel resource.http://www.vanemery.com/Linux/NFSv4/NFSv4-no-rpcsec.html D escribes the details of NFSv4with Fedora Core 2, which includes the 2.6 kernel.http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.111.4086 An excellent whitepaper onthe features and enhancements of the NFS Version 4 protocol.
Related BooksManaging NFS and NIS by Hal Stern, Mike Eisler, and Ricardo Labiaga; O'Reilly & Associates Makes an excellent reference guide for the many different NFS export and mount optionsavailable.NFS Illustrated by Brent Callaghan; Addison-Wesley Publishing Company Providescomparisons of NFS to other network file systems and shows, in detail, how NFS communicationoccurs.
[1] The rpcbi nd s ervic e rep lac es po rtmap , whic h was us ed in p revio us vers io ns o f Red HatEnterp ris e Linux to map RPC p ro g ram numb ers to IP ad d res s p o rt numb er c o mb inatio ns . Fo r mo reinfo rmatio n, refer to Sec tio n 8 .1.1, Req uired Servic es .
Chapter 9. FS-CacheFS-Cache is a persistent local cache that can be used by file systems to take data retrieved from overthe network and cache it on local disk. This helps minimize network traffic for users accessing datafrom a file system mounted over the network (for example, NFS).The following diagram is a high-level illustration of how FS-Cache works:
To provide caching services, FS-Cache needs a cache back-end. A cache back-end is a storagedriver configured to provide caching services (i.e. cachefi l es). In this case, FS-Cache requires amounted block-based file system that supports bmap and extended attributes (e.g. ext3) as its cacheback-end.FS-Cache cannot arbitrarily cache any file system, whether through the network or otherwise: theshared file system's driver must be altered to allow interaction with FS-Cache, data storage/retrieval,and metadata setup and validation. FS-Cache needs indexing keys and coherency data from thecached file system to support persistence: indexing keys to match file system objects to cacheobjects, and coherency data to determine whether the cache objects are still valid.
File systems that support functionalities required by FS-Cache cache back-end include the Red HatEnterprise Linux 7 implementations of the following file systems:ext3 (with extended attributes enabled)ext4BTRFSXFSThe host file system must support user-defined extended attributes; FS-Cache uses these attributes tostore coherency maintenance information. To enable user-defined extended attributes for ext3 filesystems (i.e. device), use:# tune2fs -o user_xattr /d ev/deviceAlternatively, extended attributes for a file system can be enabled at mount time, as in:# mo unt /d ev/device /path/to/cache -o user_xattrThe cache back-end works by maintaining a certain amount of free space on the partition hosting thecache. It grows and shrinks the cache in response to other elements of the system using up freespace, making it safe to use on the root file system (for example, on a laptop). FS-Cache sets defaultson this behavior, which can be configured via cache cull limits. For more information aboutconfiguring cache cull limits, refer to Section 9.4, Setting Cache Cull Limits .Once the configuration file is in place, start up the cachefi l esd daemon:# servi ce cachefi l esd startTo configure cachefi l esd to start at boot time, execute the following command as root:# chkco nfi g cachefi l esd o n
Opening a file from a shared file system for direct I/O will automatically bypass the cache. This isbecause this type of access must be direct to the server.Opening a file from a shared file system for writing will not work on NFS version 2 and 3. Theprotocols of these versions do not provide sufficient coherency management information for the clientto detect a concurrent write to the same file from another client.As such, opening a file from a shared file system for either direct I/O or writing will flush the cachedcopy of the file. FS-Cache will not cache the file again until it is no longer opened for direct I/O orwriting.Furthermore, this release of FS-Cache only caches regular NFS files. FS-Cache will not cachedirectories, symlinks, device files, FIFOs and sockets.
ImportantCulling depends on both bxxx and fxxx pairs simultaneously; they can not be treatedseparately.
9.6. ReferencesFor more information on cachefi l esd and how to configure it, refer to man cachefi l esd andman cachefi l esd . co nf. The following kernel documents also provide additional information:/usr/share/d o c/cachefi l esd -version-number/R EAD ME/usr/share/man/man5/cachefi l esd . co nf. 5. g z/usr/share/man/man8/cachefi l esd . 8. g zFor general information about FS-Cache, including details on its design constraints, availablestatistics, and capabilities, refer to the following kernel document:/usr/share/d o c/kernel d o c-version/D o cumentati o n/fi l esystems/cachi ng /fscache. txt
10.1. CephCeph is a storage platform that presents object, block and file systems from a single distributedcomputer cluster and scales to the exabyte level.A Ceph Storage Cluster requires at least one Ceph Monitor and at least two Ceph OSD (On-ScreenD isplay) D aemons. Also, when running a Ceph File System, the Ceph Metadata Server is essential.C ep h O SDThe Ceph OSD D aemon:stores datahandles data replicationhandles data recoveryhandles data backfillinghandles data rebalancingprovides monitoring information to Ceph Monitors by checking other Ceph OSDD aemons for a heartbeat.The reason behind requiring at least two Ceph OSD D aemons to be active is because thiscreates (at least) two copies of your data. This creates a clean state, aiding in reliability.Mo n it o rsA Ceph Monitor maintains maps of the cluster state. These include the:monitor mapthe OSD mapthe Placement Group (PG) mapCRUSH mapAny changes made in these maps are recorded in an epoch (or a history).C ep h Met ad at a Server ( MD S)An MD S stores metadata on behalf of the Ceph File System. These metadata servers allowPOSIX file system users to execute basic commands (such as l s, fi nd ) without placing anenormous burden on the Ceph Storage Cluster.
NoteThe Ceph Block D evices and Ceph Object Storage do not use the MD S.
1 0 .1 .1 .2 . Mo re Info o n Ce ph St o rageFor more information on Ceph Storage, please refer to the Ceph documentation, located athttp://docs.ceph.com/docs/master/rados/.
ImportantFor older kernels (< 2.6.33) it is required to disable the write cache if the journal is on a rawdrive. To do this, use the following command:$ sud o hd parm -W 0 /d ev/hd a 0This is not required for kernels newer than 2.6.33.
ImportantThe ceph fs new command only works after Ceph 0.84. No manual steps are required tocreate a file system prior to this release, and pools named d ata and metad ata exist bydefault.Once the file system has been created, the MD S is able to enter an active state. This can be testedwith the following command example:$ ceph mds state5: 1/1/1 up {0=a=up:active}
WarningRemoving/deleting RAID metadata from disk could potentially destroy any stored data. RedHat recommends that you back up your data before proceeding.To delete RAID metadata from the disk, use the following command:d mrai d -r -E /device/For more information about managing RAID devices, refer to man d mrai d and Chapter 18,Redundant Array of Independent Disks (RAID).
DASDDirect-access storage devices (D ASD ) cannot be added/configured during installation. Such devicesare specified in the CMS configuration file.
NoteThese filesystem checkers only guarantee metadata consistency across the filesystem; theyhave no awareness of the actual data contained within the filesystem and are not datarecovery tools.Filesystem inconsistencies can occur for various reasons, including but not limited to hardwareerrors, storage administration errors, and software bugs.Before modern metadata-journaling filesystems became common, a filesystem check was requiredany time a system crashed or lost power. This was because a filesystem update could have beeninterrupted, leading to an inconsistent state. As a result, a filesystem check is traditionally run oneach filesystem listed in /etc/fstab at boot-time. For journaling filesystems, this is usually a veryshort operation, because the filesystem's metadata journaling ensures consistence even after acrash.However, there are times when a filesystem inconsistency or corruption may occur, even forjournaling filesystems. When this happens, the filesystem checker must be used to repair thefilesystem. The following will provide best practices and other useful information when performingthis procedure.
ImportantRed Hat does not recommended this unless the machine does not boot, the file system isextremely large, or the file system is on remote storage. It is possible to disable file systemcheck at boot by setting the sixth field in /etc/fstab to 0.
NoteLater phases of consistency checking may print extra errors as it discoversinconsistencies which would have been fixed in early phases if it were running inrepair mode.O p erat e f irst o n a f ilesyst em imag eMost filesystems support the creation of a metadata image, a sparse copy of the filesystemwhich contains only metadata. Because filesystem checkers operate only on metadata,such an image can be used to perform a dry run of an actual filesystem repair, to evaluatewhat changes would actually be made. If the changes are acceptable, the repair can thenbe performed on the filesystem itself.
NoteSeverely damaged filesystems may cause problems with metadata image creation.Save a f ilesyst em imag e f o r su p p o rt in vest ig at io n sA pre-repair filesystem metadata image can often be useful for support investigations ifthere is a possibility that the corruption was due to a software bug. Patterns of corruptionpresent in the pre-repair image may aid in root-cause analysis.O p erat e o n ly o n u n mo u n t ed f ilesyst emsA filesystem repair must be run only on unmounted filesystems. The tool must have soleaccess to the filesystem or further damage may result. Most filesystem tools enforce thisrequirement in repair mode, although some only support check-only mode on a mountedfilesystem. If check-only mode is run on a mounted filesystem, it may find spurious errorsthat would not be found when run on an unmounted filesystem.D isk erro rsFilesystem check tools cannot repair hardware problems. A filesystem must be fullyreadable and writable if repair is to operate successfully. If a filesystem was corrupted dueto a hardware error, the filesystem must first be moved to a good disk, for example with thed d (8) utility.
crash.If these filesystems encounter metadata inconsistencies while mounted, they will record this fact in thefilesystem super block. If e2fsck finds that a filesystem is marked with such an error e2fsck willperform a full check after replaying the journal (if present).e2fsck may ask for user input during the run if the -p option is not specified. The -p option tellse2fsck to automatically do all repairs that may be done safely. If user intervention is required,e2fsck will indicate the unfixed problem in its output and reflect this status in the exit code.Commonly used e2fsck run-time options include:-nNo-modify mode. Check-only operation.-b su p erb lo ckSpecify block number of an alternate suprerblock if the primary one is damaged.-fForce full check even if the superblock has no recorded errors.-j jo u rn al- d evSpecify the external journal device, if any.-pAutomatically repair or " preen" the filesystem with no user input.-yAssume an answer of " yes" to all questions.All options for e2fsck are specified in the e2fsck(8) manual page.The following five basic phases are performed by e2fsck while running:1. Inode, block, and size checks.2. D irectory structure checks.3. D irectory connectivity checks.4. Reference count checks.5. Group summary info checks.The e2i mag e(8) utility can be used to create a metadata image prior to repair for diagnostic ortesting purposes. The -r option should be used for testing purposes in order to create a sparse fileof the same size as the filesystem itself. e2fsck can then operate directly on the resulting file. The -Qoption should be specified if the image is to be archived or provided for diagnostic. This creates amore compact file format suitable for transfer.
12.2.2. XFSNo repair is performed automatically at boot time. To initiate a filesystem check or repair, thexfs_repai r tool is used.
NoteAlthough an fsck. xfs binary is present in the xfsprogs package, this is present only tosatisfy initscripts that look for an fsck. filesystem binary at boot time. fsck. xfsimmediately exits with an exit code of 0.Another thing to be aware of is that older xfsprogs packages contain an xfs_check tool. Thistool is very slow and does not scale well for large filesystems. As such, it has been depreciatedin favor of xfs_repai r -n.
A clean log on a filesystem is required for xfs_repai r to operate. If the filesystem was not cleanlyunmounted, it should be mounted and unmounted prior to using xfs_repai r. If the log is corruptand cannot be replayed, the -L option may be used to zero the log.
ImportantThe -L option must only be used if the log cannot be replayed. The option discards allmetadata updates in the log and will result in further inconsistencies.It is possible to run xfs_repai r in a dry run, check-only mode by using the -n option. No changeswill be made to the filesystem when this option is specified.xfs_repai r takes very few options. Commonly used options include:-nNo modify mode. Check-only operation.-LZ ero metadata log. Use only if log cannot be replayed with mount.-m maxmemLimit memory used during run to maxmem MB. 0 can be specified to obtain a rough estimateof the minimum memory required.-l lo g d evSpecify the external log device, if present.All options for xfs_repai r are specified in the xfs_repai r(8) manual page.The following eight basic phases are performed by xfs_repai r while running:1. Inode and inode blockmap (addressing) checks.2. Inode allocation map checks.3. Inode size checks.4. D irectory checks.5. Pathname checks.
12.2.3. Bt rfsThe btrfsck tool is used to check and repair btrfs filesystems. This tool is still in early developmentand may not detect or repair all types of filesystem corruption.By default, btrfsck does not make changes to the filesystem; that is, it runs check-only mode bydefault. If repairs are desired the --repai r option must be specified.The following three basic phases are performed by btrfsck while running:1. Extent checks.2. Filesystem root checks.3. Root reference count checks.The btrfs-i mag e(8) utility can be used to create a metadata image prior to repair for diagnostic ortesting purposes.
check minor-numcp from to
hel pmkl abel labelmkfs minor-num file-system-typemkpart part-type fs-type start-mbend-mbmkpartfs part-type fs-type start-mbend-mbmo ve minor-num start-mb end-mbname minor-num namepri ntq ui trescue start-mb end-mbresi ze minor-num start-mb end-mb
C o mman d
rm minor-numsel ect deviceset minor-num flag stateto g g l e [NUMBER [FLAG]uni t UNIT
The first line contains the disk type, manufacturer, model number and interface, and the secondline displays the disk label type. The remaining output below the fourth line shows the partitiontable.In the partition table, the Minor number is the partition number. For example, the partition with minornumber 1 corresponds to /d ev/sd a1. The Start and End values are in megabytes. Valid T ype aremetadata, free, primary, extended, or logical. The Fi l esystem is the file system type, which can beany of the following:ext2ext3fat16fat32hfsjfslinux-swapntfsreiserfs
hp-ufssun-ufsxfsIf a Fi l esystem of a device shows no value, this means that its file system type is unknown.The Fl ag s column lists the flags set for the partition. Available flags are boot, root, swap, hidden,raid, lvm, or lba.
NoteTo select a different device without having to restart parted , use the sel ect commandfollowed by the device name (for example, /d ev/sd a). D oing so allows you to view orconfigure the partition table of a device.
NoteIf you use the mkpartfs command instead, the file system is created after the partition iscreated. However, parted does not support creating an ext3 file system. Thus, if you wish tocreate an ext3 file system, use mkpart and create the file system with the mkfs command asdescribed later.The changes start taking place as soon as you press Enter, so review the command beforeexecuting to it.After creating the partition, use the pri nt command to confirm that it is in the partition table with thecorrect partition type, file system type, and size. Also remember the minor number of the new partitionso that you can label any file systems on it. You should also view the output of cat/pro c/parti ti o ns after parted is closed to make sure the kernel recognizes the new partition.The maximum number of partitions parted will create is 128. While the GUID Partition Table (GPT)specification allows for more partitions by growing the area reserved for the partition table, commonpractice used by parted is to limit it to enough area for 128 partitions.
WarningFormatting the partition permanently destroys any data that currently exists on thepartition.2. Next, give the file system on the partition a label. For example, if the file system on the newpartition is /d ev/sd a6 and you want to label it /wo rk, use:# e2l abel /dev/sda6 /workBy default, the installation program uses the mount point of the partition as the label to make sure thelabel is unique. You can use any label you want.Afterwards, create a mount point (e.g. /wo rk) as root.
The first column should contain UUID = followed by the file system's UUID . The second columnshould contain the mount point for the new partition, and the next column should be the file systemtype (for example, ext3 or swap). If you need more information about the format, read the man pagewith the command man fstab.If the fourth column is the word d efaul ts, the partition is mounted at boot time. To mount thepartition without rebooting, as root, type the command:mo unt /wo rk
WarningD o not attempt to resize a partition on a device that is in use.
WarningA partition cannot be made larger than the space available on the device5. After resizing the partition, use the pri nt command to confirm that the partition has beenresized correctly, is the correct partition type, and is the correct file system type.6. After rebooting the system into normal mode, use the command d f to make sure the partitionwas mounted and is recognized with the new size.
Chapt er 1 4 . Snapper
Creating a snapshot works the same for both LVM and BTRFS. If the --type or -t option is ommitedthen a single type of snapper snapshot is created.
+File created.File deleted.cContent changed.tType of directory entry has changed (for example, a former symbolic link haschanged to a regular file with the same file name).C o lu mn 2This indicates any changes in the file's permissions. The possible values are:.No permissions changed.pPermissions changedC o lu mn 3This indicates any changes in the user ownership. The possible values are:.No user ownership changed.uUser ownership has changedC o lu mn 4This indicates any changes in the group ownership. The possible values are:.No group ownership changed.gGroup ownership has changed.C o lu mn 5This indicates any changes in the extended attributes. The possible values are:.No extended attributes changed.x
NoteBe aware that btrfs does not support swap space.Swap should equal 2x physical RAM for up to 2 GB of physical RAM, and then an additional 1xphysical RAM for any amount above 2 GB, but never less than 32 MB.
ImportantFile systems and LVM2 volumes assigned as swap space should not be in use when beingmodified. Any attempts to modify swap will fail if a system process or the kernel is using swapspace. Use the free and cat /pro c/swaps commands to verify how much and where swapis in use.Red Hat advises that you modify swap space while the system is booted in rescue mode; forinstructions on how to boot in rescue mode, refer to the Installation Guide. When prompted tomount the file system, select Ski p.
Sometimes it is necessary to add more swap space after installation. For example, you may upgradethe amount of RAM in your system from 1 GB to 2 GB, but there is only 2 GB of swap space. It mightbe advantageous to increase the amount of swap space to 4 GB if you perform memory-intenseoperations or run applications that require a large amount of memory.You have three options: create a new swap partition, create a new swap file, or extend swap on anexisting LVM2 logical volume. It is recommended that you extend an existing logical volume.
1. At a shell prompt, execute the following command to disable the swap file (where /swapfi l eis the swap file):# swapo ff -v /swapfi l e2. Remove its entry from the /etc/fstab file.3. Remove the actual file:# rm /swapfi l e
Performs a dry run. This parses the command line options and gathers system informationif necessary without actually running anything. This is mainly used for debuggingpurposes but it does not perform all checks as some are handled by the backendsthemselves.
16.2.1. ListThe ssm l i st lists information about all detected devices, pools, volumes and snapshots found onthe system. By using just ssm l i st it will show all available information, or specific sections can berequested using the following options.vo l umes o r vo lThis option lists information about all volumes found on the system.d evi ces o r d evThis option lists information about all devices found on the system.
NoteSome devices are intentionally hidden. This includes cdrom, or D M/MD devices asthey are listed as volumes.po o l s o r po o lThis option lists information about all pools found on the system.fi l esystems o r fsThis option lists information about all volumes that contain file systems on the system.snapsho ts o r snapThis option lists information about all snapshots on the system.
NoteSome backends do not support snapshots and some cannot distinguish between asnapshot and a regular volume. In this case SSM will attempt to recognize thevolume name in order to identify a snapshot. If the SSM regular expression does notmatch the snapshot pattern, then the snapshot will not be recognized.-h o r --hel pThis option shows the above information and then exits.
DeviceTotalMount point------------------------------------/dev/loop05.00 GB/dev/loop15.00 GB/dev/loop25.00 GB/dev/loop35.00 GB/dev/loop45.00 GB/dev/sda149.05 GBPARTITIONED/dev/sda119.53 GB//dev/sda278.12 GB/dev/sda31.95 GBSWAP/dev/sda41.00 KB/dev/sda549.44 GB/mnt/test-----------------------------------------------------------------------------------------------------------------VolumePoolVolume size FSFS sizeFreeTypeMount point----------------------------------------------------------------------------/dev/dm-0 dm-crypt78.12 GBext4 78.12 GB45.01 GB crypt/home/dev/sda119.53 GBext4 19.53 GB12.67 GB part//dev/sda549.44 GBext4 49.44 GB29.77 GB part/mnt/test-----------------------------------------------------------------------------
16.2.2. Creat eThe ssm create command creates a new volume with defined parameters. It can be made morespecific by specifying the following options:-s size o r --si ze sizeThis option specifies the size to allocate for the new logical volume. The following suffixesare accepted to define power of two units:K or kM or mG or gT or tP or pE or eThe default is kilobytes and SSM defaults to this if no size suffix is provided.If this option is not provided the maximum possible size will be used.-n name o r --name name
This option gives a name to the new logical volume. If this option is not provided it will usea name generated by the corresponding backend. For more information on backends, seeSection 16.3, SSM Backends .--fstype filesystem_typeThis option specifies what filesystem type the new logical volume is to be. Supported filesystems are:ext3ext4xfsbtrfsIf this option is not provided a filesystem will not be created.-r level o r --rai d levelThis option specifies a RAID level to use when creating a new volume. The supported levelsto choose from are:0110
NoteSome backends may not implement all supported RAID levels.If this option is not provided a linear volume will be created.-I stripesize o r --stri pesi ze stripesizeThis option gives the number of kilobytes for the granularity of stripes.
NoteIt is required to specify RAID level in order to use this option.If this option is not provided the default for the backend will be used. For more informationon backends, see Section 16.3, SSM Backends .-i stripes o r --stri pes stripesThis option gives the number of stripes to a new volume. This is equal to the number ofphysical volumes to scatter the logical volume.
NoteIt is required to specify RAID level in order to use this option.If this option is not provided but stripesize is set and stripes are provided to multipledevices, the number of stripes is determined automatically from the number of devices.-p pool o r --po o l poolThis option specifies the pool to use to create the new volume. If the pool exists, the newvolume will be created from that pool (optionally adding a d evi ce into the pool). If the pooldoes not exist, SSM will attempt to create a new pool with the provided d evi ce and thencreate a new volume from this pool.If the -backend option is omitted, the default SSM backend, LVM, will be used.-e o r --encryptThis option creates an encrypted volume. The extension to use can be specified. Acceptedextensions are:luksplaind evi ce deviceIf the device is specified it will be used to create the new volume, adding it to the pool priorto volume creation (refer to Section 16.2.7, Add for more information).If the specified device is already used in a different pool, SSM will ask whether to remove itfrom the original pool. If this is declined, or the removal fails, and if the size was notprovided, then the volume creation will fail. If size was provided and some devices can notbe added to the pool, then the volume creation will succeed provided there is enough spacein the pool.mo unt mount_pointIf the mount point is provided, SSM will attempt to mount the volume after creation. It will failif the mountable file system is not present on the volume.-h o r --hel pThis option shows the help message, then exits.
16.2.3. Remove
The ssm remo ve command removes an item from the system. Multiple items can be specified. If anitem can not be removed for any reason, it will be skipped.An item can be any of the following:d eviceThis removes a device from the pool. This can not be done in cases where the device iscurrently being used by the pool. The -f argument can force the removal. If the device doesnot belong to any pool, it will be skipped.poolThis removes a pool from the system including all volumes created from that pool.vo lu meThis removes a volume from the system. If the volume is mounted it will fail. It can not beforced with the -f argument.The following options can be appended to the command:-a o r --al lThis option removes all pools in the system.-h o r --hel pThis option displays the help message and then exits.
16.2.4 . Resiz eThe ssm resi ze command changes the size of the specified volume and file system. If there is nofile system then only the volume itself will be resized.
NoteThe device will only be added into the pool if the volume size is to grow.If the device is already used in a different pool, SSM will ask whether to remove it from the originalpool.In some cases the file system has to be mounted in order to resize. This is handled by SSMautomatically by mounting the volume temporarily.The following are a list of options.-s size o r --si ze size
This option sets the new size of the volume. The plus (+) or minus (-) sign with the newvalue adds or subtracts from the actual size of the volume. Without a plus or minus thevalue will be the new volume size.Adding a suffix specifies the value's size. Accepted suffixes are:K or k for kilobytes.M or m for megabytes.G or g for gigabytes.T or t for terabytes.P or p for pentabytes.If no unit is provided then the default size is kilobytes.-h o r --hel pThis option displays the help message and then exits.
16.2.5. CheckThe ssm check command checks the file system consistency on the volume. It is possible to specifymultiple volumes to check. If there is no file system on the volume, then the volume will be skipped.In some cases the file system must be mounted in order to check the file system. This will be handledby SSM automatically by temporarily mounting the volume.The following option is available for ssm check:-h o r --hel pThis option shows the help message and then exits.
16.2.6. SnapshotThe ssm snapsho t command takes a snapshot of an existing volume.
NoteThis operation will fail if the back end the volume belongs to does not support snapshotting.In some cases the file system must be mounted in order to take a snapshot of the volume. This ishandled by SSM automatically by mounting the volume temporarily.
NoteBoth name and destination options can not be specified at the same time as they aremutually exclusive.The following options are available for the ssm snapsho t command:-s size o r --si ze sizeThis gives the size to allocate the new shapshot volume. Accepted suffixes are:K or k for kilobytesM or m for megabytesG or g for gigabytesT or t for terabytesP or p for petabytesE or e for exabytesA size suffix can be used to define 'power of two' units. If the unit size is not specified thedefault is kilobytes. If this option is not specified the size is automatically determined.-d destination o r --d est destinationThis option specifies the destination of the snapshot, specified with the absolute path to beused for the new snapshop. If this option is not specified the default backend policy will beperformed.-n name o r --name nameThis option specifies the name of the new snapshot. If this is not specified the defaultbackend policy will be performed.-h o r --hel pThis option displays the help message and then exits.
16.2.7. AddThe ssm ad d command adds the device into the pool. When multiple devices are provided, all areadded into the pol.
NoteIf any devices are already a part of a different pool, then the command will fail. The user isasked whether to remove the device from its pool or not.If no pool is specified then the default pol will be chosen. If the specified pool does not exist, it willcreated using provided devices.The following options are available for the ssm ad d command:-p pool o r --po l poolThis option specifies the pool the device is to be added into. If this is not specified thedefault pool is used.-h o r --hel pThis option displays the help message and then exits.
16.2.8. MountThe ssm mo unt command will mount the volume at the specified directory. The volume can bespecified in the same way as is detailed in man mo unt(8). In addition it is also possible to specifythe volume in the format as it appears in the table printed with the command ssm l i st. For example,instead of finding out what the device and subvolume ID is of the btrfs subvolumebtrfs_po o l : vo l 0 0 1 in order to mount it, it is possible to call ssm mo untbtrfs_po o l : vo l 0 0 1 /mnt/test.The following options are available for the ssm mo unt:-o options o r --o pti o ns optionsOptions are specified with the -o flag, followed by a comma separated string of options.This option is the equivalent to the same option specified in man mo unt(8)-h o r --hel pThis option displays the help message and then exits.
any BTRFS volume in the system with SSM. However, BTRFS does not distinguish betweensubvolumes and snapshots because a snapshot is actually just a subvolume with someblocks shared with different subvolumes. This means that SSM is not able to recognizeBTRFS snapshots destination, but instead tries to recognize special name formats of theBTRFS volume. If the name specified when creating the snapshot does not match thespecific pattern, the snapshot will not be recognized and instead be listed as a regularBTRFS volume.New BTRFS snapshots can be created with the snapsho t command.D eviceBTRFS does not require any special device to be created on.
16.3.3. CryptCrypt backend in SSM uses cryptsetup and dm-crypt target to manage encrypted volumes. Cryptbackend can be used as a regular backend for creating encrypted volumes on top of regular blockdevices (or on other volumes such as LVM or MD volumes), or to create encrypted LVM volumes in asingle steps.Only volumes can be created with crypt backend; pooling is not supported and it does not requirespecial devices.The following are definitions from the crypt backend point of view.Po o lCrypt backend does not support pooling; it is not possible to create a crypt pool, nor to add
Crypt backend does not support pooling; it is not possible to create a crypt pool, nor to adda device into a pool.Vo lu meVolume in crypt backends is the volume created by d m-crypt which represents the data onthe original encrypted device in an unencrypted form. Only one device can be used tocreate a crypt volume. It does not support RAID or any device concatenation.Two modes or extensions are supported: luks and plain. Luks is used by default. For moreinformation on the extensions, refer to man cryptsetup.Sn ap sh o tCrypt backend does not support snapshotting. However, if the encrypted volume is createdon top of an LVM volume, the volume itself can be snapshotted. The snapshot can then beopened by using cryptsetup.D eviceCrypt backend does not require any special device to be created on.
NoteThe documentation is build using sphinx software to avoid duplication of documentation.Therefore if you want to modify the documentation do not modify the man page, HTMLpages, or the R EAD ME directory. Instead, modify the d o c/*. rst and d o c/src/*. rst filesaccordingly so the change is propagated to all documents.The SSM wiki can be accessed at http://storagemanager.sourceforge.net/index.html.The mailing list can be subscribed to fromhttps://lists.sourceforge.net/lists/listinfo/storagemanager-devel and mailing list archives can beaccessed at http://sourceforge.net/mailarchive/forum.php?forum_name=storagemanager-devel.The mailing list is where developers communicate. There is no user mailing list so feel free to postquestions there as well.
NoteThis chapter is for all file systems, however some file systems have their own quotamanagement tools. Refer to the corresponding description for the applicable file systems.For XFS file systems, refer to Section 6.3, XFS Quota Management .Btrfs does not have disk quotas so is not covered.
Add the usrq uo ta and/or g rpq uo ta options to the file systems that require quotas:
Examp le 17.2. Ad d q u o t as/dev/VolGroup00/LogVol00LABEL=/bootnonenonenonenone/dev/VolGroup00/LogVol021 2/dev/VolGroup00/LogVol01
ext3defaults1 1ext3defaults1 2devpts gid=5,mode=620 0 0tmpfsdefaults0 0procdefaults0 0sysfsdefaults0 0ext3defaults,usrquota,grpquota
0 0 . . .
In this example, the /ho me file system has both user and group quotas enabled.
NoteThe following examples assume that a separate /ho me partition was created during theinstallation of Red Hat Enterprise Linux. The root (/) partition can be used for setting quotapolicies in the /etc/fstab file.
NoteThe q uo tacheck command has no effect on XFS as the table of disk usage is completedautomatically at mount time. Refer to the man page xfs_q uo ta(8) for more information.
To create the quota files (aq uo ta. user and aq uo ta. g ro up) on the file system, use the -c optionof the q uo tacheck command.
The -c option specifies that the quota files should be created for each file system with quotasenabled, the -u option specifies to check for user quotas, and the -g option specifies to check forgroup quotas.If neither the -u or -g options are specified, only the user quota file is created. If only -g is specified,only the group quota file is created.After the files are created, run the following command to generate the table of current disk usage perfile system with quotas enabled:# q uo tacheck -avugThe options used are as follows:aCheck all quota-enabled, locally-mounted file systemsvD isplay verbose status information as the quota check proceedsuCheck user disk quota informationgCheck group disk quota informationAfter q uo tacheck has finished running, the quota files corresponding to the enabled quotas (userand/or group) are populated with data for each quota-enabled locally-mounted file system such as/ho me.
NoteThe text editor defined by the ED IT O R environment variable is used by ed q uo ta. To changethe editor, set the ED IT O R environment variable in your ~ /. bash_pro fi l e file to the fullpath of the editor of your choice.The first column is the name of the file system that has a quota enabled for it. The second columnshows how many blocks the user is currently using. The next two columns are used to set soft andhard block limits for the user on the file system. The i no d es column shows how many inodes theuser is currently using. The last two columns are used to set the soft and hard inode limits for theuser on the file system.The hard block limit is the absolute maximum amount of disk space that a user or group can use.Once this limit is reached, no further disk space can be used.The soft block limit defines the maximum amount of disk space that can be used. However, unlike thehard limit, the soft limit can be exceeded for a certain amount of time. That time is known as the graceperiod. The grace period can be expressed in seconds, minutes, hours, days, weeks, or months.If any of the values are set to 0, that limit is not set. In the text editor, change the desired limits.
To verify that the quota for the user has been set, use the command:# quota usernameDisk quotas for user username (uid 501):Filesystem blocksquotalimitgracegrace/dev/sdb1000*10001000
# ed q uo ta -g d evelThis command displays the existing quota for the group in the text editor:Disk quotas for group devel (gid 505):Filesystemblockssofthard/dev/VolGroup00/LogVol02 44040000
ImportantWhile other ed q uo ta commands operate on quotas for a particular user or group, the -toption operates on every file system with quotas enabled.
To enable quotas again, use the q uo tao n command with the same options.For example, to enable user and group quotas for all file systems, use the following command:# q uo tao n -vaugTo enable quotas for a specific file system, such as /ho me, use the following command:# q uo tao n -vug /ho meIf neither the -u or -g options are specified, only the user quotas are enabled. If only -g is specified,only group quotas are enabled.
NoteThe q uo tao n command is not always needed for XFS because it is performed automaticallyat mount time. Refer to the man page q uo tao n(8) for more information.
To view the disk usage report for all (option -a) quota-enabled file systems, use the command:# repq uo ta -aWhile the report is easy to read, a few points should be explained. The -- displayed after each useris a quick way to determine whether the block or inode limits have been exceeded. If either soft limit isexceeded, a + appears in place of the corresponding -; the first - represents the block limit, and thesecond represents the inode limit.The g race columns are normally blank. If a soft limit has been exceeded, the column contains a timespecification equal to the amount of time remaining on the grace period. If the grace period hasexpired, no ne appears in its place.
When a file system fails to unmount cleanly (due to a system crash, for example), it is necessary torun q uo tacheck. However, q uo tacheck can be run on a regular basis, even if the system has notcrashed. Safe methods for periodically running q uo tacheck include:En su rin g q u o t ach eck ru n s o n n ext reb o o t
WarningRunning q uo tacheck on a live file system mounted read-write is not recommendeddue to the possibility of quota file corruption.Refer to man cro n for more information about configuring cro n.
ed q uo tarepq uo taq uo taq uo tao nq uo tao ff
Firmware RAIDFirmware RAID (also known as ATARAID ) is a type of software RAID where the RAID sets can beconfigured using a firmware-based menu. The firmware used by this type of RAID also hooks into theBIOS, allowing you to boot from its RAID sets. D ifferent vendors use different on-disk metadataformats to mark the RAID set members. The Intel Matrix RAID is a good example of a firmware RAIDsystem.
Hardware RAIDThe hardware-based array manages the RAID subsystem independently from the host. It presents asingle disk per RAID array to the host.A Hardware RAID device may be internal or external to the system, with internal devices commonlyconsisting of a specialized controller card that handles the RAID tasks transparently to the operatingsystem and with external devices commonly connecting to the system via SCSI, fiber channel, iSCSI,InfiniBand, or other high speed network interconnect and presenting logical volumes to the system.RAID controller cards function like a SCSI controller to the operating system, and handle all theactual drive communications. The user plugs the drives into the RAID controller (just like a normalSCSI controller) and then adds them to the RAID controllers configuration. The operating system willnot be able to tell the difference.
Software RAIDSoftware RAID implements the various RAID levels in the kernel disk (block device) code. It offers thecheapest possible solution, as expensive disk controller cards or hot-swap chassis [2] are notrequired. Software RAID also works with cheaper ID E disks as well as SCSI disks. With today's fasterCPUs, Software RAID also generally outperforms Hardware RAID .The Linux kernel contains a multi-disk (MD ) driver that allows the RAID solution to be completelyhardware independent. The performance of a software-based array depends on the server CPUperformance and load.Here are some of the key features of the Linux software RAID stack:Multi-threaded designPortability of arrays between Linux machines without reconstructionBackgrounded array reconstruction using idle system resourcesHot-swappable drive supportAutomatic CPU detection to take advantage of certain CPU features such as streaming SIMDsupportAutomatic correction of bad sectors on disks in an arrayRegular consistency checks of RAID data to ensure the health of the arrayProactive monitoring of arrays with email alerts sent to a designated email address on importanteventsWrite-intent bitmaps which drastically increase the speed of resync events by allowing the kernelto know precisely which portions of a disk need to be resynced instead of having to resync theentire arrayResync checkpointing so that if you reboot your computer during a resync, at startup the resyncwill pick up where it left off and not start all over againThe ability to change parameters of the array after installation. For example, you can grow a 4disk RAID 5 array to a 5-disk RAID 5 array when you have a new disk to add. This grow operationis done live and does not require you to reinstall on the new array.
capacity of the smallest member disk in a Hardware RAID or the capacity of smallestmember partition in a Software RAID multiplied by the number of disks or partitions in thearray.Level 1RAID level 1, or " mirroring," has been used longer than any other form of RAID . Level 1provides redundancy by writing identical data to each member disk of the array, leaving a" mirrored" copy on each disk. Mirroring remains popular due to its simplicity and high levelof data availability. Level 1 operates with two or more disks, and provides very good datareliability and improves performance for read-intensive applications but at a relatively highcost. [3]The storage capacity of the level 1 array is equal to the capacity of the smallest mirroredhard disk in a Hardware RAID or the smallest mirrored partition in a Software RAID . Level 1redundancy is the highest possible among all RAID types, with the array being able tooperate with only a single disk present.Level 4Level 4 uses parity [4] concentrated on a single disk drive to protect data. Because thededicated parity disk represents an inherent bottleneck on all write transactions to the RAIDarray, level 4 is seldom used without accompanying technologies such as write-backcaching, or in specific circumstances where the system administrator is intentionallydesigning the software RAID device with this bottleneck in mind (such as an array that willhave little to no write transactions once the array is populated with data). RAID level 4 is sorarely used that it is not available as an option in Anaconda. However, it could be createdmanually by the user if truly needed.The storage capacity of Hardware RAID level 4 is equal to the capacity of the smallestmember partition multiplied by the number of partitions minus one. Performance of a RAIDlevel 4 array will always be asymmetrical, meaning reads will outperform writes. This isbecause writes consume extra CPU and main memory bandwidth when generating parity,and then also consume extra bus bandwidth when writing the actual data to disks becauseyou are writing not only the data, but also the parity. Reads need only read the data andnot the parity unless the array is in a degraded state. As a result, reads generate less trafficto the drives and across the busses of the computer for the same amount of data transferunder normal operating conditions.Level 5This is the most common type of RAID . By distributing parity across all of an array'smember disk drives, RAID level 5 eliminates the write bottleneck inherent in level 4. The onlyperformance bottleneck is the parity calculation process itself. With modern CPUs andSoftware RAID , that is usually not a bottleneck at all since modern CPUs can generateparity very fast. However, if you have a sufficiently large number of member devices in asoftware RAID 5 array such that the combined aggregate data transfer speed across alldevices is high enough, then this bottleneck can start to come into play.As with level 4, level 5 has asymmetrical performance, with reads substantiallyoutperforming writes. The storage capacity of RAID level 5 is calculated the same way aswith level 4.Level 6This is a common level of RAID when data redundancy and preservation, and notperformance, are the paramount concerns, but where the space inefficiency of level 1 is notacceptable. Level 6 uses a complex parity scheme to be able to recover from the loss of any
two drives in the array. This complex parity scheme creates a significantly higher CPUburden on software RAID devices and also imposes an increased burden during writetransactions. As such, level 6 is considerably more asymmetrical in performance than levels4 and 5.The total capacity of a RAID level 6 array is calculated similarly to RAID level 5 and 4,except that you must subtract 2 devices (instead of 1) from the device count for the extraparity storage space.Level 10This RAID level attempts to combine the performance advantages of level 0 with theredundancy of level 1. It also helps to alleviate some of the space wasted in level 1 arrayswith more than 2 devices. With level 10, it is possible to create a 3-drive array configured tostore only 2 copies of each piece of data, which then allows the overall array size to be 1.5times the size of the smallest devices instead of only equal to the smallest device (like itwould be with a 3-device, level 1 array).The number of options available when creating level 10 arrays (as well as the complexity ofselecting the right options for a specific use case) make it impractical to create duringinstallation. It is possible to create one manually using the command line md ad m tool. Fordetails on the options and their respective performance trade-offs, refer to man md .Lin ear R AIDLinear RAID is a simple grouping of drives to create a larger virtual drive. In linear RAID , thechunks are allocated sequentially from one member drive, going to the next drive only whenthe first is completely filled. This grouping provides no performance benefit, as it is unlikelythat any I/O operations will be split between member drives. Linear RAID also offers noredundancy and, in fact, decreases reliability if any one member drive fails, the entirearray cannot be used. The capacity is the total of all member disks.
mdraidThe md rai d subsystem was designed as a software RAID solution for Linux; it is also the preferredsolution for software RAID under Linux. This subsystem uses its own metadata format, generallyreferred to as native md rai d metadata.md rai d also supports other metadata formats, known as external metadata. Red Hat EnterpriseLinux 7 uses md rai d with external metadata to access ISW / IMSM (Intel firmware RAID ) sets.md rai d sets are configured and controlled through the md ad m utility.
dmraidDevice-mapper RAID or d mrai d refers to device-mapper kernel code that offers the mechanism topiece disks together into a RAID set. This same kernel code does not provide any RAID configurationmechanism.d mrai d is configured entirely in user-space, making it easy to support various on-disk metadataformats. As such, d mrai d is used on a wide variety of firmware RAID implementations. d mrai d alsosupports Intel firmware RAID , although Red Hat Enterprise Linux 7 uses md rai d to access Intelfirmware RAID sets.
mdadmThe md ad m command-line tool is used to manage software RAID in Linux, i.e. md rai d . Forinformation on the different md ad m modes and options, refer to man md ad m. The man page alsocontains useful examples for common operations like creating, monitoring, and assembling softwareRAID arrays.
As the name suggests, d mrai d is used to manage device-mapper RAID sets. The d mrai d tool findsATARAID devices using multiple metadata format handlers, each supporting various formats. For acomplete list of supported formats, run d mrai d -l .As mentioned earlier in Section 18.3, Linux RAID Subsystems , the d mrai d tool cannot configureRAID sets after creation. For more information about using d mrai d , refer to man d mrai d .
NoteThe limited Rescue Mode of the installer does not include man pages. Both the man md ad mand man md contain useful information for creating custom RAID arrays, and may be neededthroughout the workaround. As such, it can be helpful to either have access to a machine withthese man pages present, or to print them out prior to booting into Rescue Mode and creatingyour custom arrays.
[2] A ho t-s wap c has s is allo ws yo u to remo ve a hard d rive witho ut having to p o wer-d o wn yo ur s ys tem.[3] RAID level 1 c o mes at a hig h c o s t b ec aus e yo u write the s ame info rmatio n to all o f the d is ks in thearray, p ro vid es d ata reliab ility, b ut in a muc h les s s p ac e-effic ient manner than p arity b as ed RAID levelss uc h as level 5. Ho wever, this s p ac e ineffic ienc y c o mes with a p erfo rmanc e b enefit: p arity-b as ed RAIDlevels c o ns ume c o ns id erab ly mo re CPU p o wer in o rd er to g enerate the p arity while RAID level 1 s imp ly
Red Hat Ent erprise Linux 7 St orage Administ rat ion G uidewrites the s ame d ata mo re than o nc e to the multip le RAID memb ers with very little CPU o verhead . Ass uc h, RAID level 1 c an o utp erfo rm the p arity-b as ed RAID levels o n mac hines where s o ftware RAID isemp lo yed and CPU res o urc es o n the mac hine are c o ns is tently taxed with o p eratio ns o ther than RAIDac tivities .[4] Parity info rmatio n is c alc ulated b as ed o n the c o ntents o f the res t o f the memb er d is ks in the array.This info rmatio n c an then b e us ed to rec o ns truc t d ata when o ne d is k in the array fails . Therec o ns truc ted d ata c an then b e us ed to s atis fy I/O req ues ts to the failed d is k b efo re it is rep lac ed andto rep o p ulate the failed d is k after it has b een rep lac ed .
mo unt
On Linux, UNIX, and similar operating systems, file systems on different partitions and removabledevices (CD s, D VD s, or USB flash drives for example) can be attached to a certain point (the mountpoint) in the directory tree, and then detached again. To attach or detach a file system, use the mo untor umo unt command respectively. This chapter describes the basic use of these commands, as wellas some advanced topics, such as moving a mount point or creating shared subtrees.
When the mo unt command is run without all required information (that is, without the device name,the target directory, or the file system type), it reads the content of the /etc/fstab configuration fileto see if the given file system is listed. This file contains a list of device names and the directories inwhich the selected file systems should be mounted, as well as the file system type and mount options.Because of this, when mounting a file system that is specified in this file, you can use one of thefollowing variants of the command:mo unt [option] directorymo unt [option] deviceNote that permissions are required to mount the file systems unless the command is run as ro o t (seeSection 19.2.2, Specifying the Mount Options ).
ext2ext3ext4btrfsxfsi so 9 6 6 0
jfsnfsnfs4ntfsud fvfat
See Example 19.2, Mounting a USB Flash D rive for an example usage.
d efaul tsexecloopno autono execno userremo untrorwuser
~]# mo unt -o ro ,l o o p Fed o ra-14 -x86 _6 4 -Li ve-D eskto p. i so /med i a/cd ro mNote that ISO 9660 is by design a read-only file system.
It is now possible to verify that a mount within /med i a also appears in /mnt. Forexample, if the CD -ROM drive contains non-empty media and the /med i a/cd ro m/directory exists, run the following commands:~]# mo unt /d ev/cd ro m /med i a/cd ro m~]# l s /med i a/cd ro mEFI GPL isolinux LiveOS~]# l s /mnt/cd ro mEFI GPL isolinux LiveOSSimilarly, it is possible to verify that any file system mounted in the /mnt directory isreflected in /med i a. For instance, if a non-empty USB flash drive that uses the/d ev/sd c1 device is plugged in and the /mnt/fl ashd i sk/ directory is present, type:~]# mo unt /d ev/sd c1 /mnt/fl ashd i sk~]# l s /med i a/fl ashd i sken-US publican.cfg~]# l s /mnt/fl ashd i sken-US publican.cfgSlave Mo u n tA slave mount allows the creation of a limited duplicate of a given mount point. When amount point is marked as a slave mount, any mount within the original mount point isreflected in it, but no mount within a slave mount is reflected in its original. To change thetype of a mount point to a slave mount, type the following at a shell prompt:mo unt --make-sl ave mount_pointAlternatively, it is possible to change the mount type for the selected mount point and allmount points under it by typing:mo unt --make-rsl ave mount_pointSee Example 19.5, Creating a Slave Mount Point for an example usage.
It is also possible to verify that file systems mounted in the /mnt directory are notreflected in /med i a. For instance, if a non-empty USB flash drive that uses the/d ev/sd c1 device is plugged in and the /mnt/fl ashd i sk/ directory is present, type:~]# mo unt /d ev/sd c1 /mnt/fl ashd i sk~]# l s /med i a/fl ashd i sk~]# l s /mnt/fl ashd i sken-US publican.cfgU n b in d ab le Mo u n tIn order to prevent a given mount point from being duplicated whatsoever, an unbindablemount is used. To change the type of a mount point to an unbindable mount, type thefollowing at a shell prompt:mo unt --make-unbi nd abl e mount_pointAlternatively, it is possible to change the mount type for the selected mount point and allmount points under it:mo unt --make-runbi nd abl e mount_pointSee Example 19.7, Creating an Unbindable Mount Point for an example usage.
An NFS storage contains user directories and is already mounted in /mnt/userd i rs/. As ro o t,move this mount point to /ho me by using the following command:~]# mo unt --mo ve /mnt/userd i rs /ho meTo verify the mount point has been moved, list the content of both directories:~]# l s /mnt/userd i rs~]# l s /ho mejill joe
Examp le 19 .9 . U n mo u n t in g a C DTo unmount a CD that was previously mounted to the /med i a/cd ro m/ directory, type thefollowing at a shell prompt:~]$ umo unt /med i a/cd ro m
14 0
vo l ume_key
The volume_key function provides two tools, libvolume_key and vo l ume_key. libvolume_key is alibrary for manipulating storage volume encryption keys and storing them separately from volumes.vo l ume_key is an associated command line tool used to extract keys and passphrases in order torestore access to an encrypted hard drive.This is useful for when the primary user forgets their keys and passwords, after an employee leavesabruptly, or in order to extract data after a hardware or software failure corrupts the header of theencrypted volume. In a corporate setting, the IT help desk can use vo l ume_key to back up theencryption keys before handing over the computer to the end user.Currently, vo l ume_key only supports the LUKS volume encryption format.
Notevo l ume_key is not included in a standard install of Red Hat Enterprise Linux 7 server. Forinformation on installing it, refer tohttp://fedoraproject.org/wiki/D isk_encryption_key_escrow_use_cases.
14 1
process.--reencrypt, --secrets, an d --d umpThese three commands perform similar functions with varying output methods. They eachrequire the operand packet, and each opens the packet, decrypting it where necessary. -reencrypt then stores the information in one or more new output packets. --secretsoutputs the keys and passphrases contained in the packet. --d ump outputs the content ofthe packet, though the keys and passphrases are not output by default. This can bechanged by appending --wi th-secrets to the command. It is also possible to only dumpthe unencrypted parts of the packet, if any, by using the --unencrypted command. Thisdoes not require any passphrase or private key access.Each of these can be appended with the following options:-o , --o utput packetThis command writes the default key or passphrase to the packet. The default key orpassphrase depends on the volume format. Ensure it is one that is unlikely to expire, andwill allow --resto re to restore access to the volume.--o utput-fo rmat formatThis command uses the specified format for all output packets. Currently, format can be oneof the following:asymmetri c: uses CMS to encrypt the whole packet, and requires a certificateasymmetri c_wrap_secret_o nl y: wraps only the secret, or keys and passphrases,and requires a certificatepassphrase: uses GPG to encrypt the whole packet, and requires a passphrase--create-rand o m-passphrase packetThis command generates a random alphanumeric passphrase, adds it to the volume(without affecting other passphrases), and then stores this random passphrase into thepacket.
20.2. Using
as an individual user
As an individual user, vo l ume_key can be used to save encryption keys by using the followingprocedure.
NoteFor all examples in this file, /path/to/volume is a LUKS device, not the plaintext devicecontained within. bl ki d -s type /path/to/volume should reporttype= "crypto _LUKS".
14 2
A prompt will then appear requiring an escrow packet passphrase to protect the key.2. Save the generated escro w-packet file, ensuring that the passphrase is not forgotten.If the volume passphrase is forgotten, use the saved escrow packet to restore access to the data.Pro ced u re 20.2. R est o re access t o d at a wit h escro w p acket1. Boot the system in an environment where vo l ume_key can be run and the escrow packet isavailable (a rescue mode, for example).2. Run:vo l ume_key --resto re /path/to/volume escro w-packetA prompt will appear for the escrow packet passphrase that was used when creating theescrow packet, and for the new passphrase for the volume.3. Mount the volume using the chosen passphrase.To free up the passphrase slot in the LUKS header of the encrypted volume, remove the old, forgottenpassphrase by using the command cryptsetup l uksKi l l Sl o t.
20.3. Using
In a larger organization, using a single password known by every system administrator and keepingtrack of a separate password for each system is impractical and a security risk. To counter this,vo l ume_key can use asymmetric cryptography to minimize the number of people who know thepassword required to access encrypted data on any computer.This section will cover the procedures required for preparation before saving encryption keys, how tosave encryption keys, restoring access to a volume, and setting up emergency passphrases.
14 3
At this point it is possible to choose an NSS database password. Each NSS database canhave a different password so the designated users do not need to share a singlepassword if a separate NSS database is used by each user.C. Run:pk12uti l -d /the/nss/directory -i the-pkcs12-file4. D istribute the certificate to anyone installing systems or saving keys on existing systems.5. For saved private keys, prepare storage that allows them to be looked up by machine andvolume. For example, this can be a simple directory with one subdirectory per machine, or adatabase used for other system management tasks as well.
NoteFor all examples in this file, /path/to /vo l ume is a LUKS device, not the plaintext devicecontained within; bl ki d -s type /path/to/volume should reporttype= "crypto _LUKS".
14 4
After providing the NSS database password, the designated user chooses a passphrase forencrypting escro w-packet-o ut. This passphrase can be different every time and onlyprotects the encryption keys while they are moved from the designated user to the targetsystem.3. Obtain the escro w-packet-o ut file and the passphrase from the designated user.4. Boot the target system in an environment that can run vo l ume_key and have the escro wpacket-o ut file available, such as in a rescue mode.5. Run:vo l ume_key --resto re /path/to/volume escro w-packet-o utA prompt will appear for the packet passphrase chosen by the designated user, and for a newpassphrase for the volume.6. Mount the volume using the chosen volume passphrase.It is possible to remove the old passphrase that was forgotten by using cryptsetupl uksKi l l Sl o t, for example, to free up the passphrase slot in the LUKS header of the encryptedvolume. This is done with the command cryptsetup l uksKi l l Sl o t device key-slot. Formore information and examples see cryptsetup --hel p.
14 5
1 2
If an ext3 file system is accessed via Samba and ACLs have been enabled for it, the ACLs arerecognized because Samba has been compiled with the --wi th-acl -suppo rt option. No specialflags are required when accessing or mounting a Samba share.
21.1.1. NFSBy default, if the file system being exported by an NFS server supports ACLs and the NFS client canread ACLs, ACLs are utilized by the client system.To disable ACLs on NFS shares when configuring the server, include the no _acl option in the/etc/expo rts file. To disable ACLs on an NFS share when mounting it on a client, mount it with theno _acl option via the command line or the /etc/fstab file.
14 6
2. Per group3. Via the effective rights mask4. For users not in the user group for the fileThe setfacl utility sets ACLs for files and directories. Use the -m option to add or modify the ACL ofa file or directory:# setfacl -m rules filesRules (rules) must be specified in the following formats. Multiple rules can be specified in the samecommand if they are separated by commas.u: uid: permsSets the access ACL for a user. The user name or UID may be specified. The user may beany valid user on the system.g : gid: permsSets the access ACL for a group. The group name or GID may be specified. The group maybe any valid group on the system.m: permsSets the effective rights mask. The mask is the union of all permissions of the owning groupand all of the user and group entries.o : permsSets the access ACL for users other than the ones in the group for the file.Permissions (perms) must be a combination of the characters r, w, and x for read, write, and execute.If a file or directory already has an ACL, and the setfacl command is used, the additional rules areadded to the existing ACL or the existing rule is modified.
To remove all the permissions for a user, group, or others, use the -x option and do not specify anypermissions:# setfacl -x rules files
14 7
14 8
By default, the d ump command now preserves ACLs during a backup operation. When archiving afile or file system with tar, use the --acl s option to preserve ACLs. Similarly, when using cp to copyfiles with ACLs, include the --preserve= mo d e option to ensure that ACLs are copied across too. Inaddition, the -a option (equivalent to -d R --preserve= al l ) of cp also preserves ACLs during abackup along with other information such as timestamps, SELinux contexts, and the like. For moreinformation about d ump, tar, or cp, refer to their respective man pages.The star utility is similar to the tar utility in that it can be used to generate archives of files; however,some of its options are different. Refer to Table 21.1, Command Line Options for star for a listing ofmore commonly used options. For all available options, refer to man star. The star package isrequired to use this utility.T ab le 21.1. C o mman d Lin e O p t io n s f o r starO p t io n
-hel p-xhel p-/
NoteNot all solid-state devices in the market have d i scard support. To determine if your solidstate device has d i scard support check for/sys/bl o ck/sd a/q ueue/d i scard _g ranul ari ty.
Red Hat also warns that software RAID levels 1, 4, 5, and 6 are not recommended for use on SSD s.D uring the initialization stage of these RAID levels, some RAID management utilities (such as md ad m)write to all of the blocks on the storage device to ensure that checksums operate properly. This willcause the performance of the SSD to degrade quickly.As of Red Hat Enterprise Linux 6.4, ext4 and XFS are the only fully-supported file systems thatsupport d i scard . Previous versions of Red Hat Enterprise Linux 6 only ext4 fully supportedd i scard . To enable d i scard commands on a device, use the mo unt option d i scard . Forexample, to mount /d ev/sd a2 to /mnt with d i scard enabled, run:# mount -t ext4 -o discard /dev/sda2 /mntBy default, ext4 does not issue the d i scard command. This is mostly to avoid problems on deviceswhich may not properly implement the d i scard command. The Linux swap code will issue d i scardcommands to d i scard -enabled devices, and there is no option to control this behavior.
I/O SchedulerAny I/O scheduler should perform well with most SSD s. However, as with any other storage type, RedHat recommends benchmarking to determine the optimal configuration for a given workload.When using SSD s, Red Hat advises changing the I/O scheduler only for benchmarking particularworkloads. For more information about the different types of I/O schedulers, refer to the I/O TuningGuide (also provided by Red Hat). The following kernel document also contains instructions on howto switch between I/O schedulers:/usr/share/d o c/kernel -version/D o cumentati o n/bl o ck/swi tchi ng -sched . txtAs of Red Hat Enterprise Linux 7.0, the default I/O scheduler is now D eadline, except for use withSATA drives. In the case of SATA drives, CFQ is the default I/O scheduler. For faster storage, deadlinecan outperform CFQ leading to better I/O performance without the need for specific tuning. It ispossible, however, that default is not right for some disks (such as SAS rotational disks). In this casethe I/O scheduler will need to be changed to CFQ.
Virtual MemoryLike the I/O scheduler, virtual memory (VM) subsystem requires no special tuning. Given the fastnature of I/O on SSD , it should be possible to turn down the vm_d i rty_backg ro und _rati o andvm_d i rty_rati o settings, as increased write-out activity should not negatively impact the latencyof other operations on the disk. However, this can generate more overall I/O and so is not generallyrecommended without workload-specific testing.
SwapAn SSD can also be used as a swap device, and is likely to produce good page-out/page-inperformance.
To mitigate the risk of data corruption during power loss, some storage devices use battery-backedwrite caches. Generally, high-end arrays and some hardware controllers use battery-backed writecaches. However, because the cache's volatility is not visible to the kernel, Red Hat Enterprise Linux 7enables write barriers by default on all supported journaling file systems.
NoteWrite caches are designed to increase I/O performance. However, enabling write barriersmeans constantly flushing these caches, which can significantly reduce performance.For devices with non-volatile, battery-backed write caches and those with write-caching disabled,you can safely disable write barriers at mount time using the -o no barri er option for mo unt.However, some devices do not support write barriers; such devices will log an error message to/var/l o g /messag es (refer to Table 23.1, Write barrier error messages per file system ).T ab le 23.1. Writ e b arrier erro r messag es p er f ile syst emFile Syst em
Erro r Messag e
Most controllers use vendor-specific tools to query and manipulate target drives. For example, theLSI Megaraid SAS controller uses a battery-backed write cache; this type of controller requires theMeg aC l i 6 4 tool to manage target drives. To show the state of all back-end drives for LSI MegaraidSAS, use:# MegaCli64 -LDGetProp
-LAll -aALL
To disable the write cache of all back-end drives for LSI Megaraid SAS, use:# MegaCli64 -LDSetProp -DisDskCache -Lall -aALL
NoteHardware RAID cards recharge their batteries while the system is operational. If a system ispowered off for an extended period of time, the batteries will lose their charge, leaving storeddata vulnerable during a power failure.
High-End ArraysHigh-end arrays have various ways of protecting data in the event of a power failure. As such, thereis no need to verify the state of the internal drives in external RAID storage.
NFSNFS clients do not need to enable write barriers, since data integrity is handled by the NFS serverside. As such, NFS servers should be configured to ensure data persistence throughout a power loss(whether through write barriers or other means).
Storage vendors can also supply I/O hints about the preferred minimum unit for random I/O(mi ni mum_i o _si ze) and streaming I/O (o pti mal _i o _si ze) of a device. For example,mi ni mum_i o _si ze and o pti mal _i o _si ze may correspond to a RAID device's chunk size andstripe size respectively.
sysfs Interface/sys/block/disk/alignment_offset/sys/block/disk/partition/alignment_offset/sys/block/disk/queue/physical_block_size/sys/block/disk/queue/logical_block_size/sys/block/disk/queue/minimum_io_size/sys/block/disk/queue/optimal_io_sizeThe kernel will still export these sysfs attributes for " legacy" devices that do not provide I/Oparameters information, for example:
24 .3. St andardsThis section describes I/O standards used by ATA and SCSI devices.
ATAATA devices must report appropriate information via the ID ENT IFY D EVIC E command. ATA devicesonly report I/O parameters for physi cal _bl o ck_si ze, l o g i cal _bl o ck_si ze, andal i g nment_o ffset. The additional I/O hints are outside the scope of the ATA Command Set.
SCSII/O parameters support in Red Hat Enterprise Linux 7 requires at least version 3 of the SCSI PrimaryCommands (SPC-3) protocol. The kernel will only send an extended inquiry (which gains access to theBLO C K LIMIT S VP D page) and R EAD C AP AC IT Y (16 ) command to devices which claimcompliance with SPC-3.The R EAD C AP AC IT Y (16 ) command provides the block sizes and alignment offset:LO G IC AL BLO C K LENG T H IN BY T ES is used to derive/sys/bl o ck/disk/q ueue/physi cal _bl o ck_si zeLO G IC AL BLO C KS P ER P HY SIC AL BLO C K EXP O NENT is used to derive/sys/bl o ck/disk/q ueue/l o g i cal _bl o ck_si zeLO WEST ALIG NED LO G IC AL BLO C K AD D R ESS is used to derive:/sys/bl o ck/disk/al i g nment_o ffset/sys/bl o ck/disk/partition/al i g nment_o ffsetThe BLO C K LIMIT S VP D page (0 xb0 ) provides the I/O hints. It also uses O P T IMAL T R ANSFERLENG T H G R ANULAR IT Y and O P T IMAL T R ANSFER LENG T H to derive:/sys/bl o ck/disk/q ueue/mi ni mum_i o _si ze/sys/bl o ck/disk/q ueue/o pti mal _i o _si zeThe sg 3_uti l s package provides the sg _i nq utility, which can be used to access the BLO C KLIMIT S VP D page. To do so, run:# sg_inq -p 0xb0 disk
All layers of the Linux I/O stack have been engineered to propagate the various I/O parameters up thestack. When a layer consumes an attribute or aggregates many devices, the layer must exposeappropriate I/O parameters so that upper-layer devices or tools will have an accurate view of thestorage as it transformed. Some practical examples are:Only one layer in the I/O stack should adjust for a non-zero al i g nment_o ffset; once a layeradjusts accordingly, it will export a device with an al i g nment_o ffset of zero.A striped D evice Mapper (D M) device created with LVM must export a mi ni mum_i o _si ze ando pti mal _i o _si ze relative to the stripe count (number of disks) and user-provided chunk size.In Red Hat Enterprise Linux 7, D evice Mapper and Software Raid (MD ) device drivers can be used toarbitrarily combine devices with different I/O parameters. The kernel's block layer will attempt toreasonably combine the I/O parameters of the individual devices. The kernel will not preventcombining heterogeneous devices; however, be aware of the risks associated with doing so.For instance, a 512-byte device and a 4K device may be combined into a single logical D M device,which would have a l o g i cal _bl o ck_si ze of 4K. File systems layered on such a hybrid deviceassume that 4K will be written atomically, but in reality it will span 8 logical block addresses whenissued to the 512-byte device. Using a 4K l o g i cal _bl o ck_si ze for the higher-level D M deviceincreases potential for a partial write to the 512-byte device if there is a system crash.If combining the I/O parameters of multiple devices results in a conflict, the block layer may issue awarning that the device is susceptible to partial writes and/or is misaligned.
determine the I/O parameters of a device for optimal placement of all partitions. The fd i sk utility willalign all partitions on a 1MB boundary.
NoteRed Hat Enterprise Linux 7 cannot distinguish between devices that don't provide I/O hintsand those that do so with al i g nment_o ffset= 0 and o pti mal _i o _si ze= 0 . Such adevice might be a single SAS 4K device; as such, at worst 1MB of space is lost at the startof the disk.
class "pxeclients" {match if substring(option vendor-class-identifier, 0, 9) ="PXEClient";next-server server-ip;filename "pxelinux.0";}Replace server-ip with the IP address of the host machine on which the tftp and D HCP servicesreside. Now that tftp and D HCP are configured, all that remains is to configure NFS and theexported file system; refer to Section 25.3, Configuring an Exported File System for D iskless Clientsfor instructions.
Copy the resulting i ni tramfs-kernel-version. i mg into the tftp boot directory as well.4. Edit the default boot configuration to use the i ni trd and kernel inside/var/l i b/tftpbo o t. This configuration should instruct the diskless client's root to mountthe exported file system (/exported/root/directory) as read-write. To do this, configure/var/l i b/tftpbo o t/pxel i nux. cfg /d efaul t with the following:default rhel7label rhel7kernel vmlinuz-kernel-versionappend initrd=initramfs-kernel-version.img root=nfs:serverip:/exported/root/directory rwReplace server-ip with the IP address of the host machine on which the tftp and D HCPservices reside.The NFS share is now ready for exporting to diskless clients. These clients can boot over the networkvia PXE.
WarningOnline storage reconfiguration must be done carefully. System failures or interruptions duringthe process can lead to unexpected results. Red Hat advises that you reduce system load tothe maximum extent possible during the change operations. This will reduce the chance of I/Oerrors, out-of-memory errors, or similar errors occurring in the midst of a configuration change.The following sections provide more specific guidelines regarding this.In addition, Red Hat recommends that you back up all data before reconfiguring onlinestorage.
Notetarg etcl i 's hierachy does not always match up to the kernel interface. This is because it isdesigned to be simplified where possible.
NoteIn Red Hat Enterprise Linux 7.0, running a targ etcl i command from bash (for example,targ etcl i i scsi / create) would not work, but would also give no error code. This hasbeen fixed in Red Hat Enterprise Linux 7.1 to provide an error status code, thus making it moreuseful to use with shell scripts.
NoteIn Red Hat Enterprise Linux 6 the term 'backing-store' is used to refer to the mappings created.However, to avoid confusion between the various ways 'backstores' can be used, in Red HatEnterprise Linux 7 the term 'storage objects' refers to the mappings created and 'backstores' isused to describe the different types of backing devices.The backstore devices that LIO supports are:FILEIO ( Lin u x f ile- b acked st o rag e)FILEIO storage objects can support either wri te_back or wri te_thru operation. Thewri te_back enables the local file system cache. This improves performance but increasesthe risk of data loss. It is recommended to use wri te_back= fal se to disablewri te_back in favor of wri te_thru.
To create a fileio storage object, run the command /backsto res/fi l ei o createfile_name file_location file_size wri te_back= fal se. For example:/> /backstores/fileio create file1 /tmp/disk1.img 200Mwrite_back=falseCreated fileio file1 with size 209715200B LO C K ( Lin u x B LO C K d evices)The block driver allows the use of any block device that appears in the /sys/bl o ck to beused with LIO. This includes physical devices (for example, HD D s, SSD s, CD s, D VD s) andlogical devices (for example, software or hardware RAID volumes, or LVM volumes).
NoteBLOCK backstores usually provide the best performance.To create a BLOCK backstore using the /d ev/sd b block device, use the followingcommand:/> /backstores/fileio create name=block_backend dev=/dev/sdbGenerating a wwn serial.Created block storage object block_backend using /dev/sdb.PSC SI ( Lin u x p ass- t h ro u g h SC SI d evices)Any storage object that supports direct pass-through of SCSI commands without SCSIemulation, and with an underlying SCSI device that appears with lsscsi in/pro c/scsi /scsi (such as a SAS hard drive) can be configured as a backstore. SCSI-3and higher is supported with this subsystem.
WarningPSCSI should only be used by advanced users. Advanced SCSI commands such asfor Aysmmetric Logical Unit Assignment (ALUAs) or Persistent Reservations (forexample, those used by VMware ESX, and vSphere) are usually not implemented inthe device firmware and can cause malfunctions or crashes. When in doubt, useBLOCK for production setups instead.To create a PSCSI backstore for a physical SCSI device, a T Y P E_R O M device using/d ev/sr0 in this example, use:/> backstores/pscsi/ create name=pscsi_backend dev=/dev/sr0Generating a wwn serial.Created pscsi storage object pscsi_backend using /dev/sr0Memo ry C o p y R AM d isk ( Lin u x R AMD ISK _MC P)
Memory Copy RAM disks (ramd i sk) provide RAM disks with full SCSI emulation andseparate memory mappings using memory copy for initiators. This provides capability formulti-sessions and is particularly useful for fast, volatile mass storage for productionpurposes.To create a 1GB RAM disk backstore, use the following command:/> backstores/ramdisk/ create name=rd_backend size=1GBGenerating a wwn serial.Created rd_mcp ramdisk rd_backend with size 1GB.
NoteThe cd command is also accepted to change directories, as well as simply listing thepath to move into.3. Create an iSCSI target using a default target name./iscsi> createCreated targetiqn.2003-01.org.linux-iscsi.hostname.x8664:sn.78b473f296ffCreated TPG1Or create an iSCSI target using a specified name./iscsi > create iqn.2006-04.com.example:444Created target iqn.2006-04.com.example:444Created TPG14. Verify that the newly created target is visible when targets are listed with l s./iscsi > lso- iscsi.......................................[1 Target]o- iqn.2006-04.com.example:444................[1 TPG]o- tpg1...........................[enabled, auth]o- acls...............................[0 ACL]o- luns...............................[0 LUN]o- portals.........................[0 Portal]
NoteAs of Red Hat Enterprise Linux 7.1, whenever a target is created, a default portal is alsocreated.
NoteThe above example's behavior depends on the setting used. In this case, the globalsetting auto _ad d _mapped _l uns is used. This automatically maps LUNs to anycreated ACL.
ImportantBefore proceeding, refer to Section 26.3, Configuring a Fibre-Channel Over EthernetInterface and verify that basic FCoE setup is completed, and that fco ead m -i displaysconfigured FCoE interfaces.
NoteThis was uploaded on February 28, 2012. As such, the service name has changedfrom targ etcl i to targ et.
ImportantIf your system is using multipath software, Red Hat recommends that you consult yourhardware vendor before changing any of the values described in this section.T ran sp o rt : /sys/cl ass/fc_transpo rt/targ etH: B: T/po rt_i d 24-bit port ID /addressno d e_name 64-bit node namepo rt_name 64-bit port nameR emo t e Po rt : /sys/cl ass/fc_remo te_po rts/rpo rt-H: B-R/po rt_i dno d e_name
po rt_named ev_l o ss_tmo number of seconds to wait before marking a link as " bad" . Once alink is marked bad, I/O running on its corresponding path (along with any new I/O onthat path) will be failed.The default d ev_l o ss_tmo value varies, depending on which driver/device is used. If aQlogic adapter is used, the default is 35 seconds, while if an Emulex adapter is used, itis 30 seconds. The d ev_l o ss_tmo value can be changed via thescsi _transpo rt_fc module parameter d ev_l o ss_tmo , although the driver canoverride this timeout value.The maximum d ev_l o ss_tmo value is 600 seconds. If d ev_l o ss_tmo is set to zero orany value greater than 600, the driver's internal timeouts will be used instead.fast_i o _fai l _tmo length of time to wait before failing I/O executed when a linkproblem is detected. I/O that reaches the driver will fail. If I/O is in a blocked queue, it willnot be failed until d ev_l o ss_tmo expires and the queue is unblocked.H o st : /sys/cl ass/fc_ho st/ho stH/po rt_i di ssue_l i p instructs the driver to rediscover remote ports.
Transportpo rt_i dTransportno d e_nameTransportpo rt_nameRemote Portd ev_l o ss_tmoRemote Portfast_i o _fai l _tmo
l pfc
q l a2xxx
X [a]
X [b ]
Host po rt_i dHosti ssue_l i p
Modify the contents of cfg -ethX as necessary. Of note, D C B_R EQ UIR ED should be set tono for networking interfaces that implement a hardware D CBX client.2. If you want the device to automatically load during boot time, set O NBO O T = yes in thecorresponding /etc/sysco nfi g /netwo rk-scri pts/i fcfg -ethX file. For example, if theFCoE device is eth2, then edit /etc/sysco nfi g /netwo rk-scri pts/i fcfg -eth2accordingly.3. Start the data center bridging daemon (d cbd ) using the following command:# /etc/init.d/lldpad start4. For networking interfaces that implement a hardware D CBX client, skip this step and move onto the next.For interfaces that require a software D CBX client, enable data center bridging on theEthernet interface using the following commands:# dcbtool sc ethX dcb onThen, enable FCoE on the Ethernet interface by running:# dcbtool sc ethX app:fcoe e:1
NoteThese commands will only work if the d cbd settings for the Ethernet interface were notchanged.5. Load the FCoE device now using:# ifconfig ethX up6. Start FCoE using:# service fcoe startThe FCoE device should appear shortly, assuming all other settings on the fabric are correct.To view configured FCoE devices, run:# fcoeadm -iAfter correctly configuring the Ethernet interface to use FCoE, Red Hat recommends that you set FCoEand l l d pad to run at startup. To do so, use chkco nfi g , as in:# chkconfig lldpad on# chkconfig fcoe on
NoteUsing systemctl sto p fco e. servi ce will stop the daemon but will not deconfigure FCoEinterfaces. To do so, a superuser will need to use the systemctl -s SIG HUP ki l lfco e. servi ce command.As of Red Hat Enterprise Linux 7, Network Manager has the ability to query and set the D CB settingsof a D CB capable ethernet interface.
26.4 . Configuring an FCoE Int erface t o Aut omat ically Mount at BootNoteThe instructions in this section are available in /usr/share/d o c/fco euti l s-version/R EAD ME as of Red Hat Enterprise Linux 6.1. Refer to that document for anypossible changes throughout minor releases.You can mount newly discovered disks via ud ev rules, auto fs, and other similar methods.Sometimes, however, a specific service might require the FCoE disk to be mounted at boot-time. Insuch cases, the FCoE disk should be mounted as soon as the fco e service runs and before theinitiation of any service that requires the FCoE disk.
To configure an FCoE disk to automatically mount at boot, add proper FCoE mounting code to thestartup script for the fco e service. The fco e startup script is /etc/i ni t. d /fco e.The FCoE mounting code is different per system configuration, whether you are using a simpleformatted FCoE disk, LVM, or multipathed device node.
Examp le 26 .2. FC o E mo u n t in g co d eThe following is a sample FCoE mounting code for mounting file systems specified via wild cardsin /etc/fstab:mount_fcoe_disks_from_fstab(){local timeout=20local done=1local fcoe_disks=($(egrep 'by-path\/fc-.*_netdev' /etc/fstab | cut-d ' ' -f1))test -z $fcoe_disks & & return 0echo -n "Waiting for fcoe disks . "while [ $timeout -gt 0 ]; dofor disk in ${fcoe_disks[*]}; doif ! test -b $disk; thendone=0breakfidonetest $done -eq 1 & & break;sleep 1echo -n ". "done=1let timeout-doneif test $timeout -eq 0; thenecho "timeout!"elseecho "done!"fi# mount any newly discovered diskmount -a 2>/dev/null}
The mo unt_fco e_d i sks_fro m_fstab function should be invoked after the fco e service scriptstarts the fco emo n daemon. This will mount FCoE disks specified by the following paths in/etc/fstab:/dev/disk/by-path/fc-0xXX:0xXX /mnt/fcoe-disk1 ext30 0/dev/disk/by-path/fc-0xYY:0xYY /mnt/fcoe-disk2 ext30 0
Entries with fc- and _netd ev sub-strings enable the mo unt_fco e_d i sks_fro m_fstab functionto identify FCoE disk mount entries. For more information on /etc/fstab entries, refer to man 5fstab.
NoteThe fco e service does not implement a timeout for FCoE disk discovery. As such, the FCoEmounting code should implement its own timeout period.
26.5. iSCSIThis section describes the iSCSI API and the i scsi ad m utility. Before using the i scsi ad m utility,install the i scsi -i ni ti ato r-uti l s package first by running yum i nstal l i scsi i ni ti ato r-uti l s.In Red Hat Enterprise Linux 7, the iSCSI service is lazily started by default. If root is not on an iSCSIdevice or there are no nodes marked with no d e. startup = auto mati c then the iSCSI service willnot start until an i scsi ad m command is run that requires iscsid or the iscsi kernel modules to bestarted. For example, running the discovery command i scsi ad m -m d i sco very -t st -pi p: po rt will cause iscsiadmin to start the iSCSI service.To force the iscsid daemon to run and iSCSI kernel modules to load, run servi ce i scsi dfo rce-start.
# iscsiadm -m sessiontcp [2] 10.15.84.19:3260,2 iqn.1992-08.com.netapp:sn.33615311tcp [3] 10.15.85.19:3260,3 iqn.1992-08.com.netapp:sn.33615311
For more information about the iSCSI API, refer to /usr/share/d o c/i scsi -i ni ti ato ruti l s-version/R EAD ME.
The World Wide Identifier (WWID ) can be used in reliably identifying devices. It is a persistent, systemindependent ID that the SCSI Standard requires from all SCSI devices. The WWID identifier isguaranteed to be unique for every storage device, and independent of the path that is used to accessthe device.This identifier can be obtained by issuing a SCSI Inquiry to retrieve the Device Identification VitalProduct Data (page 0 x83) or Unit Serial Number (page 0 x80 ). The mappings from these WWID s to thecurrent /d ev/sd names can be seen in the symlinks maintained in the /d ev/d i sk/by-i d /directory.
Examp le 26 .4 . WWIDFor example, a device with a page 0 x83 identifier would have:scsi-3600508b400105e210000900000490000 -> ../../sdaOr, a device with a page 0 x80 identifier would have:scsi-SSEAGATE_ST373453LW_3HW1RHM6 -> ../../sda
Red Hat Enterprise Linux automatically maintains the proper mapping from the WWID -based devicename to a current /d ev/sd name on that system. Applications can use the /d ev/d i sk/by-i d /name to reference the data on the disk, even if the path to the device changes, and even whenaccessing the device from different systems.If there are multiple paths from a system to a device, d evice- map p er- mu lt ip at h uses the WWID todetect this. D evice- map p er- mu lt ip at h then presents a single " pseudo-device" in/d ev/mapper/wwi d , such as /d ev/mapper/36 0 0 50 8b4 0 0 10 5d f70 0 0 0 e0 0 0 0 0 ac0 0 0 0 .The command mul ti path -l shows the mapping to the non-persistent identifiers:Host: Channel: Target: LUN, /d ev/sd name, and the majo r: mi no r number.3600508b400105df70000e00000ac0000 dm-2 vendor,product[size=20G][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=0][active]\_ 5:0:1:1 sdc 8:32 [active][undef]\_ 6:0:1:1 sdg 8:96 [active][undef]\_ round-robin 0 [prio=0][enabled]\_ 5:0:0:1 sdb 8:16 [active][undef]\_ 6:0:0:1 sdf 8:80 [active][undef]D evice- map p er- mu lt ip at h automatically maintains the proper mapping of each WWID -baseddevice name to its corresponding /d ev/sd name on the system. These names are persistent acrosspath changes, and they are consistent when accessing the device from different systems.When the user_fri end l y_names feature (of d evice- map p er- mu lt ip at h ) is used, the WWID ismapped to a name of the form /d ev/mapper/mpathn. By default, this mapping is maintained in thefile /etc/mul ti path/bi nd i ng s. These mpathn names are persistent as long as that file ismaintained.
ImportantIf you use user_fri end l y_names, then additional steps are required to obtain consistentnames in a cluster. Refer to the Consistent Multipath D evice Names in a Cluster section in theUsing DM Multipath Configuration and Administration book.In addition to these persistent names provided by the system, you can also use ud ev rules toimplement persistent names of your own, mapped to the WWID of the storage. For more informationabout this, refer to http://kbase.redhat.com/faq/docs/D OC-7319.
3. Remove the device from any md and LVM volume using it. If the device is a member of an LVMVolume group, then it may be necessary to move data off the device using the pvmo vecommand, then use the vg red uce command to remove the physical volume, and (optionally)pvremo ve to remove the LVM metadata from the disk.4. If the device uses multipathing, run mul ti path -l and note all the paths to the device.Afterwards, remove the multipathed device using mul ti path -f d evi ce.5. Run bl o ckd ev --fl ushbufs device to flush any outstanding I/O to all paths to thedevice. This is particularly important for raw devices, where there is no umo unt or vg red uceoperation to cause an I/O flush.6. Remove any reference to the device's path-based name, like /d ev/sd , /d ev/d i sk/bypath or the majo r: mi no r number, in applications, scripts, or utilities on the system. This isimportant in ensuring that different devices added in the future will not be mistaken for thecurrent device.7. Finally, remove each path to the device from the SCSI subsystem. To do so, use the commandecho 1 > /sys/bl o ck/device-name/d evi ce/d el ete where device-name may besd e, for example.Another variation of this operation is echo 1 >/sys/cl ass/scsi _d evi ce/h: c: t: l/d evi ce/d el ete, where h is the HBA number, c isthe channel on the HBA, t is the SCSI target ID , and l is the LUN.
NoteThe older form of these commands, echo "scsi remo ve-si ng l e-d evi ce 0 0 00 " > /pro c/scsi /scsi , is deprecated.You can determine the device-name, HBA number, HBA channel, SCSI target ID and LUN for adevice from various commands, such as l sscsi , scsi _i d , mul ti path -l , and l s -l/d ev/d i sk/by-*.After performing Procedure 26.7, Ensuring a Clean D evice Removal , a device can be physicallyremoved safely from a running system. It is not necessary to stop I/O to other devices while doing so.Other procedures, such as the physical removal of the device, followed by a rescan of the SCSI bus(as described in Section 26.10, Scanning Storage Interconnects ) to cause the operating systemstate to be updated to reflect the change, are not recommended. This will cause delays due to I/Otimeouts, and devices may be removed unexpectedly. If it is necessary to perform a rescan of aninterconnect, it must be done while I/O is paused, as described in Section 26.10, Scanning StorageInterconnects .
2. Take the path offline using echo o ffl i ne > /sys/bl o ck/sd a/d evi ce/state.This will cause any subsequent I/O sent to the device on this path to be failed immediately.D evice- map p er- mu lt ip at h will continue to use the remaining paths to the device.3. Remove the path from the SCSI subsystem. To do so, use the command echo 1 >/sys/bl o ck/device-name/d evi ce/d el ete where device-name may be sd e, forexample (as described in Procedure 26.7, Ensuring a Clean D evice Removal ).After performing Procedure 26.8, Removing a Path to a Storage D evice , the path can be safelyremoved from the running system. It is not necessary to stop I/O while this is done, as d evicemap p er- mu lt ip at h will re-route I/O to remaining paths according to the configured path groupingand failover policies.Other procedures, such as the physical removal of the cable, followed by a rescan of the SCSI bus tocause the operating system state to be updated to reflect the change, are not recommended. This willcause delays due to I/O timeouts, and devices may be removed unexpectedly. If it is necessary toperform a rescan of an interconnect, it must be done while I/O is paused, as described inSection 26.10, Scanning Storage Interconnects .
In the previous command, h is the HBA number, c is the channel on the HBA, t is the SCSItarget ID , and l is the LUN.
NoteThe older form of this command, echo "scsi ad d -si ng l e-d evi ce 0 0 0 0 " >/pro c/scsi /scsi , is deprecated.
a. In some Fibre Channel hardware, a newly created LUN on the RAID array may not bevisible to the operating system until a Loop Initialization Protocol (LIP) operation isperformed. Refer to Section 26.10, Scanning Storage Interconnects for instructionson how to do this.
ImportantIt will be necessary to stop I/O while this operation is executed if an LIP isrequired.b. If a new LUN has been added on the RAID array but is still not being configured bythe operating system, confirm the list of LUNs being exported by the array using thesg _l uns command, part of the sg3_utils package. This will issue the SC SI R EP O R TLUNS command to the RAID array and return a list of LUNs that are present.For Fibre Channel storage servers that implement a single WWNN for all ports, you candetermine the correct h,c,and t values (i.e. HBA number, HBA channel, and SCSI target ID )by searching for the WWNN in sysfs.
3. After adding all the SCSI paths to the device, execute the mul ti path command, and check tosee that the device has been properly configured. At this point, the device can be added tomd , LVM, mkfs, or mo unt, for example.If the steps above are followed, then a device can safely be added to a running system. It is notnecessary to stop I/O to other devices while this is done. Other procedures involving a rescan (or areset) of the SCSI bus, which cause the operating system to update its state to reflect the currentdevice connectivity, are not recommended while storage I/O is in progress.
This chapter describes how to set up iSCSI interfaces in order to bind a session to a NIC port whenusing software iSCSI. It also describes how to set up interfaces for use with network devices thatsupport offloading.The network subsystem can be configured to determine the path/NIC that iSCSI interfaces should usefor binding. For example, if portals and NICs are set up on different subnets, then it is not necessaryto manually configure iSCSI interfaces for binding.Before attempting to configure an iSCSI interface for binding, run the following command first:$ ping -I ethX target_IPIf pi ng fails, then you will not be able to bind a session to a NIC. If this is the case, check the networksettings first.
i face_nametranspo rt_namehard ware_ad d ressi p_ad d ressnet_i face_name
Set t in g
i ni ti ato r_name
For software iSCSI, each i face configuration must have a unique name (with less than 65characters). The i face_name for network devices that support offloading appears in the formattransport_name. hardware_name.
It is also possible to display the settings of a specific i face configuration in a more friendly way. Todo so, use the option -I iface_name. This will display the settings in the following format:iface.setting = value
As mentioned earlier, an i face configuration is required for each network object that will be used tobind a session.BeforeTo create an i face configuration for software iSCSI, run the following command:# iscsiadm -m iface -I iface_name --op=newThis will create a new empty i face configuration with a specified iface_name. If an existing i faceconfiguration already has the same iface_name, then it will be overwritten with a new, empty one.To configure a specific setting of an i face configuration, use the following command:# iscsiadm -m iface -I iface_name --op=update -n iface.setting -vhw_address
WarningD o not use d efaul t or i ser as i face names. Both strings are special values used byi scsi ad m for backward compatibility. Any manually-created i face configurations namedd efaul t or i ser will disable backwards compatibility.
By default, the i scsi ad m utility will not automatically bind any portals to i face configurations thatuse offloading. This is because such i face configurations will not have i face. transpo rt set totcp. As such, the i face configurations need to be manually bound to discovered portals.It is also possible to prevent a portal from binding to any existing i face. To do so, use d efaul t asthe iface_name, as in:# iscsiadm -m discovery -t st -p IP:port -I default -P 1To remove the binding between a target and i face, use:# iscsiadm -m node -targetname proper_target_name -I iface0 --op=delete [6]To delete all bindings for a specific i face, use:# iscsiadm -m node -I iface_name --op=deleteTo delete bindings for a specific portal (e.g. for Equalogic targets), use:# iscsiadm -m node -p IP:port -I iface_name --op=delete
NoteIf there are no i face configurations defined in /var/l i b/i scsi /i face and the -I optionis not used, i scsi ad m will allow the network subsystem to decide which device a specificportal should use.
However, if the targets do not send an iSCSI async event, you need to manually scan them using thei scsi ad m utility. Before doing so, however, you need to first retrieve the proper --targ etname andthe --po rtal values. If your device model supports only a single logical unit and portal per target,use i scsi ad m to issue a send targ ets command to the host, as in:# iscsiadm -m discovery -t sendtargets -p target_IP:port[5]
ImportantThe send targ ets command used to retrieve --targ etname and --po rtal valuesoverwrites the contents of the /var/l i b/i scsi /no d es database. This database will thenbe repopulated using the settings in /etc/i scsi /i scsi d . co nf. However, this will notoccur if a session is currently logged in and in use.To safely add new targets/portals or delete old ones, use the -o new or -o d el ete options,respectively. For example, to add new targets/portals without overwriting/var/l i b/i scsi /no d es, use the following command:iscsiadm -m discovery -t st -p target_IP -o newTo delete /var/l i b/i scsi /no d es entries that the target did not display during discovery,use:iscsiadm -m discovery -t st -p target_IP -o deleteYou can also perform both tasks simultaneously, as in:iscsiadm -m discovery -t st -p target_IP -o delete -o new
The send targ ets command will yield the following output:ip:port,target_portal_group_tag proper_target_name
At this point, you now have the proper --targ etname and --po rtal values needed to manuallyscan for iSCSI devices. To do so, run the following command:# iscsiadm --mode node --targetname proper_target_name --portalip:port,target_portal_group_tag \ --login [8]
NoteThe proper_target_name and target_IP:port refer to the full name and IP address/portcombination of a target. For more information, refer to Section 26.5.1, iSCSI API andSection 26.13, Scanning iSCSI Interconnects .
NoteIn order to resize an online file system, the file system must not reside on a partitioned device.
ImportantTo re-scan fibre channel logical units on a system that uses multipathing, execute theaforementioned command for each sd device (i.e. sd 1, sd 2, and so on) that represents a pathfor the multipathed logical unit. To determine which devices are paths for a multipath logicalunit, use mul ti path -l l ; then, find the entry that matches the logical unit being resized. It isadvisable that you refer to the WWID of each entry to make it easier to find which one matchesthe logical unit being resized.
NoteYou can also re-scan iSCSI logical units using the following command:# iscsiadm -m node -R -I interfaceReplace interface with the corresponding interface name of the resized logical unit (forexample, i face0 ). This command performs two operations:It scans for new devices in the same way that the command echo "- - -" >/sys/cl ass/scsi _ho st/host/scan does (refer to Section 26.13, Scanning iSCSIInterconnects ).It re-scans for new/modified logical units the same way that the command echo 1 >/sys/bl o ck/sd X/d evi ce/rescan does. Note that this command is the same one usedfor re-scanning fibre-channel logical units.
ImportantD o not use mul ti pathd -k"resi ze map multipath_device" if there are any commandsqueued to multipath_device. That is, do not use this command when the no _path_retryparameter (in /etc/mul ti path. co nf) is set to "q ueue", and there are no active paths tothe device.For more information about multipathing, refer to the Red Hat Enterprise Linux 7 DM Multipath guide.
Read/Write (R/W) to Read-Only (RO), and from RO to R/W. This is typically done through amanagement interface on the storage device. The operating system will not automatically update itsview of the state of the device when a change is made. Follow the procedures described in thischapter to make the operating system aware of the change.Run the following command, replacing XYZ with the desired device designator, to determine theoperating system's current view of the R/W state of a device:# bl o ckd ev --g etro /d ev/sd XYZThe following command is also available for Red Hat Enterprise Linux 7:# cat /sys/bl o ck/sd XYZ/ro 1 = read -o nl y 0 = read -wri teWhen using multipath, refer to the ro or rw field in the second line of output from the mul ti path -l lcommand. For example:36001438005deb4710000500000640000 dm-8 GZ,GZ500[size=20G][features=0][hwhandler=0][ro]\_ round-robin 0 [prio=200][active]\_ 6:0:4:1 sdax 67:16 [active][ready]\_ 6:0:5:1 sday 67:32 [active][ready]\_ round-robin 0 [prio=40][enabled]\_ 6:0:6:1 sdaz 67:48 [active][ready]\_ 6:0:7:1 sdba 67:64 [active][ready]To change the R/W state, use the following procedure:Pro ced u re 26 .10. C h an g e t h e R /W st at e1. To move the device from RO to R/W, see step 2.To move the device from R/W to RO, ensure no further writes will be issued. D o this bystopping the application, or through the use of an appropriate, application-specific action.Ensure that all outstanding write I/Os are complete with the following command:# bl o ckd ev --fl ushbufs /d ev/deviceReplace device with the desired designator; for a device mapper multipath, this is the entry foryour device in d ev/mapper. For example, /d ev/mapper/mpath3.2. Use the management interface of the storage device to change the state of the logical unitfrom R/W to RO, or from RO to R/W. The procedure for this differs with each array. Consultapplicable storage array vendor documentation for more information.3. Perform a re-scan of the device to update the operating system's view of the R/W state of thedevice. If using a device mapper multipath, perform this re-scan for each path to the devicebefore issuing the command telling multipath to reload its device maps.This process is explained in further detail in Section 18.104.22.168, Rescanning logical units .
After modifying the online logical unit Read/Write state, as described in Section 26.15.4, Changingthe Read/Write State of an Online Logical Unit , re-scan the logical unit to ensure the system detectsthe updated state with the following command:# echo 1 > /sys/bl o ck/sd X/d evi ce/rescanTo re-scan logical units on a system that uses multipathing, execute the above command for each sddevice that represents a path for the multipathed logical unit. For example, run the command on sd1,sd2 and all other sd devices. To determine which devices are paths for a multipath unit, usemul ti path -11, then find the entry that matches the logical unit to be changed.
2 6 .1 5 .4 .3. Do cum e nt at io nFurther information can be found in the Red Hat Knowledgebase. To access this, navigate tohttps://www.redhat.com/wapps/sso/login.html?redirect=https://access.redhat.com/knowledge/ and login. Then access the article at https://access.redhat.com/kb/docs/D OC-32850.
The rescan-scsi -bus. sh can only detect the first mapped logical unit if it is LUN0 . Therescan-scsi -bus. sh will not be able to scan any other logical unit unless it detects the firstmapped logical unit even if you use the --no o ptscan option.A race condition requires that rescan-scsi -bus. sh be run twice if logical units are mapped forthe first time. D uring the first scan, rescan-scsi -bus. sh only adds LUN0 ; all other logical unitsare added in the second scan.A bug in the rescan-scsi -bus. sh script incorrectly executes the functionality for recognizing achange in logical unit size when the --remo ve option is used.The rescan-scsi -bus. sh script does not recognize ISCSI logical unit removals.
Typically, the fibre channel class will leave the device as is; i.e. /d ev/sd x will remain /d ev/sd x.This is because the target binding is saved by the fibre channel driver so when the target portreturns, the SCSI addresses are recreated faithfully. However, this cannot be guaranteed; the sd x willbe restored only if no additional change on in-storage box configuration of LUNs is made.
"1 queue_if_no_path"
This ensures that I/O errors are retried and queued if all paths are failed in the d m-mul ti path layer.You may need to adjust iSCSI timers further to better monitor your SAN for problems. Available iSCSItimers you can configure are NOP-Out Interval/Timeouts and repl acement_ti meo ut, which arediscussed in the following sections.
ImportantWhether your considerations are failover speed or security, the recommended value forrepl acement_ti meo ut will depend on other factors. These factors include the network,target, and system workload. As such, it is recommended that you thoroughly test any newconfigurations to repl acements_ti meo ut before applying it to a mission-critical system.
Device St at es
You can also configure timeouts for a specific session and make them non-persistent (instead ofusing /etc/i scsi /i scsi d . co nf). To do so, run the following command (replace the variablesaccordingly):# iscsiadm -m node -T target_name -p target_IP:port -o update -nnode.session.timeo.replacement_timeout -v $timeout_value
ImportantThe configuration described here is recommended for iSCSI sessions involving root partitionaccess. For iSCSI sessions involving access to other types of storage (namely, in systems thatuse d m-mul ti path), refer to Section 26.17.2, iSCSI Settings With d m-mul ti path .
Device StatesTo display the state of a device, use:$ cat /sys/block/device-name/device/stateTo set a device to runni ng state, use:$ echo running > /sys/block/device-name/device/state
Command Timer199
To control the command timer, you can write to /sys/bl o ck/device-name/d evi ce/ti meo ut. Todo so, run:echo value /sys/bl o ck/device-name/d evi ce/ti meo utHere, value is the timeout value (in seconds) you want to implement.
[5] The target_IP and port variab les refer to the IP ad d res s and p o rt c o mb inatio n o f a targ et/p o rtal,res p ec tively. Fo r mo re info rmatio n, refer to Sec tio n 26 .5.1, iSCSI API and Sec tio n 26 .13, Sc anningiSCSI Interc o nnec ts .[6 ] Refer to Sec tio n 26 .13, Sc anning iSCSI Interc o nnec ts fo r info rmatio n o n proper_target_name .[7] Fo r info rmatio n o n ho w to retrieve a s es s io n' s SID value, refer to Sec tio n 26 .5.1, iSCSI API .[8 ] This is a s ing le c o mmand s p lit into multip le lines , to ac c o mmo d ate p rinted and PDF vers io ns o f thisd o c ument. All c o nc atenated lines p rec ed ed b y the b ac ks las h (\) s ho uld b e treated as o nec o mmand , s ans b ac ks las hes .[9 ] Prio r to Red Hat Enterp ris e Linux 5.4, the d efault NO P-O ut req ues ts time o ut was 15 s ec o nd s .
ImportantFor comprehensive information on the deployment and configuration of virtual storageinstances in your environment, refer to the Virtualization Deployment and Administration Guideprovided by Red Hat.
Chapt er 2 7 . Device Mapper Mult ipat hing and Virt ual St orage
ImportantFor comprehensive information on the deployment and configuration of D M-Multipath in yourenvironment, refer to the Using DM-Multipath guide provided by Red Hat.
28.1. What is
The l i bSto rag eMg mt package is a storage array independent Application Programming Interface(API). It provides a stable and consistent API that allows developers the ability to programmaticallymanage different storage arrays and leverage the hardware accelerated features provided.This library is used as a building block for other higher level management tools and applications.End system administrators can also use it as a tool to manually manage storage and automatestorage management tasks with the use of scripts.The l i bSto rag eMg mt package allows operations such as:List storage pools, volumes, access groups, or file systems.Create and delete volumes, access groups, file systems, or NFS exports.Grant and remove access to volumes, access groups, or initiators.Replicate volumes with snapshots, clones, and copies.Create and delete access groups and edit members of a group.Server resources such as CPU and interconnect bandwidth are not utilized because the operationsare all done on the array.The package provides:A stable C and Python API for client application and plug-in developers.A commandl line interface that utilizes the library (l smcl i ).A daemon that executes the plug-in (l smd ).A simulator plug-in that allows the testing of client applications (si m).Plug-in architecture for interfacing with arrays.
WarningThis library and its associated tool have the ability to destroy any and all data located on thearrays it manages. It is highly recommended to develop and test applications and scriptsagainst the storage simulator pulug-in to remove any logic errors before working withproduction systems. Testing applications and scripts on actual non-production hardwarebefore deploying to production is also strongly encouraged if possible.The l i bSto rag eMg mt package in Red Hat Enterprise Linux 7 adds a default udev rule to handle theREPORTED LUNS D ATA HAS CHANGED unit attention.When a storage configuration change has taken place, one of several Unit Attention ASC/ASCQcodes reports the change. A uevent is then generated and is rescanned automatically with sysfs.
The file /l i b/ud ev/rul es. d /9 0 -scsi -ua. rul es contains example rules to enumerate otherevents that the kernel can generate.The l i bSto rag eMg mt library uses a plug-in architecture to accommodate differences in storagearrays. For more information on l i bSto rag eMg mt plug-ins and how to write them, refer to Red Hat'sDeveloper Guide.
D ifferent array vendors and storage standards use different terminology to refer to similarfunctionality. This library uses the following terminology.St o rag e arrayAny storage system that provides block access (FC, FCoE, iSCSI) or file access throughNetwork Attached Storage (NAS).Vo lu meStorage Area Network (SAN) Storage Arrays can expose a volume to the Host Bus Adapter(HBA) over different transports, such as FC, iSCSI, or FCoE. The host OS treats it as blockdevices. One volume can be exposed to many disks if mul ti path[2] is enabled).This is also known as the Logical Unit Number (LUN), StorageVolume with SNIAterminology, or virtual disk.Po o lA group of storage spaces. File systems or volumes can be created from a pool. Pools canbe created from disks, volumes, and other pools. A pool may also hold RAID settings or thinprovisioning settings.This is also known as a StoragePool with SNIA Terminology.Sn ap sh o tA point in time, read only, space efficent copy of data.This is also known as a read only snapshot.C lo n eA point in time, read writeable, space efficent copy of data.This is also known as a read writeable snapshot.CopyA full bitwise copy of the data. It occupies the full space.Mirro rA continuously updated copy (synchronous and asynchronous).Access g ro u pCollections of iSCSI, FC, and FCoE initiators which are granted access to one or morestorage volumes. This ensures that only storage volumes are accessibly by the specifiedinitiators.
To install l i bSto rag eMg mt for use with hardware arrays, select one or more of the appropriateplug-in packages with the following command:$ sudo yum install libstoragemgmt-name-pluginThe following plug-ins that are available include:lib st o rag emg mt - smis- p lu g inGeneric SMI-S array support.lib st o rag emg mt - n et ap p - p lu g inSpecific support for NetApp files.lib st o rag emg mt - n st o r- p lu g inSpecific support for NexentaStor.lib st o rag emg mt - t arg et d - p lu g inSpecific support for targetd.The daemon is then installed and configured to run at start up but will not do so until the next reboot.To use it immediately without rebooting, start the daemon manually.To manage an array requires support through a plug-in. The base install package includes opensource plug-ins for a number of different vendors. Additional plug-in packages will be availableseparately as array support improves. Currently supported hardware is constantly changing andimproving. The project web page has the latest information about which arrays and which specificfeatures of each array are supported, and can be accessed athttp://libstoragemgmt.sourceforge.net/supported_hardware.html.The libStorageMgmt daemon (l smd ) behaves like any standard service for the system.To check on the status of libStorageMgmt use:$ sudo systemctl status libstoragemgmt.serviceTo stop the service use:$ sudo systemctl stop libstoragemgmt.serviceTo start the service use:$ sudo systemctl start libstoragemgmt.service
28.4 . Use of t he
The design of the library provides for a process separation between the client and the plug-in bymeans of inter-process communication (IPC). This prevents bugs in the plug-in from crashing the clientapplication. It also provides a means for plug-in writers to write plug-ins with a license of their ownchoosing. When a client opens the library passing a URI, the client library looks at the URI todetermine which plug-in should be used.The plug-ins are technically stand alone applications but they are designed to have a file descriptorpassed to them on the command line. The client library then opens the appropriate Unix domainsocket which causes the daemon to fork and execute the plug-in. This gives the client library a pointto point communcation channel with the plug-in. The daemon can be restarted without affectingexisting clients. While the client has the library open for that plug-in, the plug-in process is running.After one or more commands are sent and the plug-in is closed, the plug-in process cleans up andthen exits.For a sequence diagram on this process seehttps://sourceforge.net/p/libstoragemgmt/wiki/Architecture/.The default behavior of lsmcli is to wait until the operation is completee. D epending on the requestedoperations, this could potentially could take many hours. To allow a return to normal usage, it ispossible to use the -b option on the command line. If the exit code is 0 the command is completed. Ifthe exit code is 7 the command is in progress and a job identifier is written to standard output. Theuser or script can then take the job ID and query the status of the command as needed by usingl smcl i --jo bstatus JobID. If the job is now completed, the exit value will be 0 and the resultsprinted to standard output. If the command is still in progress, the return value will be 7 and thepercentage complete will be printed to the standard output.
| Block Size
| ...-----+---------------+----------------------------------+------------+----Vol2 | async_created | 855C9BA51991B0CC122A3791996F6B15 | 512|...
For scripting, pass the -t SeparatorCharacters option. This will make it easier to parse theoutput.
Revision HistoryR evisio n 3- 28T h u Jan 22 2015Version for asynchronous release
R evisio n 3- 27T h u Jan 22 2015Several minor changes BZ #1114428
R evisio n 3- 26Wed Jan 21 2015Added overview of Ceph BZ #1136436
R evisio n 3- 25T h u D ec 11 2014Updated the targetcli chapter for 7.1 changes BZ #1119478
R evisio n 3- 24Wed D ec 10 2014Jacq u elyn n EastCorrected a mistake in the targetcli chapter BZ #1078259Clarified several sections in the pNFS section BZ #1165879Removed warning and added paragraph about NM allowing D CB as of RHEL7 BZ #804729Added important note to the fsck section BZ #1031865R evisio n 3- 227.1 Beta
T h u D ec 4 2014
R evisio n 3- 21T u e N o v 18 2014Jacq u elyn n EastMinor edits BZ #1112876Renamed chapter and section titles to remove questions BZ #1099301R evisio n 3- 20T u e N o v 4 2014Jacq u elyn n EastMinor edits BZ #1108154Fixed typographical error BZ #1123026Edited paragraph in SSD D eployment Considerations BZ #1128059R evisio n 3- 19Mo n O ct 27 2014Jacq u elyn n EastRemoved iSCSI target secion that is now covered in the targetcli chapter BZ #1155407R evisio n 3- 17T u e O ct 21 2014Version for asynchronous release
R evisio n 3- 16Mo n O ct 20 2014Several updates to targetcli chapter BZ #1115223
R evisio n 3- 15Mo n O ct 13 2014Jacq u elyn n EastEdited the wording in Behavioral and Administrative D ifferences Between Ext3/4 and XFS chapter forclarity BZ #1097462Multiple edits regarding btrfs BZ #836074R evisio n 3- 13Wed O ct 1 2014Jacq u elyn n EastAdded an extra step to creating a swapfile for added security BZ #1104143R evisio n 3- 12
Wed O ct 1 2014
R evisio n 3- 9Mo n Au g 18 2014More edits to the targetcli chapter BZ #1115223
R evisio n 3- 8Mo n Au g 18 2014Jacq u elyn n EastAdded information about a change in libStorageMgmt BZ #1074699R evisio n 3- 7Fri Au g 8 2014Edited targetcli chapter BZ #1115223
R evisio n 3- 6T u e Ju l 29 2014Edited targetcli chapter BZ #1115223
R evisio n 3- 4T h u Ju l 17 2014Added new chapter on targetcli BZ #1115223
R evisio n 3- 2Fixed broken links
Wed Ju n 11 2014
R evisio n 3- 1Version for 7.0 GA release
T u e Ju n 3 2014
R evisio n 2- 6 1Wed May 21 2014Fixed incorrect screen output in ext4 section BZ #1084303
R evisio n 2- 59Rebuild for style changes
T u e May 13 2014
R evisio n 2- 57Mo n May 06 2014Lau ra B aileyAdding note about the undochange functionality of snapper BZ 959504.R evisio n 2- 56Minor edits
T h u Ap r 10 2014
R evisio n 2- 55Beta refresh
Wed Ap r 9 2014
R evisio n 2- 54Wed Ap r 9 2014Lau ra B aileyCorrected instructions for FCoE target configuration BZ #844815R evisio n 2- 53Wed Ap r 9 2014Fixed file system size supports in ext4 BZ #1050839
R evisio n 2- 52T u e Ap r 8 2014Lau ra B aileyUpdated NFS introduction to reflect support for NFSv4.1 BZ #795247Corrected paths to reflect the movement of /bin, /sbin, /lib, /lib64 under /usr BZ #952478
R evisio n 2- 51T u e Ap r 8 2014Updated BTRFS warnings from beta BZ #1025572
R evisio n 2- 51Mo n Ap r 7 2014Jacq u elyn n EastAdded that md RAID is available as an LVM target BZ #952417Added new snapper chapter BZ #959504Removed NFSoRD MA server instructions and updated client instructions BZ #978110 BZ #1078802Added paragraph about new unified extN driver BZ #795284Fixed the quota commands BZ #1071147Minor edits to File System structure BZ #1084303Stated that bigalloc is not supported BZ #1050839Fixed incorrect references to the Virtualization guides BZ #1070567R evisio n 2- 4 8T h u Ap r 2 2014Jacq u elyn n EastAdded a reference to D eveloper Guide regarding writing plug-ins for libStorageMgmt BZ #795222R evisio n 2- 4 7T u e Ap r 1 2014Removed references to NFSv2 BZ #998954Updated examples in libStorageMgmt BZ #1064035
R evisio n 2- 4 5Fri Mar 28 2014Updated the pNFS chapter BZ #795249, BZ #795250
R evisio n 2- 4 3T h u Mar 27 2014Jacq u elyn n EastRemoved the eCrypt chapter as this is not supported in Red Hat Enerprise Linux 7 BZ #1056152.Updated the overview chapter to include new features in Red Hat Enterprise Linux 7. Also removedold Red Hat Enterprise Linux 6 references. BZ #1044055.R evisio n 2- 4 2T h u Mar 27 2014Jacq u elyn n EastChanged references of default file system from ext4 to xfs BZ #1044059.R evisio n 2- 4 1T u e Mar 25 2014Several changes to SSM chapter BZ #952423.
R evisio n 2- 39Wed Mar 5 2014Minor edit to xfs_metadump description BZ #1068993.
R evisio n 2- 34T u e Mar 4 2014Added SSM chapter BZ #952423.
R evisio n 2- 33Fri Feb 7 2014Minor edits to xfs chapter BZ #1058529.
R evisio n 2- 32Wed Feb 5 2014Several edits in ext3 section BZ #1016367.
R evisio n 2- 31T u e Jan 21 2014Removed system-config-lvm chapter BZ #1055547.
R evisio n 2- 30Mo n Jan 20 2014Added libStorageMgmt chapter BZ #795220.
R evisio n 2- 29Mo n Jan 6 2014Jacq u elyn n EastAdded link to support limits of file systems (Red Hat Enterprise Linux 6 version).R evisio n 2- 27Fri D ec 13 2013Several minor edits BZ #1014902Several minor edits BZ #1006628Several minor edits BZ #795261
R evisio n 2- 25Mo n D ec 9 2013Added info on I/O scheduler performance increase BZ #795292
R evisio n 2- 23T h u D ec 5 2013Added moving from ext4 to xfs section BZ #1006628
R evisio n 2- 22Minor edits BZ #1036919
Wed D ec 4 2013
R evisio n 2- 21Minor edits BZ #1028825Minor edits BZ #1028820Minor edits BZ #1028821Minor edits BZ #1028823
T u e D ec 3 2013
R evisio n 2- 20Pre-beta build
Wed N o v 27 2013
R evisio n 2- 19Fri N o v 22 2013Jacq u elyn n EastAdded important regarding use cases with btrfs to be removed after beta BZ #1007665R evisio n 2- 18T u e N o v 19 2013Added important note regarding xfs_quota BZ #1014902
R evisio n 2- 17Fri N o v 15 2013Added note regarding systemctl stop fcoe.service BZ #860097
R evisio n 2- 16Snapshot Build
T h u N o v 14 2013
R evisio n 2- 15T u e N o v 5 2013Jacq u elyn n EastRemoved suport details about file systems and linked to official article instead BZ #1015936R evisio n 2- 14Mo n N o v 4 2013Jacq u elyn n EastAdded information and corrections to Ext3 and Ext4 chapters BZ #1016367R evisio n 2- 13Wed O ct 30 2013Added btrfs multiple devices section BZ #795261Fixed typo BZ #1007620
R evisio n 2- 11T h u O ct 17 2013Added btrfs SSD optimization section BZ #795268
R evisio n 2- 10
Mo n O ct 14 2013
Added btrfs sections: what is btrfs; creating a btrfs file system; mounting a btrfs file system; resizing abtrfs file system; btrfs references BZ #836074R evisio n 2- 7Wed O ct 9 2013Jacq u elyn n EastAdded sentence saying xattr is enabled by default; removed info on inode64 mounting as this is nowdefault; explained that mkfs. xfs does not exist; fixed xfsrestore command BZ #1015936R evisio n 2- 6T h u Sep 26 2013Jacq u elyn n EastAdded notes to disk quotas chapter to cover XFS and Btrfs BZ #10007620R evisio n 2- 5Wed Sep 25 2013Updated support info for ext4 BZ #795282
R evisio n 2- 3Wed Sep 18 2013Jacq u elyn n EastRearranged chapters to match Red Hat Enterprise Linux 6 version.Added note regarding an ENOSPC write fail error. BZ #911466R evisio n 2- 1T u e Ju n 19 2013Branched for Red Hat Enterprise Linux 7
IndexSymbols/b o o t / d irect o ry, T h e /b o o t / D irect o ry/d ev/d isk- persistent naming, Persistent Naming/d ev/sh m, G at h erin g File Syst em In f o rmat io n/et c/f st ab , C o n vert in g t o an Ext 3 File Syst em, Mo u n t in g N FS File Syst ems u sin g/et c/f st ab , Mo u n t in g a File Syst em/et c/f st ab f ile- enabling disk quotas with, Enabling Quotas/lo cal/d irect o ry ( clien t co n f ig u rat io n , mo u n t in g )- NFS, NFS Client Configuration/p ro c-
/p ro c/d evices- virtual file system (/proc), The /proc Virtual File System/p ro c/f ilesyst ems- virtual file system (/proc), The /proc Virtual File System
/p ro c/md st at- virtual file system (/proc), The /proc Virtual File System/p ro c/mo u n t s- virtual file system (/proc), The /proc Virtual File System/p ro c/mo u n t s/- virtual file system (/proc), The /proc Virtual File System/p ro c/p art it io n s- virtual file system (/proc), The /proc Virtual File System/remo t e/exp o rt ( clien t co n f ig u rat io n , mo u n t in g )- NFS, NFS Client Configuration
AAccess C o n t ro l List s ( see AC Ls)AC Ls-
au t o f s , au t o f s, au t o f s C o n f ig u rat io n- (see also NFS)au t o f s versio n 5- NFS, Improvements in autofs Version 5 over Version 4
Bb acku p /rest o rat io n- XFS, Backup and Restoration of XFS File Systemsb at t ery- b acked writ e cach es- write barriers, Battery-Backed Write Cachesb cu ll ( cach e cu ll limit s set t in g s)- FS-Cache, Setting Cache Cull Limitsb in d in g /u n b in d in g an if ace t o a p o rt al- offload and interface binding- iSCSI, Binding/Unbinding an iface to a Portalb lo ck d evice io ct ls ( u sersp ace access)- I/O alignment and size, Block D evice ioctlsb lo cked d evice, verif yin g- fibre channel- modifying link loss behavior, Fibre Channelb ru n ( cach e cu ll limit s set t in g s)- FS-Cache, Setting Cache Cull Limitsb st o p ( cach e cu ll limit s set t in g s)- FS-Cache, Setting Cache Cull LimitsB t rf s- File System, Btrfs (Technology Preview)
Ccach e b ack- en d- FS-Cache, FS-Cachecach e cu ll limit s- FS-Cache, Setting Cache Cull Limitscach e limit at io n s wit h N FS- FS-Cache, Cache Limitations With NFScach e set u p- FS-Cache, Setting Up a Cachecach e sh arin g- FS-Cache, Cache Sharingcach ef iles
- FS-Cache, FS-Cachecach ef ilesd- FS-Cache, Setting Up a CacheC C W, ch an n el co mman d wo rd- storage considerations during installation, D ASD and zFCP D evices on IBM SystemZch an g in g d ev_lo ss_t mo- fibre channel- modifying link loss behavior, Fibre ChannelC h an g in g t h e read /writ e st at e- Online logical units, Changing the Read/Write State of an Online Logical Unitch an n el co mman d wo rd ( C C W)- storage considerations during installation, D ASD and zFCP D evices on IBM SystemZco h eren cy d at a- FS-Cache, FS-Cacheco mman d t imer ( SC SI)- Linux SCSI layer, Command Timerco mman d s- volume_key, volume_key Commandsco n f ig u rat io n- discovery- iSCSI, iSCSI D iscovery Configurationco n f ig u rin g a t f t p service f o r d iskless clien t s- diskless systems, Configuring a tftp Service for D iskless Clientsco n f ig u rin g an Et h ern et in t erf ace t o u se FC o E- FCoE, Configuring a Fibre-Channel Over Ethernet Interfaceco n f ig u rin g D H C P f o r d iskless clien t s- diskless systems, Configuring D HCP for D iskless Clientsco n f ig u rin g R AID set s- RAID , Configuring RAID Setsco n t ro llin g SC SI co mman d t imer an d d evice st at u s- Linux SCSI layer, Controlling the SCSI Command Timer and D evice Statuscreat in g- ext4, Creating an Ext4 File System- XFS, Creating an XFS File Systemcu mu lat ive mo d e ( xf srest o re)- XFS, Cumulative Mode for xfsrestore
DD ASD an d z FC P d evices o n IB M Syst em z- storage considerations during installation, D ASD and zFCP D evices on IBM SystemZd eb u g f s ( o t h er ext 4 f ile syst em u t ilit ies)- ext4, Other Ext4 File System Utilitiesd ep lo ymen t- solid state disks, D eployment Considerationsd ep lo ymen t g u id elin es- solid state disks, Solid-State D isk D eployment Guidelinesd et ermin in g remo t e p o rt st at es- fibre channel- modifying link loss behavior, Fibre Channeld ev d irect o ry, T h e /d ev/ D irect o ryd evice st at u s- Linux SCSI layer, D evice Statesd evice- map p er mu lt ip at h in g , D M- Mu lt ip at hd evices, remo vin g , R emo vin g a St o rag e D eviced ev_lo ss_t mo- fibre channel- modifying link loss behavior, Fibre Channel- fibre channel API, Fibre Channel APId ev_lo ss_t mo , ch an g in g- fibre channel- modifying link loss behavior, Fibre Channeld f , G at h erin g File Syst em In f o rmat io nD H C P, co n f ig u rin g- diskless systems, Configuring D HCP for D iskless ClientsD IF/D IX- en ab led b lo ck d evices- storage considerations during installation, Block D evices with D IF/D IX Enabledd irect map su p p o rt ( au t o f s versio n 5)- NFS, Improvements in autofs Version 5 over Version 4d irect o ries- /boot/, The /boot/ D irectory- /dev/, The /dev/ D irectory- /etc/, The /etc/ D irectory- /media/, The /media/ D irectory- /mnt/, The /mnt/ D irectory- /opt/, The /opt/ D irectory- /proc/, The /proc/ D irectory- /srv/, The /srv/ D irectory
Ee2f sck, R evert in g t o an Ext 2 File Syst eme2imag e ( o t h er ext 4 f ile syst em u t ilit ies)- ext4, Other Ext4 File System Utilitiese2lab el- ext4, Other Ext4 File System Utilitiese2lab el ( o t h er ext 4 f ile syst em u t ilit ies)- ext4, Other Ext4 File System Utilitiesen ab lin d /d isab lin g- write barriers, Enabling/D isabling Write Barriersen h an ced LD AP su p p o rt ( au t o f s versio n 5)- NFS, Improvements in autofs Version 5 over Version 4erro r messag es- write barriers, Enabling/D isabling Write Barrierset c d irect o ry, T h e /et c/ D irect o ryexp ert mo d e ( xf s_q u o t a)- XFS, XFS Quota Managementexp o rt ed f ile syst ems- diskless systems, Configuring an Exported File System for D iskless Clientsext 2- reverting from ext3, Reverting to an Ext2 File Systemext 3- converting from ext2, Converting to an Ext3 File System- creating, Creating an Ext3 File System- features, The Ext3 File Systemext 4-
Ff ast _io _f ail_t mo- fibre channel API, Fibre Channel APIFC o E- configuring an Ethernet interface to use FCoE, Configuring a Fibre-Channel OverEthernet Interface- fibre channel over ethernet, Configuring a Fibre-Channel Over Ethernet Interface- required packages, Configuring a Fibre-Channel Over Ethernet InterfaceFH S, O verview o f Filesyst em H ierarch y St an d ard ( FH S) , FH S O rg an iz at io n- (see also file system)f ib re ch an n el- online storage, Fibre Channelf ib re ch an n el API, Fib re C h an n el APIf ib re ch an n el d rivers ( n at ive) , N at ive Fib re C h an n el D rivers an d C ap ab ilit iesf ib re ch an n el o ver et h ern et- FCoE, Configuring a Fibre-Channel Over Ethernet Interfacef ile syst em- FHS standard, FHS Organization- hierarchy, Overview of Filesystem Hierarchy Standard (FHS)- organization, FHS Organization- structure, File System Structure and MaintenanceFile Syst em- Btrfs, Btrfs (Technology Preview)f ile syst em t yp es- ext4, The Ext4 File System- GFS2, Global File System 2- XFS, The XFS File Systemf ile syst ems, G at h erin g File Syst em In f o rmat io n- ext2 (see ext2)- ext3 (see ext3)f in d mn t ( co mman d )- listing mounts, Listing Currently Mounted File SystemsFS- C ach e- bcull (cache cull limits settings), Setting Cache Cull Limits
f syn c( )- ext4, The Ext4 File System- XFS, The XFS File System
Gg et f acl , R et rievin g AC LsG FS2- file system types, Global File System 2- gfs2.ko, Global File System 2- maximum size, Global File System 2G FS2 f ile syst em maximu m siz e, G lo b al File Syst em 2g f s2.ko- GFS2, Global File System 2G lo b al File Syst em 2- file system types, Global File System 2- gfs2.ko, Global File System 2- maximum size, Global File System 2g q u o t a/g q n o en f o rce- XFS, XFS Quota Management
HH ard ware R AID ( see R AID )h ard ware R AID co n t ro ller d rivers- RAID , Linux Hardware RAID controller driversh ierarch y, f ile syst em, O verview o f Filesyst em H ierarch y St an d ard ( FH S)h ig h - en d arrays- write barriers, High-End Arraysh o st- fibre channel API, Fibre Channel APIh o w writ e b arriers wo rk- write barriers, How Write Barriers Work
II/O alig n men t an d siz e, St o rag e I/O Alig n men t an d Siz e- ATA standards, ATA- block device ioctls (userspace access), Block D evice ioctls- Linux I/O stack, Storage I/O Alignment and Size- logical_block_size, Userspace Access- LVM, Logical Volume Manager- READ CAPACITY(16), SCSI- SCSI standards, SCSI- stacking I/O parameters, Stacking I/O Parameters- storage access parameters, Parameters for Storage Access- sysfs interface (userspace access), sysfs Interface- tools (for partitioning and other file system functions), Partition and File SystemTools- userspace access, Userspace AccessI/O p aramet ers st ackin g- I/O alignment and size, Stacking I/O ParametersI/O sch ed u ler ( t u n in g )- solid state disks, I/O Schedulerif ace ( co n f ig u rin g f o r iSC SI o f f lo ad )- offload and interface binding- iSCSI, Configuring an iface for iSCSI Offloadif ace b in d in g /u n b in d in g- offload and interface binding- iSCSI, Binding/Unbinding an iface to a Portalif ace co n f ig u rat io n s, viewin g- offload and interface binding- iSCSI, Viewing Available iface Configurationsif ace f o r so f t ware iSC SI- offload and interface binding- iSCSI, Configuring an iface for Software iSCSIif ace set t in g s- offload and interface binding- iSCSI, Viewing Available iface Configurationsimp o rt an ce o f writ e b arriers- write barriers, Importance of Write Barriersin creasin g f ile syst em siz e- XFS, Increasing the Size of an XFS File Systemin d exin g keys- FS-Cache, FS-Cachein d ivid u al u ser- volume_key, Using volume_key as an individual user
iSC SI ro o t- iSCSI configuration, iSCSI Rootissu e_lip- fibre channel API, Fibre Channel API
Kkn o wn issu es- adding/removing- LUN (logical unit number), Known Issues With rescan-scsi-bus.sh
Llaz y mo u n t /u n mo u n t su p p o rt ( au t o f s versio n 5)- NFS, Improvements in autofs Version 5 over Version 4levels- RAID , RAID Levels and Linear Supportlimit ( xf s_q u o t a exp ert mo d e)- XFS, XFS Quota Managementlin ear R AID- RAID , RAID Levels and Linear SupportLin u x I/O st ack- I/O alignment and size, Storage I/O Alignment and Sizelo g g in g in- iSCSI targets, Logging in to an iSCSI Targetlo g ical_b lo ck_siz e- I/O alignment and size, Userspace AccessLU K S/d m- cryp t , en cryp t in g b lo ck d evices u sin g- storage considerations during installation, Encrypting Block D evices Using LUKSLU N ( lo g ical u n it n u mb er)- adding/removing, Adding/Removing a Logical Unit Through rescan-scsi-bus.sh- known issues, Known Issues With rescan-scsi-bus.sh- required packages, Adding/Removing a Logical Unit Through rescan-scsibus.sh- rescan-scsi-bus.sh, Adding/Removing a Logical Unit Through rescan-scsibus.shLVM- I/O alignment and size, Logical Volume Manager
Mmain f eat u res- ext4, The Ext4 File System- XFS, The XFS File Systemmaximu m siz e- GFS2, Global File System 2
Nn at ive f ib re ch an n el d rivers, N at ive Fib re C h an n el D rivers an d C ap ab ilit iesn et wo rk b o o t in g service- diskless systems, Setting Up A Remote D iskless SystemN et wo rk File Syst em ( see N FS)
- autofs- augmenting, Overriding or Augmenting Site Configuration Files- configuration, autofs Configuration- LD AP, Using LD AP to Store Automounter Maps- autofs version 5, Improvements in autofs Version 5 over Version 4- client- autofs , autofs- configuration, NFS Client Configuration- mount options, Common NFS Mount Options- condrestart, Starting and Stopping NFS- configuration with firewall, Running NFS Behind a Firewall- direct map support (autofs version 5), Improvements in autofs Version 5 over Version4- enhanced LD AP support (autofs version 5), Improvements in autofs Version 5 overVersion 4- FS-Cache, Using the Cache With NFS- hostname formats, Hostname Formats- how it works, How NFS Works- introducing, Network File System (NFS)- lazy mount/unmount support (autofs version 5), Improvements in autofs Version 5over Version 4- mount (client configuration), NFS Client Configuration- multiple master map entries per autofs mount point (autofs version 5), Improvementsin autofs Version 5 over Version 4- options (client configuration, mounting), NFS Client Configuration- overriding/augmenting site configuration files (autofs), autofs Configuration- proper nsswitch configuration (autofs version 5), use of, Improvements in autofsVersion 5 over Version 4- RD MA, NFS over RD MA- reloading, Starting and Stopping NFS- required services, Required Services- restarting, Starting and Stopping NFS- rfc2307bis (autofs), Using LD AP to Store Automounter Maps- rpcbind , NFS and rpcbind- security, Securing NFS- file permissions, File Permissions- NFSv3 host access, NFS Security with AUTH_SYS and export controls- NFSv4 host access, NFS security with AUTH_GSS- server (client configuration, mounting), NFS Client Configuration- server configuration, NFS Server Configuration- /etc/exports , The /etc/exports Configuration File- exportfs command, The exportfs Command- exportfs command with NFSv4, Using exportfs with NFSv4- starting, Starting and Stopping NFS
Oo f f lin e st at u s- Linux SCSI layer, Controlling the SCSI Command Timer and D evice Statuso f f lo ad an d in t erf ace b in d in g- iSCSI, Configuring iSCSI Offload and Interface BindingO n lin e lo g ical u n it s- Changing the read/write state, Changing the Read/Write State of an Online LogicalUnito n lin e st o rag e- fibre channel, Fibre Channel- overview, Online Storage Management- sysfs, Online Storage Management- troubleshooting, Online Storage Configuration Troubleshootingo p t d irect o ry, T h e /o p t / D irect o ryo p t io n s ( clien t co n f ig u rat io n , mo u n t in g )- NFS, NFS Client Configurationo t h er f ile syst em u t ilit ies- ext4, Other Ext4 File System Utilitieso verrid in g /au g men t in g sit e co n f ig u rat io n f iles ( au t o f s)- NFS, autofs Configuration
o verview, O verview- online storage, Online Storage Management
PParallel N FS- pNFS, pNFSp aramet ers f o r st o rag e access- I/O alignment and size, Parameters for Storage Accessp arit y- RAID , RAID Levels and Linear Supportp art ed , Part it io n s- creating partitions, Creating a Partition- overview, Partitions- removing partitions, Removing a Partition- resizing partitions, Resizing a Partition- selecting device, Viewing the Partition Table- table of commands, Partitions- viewing partition table, Viewing the Partition Tablep art it io n t ab le- viewing, Viewing the Partition Tablep art it io n s- creating, Creating a Partition- formatting- mkfs , Formatting and Labeling the Partition- making- mkpart , Making the Partition- removing, Removing a Partition- resizing, Resizing a Partition- viewing list, Viewing the Partition Tablep at h t o st o rag e d evices, ad d in g , Ad d in g a St o rag e D evice o r Pat hp at h t o st o rag e d evices, remo vin g , R emo vin g a Pat h t o a St o rag e D evicep erf o rman ce g u aran t ee- FS-Cache, Performance Guaranteep ersist en t n amin g , Persist en t N amin gp N FS- Parallel NFS, pNFSp o rt st at es ( remo t e) , d et ermin in g- fibre channel- modifying link loss behavior, Fibre Channelp q u o t a/p q n o en f o rce- XFS, XFS Quota Managementp rivat e mo u n t , Sh arin g Mo u n t s
Qq u eu e_if _n o _p at h- iSCSI configuration, iSCSI Settings With dm-multipath- modifying link loss- iSCSI configuration, replacement_timeoutq u o t a ( o t h er ext 4 f ile syst em u t ilit ies)- ext4, Other Ext4 File System Utilitiesq u o t a man ag emen t- XFS, XFS Quota Managementq u o t ach eck , C reat in g t h e Q u o t a D at ab ase Filesq u o t ach eck co mman d- checking quota accuracy with, Keeping Quotas Accurateq u o t ao f f , En ab lin g an d D isab lin gq u o t ao n , En ab lin g an d D isab lin g
R D MA- NFS, NFS over RD MA
R EAD C APAC IT Y( 16 )- I/O alignment and size, SCSIreco rd t yp es- discovery- iSCSI, iSCSI D iscovery ConfigurationR ed H at En t erp rise Lin u x- sp ecif ic f ile lo cat io n s- /etc/sysconfig/, Special Red Hat Enterprise Linux File Locations- (see also sysconfig directory)- /var/cache/yum, Special Red Hat Enterprise Linux File Locations- /var/lib/rpm/, Special Red Hat Enterprise Linux File Locationsremo t e d iskless syst ems- diskless systems, Setting Up A Remote D iskless Systemremo t e p o rt- fibre channel API, Fibre Channel APIremo t e p o rt st at es, d et ermin in g- fibre channel- modifying link loss behavior, Fibre Channelremo vin g d evices, R emo vin g a St o rag e D eviceremo vin g p at h s t o a st o rag e d evice, R emo vin g a Pat h t o a St o rag e D evicerep airin g f ile syst em- XFS, Repairing an XFS File Systemrep airin g XFS f ile syst ems wit h d irt y lo g s- XFS, Repairing an XFS File Systemrep lacemen t _t imeo u t- modifying link loss- iSCSI configuration, SCSI Error Handler, replacement_timeoutrep lacemen t _t imeo u t M- iSCSI configuration, iSCSI Rootrep o rt ( xf s_q u o t a exp ert mo d e)- XFS, XFS Quota Managementreq u ired p ackag es- adding/removing- LUN (logical unit number), Adding/Removing a Logical Unit Throughrescan-scsi-bus.sh- diskless systems, Setting Up A Remote D iskless System- FCoE, Configuring a Fibre-Channel Over Ethernet Interfacerescan - scsi- b u s.sh- adding/removing- LUN (logical unit number), Adding/Removing a Logical Unit Throughrescan-scsi-bus.sh
Sscan n in g in t erco n n ect s- iSCSI, Scanning iSCSI Interconnectsscan n in g st o rag e in t erco n n ect s, Scan n in g St o rag e In t erco n n ect sSC SI co mman d t imer- Linux SCSI layer, Command TimerSC SI Erro r H an d ler- modifying link loss- iSCSI configuration, SCSI Error HandlerSC SI st an d ard s- I/O alignment and size, SCSIsep arat e p art it io n s ( f o r /h o me, /o p t , /u sr/lo cal)- storage considerations during installation, Separate Partitions for /home, /opt,/usr/localserver ( clien t co n f ig u rat io n , mo u n t in g )- NFS, NFS Client Configurationset f acl , Set t in g Access AC Ls
set t in g u p a cach e- FS-Cache, Setting Up a Cachesh ared mo u n t , Sh arin g Mo u n t ssh ared su b t rees, Sh arin g Mo u n t s- private mount, Sharing Mounts- shared mount, Sharing Mounts- slave mount, Sharing Mounts- unbindable mount, Sharing Mountssimp le mo d e ( xf srest o re)- XFS, Simple Mode for xfsrestoreslave mo u n t , Sh arin g Mo u n t sso f t ware iSC SI- iSCSI, Configuring an iface for Software iSCSI- offload and interface binding- iSCSI, Configuring an iface for Software iSCSISo f t ware R AID ( see R AID )so lid st at e d isks- deployment, D eployment Considerations- deployment guidelines, Solid-State D isk D eployment Guidelines- I/O scheduler (tuning), I/O Scheduler- SSD , Solid-State D isk D eployment Guidelines- swap (tuning), Swap- throughput classes, Solid-State D isk D eployment Guidelines- TRIM command, Solid-State D isk D eployment Guidelines- tuning, Tuning Considerations- virtual memory (tuning), Virtual Memorysp ecif ic sessio n t imeo u t s, co n f ig u rin g- iSCSI configuration, Configuring Timeouts for a Specific Sessionsrv d irect o ry, T h e /srv/ D irect o rySSD- solid state disks, Solid-State D isk D eployment GuidelinesSSM- System Storage Manager, System Storage Manager (SSM)- add command, Add- Backends, SSM Backends- BTRFS Backend, BTRFS Backend- check command, Check- Commands, SSM Commands- create command, Create- crypt, Crypt- Installation, SSM Installation- list command, List- LVM Backend, LVM Backend- mount command, Mount- References, SSM References- remove command, Remove- resize command, Resize
sw ( mkf s.xf s su b - o p t io n s)- XFS, Creating an XFS File Systemswap ( t u n in g )- solid state disks, Swapswap sp ace, Swap Sp ace- creating, Adding Swap Space- expanding, Adding Swap Space- file- creating, Creating a Swap File, Removing a Swap File- LVM2-
Tt arg et s- iSCSI, Logging in to an iSCSI Targett f t p service, co n f ig u rin g- diskless systems, Configuring a tftp Service for D iskless Clientst h ro u g h p u t classes- solid state disks, Solid-State D isk D eployment Guidelinest imeo u t s f o r a sp ecif ic sessio n , co n f ig u rin g- iSCSI configuration, Configuring Timeouts for a Specific Sessiont o o ls ( f o r p art it io n in g an d o t h er f ile syst em f u n ct io n s)- I/O alignment and size, Partition and File System Toolst rackin g st at ist ical in f o rmat io n- FS-Cache, Statistical Informationt ran sp o rt- fibre channel API, Fibre Channel APIT R IM co mman d- solid state disks, Solid-State D isk D eployment Guidelinest ro u b lesh o o t in g- online storage, Online Storage Configuration Troubleshootingt ro u b lesh o o t in g N FS an d rp cb in d- NFS, Troubleshooting NFS and rpcbindt u n e2f s- converting to ext3 with, Converting to an Ext3 File System- reverting to ext2 with, Reverting to an Ext2 File Systemt u n e2f s ( mo u n t in g )- ext4, Mounting an Ext4 File Systemt u n e2f s ( set t in g u p a cach e)- FS-Cache, Setting Up a Cachet u n in g- solid state disks, Tuning Considerations
Uu d ev- persistent naming, WWIDu d ev ru le ( t imeo u t )- command timer (SCSI), Command Timeru mo u n t , U n mo u n t in g a File Syst emu n b in d ab le mo u n t , Sh arin g Mo u n t s
U n iversally U n iq u e Id en t if ier ( U U ID )- persistent naming, UUID and Other Persistent Identifiersu n mo u n t in g , U n mo u n t in g a File Syst emu p d at es- storage considerations during installation, Storage Considerations D uringInstallationu q u o t a/u q n o en f o rce- XFS, XFS Quota Managementu sersp ace access- I/O alignment and size, Userspace Accessu sersp ace API f iles- fibre channel API, Fibre Channel APIu sr d irect o ry, T h e /u sr/ D irect o ryU U ID- persistent naming, UUID and Other Persistent Identifiers
Vvar d irect o ry, T h e /var/ D irect o ryvar/lib /rp m/ d irect o ry, Sp ecial R ed H at En t erp rise Lin u x File Lo cat io n svar/sp o o l/u p 2d at e/ d irect o ry, Sp ecial R ed H at En t erp rise Lin u x File Lo cat io n sverif yin g if a d evice is b lo cked- fibre channel- modifying link loss behavior, Fibre Channelversio n- what is new- autofs, Improvements in autofs Version 5 over Version 4viewin g availab le if ace co n f ig u rat io n s- offload and interface binding- iSCSI, Viewing Available iface Configurationsvirt u al f ile syst em ( /p ro c)- /proc/devices, The /proc Virtual File System- /proc/filesystems, The /proc Virtual File System- /proc/mdstat, The /proc Virtual File System- /proc/mounts, The /proc Virtual File System- /proc/mounts/, The /proc Virtual File System- /proc/partitions, The /proc Virtual File Systemvirt u al memo ry ( t u n in g )- solid state disks, Virtual Memoryvirt u al st o rag e, Virt u al St o rag evo lu me_key- commands, volume_key Commands- individual user, Using volume_key as an individual user
24 0
Wwh at ' s n ew- storage considerations during installation, Storage Considerations D uringInstallationWo rld Wid e Id en t if ier ( WWID )- persistent naming, WWIDwrit e b arriers- battery-backed write caches, Battery-Backed Write Caches- definition, Write Barriers- disabling write caches, D isabling Write Caches- enablind/disabling, Enabling/D isabling Write Barriers- error messages, Enabling/D isabling Write Barriers- ext4, Mounting an Ext4 File System- high-end arrays, High-End Arrays- how write barriers work, How Write Barriers Work- importance of write barriers, Importance of Write Barriers- NFS, NFS- XFS, Write Barrierswrit e cach es, d isab lin g- write barriers, D isabling Write CachesWWID- persistent naming, WWID
24 1
xf sd u mp- XFS, Backup and Restoration of XFS File Systemsxf sp ro g s- XFS, Suspending an XFS File Systemxf srest o re- XFS, Backup and Restoration of XFS File Systemsxf s_ad min- XFS, Other XFS File System Utilitiesxf s_b map- XFS, Other XFS File System Utilitiesxf s_co p y- XFS, Other XFS File System Utilitiesxf s_d b- XFS, Other XFS File System Utilitiesxf s_f reez e- XFS, Suspending an XFS File Systemxf s_f sr- XFS, Other XFS File System Utilitiesxf s_g ro wf s- XFS, Increasing the Size of an XFS File Systemxf s_in f o- XFS, Other XFS File System Utilitiesxf s_md rest o re- XFS, Other XFS File System Utilities
24 2
xf s_met ad u mp- XFS, Other XFS File System Utilitiesxf s_q u o t a- XFS, XFS Quota Managementxf s_rep air- XFS, Repairing an XFS File System
24 3