{"id":5996,"date":"2021-07-18T16:54:14","date_gmt":"2021-07-18T16:54:14","guid":{"rendered":"https:\/\/really.zonky.org\/?p=5996"},"modified":"2021-07-18T16:54:14","modified_gmt":"2021-07-18T16:54:14","slug":"zfs-replacing-a-functional-mirrored-disk","status":"publish","type":"post","link":"https:\/\/really.zonky.org\/?p=5996","title":{"rendered":"ZFS: Replacing A Functional Mirrored Disk"},"content":{"rendered":"\n

This is a procedure to replace one working<\/em> drive in a fully functional mirror vdev; if you are replacing a failed disk there is no advantages in following this procedure. Although if you have a somewhat<\/em> functional disk it may be worth trying. <\/p>\n\n\n\n

So why not simply yank out the working disk you want to replace? Well, you can of course and that would work but there is nothing Murphy likes more than a mirrored vdev temporarily down to a single disk – resilvering onto a new disk guarantees<\/em> a higher chance of failure of the previously working disk (I have actually seen this happening). <\/p>\n\n\n\n

So I’m going to describe how to make a three-way mirror with three disks and then detach the disk you wanted to replace. <\/p>\n\n\n\n

To do this there are some prerequisites :-<\/p>\n\n\n\n

  1. You will need space to install an additional disk into your system; perhaps temporarily in an “unsuitable” location.<\/li>
  2. You will need a spare SATA controller port to plug the new disk into. If necessary with an additional PCIe SATA controller (which sounds expensive but safety is worth the cost).<\/li>
  3. You will need a SATA data cable and a SATA power cable. <\/li><\/ol>\n\n\n\n

    The first step is to make very careful note of what devices you are going to “swap over” – ideally using their WWNs. If you don’t use WWNs, sorting out which disk is which is going to be a bit trickier. <\/p>\n\n\n\n

    The second step is to practice the steps involved using a ‘fake’ storage pool backed up by tiny disk files :-<\/p>\n\n\n\n

    # cd \/pool1\/temp\n# for w in one two three\ndo\n  dd if=\/dev\/zero of=test-disk-${w}.img bs=1M count=1000\ndone\n# zpool create test mirror \/pool1\/temp\/test-disk-one.img \/pool1\/temp\/test-disk-two.img\n# zpool attach test \/pool1\/temp\/test-disk-one.img \/pool\/temp\/test-disk-three.img\n# zpool detach test \/pool1\/temp\/test-disk-one.img<\/code><\/pre>\n\n\n\n
    That’s pretty much it in a nutshell.<\/p>\n\n\n\n
    The real process is a bit more disturbing of course and most of the work is physical. The first difference from practice is that when you attach<\/em> the new disk to one or other of the existing devices within the mirror, you will have to wait until the resilvering process is complete. <\/p>\n\n\n\n
    Whilst you will receive an estimate for that if you run zpool status<\/em>, the estimate that you get :-<\/p>\n\n\n\n
      scan: resilver in progress since Sun Jul 18 08:20:54 2021\n\t8.25T scanned at 1.09G\/s, 7.28T issued at 981M\/s, 8.25T total\n\t995G resilvered, 88.23% done, 0 days 00:17:16 to go<\/code><\/pre>\n\n\n\n
    (Only showing the relevant part as the full output from my system is confusing and deceptive)<\/p>\n\n\n\n
    Is wildly inaccurate – partially because the resilvering process takes second place to any ordinary file system activity. My own estimate (1 hour per Tbyte) is probably also wildly inaccurate; basically it is done when it is done. <\/p>\n\n\n\n
    Detaching the old device is fast – you won’t need to sit down to wait for it. <\/p>\n","protected":false},"excerpt":{"rendered":"
    This is a procedure to replace one working drive in a fully functional mirror vdev; if you are replacing a failed disk there is no advantages in following this procedure. Although if you have a somewhat functional disk it may be worth trying. So why not simply yank out the working disk you want to […]<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"_share_on_mastodon":"0"},"categories":[4,209,226],"tags":[2034,2033,88],"class_list":["post-5996","post","type-post","status-publish","format-standard","hentry","category-it","category-linux-it","category-working-notes","tag-mirror","tag-resilvering","tag-zfs","category-4-id","category-209-id","category-226-id","post-seq-1","post-parity-odd","meta-position-corners","fix"],"share_on_mastodon":{"url":"","error":""},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p1f2KI-1yI","_links":{"self":[{"href":"https:\/\/really.zonky.org\/index.php?rest_route=\/wp\/v2\/posts\/5996","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/really.zonky.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/really.zonky.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/really.zonky.org\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/really.zonky.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=5996"}],"version-history":[{"count":3,"href":"https:\/\/really.zonky.org\/index.php?rest_route=\/wp\/v2\/posts\/5996\/revisions"}],"predecessor-version":[{"id":5999,"href":"https:\/\/really.zonky.org\/index.php?rest_route=\/wp\/v2\/posts\/5996\/revisions\/5999"}],"wp:attachment":[{"href":"https:\/\/really.zonky.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=5996"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/really.zonky.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=5996"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/really.zonky.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=5996"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}