{"action":"created","issue":{"url":"https://api.github.com/repos/zfsonlinux/zfs/issues/5018","repository_url":"https://api.github.com/repos/zfsonlinux/zfs","labels_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/5018/labels{/name}","comments_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/5018/comments","events_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/5018/events","html_url":"https://github.com/zfsonlinux/zfs/issues/5018","id":172900339,"number":5018,"title":"Random checksum errors on raidz1 and mirror pools -- It's not a hardware fault","user":{"login":"JuliaVixen","id":19146207,"avatar_url":"https://avatars.githubusercontent.com/u/19146207?v=3","gravatar_id":"","url":"https://api.github.com/users/JuliaVixen","html_url":"https://github.com/JuliaVixen","followers_url":"https://api.github.com/users/JuliaVixen/followers","following_url":"https://api.github.com/users/JuliaVixen/following{/other_user}","gists_url":"https://api.github.com/users/JuliaVixen/gists{/gist_id}","starred_url":"https://api.github.com/users/JuliaVixen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JuliaVixen/subscriptions","organizations_url":"https://api.github.com/users/JuliaVixen/orgs","repos_url":"https://api.github.com/users/JuliaVixen/repos","events_url":"https://api.github.com/users/JuliaVixen/events{/privacy}","received_events_url":"https://api.github.com/users/JuliaVixen/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":24,"created_at":"2016-08-24T09:02:35Z","updated_at":"2017-02-06T04:41:59Z","closed_at":null,"body":"I was backing up a pool (with `zfs send|recv`), and it happened again. Two checksum errors on a raidz1 pool, with zero checksum errors on any device. Against my own paranoid judgement, I imported this pool read-write, rather than read-only like I usually do, so i can't reboot, swap the drives around, reimport it, and see if it faults on the same file at the same offset... but, I did do that last time, twice, and the fault was always in the exact same spot. I'm using ECC memory and I just tested it for several hours a few days ago. There are no error messages in Linux `dmesg`, no SMART reports, everything claims to be functioning without error. (And these are _not_ the crappy Seagate drives.)\n\nSo... How did this checksum error happen?\n\nAbout month ago, I copied most of the files off this pool with a regular `cp -avi` copy, rather than a `zfs send|recv`, there was no checksum error at that time. The disks have been mostly powered off since then, and I only plugged them back in today to make sure I really got all the data off them.\n\nSo, what do I need to do with `zdb` to debug this, and is anyone else noticing the same thing?\n\nI have about 50T of corrupt zpools now, which I need to \"restore the entire pool from backup\" because of checksums errors on the top level vdev, when the actual devices themselves are fine, and there's at least an extra disk of parity! How does this happen? \n\nI think this has only started happening this month, so maybe there was a Github commit in July that did something?\n\n```\nlocalhost ~ # zpool status -v n\n  pool: n\n state: ONLINE\nstatus: One or more devices has experienced an error resulting in data\n    corruption.  Applications may be affected.\naction: Restore the file in question if possible.  Otherwise restore the\n    entire pool from backup.\n   see: http://zfsonlinux.org/msg/ZFS-8000-8A\n  scan: none requested\nconfig:\n\n    NAME                                   STATE     READ WRITE CKSUM\n    n                                      ONLINE       0     0     1\n      raidz1-0                             ONLINE       0     0     2\n        ata-WDC_WD80EFZX-68UW8N0_VKGNH8BX  ONLINE       0     0     0\n        ata-WDC_WD80EFZX-68UW8N0_VKHJK9ZX  ONLINE       0     0     0\n        ata-WDC_WD80EFZX-68UW8N0_VKHNJWBX  ONLINE       0     0     0\n\nerrors: Permanent errors have been detected in the following files:\n\n        n@Aug_23_2016:/backed_up/ST31000340AS_9QJ0J2GY_Reiserfs.img\n```\n\n```\nlocalhost ~ # zpool history n\nHistory for 'n':\n2016-05-19.08:41:14 zpool create -O atime=off -o ashift=12 -o feature@lz4_compress=enabled -o feature@embedded_data=enabled n -f raidz1 /dev/disk/by-id/ata-WDC_WD80EFZX-68UW8N0_VKGNH8BX /dev/disk/by-id/ata-WDC_WD80EFZX-68UW8N0_VKHJK9ZX /dev/disk/by-id/ata-WDC_WD80EFZX-68UW8N0_VKHNJWBX\n2016-05-19.13:51:10 zfs recv -evF n\n2016-05-19.23:59:16 zfs recv -evF n\n2016-05-20.00:12:53 zfs recv -evF n\n2016-05-20.03:22:21 zpool export n\n2016-06-10.06:27:22 zpool import n\n2016-06-10.06:41:15 zfs create n/Seagate_750G_Photos_Sep_2007\n2016-06-12.08:10:03 zfs create n/Seagate_750G_copied_Mar_2009\n2016-06-13.04:13:54 zfs create n/backedup_SSD\n2016-06-14.01:24:08 zfs create n/NEFs\n2016-06-24.03:54:36 zpool export n\n2016-07-17.01:16:48 zpool import n\n2016-07-17.05:00:20 zpool export n\n2016-08-03.10:37:48 zpool import n\n2016-08-03.10:38:29 zfs destroy n/backedup_SSD\n2016-08-03.10:40:03 zfs snapshot n/Seagate_750G_Photos_Sep_2007@send\n2016-08-03.10:40:20 zfs snapshot n/Seagate_750G_copied_Mar_2009@send\n2016-08-03.10:41:04 zpool export n\n2016-08-11.06:48:17 zpool import n\n2016-08-11.20:00:57 zpool export n\n2016-08-24.05:14:46 zpool import n\n2016-08-24.05:15:45 zfs snapshot -r n@Aug_23_2016\n```\n\n```\nlocalhost ~ # zfs get all n  \nNAME  PROPERTY              VALUE                  SOURCE\nn     type                  filesystem             -\nn     creation              Thu May 19  8:41 2016  -\nn     used                  14.0T                  -\nn     available             82.1G                  -\nn     referenced            5.63T                  -\nn     compressratio         1.00x                  -\nn     mounted               yes                    -\nn     quota                 none                   default\nn     reservation           none                   default\nn     recordsize            128K                   default\nn     mountpoint            /n                     default\nn     sharenfs              off                    default\nn     checksum              on                     default\nn     compression           off                    default\nn     atime                 off                    local\nn     devices               on                     default\nn     exec                  on                     default\nn     setuid                on                     default\nn     readonly              off                    default\nn     zoned                 off                    default\nn     snapdir               hidden                 default\nn     aclinherit            restricted             default\nn     canmount              on                     default\nn     xattr                 on                     default\nn     copies                1                      default\nn     version               5                      -\nn     utf8only              off                    -\nn     normalization         none                   -\nn     casesensitivity       sensitive              -\nn     vscan                 off                    default\nn     nbmand                off                    default\nn     sharesmb              off                    default\nn     refquota              none                   default\nn     refreservation        none                   default\nn     primarycache          all                    default\nn     secondarycache        all                    default\nn     usedbysnapshots       0                      -\nn     usedbydataset         5.63T                  -\nn     usedbychildren        8.32T                  -\nn     usedbyrefreservation  0                      -\nn     logbias               latency                default\nn     dedup                 off                    default\nn     mlslabel              none                   default\nn     sync                  standard               default\nn     dnodesize             legacy                 default\nn     refcompressratio      1.00x                  -\nn     written               0                      -\nn     logicalused           13.9T                  -\nn     logicalreferenced     5.64T                  -\nn     filesystem_limit      none                   default\nn     snapshot_limit        none                   default\nn     filesystem_count      none                   default\nn     snapshot_count        none                   default\nn     snapdev               hidden                 default\nn     acltype               off                    default\nn     context               none                   default\nn     fscontext             none                   default\nn     defcontext            none                   default\nn     rootcontext           none                   default\nn     relatime              off                    default\nn     redundant_metadata    all                    default\nn     overlay               off                    default\n```\n\n```\nlocalhost ~ # zpool get all n\nNAME  PROPERTY                    VALUE                       SOURCE\nn     size                        21.8T                       -\nn     capacity                    96%                         -\nn     altroot                     -                           default\nn     health                      ONLINE                      -\nn     guid                        13306438682543689121        default\nn     version                     -                           default\nn     bootfs                      -                           default\nn     delegation                  on                          default\nn     autoreplace                 off                         default\nn     cachefile                   -                           default\nn     failmode                    wait                        default\nn     listsnapshots               off                         default\nn     autoexpand                  off                         default\nn     dedupditto                  0                           default\nn     dedupratio                  1.00x                       -\nn     free                        819G                        -\nn     allocated                   20.9T                       -\nn     readonly                    off                         -\nn     ashift                      12                          local\nn     comment                     -                           default\nn     expandsize                  -                           -\nn     freeing                     0                           default\nn     fragmentation               56%                         -\nn     leaked                      0                           default\nn     feature@async_destroy       enabled                     local\nn     feature@empty_bpobj         active                      local\nn     feature@lz4_compress        active                      local\nn     feature@spacemap_histogram  active                      local\nn     feature@enabled_txg         active                      local\nn     feature@hole_birth          active                      local\nn     feature@extensible_dataset  enabled                     local\nn     feature@embedded_data       active                      local\nn     feature@bookmarks           enabled                     local\nn     feature@filesystem_limits   enabled                     local\nn     feature@large_blocks        enabled                     local\nn     feature@large_dnode         disabled                    local\n```\n\nHere's all the SMART stuff for these drives...\n\n```\nlocalhost ~ # smartctl -x /dev/disk/by-id/ata-WDC_WD80EFZX-68UW8N0_VKGNH8BX\nsmartctl 6.4 2015-06-04 r4109 [x86_64-linux-4.4.6-gentoo-debug2] (local build)\nCopyright (C) 2002-15, Bruce Allen, Christian Franke, www.smartmontools.org\n\n=== START OF INFORMATION SECTION ===\nDevice Model:     WDC WD80EFZX-68UW8N0\nSerial Number:    VKGNH8BX\nLU WWN Device Id: 5 000cca 254c950be\nFirmware Version: 83.H0A83\nUser Capacity:    8,001,563,222,016 bytes [8.00 TB]\nSector Sizes:     512 bytes logical, 4096 bytes physical\nRotation Rate:    5400 rpm\nForm Factor:      3.5 inches\nDevice is:        Not in smartctl database [for details use: -P showall]\nATA Version is:   ACS-2, ATA8-ACS T13/1699-D revision 4\nSATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)\nLocal Time is:    Wed Aug 24 08:30:43 2016 Local time zone must be set--see zic m\nSMART support is: Available - device has SMART capability.\nSMART support is: Enabled\nAAM feature is:   Unavailable\nAPM level is:     164 (intermediate level without standby)\nRd look-ahead is: Enabled\nWrite cache is:   Enabled\nATA Security is:  Disabled, NOT FROZEN [SEC1]\nWt Cache Reorder: Enabled\n\n=== START OF READ SMART DATA SECTION ===\nSMART overall-health self-assessment test result: PASSED\n\nGeneral SMART Values:\nOffline data collection status:  (0x80) Offline data collection activity\n                    was never started.\n                    Auto Offline Data Collection: Enabled.\nSelf-test execution status:      (   0) The previous self-test routine completed\n                    without error or no self-test has ever \n                    been run.\nTotal time to complete Offline \ndata collection:        (  101) seconds.\nOffline data collection\ncapabilities:            (0x5b) SMART execute Offline immediate.\n                    Auto Offline data collection on/off support.\n                    Suspend Offline collection upon new\n                    command.\n                    Offline surface scan supported.\n                    Self-test supported.\n                    No Conveyance Self-test supported.\n                    Selective Self-test supported.\nSMART capabilities:            (0x0003) Saves SMART data before entering\n                    power-saving mode.\n                    Supports SMART auto save timer.\nError logging capability:        (0x01) Error logging supported.\n                    General Purpose Logging supported.\nShort self-test routine \nrecommended polling time:    (   2) minutes.\nExtended self-test routine\nrecommended polling time:    (1133) minutes.\nSCT capabilities:          (0x003d) SCT Status supported.\n                    SCT Error Recovery Control supported.\n                    SCT Feature Control supported.\n                    SCT Data Table supported.\n\nSMART Attributes Data Structure revision number: 16\nVendor Specific SMART Attributes with Thresholds:\nID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE\n  1 Raw_Read_Error_Rate     PO-R--   100   100   016    -    0\n  2 Throughput_Performance  P-S---   131   131   054    -    116\n  3 Spin_Up_Time            POS---   147   147   024    -    447 (Average 446)\n  4 Start_Stop_Count        -O--C-   100   100   000    -    15\n  5 Reallocated_Sector_Ct   PO--CK   100   100   005    -    0\n  7 Seek_Error_Rate         PO-R--   100   100   067    -    0\n  8 Seek_Time_Performance   P-S---   128   128   020    -    18\n  9 Power_On_Hours          -O--C-   100   100   000    -    457\n 10 Spin_Retry_Count        PO--C-   100   100   060    -    0\n 12 Power_Cycle_Count       -O--CK   100   100   000    -    15\n 22 Unknown_Attribute       PO---K   100   100   025    -    100\n192 Power-Off_Retract_Count -O--CK   100   100   000    -    66\n193 Load_Cycle_Count        -O--C-   100   100   000    -    66\n194 Temperature_Celsius     -O----   125   125   000    -    48 (Min/Max 25/55)\n196 Reallocated_Event_Count -O--CK   100   100   000    -    0\n197 Current_Pending_Sector  -O---K   100   100   000    -    0\n198 Offline_Uncorrectable   ---R--   100   100   000    -    0\n199 UDMA_CRC_Error_Count    -O-R--   200   200   000    -    0\n                            ||||||_ K auto-keep\n                            |||||__ C event count\n                            ||||___ R error rate\n                            |||____ S speed/performance\n                            ||_____ O updated online\n                            |______ P prefailure warning\n\nGeneral Purpose Log Directory Version 1\nSMART           Log Directory Version 1 [multi-sector log support]\nAddress    Access  R/W   Size  Description\n0x00       GPL,SL  R/O      1  Log Directory\n0x01           SL  R/O      1  Summary SMART error log\n0x02           SL  R/O      1  Comprehensive SMART error log\n0x03       GPL     R/O      1  Ext. Comprehensive SMART error log\n0x04       GPL,SL  R/O      8  Device Statistics log\n0x06           SL  R/O      1  SMART self-test log\n0x07       GPL     R/O      1  Extended self-test log\n0x08       GPL     R/O      2  Power Conditions log\n0x09           SL  R/W      1  Selective self-test log\n0x10       GPL     R/O      1  SATA NCQ Queued Error log\n0x11       GPL     R/O      1  SATA Phy Event Counters log\n0x12       GPL     R/O      1  SATA NCQ NON-DATA log\n0x15       GPL,SL  R/W      1  SATA Rebuild Assist log\n0x21       GPL     R/O      1  Write stream error log\n0x22       GPL     R/O      1  Read stream error log\n0x24       GPL     R/O    256  Current Device Internal Status Data log\n0x25       GPL     R/O    256  Saved Device Internal Status Data log\n0x30       GPL,SL  R/O      9  IDENTIFY DEVICE data log\n0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log\n0xe0       GPL,SL  R/W      1  SCT Command/Status\n0xe1       GPL,SL  R/W      1  SCT Data Transfer\n\nSMART Extended Comprehensive Error Log Version: 1 (1 sectors)\nNo Errors Logged\n\nSMART Extended Self-test Log Version: 1 (1 sectors)\nNo self-tests have been logged.  [To run self-tests, use: smartctl -t]\n\nSMART Selective self-test log data structure revision number 1\n SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS\n    1        0        0  Not_testing\n    2        0        0  Not_testing\n    3        0        0  Not_testing\n    4        0        0  Not_testing\n    5        0        0  Not_testing\nSelective self-test flags (0x0):\n  After scanning selected spans, do NOT read-scan remainder of disk.\nIf Selective self-test is pending on power-up, resume after 0 minute delay.\n\nSCT Status Version:                  3\nSCT Version (vendor specific):       256 (0x0100)\nSCT Support Level:                   1\nDevice State:                        Active (0)\nCurrent Temperature:                    48 Celsius\nPower Cycle Min/Max Temperature:     29/49 Celsius\nLifetime    Min/Max Temperature:     25/55 Celsius\nUnder/Over Temperature Limit Count:   0/0\n\nSCT Temperature History Version:     2\nTemperature Sampling Period:         1 minute\nTemperature Logging Interval:        1 minute\nMin/Max recommended Temperature:      0/60 Celsius\nMin/Max Temperature Limit:           -40/70 Celsius\nTemperature History Size (Index):    128 (8)\n\nIndex    Estimated Time   Temperature Celsius\n   9    2016-08-24 06:23    49  ******************************\n ...    ..( 12 skipped).    ..  ******************************\n  22    2016-08-24 06:36    49  ******************************\n  23    2016-08-24 06:37    48  *****************************\n ...    ..( 73 skipped).    ..  *****************************\n  97    2016-08-24 07:51    48  *****************************\n  98    2016-08-24 07:52    47  ****************************\n ...    ..(  3 skipped).    ..  ****************************\n 102    2016-08-24 07:56    47  ****************************\n 103    2016-08-24 07:57    46  ***************************\n ...    ..(  6 skipped).    ..  ***************************\n 110    2016-08-24 08:04    46  ***************************\n 111    2016-08-24 08:05    47  ****************************\n ...    ..( 14 skipped).    ..  ****************************\n 126    2016-08-24 08:20    47  ****************************\n 127    2016-08-24 08:21    48  *****************************\n ...    ..(  7 skipped).    ..  *****************************\n   7    2016-08-24 08:29    48  *****************************\n   8    2016-08-24 08:30    49  ******************************\n\nSCT Error Recovery Control:\n           Read:     70 (7.0 seconds)\n          Write:     70 (7.0 seconds)\n\nDevice Statistics (GP Log 0x04)\nPage  Offset Size        Value Flags Description\n0x01  =====  =               =  ===  == General Statistics (rev 2) ==\n0x01  0x008  4              15  ---  Lifetime Power-On Resets\n0x01  0x018  6     16078448412  ---  Logical Sectors Written\n0x01  0x020  6        76148289  ---  Number of Write Commands\n0x01  0x028  6     11633444836  ---  Logical Sectors Read\n0x01  0x030  6        61332439  ---  Number of Read Commands\n0x01  0x038  6      1647367400  ---  Date and Time TimeStamp\n0x03  =====  =               =  ===  == Rotating Media Statistics (rev 1) ==\n0x03  0x008  4             258  ---  Spindle Motor Power-on Hours\n0x03  0x010  4             258  ---  Head Flying Hours\n0x03  0x018  4              66  ---  Head Load Events\n0x03  0x020  4               0  ---  Number of Reallocated Logical Sectors\n0x03  0x028  4          945927  ---  Read Recovery Attempts\n0x03  0x030  4               0  ---  Number of Mechanical Start Failures\n0x04  =====  =               =  ===  == General Errors Statistics (rev 1) ==\n0x04  0x008  4               0  ---  Number of Reported Uncorrectable Errors\n0x04  0x010  4               0  ---  Resets Between Cmd Acceptance and Completion\n0x05  =====  =               =  ===  == Temperature Statistics (rev 1) ==\n0x05  0x008  1              48  ---  Current Temperature\n0x05  0x010  1              45  N--  Average Short Term Temperature\n0x05  0x018  1              41  N--  Average Long Term Temperature\n0x05  0x020  1              55  ---  Highest Temperature\n0x05  0x028  1              25  ---  Lowest Temperature\n0x05  0x030  1              52  N--  Highest Average Short Term Temperature\n0x05  0x038  1              25  N--  Lowest Average Short Term Temperature\n0x05  0x040  1              41  N--  Highest Average Long Term Temperature\n0x05  0x048  1              25  N--  Lowest Average Long Term Temperature\n0x05  0x050  4               0  ---  Time in Over-Temperature\n0x05  0x058  1              60  ---  Specified Maximum Operating Temperature\n0x05  0x060  4               0  ---  Time in Under-Temperature\n0x05  0x068  1               0  ---  Specified Minimum Operating Temperature\n0x06  =====  =               =  ===  == Transport Statistics (rev 1) ==\n0x06  0x008  4              31  ---  Number of Hardware Resets\n0x06  0x010  4               1  ---  Number of ASR Events\n0x06  0x018  4               0  ---  Number of Interface CRC Errors\n                                |||_ C monitored condition met\n                                ||__ D supports DSN\n                                |___ N normalized value\n\nSATA Phy Event Counters (GP Log 0x11)\nID      Size     Value  Description\n0x0001  2            0  Command failed due to ICRC error\n0x0002  2            0  R_ERR response for data FIS\n0x0003  2            0  R_ERR response for device-to-host data FIS\n0x0004  2            0  R_ERR response for host-to-device data FIS\n0x0005  2            0  R_ERR response for non-data FIS\n0x0006  2            0  R_ERR response for device-to-host non-data FIS\n0x0007  2            0  R_ERR response for host-to-device non-data FIS\n0x0008  2            0  Device-to-host non-data FIS retries\n0x0009  2           33  Transition from drive PhyRdy to drive PhyNRdy\n0x000a  2           30  Device-to-host register FISes sent due to a COMRESET\n0x000b  2            0  CRC errors within host-to-device FIS\n0x000d  2            0  Non-CRC errors within host-to-device FIS\n\nlocalhost ~ # smartctl -x /dev/disk/by-id/ata-WDC_WD80EFZX-68UW8N0_VKHJK9ZX\nsmartctl 6.4 2015-06-04 r4109 [x86_64-linux-4.4.6-gentoo-debug2] (local build)\nCopyright (C) 2002-15, Bruce Allen, Christian Franke, www.smartmontools.org\n\n=== START OF INFORMATION SECTION ===\nDevice Model:     WDC WD80EFZX-68UW8N0\nSerial Number:    VKHJK9ZX\nLU WWN Device Id: 5 000cca 254d59e77\nFirmware Version: 83.H0A83\nUser Capacity:    8,001,563,222,016 bytes [8.00 TB]\nSector Sizes:     512 bytes logical, 4096 bytes physical\nRotation Rate:    5400 rpm\nForm Factor:      3.5 inches\nDevice is:        Not in smartctl database [for details use: -P showall]\nATA Version is:   ACS-2, ATA8-ACS T13/1699-D revision 4\nSATA Version is:  SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s)\nLocal Time is:    Wed Aug 24 08:30:57 2016 Local time zone must be set--see zic m\nSMART support is: Available - device has SMART capability.\nSMART support is: Enabled\nAAM feature is:   Unavailable\nAPM level is:     164 (intermediate level without standby)\nRd look-ahead is: Enabled\nWrite cache is:   Enabled\nATA Security is:  Disabled, frozen [SEC2]\nWt Cache Reorder: Enabled\n\n=== START OF READ SMART DATA SECTION ===\nSMART overall-health self-assessment test result: PASSED\n\nGeneral SMART Values:\nOffline data collection status:  (0x80) Offline data collection activity\n                    was never started.\n                    Auto Offline Data Collection: Enabled.\nSelf-test execution status:      (   0) The previous self-test routine completed\n                    without error or no self-test has ever \n                    been run.\nTotal time to complete Offline \ndata collection:        (  101) seconds.\nOffline data collection\ncapabilities:            (0x5b) SMART execute Offline immediate.\n                    Auto Offline data collection on/off support.\n                    Suspend Offline collection upon new\n                    command.\n                    Offline surface scan supported.\n                    Self-test supported.\n                    No Conveyance Self-test supported.\n                    Selective Self-test supported.\nSMART capabilities:            (0x0003) Saves SMART data before entering\n                    power-saving mode.\n                    Supports SMART auto save timer.\nError logging capability:        (0x01) Error logging supported.\n                    General Purpose Logging supported.\nShort self-test routine \nrecommended polling time:    (   2) minutes.\nExtended self-test routine\nrecommended polling time:    (1225) minutes.\nSCT capabilities:          (0x003d) SCT Status supported.\n                    SCT Error Recovery Control supported.\n                    SCT Feature Control supported.\n                    SCT Data Table supported.\n\nSMART Attributes Data Structure revision number: 16\nVendor Specific SMART Attributes with Thresholds:\nID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE\n  1 Raw_Read_Error_Rate     PO-R--   100   100   016    -    0\n  2 Throughput_Performance  P-S---   132   132   054    -    112\n  3 Spin_Up_Time            POS---   144   144   024    -    458 (Average 456)\n  4 Start_Stop_Count        -O--C-   100   100   000    -    16\n  5 Reallocated_Sector_Ct   PO--CK   100   100   005    -    0\n  7 Seek_Error_Rate         PO-R--   100   100   067    -    0\n  8 Seek_Time_Performance   P-S---   128   128   020    -    18\n  9 Power_On_Hours          -O--C-   100   100   000    -    453\n 10 Spin_Retry_Count        PO--C-   100   100   060    -    0\n 12 Power_Cycle_Count       -O--CK   100   100   000    -    16\n 22 Unknown_Attribute       PO---K   100   100   025    -    100\n192 Power-Off_Retract_Count -O--CK   100   100   000    -    64\n193 Load_Cycle_Count        -O--C-   100   100   000    -    64\n194 Temperature_Celsius     -O----   133   133   000    -    45 (Min/Max 25/55)\n196 Reallocated_Event_Count -O--CK   100   100   000    -    0\n197 Current_Pending_Sector  -O---K   100   100   000    -    0\n198 Offline_Uncorrectable   ---R--   100   100   000    -    0\n199 UDMA_CRC_Error_Count    -O-R--   200   200   000    -    0\n                            ||||||_ K auto-keep\n                            |||||__ C event count\n                            ||||___ R error rate\n                            |||____ S speed/performance\n                            ||_____ O updated online\n                            |______ P prefailure warning\n\nGeneral Purpose Log Directory Version 1\nSMART           Log Directory Version 1 [multi-sector log support]\nAddress    Access  R/W   Size  Description\n0x00       GPL,SL  R/O      1  Log Directory\n0x01           SL  R/O      1  Summary SMART error log\n0x02           SL  R/O      1  Comprehensive SMART error log\n0x03       GPL     R/O      1  Ext. Comprehensive SMART error log\n0x04       GPL,SL  R/O      8  Device Statistics log\n0x06           SL  R/O      1  SMART self-test log\n0x07       GPL     R/O      1  Extended self-test log\n0x08       GPL     R/O      2  Power Conditions log\n0x09           SL  R/W      1  Selective self-test log\n0x10       GPL     R/O      1  SATA NCQ Queued Error log\n0x11       GPL     R/O      1  SATA Phy Event Counters log\n0x12       GPL     R/O      1  SATA NCQ NON-DATA log\n0x15       GPL,SL  R/W      1  SATA Rebuild Assist log\n0x21       GPL     R/O      1  Write stream error log\n0x22       GPL     R/O      1  Read stream error log\n0x24       GPL     R/O    256  Current Device Internal Status Data log\n0x25       GPL     R/O    256  Saved Device Internal Status Data log\n0x30       GPL,SL  R/O      9  IDENTIFY DEVICE data log\n0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log\n0xe0       GPL,SL  R/W      1  SCT Command/Status\n0xe1       GPL,SL  R/W      1  SCT Data Transfer\n\nSMART Extended Comprehensive Error Log Version: 1 (1 sectors)\nNo Errors Logged\n\nSMART Extended Self-test Log Version: 1 (1 sectors)\nNo self-tests have been logged.  [To run self-tests, use: smartctl -t]\n\nSMART Selective self-test log data structure revision number 1\n SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS\n    1        0        0  Not_testing\n    2        0        0  Not_testing\n    3        0        0  Not_testing\n    4        0        0  Not_testing\n    5        0        0  Not_testing\nSelective self-test flags (0x0):\n  After scanning selected spans, do NOT read-scan remainder of disk.\nIf Selective self-test is pending on power-up, resume after 0 minute delay.\n\nSCT Status Version:                  3\nSCT Version (vendor specific):       256 (0x0100)\nSCT Support Level:                   1\nDevice State:                        Active (0)\nCurrent Temperature:                    45 Celsius\nPower Cycle Min/Max Temperature:     29/46 Celsius\nLifetime    Min/Max Temperature:     25/55 Celsius\nUnder/Over Temperature Limit Count:   0/0\n\nSCT Temperature History Version:     2\nTemperature Sampling Period:         1 minute\nTemperature Logging Interval:        1 minute\nMin/Max recommended Temperature:      0/60 Celsius\nMin/Max Temperature Limit:           -40/70 Celsius\nTemperature History Size (Index):    128 (9)\n\nIndex    Estimated Time   Temperature Celsius\n  10    2016-08-24 06:23    46  ***************************\n ...    ..(  7 skipped).    ..  ***************************\n  18    2016-08-24 06:31    46  ***************************\n  19    2016-08-24 06:32    45  **************************\n ...    ..( 17 skipped).    ..  **************************\n  37    2016-08-24 06:50    45  **************************\n  38    2016-08-24 06:51    44  *************************\n ...    ..( 30 skipped).    ..  *************************\n  69    2016-08-24 07:22    44  *************************\n  70    2016-08-24 07:23    45  **************************\n  71    2016-08-24 07:24    45  **************************\n  72    2016-08-24 07:25    45  **************************\n  73    2016-08-24 07:26    44  *************************\n  74    2016-08-24 07:27    45  **************************\n ...    ..( 20 skipped).    ..  **************************\n  95    2016-08-24 07:48    45  **************************\n  96    2016-08-24 07:49    44  *************************\n ...    ..(  4 skipped).    ..  *************************\n 101    2016-08-24 07:54    44  *************************\n 102    2016-08-24 07:55    43  ************************\n ...    ..( 14 skipped).    ..  ************************\n 117    2016-08-24 08:10    43  ************************\n 118    2016-08-24 08:11    44  *************************\n ...    ..( 10 skipped).    ..  *************************\n   1    2016-08-24 08:22    44  *************************\n   2    2016-08-24 08:23    45  **************************\n ...    ..(  5 skipped).    ..  **************************\n   8    2016-08-24 08:29    45  **************************\n   9    2016-08-24 08:30    46  ***************************\n\nSCT Error Recovery Control:\n           Read:     70 (7.0 seconds)\n          Write:     70 (7.0 seconds)\n\nDevice Statistics (GP Log 0x04)\nPage  Offset Size        Value Flags Description\n0x01  =====  =               =  ===  == General Statistics (rev 2) ==\n0x01  0x008  4              16  ---  Lifetime Power-On Resets\n0x01  0x018  6     16078777451  ---  Logical Sectors Written\n0x01  0x020  6        76231268  ---  Number of Write Commands\n0x01  0x028  6     11931313928  ---  Logical Sectors Read\n0x01  0x030  6        60830864  ---  Number of Read Commands\n0x01  0x038  6      1632365950  ---  Date and Time TimeStamp\n0x03  =====  =               =  ===  == Rotating Media Statistics (rev 1) ==\n0x03  0x008  4             258  ---  Spindle Motor Power-on Hours\n0x03  0x010  4             258  ---  Head Flying Hours\n0x03  0x018  4              64  ---  Head Load Events\n0x03  0x020  4               0  ---  Number of Reallocated Logical Sectors\n0x03  0x028  4          939802  ---  Read Recovery Attempts\n0x03  0x030  4               0  ---  Number of Mechanical Start Failures\n0x04  =====  =               =  ===  == General Errors Statistics (rev 1) ==\n0x04  0x008  4               0  ---  Number of Reported Uncorrectable Errors\n0x04  0x010  4               1  ---  Resets Between Cmd Acceptance and Completion\n0x05  =====  =               =  ===  == Temperature Statistics (rev 1) ==\n0x05  0x008  1              45  ---  Current Temperature\n0x05  0x010  1              42  N--  Average Short Term Temperature\n0x05  0x018  1              42  N--  Average Long Term Temperature\n0x05  0x020  1              55  ---  Highest Temperature\n0x05  0x028  1              25  ---  Lowest Temperature\n0x05  0x030  1              49  N--  Highest Average Short Term Temperature\n0x05  0x038  1              25  N--  Lowest Average Short Term Temperature\n0x05  0x040  1              42  N--  Highest Average Long Term Temperature\n0x05  0x048  1              25  N--  Lowest Average Long Term Temperature\n0x05  0x050  4               0  ---  Time in Over-Temperature\n0x05  0x058  1              60  ---  Specified Maximum Operating Temperature\n0x05  0x060  4               0  ---  Time in Under-Temperature\n0x05  0x068  1               0  ---  Specified Minimum Operating Temperature\n0x06  =====  =               =  ===  == Transport Statistics (rev 1) ==\n0x06  0x008  4              12  ---  Number of Hardware Resets\n0x06  0x010  4               7  ---  Number of ASR Events\n0x06  0x018  4               0  ---  Number of Interface CRC Errors\n                                |||_ C monitored condition met\n                                ||__ D supports DSN\n                                |___ N normalized value\n\nSATA Phy Event Counters (GP Log 0x11)\nID      Size     Value  Description\n0x0001  2            0  Command failed due to ICRC error\n0x0002  2            0  R_ERR response for data FIS\n0x0003  2            0  R_ERR response for device-to-host data FIS\n0x0004  2            0  R_ERR response for host-to-device data FIS\n0x0005  2            0  R_ERR response for non-data FIS\n0x0006  2            0  R_ERR response for device-to-host non-data FIS\n0x0007  2            0  R_ERR response for host-to-device non-data FIS\n0x0008  2            0  Device-to-host non-data FIS retries\n0x0009  2           12  Transition from drive PhyRdy to drive PhyNRdy\n0x000a  2           13  Device-to-host register FISes sent due to a COMRESET\n0x000b  2            0  CRC errors within host-to-device FIS\n0x000d  2            0  Non-CRC errors within host-to-device FIS\n\nlocalhost ~ # smartctl -x /dev/disk/by-id/ata-WDC_WD80EFZX-68UW8N0_VKHNJWBX\nsmartctl 6.4 2015-06-04 r4109 [x86_64-linux-4.4.6-gentoo-debug2] (local build)\nCopyright (C) 2002-15, Bruce Allen, Christian Franke, www.smartmontools.org\n\n=== START OF INFORMATION SECTION ===\nDevice Model:     WDC WD80EFZX-68UW8N0\nSerial Number:    VKHNJWBX\nLU WWN Device Id: 5 000cca 254d76e4d\nFirmware Version: 83.H0A83\nUser Capacity:    8,001,563,222,016 bytes [8.00 TB]\nSector Sizes:     512 bytes logical, 4096 bytes physical\nRotation Rate:    5400 rpm\nForm Factor:      3.5 inches\nDevice is:        Not in smartctl database [for details use: -P showall]\nATA Version is:   ACS-2, ATA8-ACS T13/1699-D revision 4\nSATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)\nLocal Time is:    Wed Aug 24 08:31:15 2016 Local time zone must be set--see zic m\nSMART support is: Available - device has SMART capability.\nSMART support is: Enabled\nAAM feature is:   Unavailable\nAPM level is:     164 (intermediate level without standby)\nRd look-ahead is: Enabled\nWrite cache is:   Enabled\nATA Security is:  Disabled, NOT FROZEN [SEC1]\nWt Cache Reorder: Unknown\n\n=== START OF READ SMART DATA SECTION ===\nSMART Status command failed\nPlease get assistance from http://smartmontools.sourceforge.net/\nRegister values returned from SMART Status command are:\n ERR=0x00, SC=0x00, LL=0x00, LM=0x00, LH=0x00, DEV=0x00, STS=0x50\nSMART Status not supported: Invalid ATA output register values\nSMART overall-health self-assessment test result: PASSED\nWarning: This result is based on an Attribute check.\n\nGeneral SMART Values:\nOffline data collection status:  (0x80) Offline data collection activity\n                    was never started.\n                    Auto Offline Data Collection: Enabled.\nSelf-test execution status:      (   0) The previous self-test routine completed\n                    without error or no self-test has ever \n                    been run.\nTotal time to complete Offline \ndata collection:        (  101) seconds.\nOffline data collection\ncapabilities:            (0x5b) SMART execute Offline immediate.\n                    Auto Offline data collection on/off support.\n                    Suspend Offline collection upon new\n                    command.\n                    Offline surface scan supported.\n                    Self-test supported.\n                    No Conveyance Self-test supported.\n                    Selective Self-test supported.\nSMART capabilities:            (0x0003) Saves SMART data before entering\n                    power-saving mode.\n                    Supports SMART auto save timer.\nError logging capability:        (0x01) Error logging supported.\n                    General Purpose Logging supported.\nShort self-test routine \nrecommended polling time:    (   2) minutes.\nExtended self-test routine\nrecommended polling time:    (1292) minutes.\nSCT capabilities:          (0x003d) SCT Status supported.\n                    SCT Error Recovery Control supported.\n                    SCT Feature Control supported.\n                    SCT Data Table supported.\n\nSMART Attributes Data Structure revision number: 16\nVendor Specific SMART Attributes with Thresholds:\nID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE\n  1 Raw_Read_Error_Rate     PO-R--   100   100   016    -    0\n  2 Throughput_Performance  P-S---   131   131   054    -    114\n  3 Spin_Up_Time            POS---   100   100   024    -    0\n  4 Start_Stop_Count        -O--C-   100   100   000    -    8\n  5 Reallocated_Sector_Ct   PO--CK   100   100   005    -    0\n  7 Seek_Error_Rate         PO-R--   100   100   067    -    0\n  8 Seek_Time_Performance   P-S---   128   128   020    -    18\n  9 Power_On_Hours          -O--C-   100   100   000    -    416\n 10 Spin_Retry_Count        PO--C-   100   100   060    -    0\n 12 Power_Cycle_Count       -O--CK   100   100   000    -    8\n 22 Unknown_Attribute       PO---K   100   100   025    -    100\n192 Power-Off_Retract_Count -O--CK   100   100   000    -    44\n193 Load_Cycle_Count        -O--C-   100   100   000    -    44\n194 Temperature_Celsius     -O----   122   122   000    -    49 (Min/Max 25/55)\n196 Reallocated_Event_Count -O--CK   100   100   000    -    0\n197 Current_Pending_Sector  -O---K   100   100   000    -    0\n198 Offline_Uncorrectable   ---R--   100   100   000    -    0\n199 UDMA_CRC_Error_Count    -O-R--   200   200   000    -    0\n                            ||||||_ K auto-keep\n                            |||||__ C event count\n                            ||||___ R error rate\n                            |||____ S speed/performance\n                            ||_____ O updated online\n                            |______ P prefailure warning\n\nGeneral Purpose Log Directory Version 1\nSMART           Log Directory Version 1 [multi-sector log support]\nAddress    Access  R/W   Size  Description\n0x00       GPL,SL  R/O      1  Log Directory\n0x01           SL  R/O      1  Summary SMART error log\n0x02           SL  R/O      1  Comprehensive SMART error log\n0x03       GPL     R/O      1  Ext. Comprehensive SMART error log\n0x04       GPL,SL  R/O      8  Device Statistics log\n0x06           SL  R/O      1  SMART self-test log\n0x07       GPL     R/O      1  Extended self-test log\n0x08       GPL     R/O      2  Power Conditions log\n0x09           SL  R/W      1  Selective self-test log\n0x10       GPL     R/O      1  SATA NCQ Queued Error log\n0x11       GPL     R/O      1  SATA Phy Event Counters log\n0x12       GPL     R/O      1  SATA NCQ NON-DATA log\n0x15       GPL,SL  R/W      1  SATA Rebuild Assist log\n0x21       GPL     R/O      1  Write stream error log\n0x22       GPL     R/O      1  Read stream error log\n0x24       GPL     R/O    256  Current Device Internal Status Data log\n0x25       GPL     R/O    256  Saved Device Internal Status Data log\n0x30       GPL,SL  R/O      9  IDENTIFY DEVICE data log\n0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log\n0xe0       GPL,SL  R/W      1  SCT Command/Status\n0xe1       GPL,SL  R/W      1  SCT Data Transfer\n\nSMART Extended Comprehensive Error Log Version: 1 (1 sectors)\nNo Errors Logged\n\nSMART Extended Self-test Log Version: 1 (1 sectors)\nNo self-tests have been logged.  [To run self-tests, use: smartctl -t]\n\nSMART Selective self-test log data structure revision number 1\n SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS\n    1        0        0  Not_testing\n    2        0        0  Not_testing\n    3        0        0  Not_testing\n    4        0        0  Not_testing\n    5        0        0  Not_testing\nSelective self-test flags (0x0):\n  After scanning selected spans, do NOT read-scan remainder of disk.\nIf Selective self-test is pending on power-up, resume after 0 minute delay.\n\nSCT Status Version:                  3\nSCT Version (vendor specific):       256 (0x0100)\nSCT Support Level:                   1\nDevice State:                        Active (0)\nCurrent Temperature:                    49 Celsius\nPower Cycle Min/Max Temperature:     27/51 Celsius\nLifetime    Min/Max Temperature:     25/55 Celsius\nUnder/Over Temperature Limit Count:   0/0\n\nSCT Temperature History Version:     2\nTemperature Sampling Period:         1 minute\nTemperature Logging Interval:        1 minute\nMin/Max recommended Temperature:      0/60 Celsius\nMin/Max Temperature Limit:           -40/70 Celsius\nTemperature History Size (Index):    128 (111)\n\nIndex    Estimated Time   Temperature Celsius\n 112    2016-08-24 06:24    50  *******************************\n 113    2016-08-24 06:25    50  *******************************\n 114    2016-08-24 06:26    49  ******************************\n 115    2016-08-24 06:27    49  ******************************\n 116    2016-08-24 06:28    48  *****************************\n 117    2016-08-24 06:29    48  *****************************\n 118    2016-08-24 06:30    48  *****************************\n 119    2016-08-24 06:31    47  ****************************\n 120    2016-08-24 06:32    47  ****************************\n 121    2016-08-24 06:33    47  ****************************\n 122    2016-08-24 06:34    46  ***************************\n ...    ..(  4 skipped).    ..  ***************************\n 127    2016-08-24 06:39    46  ***************************\n   0    2016-08-24 06:40    45  **************************\n ...    ..(  4 skipped).    ..  **************************\n   5    2016-08-24 06:45    45  **************************\n   6    2016-08-24 06:46    44  *************************\n ...    ..(  5 skipped).    ..  *************************\n  12    2016-08-24 06:52    44  *************************\n  13    2016-08-24 06:53    45  **************************\n ...    ..(  8 skipped).    ..  **************************\n  22    2016-08-24 07:02    45  **************************\n  23    2016-08-24 07:03    46  ***************************\n ...    ..(  9 skipped).    ..  ***************************\n  33    2016-08-24 07:13    46  ***************************\n  34    2016-08-24 07:14    47  ****************************\n ...    ..( 10 skipped).    ..  ****************************\n  45    2016-08-24 07:25    47  ****************************\n  46    2016-08-24 07:26    48  *****************************\n ...    ..( 28 skipped).    ..  *****************************\n  75    2016-08-24 07:55    48  *****************************\n  76    2016-08-24 07:56    47  ****************************\n ...    ..( 13 skipped).    ..  ****************************\n  90    2016-08-24 08:10    47  ****************************\n  91    2016-08-24 08:11    48  *****************************\n ...    ..( 10 skipped).    ..  *****************************\n 102    2016-08-24 08:22    48  *****************************\n 103    2016-08-24 08:23    49  ******************************\n ...    ..(  6 skipped).    ..  ******************************\n 110    2016-08-24 08:30    49  ******************************\n 111    2016-08-24 08:31    51  ********************************\n\nSCT Error Recovery Control:\n           Read: Disabled\n          Write: Disabled\n\nDevice Statistics (GP Log 0x04)\nPage  Offset Size        Value Flags Description\n0x01  =====  =               =  ===  == General Statistics (rev 2) ==\n0x01  0x008  4               8  ---  Lifetime Power-On Resets\n0x01  0x018  6     16076759236  ---  Logical Sectors Written\n0x01  0x020  6        76063310  ---  Number of Write Commands\n0x01  0x028  6      6196383585  ---  Logical Sectors Read\n0x01  0x030  6        28576331  ---  Number of Read Commands\n0x01  0x038  6      1499518400  ---  Date and Time TimeStamp\n0x03  =====  =               =  ===  == Rotating Media Statistics (rev 1) ==\n0x03  0x008  4             234  ---  Spindle Motor Power-on Hours\n0x03  0x010  4             234  ---  Head Flying Hours\n0x03  0x018  4              44  ---  Head Load Events\n0x03  0x020  4               0  ---  Number of Reallocated Logical Sectors\n0x03  0x028  4          939662  ---  Read Recovery Attempts\n0x03  0x030  4               0  ---  Number of Mechanical Start Failures\n0x04  =====  =               =  ===  == General Errors Statistics (rev 1) ==\n0x04  0x008  4               0  ---  Number of Reported Uncorrectable Errors\n0x04  0x010  4               0  ---  Resets Between Cmd Acceptance and Completion\n0x05  =====  =               =  ===  == Temperature Statistics (rev 1) ==\n0x05  0x008  1              49  ---  Current Temperature\n0x05  0x010  1              43  N--  Average Short Term Temperature\n0x05  0x018  1               -  N--  Average Long Term Temperature\n0x05  0x020  1              55  ---  Highest Temperature\n0x05  0x028  1              25  ---  Lowest Temperature\n0x05  0x030  1              52  N--  Highest Average Short Term Temperature\n0x05  0x038  1              25  N--  Lowest Average Short Term Temperature\n0x05  0x040  1               -  N--  Highest Average Long Term Temperature\n0x05  0x048  1               -  N--  Lowest Average Long Term Temperature\n0x05  0x050  4               0  ---  Time in Over-Temperature\n0x05  0x058  1              60  ---  Specified Maximum Operating Temperature\n0x05  0x060  4               0  ---  Time in Under-Temperature\n0x05  0x068  1               0  ---  Specified Minimum Operating Temperature\n0x06  =====  =               =  ===  == Transport Statistics (rev 1) ==\n0x06  0x008  4               1  ---  Number of Hardware Resets\n0x06  0x010  4               0  ---  Number of ASR Events\n0x06  0x018  4               0  ---  Number of Interface CRC Errors\n                                |||_ C monitored condition met\n                                ||__ D supports DSN\n                                |___ N normalized value\n\nSATA Phy Event Counters (GP Log 0x11)\nID      Size     Value  Description\n0x0001  2            0  Command failed due to ICRC error\n0x0002  2            0  R_ERR response for data FIS\n0x0003  2            0  R_ERR response for device-to-host data FIS\n0x0004  2            0  R_ERR response for host-to-device data FIS\n0x0005  2            0  R_ERR response for non-data FIS\n0x0006  2            0  R_ERR response for device-to-host non-data FIS\n0x0007  2            0  R_ERR response for host-to-device non-data FIS\n0x0008  2            0  Device-to-host non-data FIS retries\n0x0009  2            2  Transition from drive PhyRdy to drive PhyNRdy\n0x000a  2            2  Device-to-host register FISes sent due to a COMRESET\n0x000b  2            0  CRC errors within host-to-device FIS\n0x000d  2            0  Non-CRC errors within host-to-device FIS\n```\n\nThis is what the pool status looked like a few hours ago... It was created on the previous stable release version of ZFS, and I never upgraded it.\n\n```\n  pool: n\n state: ONLINE\nstatus: Some supported features are not enabled on the pool. The pool can\n    still be used, but some features are unavailable.\naction: Enable all features using 'zpool upgrade'. Once this is done,\n    the pool may no longer be accessible by software that does not support\n    the features. See zpool-features(5) for details.\n  scan: none requested\nconfig:\n\n    NAME                                   STATE     READ WRITE CKSUM\n    n                                      ONLINE       0     0     0\n      raidz1-0                             ONLINE       0     0     0\n        ata-WDC_WD80EFZX-68UW8N0_VKGNH8BX  ONLINE       0     0     0\n        ata-WDC_WD80EFZX-68UW8N0_VKHJK9ZX  ONLINE       0     0     0\n        ata-WDC_WD80EFZX-68UW8N0_VKHNJWBX  ONLINE       0     0     0\n\nerrors: No known data errors\n```\n\nI have another pool which did this too. It had two checksum errors on the top level mirror, and zero checksum (or any kind of) errors on the individual drives. I ran a `zfs clear` in the hope that it would forget about this checksum error until the next time the checksum really fails (to see if it was just a transient (cosmic ray) error or something). But it didn't forget about the I/O error on the file, but now it just says 0 errors... probably until I scrub it again... anyway...\nNo hardware faults reported or detected....\n\nThis checksum error also occurred when I was running the latest \"master\" version of ZFS and SPL from Github, as of about the end of July, beginning of Aug.\n\n```\nlocalhost ~ # zpool status -v o\n  pool: o\n state: ONLINE\nstatus: One or more devices has experienced an error resulting in data\n    corruption.  Applications may be affected.\naction: Restore the file in question if possible.  Otherwise restore the\n    entire pool from backup.\n   see: http://zfsonlinux.org/msg/ZFS-8000-8A\n  scan: scrub repaired 0 in 172h43m with 1 errors on Thu Aug 11 09:28:55 2016\nconfig:\n\n    NAME                                   STATE     READ WRITE CKSUM\n    o                                      ONLINE       0     0     0\n      mirror-0                             ONLINE       0     0     0\n        ata-WDC_WD80EFZX-68UW8N0_VKHMZSWX  ONLINE       0     0     0\n        ata-WDC_WD80EFZX-68UW8N0_VKHATXMX  ONLINE       0     0     0\n\nerrors: Permanent errors have been detected in the following files:\n\n        o@2016_Jun_28:/Maxtor_3H500F0_H80KP5MH_blocks_505765120+7.img\n\nlocalhost ~ # zpool history o\nHistory for 'o':\n2016-06-24.04:01:08 zpool create -O atime=off -o ashift=12 -o feature@lz4_compress=enabled -o feature@embedded_data=enabled o -f mirror /dev/disk/by-id/ata-WDC_WD80EFZX-68UW8N0_VKHMZSWX /dev/disk/by-id/ata-WDC_WD80EFZX-68UW8N0_VKHATXMX\n2016-06-24.04:01:37 zfs set compression=on o\n2016-06-24.04:01:52 zfs set exec=off o\n2016-06-24.04:02:00 zfs set devices=off o\n2016-06-24.04:02:08 zfs set setuid=off o\n2016-06-24.04:03:10 zpool add o cache /dev/disk/by-id/ata-SAMSUNG_MZHPU512HCGL-00004_S1NDNYAFC00958\n2016-06-25.19:07:48 zpool import -a\n2016-06-26.00:02:30 zfs set compression=lz4 o\n2016-06-28.23:43:45 zfs set dedup=sha256 o\n2016-06-28.23:44:57 zfs snapshot o@2016_Jun_28\n2016-06-29.20:39:22 zpool import -a\n2016-06-29.20:40:36 zpool remove o /dev/disk/by-id/ata-SAMSUNG_MZHPU512HCGL-00004_S1NDNYAFC00958\n2016-06-29.20:40:38 zpool export o\n2016-08-04.03:08:17 zpool import o\n2016-08-04.04:45:32 zpool clear o\n2016-08-04.04:45:50 zpool scrub o\n2016-08-10.22:24:43 zpool import -c /etc/zfs/zpool.cache -N -a\n2016-08-11.20:00:52 zpool export o\n```\n"},"comment":{"url":"https://api.github.com/repos/zfsonlinux/zfs/issues/comments/277585476","html_url":"https://github.com/zfsonlinux/zfs/issues/5018#issuecomment-277585476","issue_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/5018","id":277585476,"user":{"login":"chrisrd","id":1211482,"avatar_url":"https://avatars.githubusercontent.com/u/1211482?v=3","gravatar_id":"","url":"https://api.github.com/users/chrisrd","html_url":"https://github.com/chrisrd","followers_url":"https://api.github.com/users/chrisrd/followers","following_url":"https://api.github.com/users/chrisrd/following{/other_user}","gists_url":"https://api.github.com/users/chrisrd/gists{/gist_id}","starred_url":"https://api.github.com/users/chrisrd/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chrisrd/subscriptions","organizations_url":"https://api.github.com/users/chrisrd/orgs","repos_url":"https://api.github.com/users/chrisrd/repos","events_url":"https://api.github.com/users/chrisrd/events{/privacy}","received_events_url":"https://api.github.com/users/chrisrd/received_events","type":"User","site_admin":false},"created_at":"2017-02-06T04:41:59Z","updated_at":"2017-02-06T04:41:59Z","body":"Yes, I just started looking into how Seagate reports that SMART parameter myself. In case anyone else is following along, the reference I found is: http://www.users.on.net/~fzabkar/HDD/Seagate_SER_RRER_HEC.html\r\n\r\nI agree your disks aren't displaying any seek errors.\r\n\r\nSorry, I thought I might have been onto something there!"}}
{"actor":{"display_login":"chrisrd"}}