{"action":"created","issue":{"url":"https://api.github.com/repos/bitcoin/bitcoin/issues/10647","repository_url":"https://api.github.com/repos/bitcoin/bitcoin","labels_url":"https://api.github.com/repos/bitcoin/bitcoin/issues/10647/labels{/name}","comments_url":"https://api.github.com/repos/bitcoin/bitcoin/issues/10647/comments","events_url":"https://api.github.com/repos/bitcoin/bitcoin/issues/10647/events","html_url":"https://github.com/bitcoin/bitcoin/issues/10647","id":237650154,"number":10647,"title":"Bitcoin Core not using configured database cache size and taking way too long to sync","user":{"login":"fresheneesz","id":149531,"avatar_url":"https://avatars0.githubusercontent.com/u/149531?v=3","gravatar_id":"","url":"https://api.github.com/users/fresheneesz","html_url":"https://github.com/fresheneesz","followers_url":"https://api.github.com/users/fresheneesz/followers","following_url":"https://api.github.com/users/fresheneesz/following{/other_user}","gists_url":"https://api.github.com/users/fresheneesz/gists{/gist_id}","starred_url":"https://api.github.com/users/fresheneesz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fresheneesz/subscriptions","organizations_url":"https://api.github.com/users/fresheneesz/orgs","repos_url":"https://api.github.com/users/fresheneesz/repos","events_url":"https://api.github.com/users/fresheneesz/events{/privacy}","received_events_url":"https://api.github.com/users/fresheneesz/received_events","type":"User","site_admin":false},"labels":[{"id":326918230,"url":"https://api.github.com/repos/bitcoin/bitcoin/labels/Resource%20usage","name":"Resource usage","color":"981023","default":false},{"id":234877,"url":"https://api.github.com/repos/bitcoin/bitcoin/labels/Windows","name":"Windows","color":"884400","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":8,"created_at":"2017-06-21T20:09:43Z","updated_at":"2017-07-03T06:07:14Z","closed_at":null,"body":"### Describe the issue\r\n\r\nMy bitcoin core wallet is taking a ridiculously long time to catch up to the blockchain. I've been running it on and off for a MONTH now, and I'm still 41 weeks behind. Looking at my Task Manager, I see that my network is being almost totally unutilized at 0.1Mbps, tho my disk is being used at 1.5 MBps. It also says the Bitcoin Core (GUI node for Bitcoin) is only using 280 MB of memory, even tho I have `-dbcache=700` set in the config (which I can see it recognizes in the options since it lists it under \"Active command-line options that override above options\"). \r\n\r\nI have also tried removing `-dbcache=700` from my bitcoin.conf and instead setting 1000 MB as the database cache option in the GUI options window (and restarting of course). Neither the bitcoin.conf nor the database cache size option in the options seem to affect how much memory is allocated to bitcoin core, even tho I have at least 800MB free. \r\n\r\n### What version of bitcoin-core are you using?\r\nBitcoin Core version v0.14.1 (64-bit), official\r\n\r\n### Machine specs:\r\n- OS: Windows 8.1 64-bit\r\n- CPU: Intel Core i7-4702HQ  @ 2.2 GHz\r\n- RAM: 8 GB\r\n- Disk size: 4TB HD for the blockchain, 250 GB SSD for the program and other (smaller) data\r\n- Network Speed: 50 Mbps\r\n\r\n### Any extra information that might be useful in the debugging process.\r\nMy debug.log (note the weird whitespace is verbatim):\r\n\r\n```\r\n2017-06-21 19:59:18 \r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n2017-06-21 19:59:18 Bitcoin version v0.14.2\r\n2017-06-21 19:59:18 InitParameterInteraction: parameter interaction: -whitelistforcerelay=1 -> setting -whitelistrelay=1\r\n2017-06-21 19:59:18 Assuming ancestors of block 00000000000000000013176bf8d7dfeab4e1db31dc93bc311b436e82ab226b90 have valid signatures.\r\n2017-06-21 19:59:18 GUI: \"registerShutdownBlockReason: Successfully registered: Bitcoin Core didn't yet exit safely...\"\r\n2017-06-21 19:59:18 Default data directory C:\\Users\\fresheneesz\\AppData\\Roaming\\Bitcoin\r\n2017-06-21 19:59:18 Using data directory H:\\Bitcoin\r\n2017-06-21 19:59:18 Using config file H:\\Bitcoin\\bitcoin.conf\r\n2017-06-21 19:59:18 Using at most 125 automatic connections (2048 file descriptors available)\r\n2017-06-21 19:59:18 Using 32 MiB out of 32 requested for signature cache, able to store 1048576 elements\r\n2017-06-21 19:59:18 Using 4 threads for script verification\r\n2017-06-21 19:59:18 Using BerkeleyDB version Berkeley DB 4.8.30: (April  9, 2010)\r\n2017-06-21 19:59:18 Using wallet wallet.dat\r\n2017-06-21 19:59:18 scheduler thread start\r\n2017-06-21 19:59:18 init message: Verifying wallet...\r\n2017-06-21 19:59:18 CDBEnv::Open: LogDir=H:\\Bitcoin\\database ErrorFile=H:\\Bitcoin\\db.log\r\n2017-06-21 19:59:18 Bound to [::]:8333\r\n2017-06-21 19:59:18 Bound to 0.0.0.0:8333\r\n2017-06-21 19:59:18 Cache configuration:\r\n2017-06-21 19:59:18 * Using 2.0MiB for block index database\r\n2017-06-21 19:59:18 * Using 8.0MiB for chain state database\r\n2017-06-21 19:59:18 * Using 990.0MiB for in-memory UTXO set (plus up to 286.1MiB of unused mempool space)\r\n2017-06-21 19:59:18 init message: Loading block index...\r\n2017-06-21 19:59:18 Opening LevelDB in H:\\Bitcoin\\blocks\\index\r\n2017-06-21 19:59:18 Opened LevelDB successfully\r\n2017-06-21 19:59:18 Using obfuscation key for H:\\Bitcoin\\blocks\\index: 0000000000000000\r\n2017-06-21 19:59:18 Opening LevelDB in H:\\Bitcoin\\chainstate\r\n2017-06-21 19:59:19 Opened LevelDB successfully\r\n2017-06-21 19:59:19 Using obfuscation key for H:\\Bitcoin\\chainstate: 2e24172acafd25a5\r\n2017-06-21 19:59:24 LoadBlockIndexDB: last block file = 614\r\n2017-06-21 19:59:24 LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=156, size=118252833, heights=427663...428149, time=2016-08-31...2016-09-03)\r\n2017-06-21 19:59:24 Checking all blk files are present...\r\n2017-06-21 19:59:24 LoadBlockIndexDB: transaction index disabled\r\n2017-06-21 19:59:24 LoadBlockIndexDB: hashBestChain=000000000000000000784ef695fdd8651da8810ee6864155b45858d987bb8ab6 height=428092 date=2016-09-03 13:35:38 progress=0.664512\r\n2017-06-21 19:59:24 init message: Rewinding blocks...\r\n2017-06-21 19:59:26 init message: Verifying blocks...\r\n2017-06-21 19:59:26 Verifying last 6 blocks at level 3\r\n2017-06-21 19:59:26 [0%]...[16%]...[33%]...[50%]...[66%]...[83%]...[99%]...[DONE].\r\n2017-06-21 19:59:31 No coin database inconsistencies in last 7 blocks (15977 transactions)\r\n2017-06-21 19:59:31  block index           12566ms\r\n2017-06-21 19:59:31 init message: Loading wallet...\r\n2017-06-21 19:59:31 nFileVersion = 140200\r\n2017-06-21 19:59:31 Keys: 102 plaintext, 0 encrypted, 102 w/ metadata, 102 total\r\n2017-06-21 19:59:31  wallet                  309ms\r\n2017-06-21 19:59:31 setKeyPool.size() = 100\r\n2017-06-21 19:59:31 mapWallet.size() = 0\r\n2017-06-21 19:59:31 mapAddressBook.size() = 1\r\n2017-06-21 19:59:31 mapBlockIndex.size() = 472291\r\n2017-06-21 19:59:31 nBestHeight = 428092\r\n2017-06-21 19:59:31 torcontrol thread start\r\n2017-06-21 19:59:31 AddLocal([2001:0:9d38:90d7:204a:599:b399:e59e]:8333,1)\r\n2017-06-21 19:59:31 Discover: btbot - 2001:0:9d38:90d7:204a:599:b399:e59e\r\n2017-06-21 19:59:31 init message: Loading addresses...\r\n2017-06-21 19:59:31 Loaded 59991 addresses from peers.dat  307ms\r\n2017-06-21 19:59:31 init message: Loading banlist...\r\n2017-06-21 19:59:31 init message: Starting network threads...\r\n2017-06-21 19:59:31 net thread start\r\n2017-06-21 19:59:31 init message: Done loading\r\n2017-06-21 19:59:31 addcon thread start\r\n2017-06-21 19:59:31 msghand thread start\r\n2017-06-21 19:59:31 opencon thread start\r\n2017-06-21 19:59:31 dnsseed thread start\r\n2017-06-21 19:59:42 Loading addresses from DNS seeds (could take a while)\r\n2017-06-21 19:59:48 95 addresses found from DNS seeds\r\n2017-06-21 19:59:48 dnsseed thread exit\r\n2017-06-21 20:01:42 UpdateTip: new best=000000000000000003f1d2605ca6630a0060acc0c888ddccad3d6d4125e82cd6 height=428093 version=0x20000000 log2_work=85.219583 tx=153653276 date='2016-09-03 13:38:30' progress=0.664519 cache=6.8MiB(5297tx)\r\n2017-06-21 20:01:42 GUI: Platform customization: \"windows\"\r\n2017-06-21 20:01:42 GUI: PaymentServer::LoadRootCAs: Loaded  37  root certificates\r\n2017-06-21 20:02:09 UpdateTip: new best=0000000000000000045ee23efcc1c09475d57eefa22c67c27b38f39cfc3c34e0 height=428094 version=0x20000000 log2_work=85.219613 tx=153655485 date='2016-09-03 13:49:02' progress=0.664529 cache=21.0MiB(11163tx)\r\n2017-06-21 20:02:44 version handshake timeout from 1\r\n2017-06-21 20:03:11 UpdateTip: new best=000000000000000003f7dcc2461a28778770029276ee304f3eb13fba6b1cea2f height=428095 version=0x20000000 log2_work=85.219643 tx=153658301 date='2016-09-03 14:12:24' progress=0.664540 cache=27.4MiB(16845tx)\r\n2017-06-21 20:03:11 receive version message: /Satoshi:0.14.1/UASF-Segwit:0.3(BIP148)/: version 70015, blocks=472293, us=76.102.26.97:57474, peer=1\r\n2017-06-21 20:04:06 UpdateTip: new best=0000000000000000022ac5bddcee7c811e7a0e6e5a89429acf1d58d3e8539320 height=428096 version=0x20000000 log2_work=85.219674 tx=153659848 date='2016-09-03 14:14:18' progress=0.664547 cache=31.4MiB(21399tx)\r\n2017-06-21 20:04:06 version handshake timeout from 2\r\n2017-06-21 20:04:06 receive version message: /Satoshi:0.14.1(UASF-SegWit-BIP148)/: version 70015, blocks=472293, us=76.102.26.97:57528, peer=2\r\n2017-06-21 20:04:31 UpdateTip: new best=0000000000000000043ed79f2ffb751e3728f78664afad371e4500e10762b484 height=428097 version=0x20000000 log2_work=85.219704 tx=153661418 date='2016-09-03 14:19:31' progress=0.664553 cache=34.5MiB(25507tx)\r\n2017-06-21 20:04:31 version handshake timeout from 3\r\n2017-06-21 20:04:31 receive version message: /Satoshi:0.14.0/: version 70015, blocks=472293, us=76.102.26.97:57617, peer=3\r\n2017-06-21 20:05:09 UpdateTip: new best=000000000000000002595133c1941973fcc9c1a8ce9c6fda28b8e0e344d03032 height=428098 version=0x20000000 log2_work=85.219734 tx=153662935 date='2016-09-03 14:24:49' progress=0.664559 cache=37.1MiB(29873tx)\r\n2017-06-21 20:05:09 receive version message: /Satoshi:0.14.1/: version 70015, blocks=472293, us=76.102.26.97:57711, peer=4\r\n2017-06-21 20:05:30 Pre-allocating up to position 0xb00000 in rev00614.dat\r\n2017-06-21 20:05:31 UpdateTip: new best=00000000000000000361dcadc5ce630cabbe7636c5a62440273821f86f3bc95d height=428099 version=0x20000000 log2_work=85.219765 tx=153663979 date='2016-09-03 14:28:16' progress=0.664564 cache=50.8MiB(32801tx)\r\n2017-06-21 20:05:52 GUI:   OpenType support missing for script 11\r\n2017-06-21 20:05:52 GUI:   OpenType support missing for script 11\r\n2017-06-21 20:05:52 GUI:   OpenType support missing for script 11\r\n2017-06-21 20:05:52 GUI:   OpenType support missing for script 11\r\n2017-06-21 20:05:52 GUI:   OpenType support missing for script 11\r\n2017-06-21 20:05:52 GUI:   OpenType support missing for script 11\r\n2017-06-21 20:05:52 GUI:   OpenType support missing for script 16\r\n2017-06-21 20:05:52 GUI:   OpenType support missing for script 16\r\n2017-06-21 20:05:52 GUI:   OpenType support missing for script 16\r\n2017-06-21 20:05:52 GUI:   OpenType support missing for script 16\r\n2017-06-21 20:05:52 GUI:   OpenType support missing for script 16\r\n2017-06-21 20:05:52 GUI:   OpenType support missing for script 16\r\n2017-06-21 20:05:54 UpdateTip: new best=0000000000000000033987fb36873b88b55f486379f69fadcaf75b9aaa3af305 height=428100 version=0x20000000 log2_work=85.219795 tx=153666230 date='2016-09-03 14:42:09' progress=0.664573 cache=56.1MiB(36737tx)\r\n2017-06-21 20:06:19 UpdateTip: new best=000000000000000002e37eb2991f40b37be13c387f3fe7c8ba757c5914b731f8 height=428101 version=0x20000000 log2_work=85.219826 tx=153667728 date='2016-09-03 14:49:17' progress=0.664579 cache=65.6MiB(40254tx)\r\n2017-06-21 20:06:19 receive version message: /Satoshi:0.14.0/UASF-Segwit:0.3(BIP148)/: version 70015, blocks=472293, us=76.102.26.97:57850, peer=7\r\n2017-06-21 20:06:46 UpdateTip: new best=000000000000000002c4a8db38b8e5d5c64f375029fd9831f2227edce35990d2 height=428102 version=0x20000000 log2_work=85.219856 tx=153669727 date='2016-09-03 15:38:00' progress=0.664588 cache=75.5MiB(44108tx)\r\n2017-06-21 20:07:28 version handshake timeout from 8\r\n2017-06-21 20:07:40 UpdateTip: new best=00000000000000000127105f5c7acd04fe94c9b72ff5b4df838f418dfb640d1d height=428103 version=0x20000000 log2_work=85.219886 tx=153671822 date='2016-09-03 15:41:00' progress=0.664596 cache=99.1MiB(49248tx)\r\n2017-06-21 20:08:11 UpdateTip: new best=00000000000000000243fb7f67177714cb56bc3f060df96ddb363b2ac3549645 height=428104 version=0x20000000 log2_work=85.219917 tx=153674491 date='2016-09-03 15:42:50' progress=0.664608 cache=102.3MiB(53947tx)\r\n2017-06-21 20:08:11 version handshake timeout from 9\r\n2017-06-21 20:08:11 receive version message: /Satoshi:0.14.1(UASF-SegWit-BIP148)/: version 70015, blocks=472293, us=76.102.26.97:58008, peer=9\r\n2017-06-21 20:08:30 UpdateTip: new best=0000000000000000026df6ad986b718f2f326700d7c49ba4ceac08ddb5a1de8c height=428105 version=0x20000000 log2_work=85.219947 tx=153676187 date='2016-09-03 15:46:25' progress=0.664615 cache=103.4MiB(56761tx)\r\n2017-06-21 20:08:30 receive version message: /Satoshi:0.14.1/: version 70015, blocks=472293, us=[2001:0:9d38:90d7:204a:599:b399:e59e]:58087, peer=10\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/bitcoin/bitcoin/issues/comments/312557034","html_url":"https://github.com/bitcoin/bitcoin/issues/10647#issuecomment-312557034","issue_url":"https://api.github.com/repos/bitcoin/bitcoin/issues/10647","id":312557034,"user":{"login":"sipa","id":548488,"avatar_url":"https://avatars2.githubusercontent.com/u/548488?v=3","gravatar_id":"","url":"https://api.github.com/users/sipa","html_url":"https://github.com/sipa","followers_url":"https://api.github.com/users/sipa/followers","following_url":"https://api.github.com/users/sipa/following{/other_user}","gists_url":"https://api.github.com/users/sipa/gists{/gist_id}","starred_url":"https://api.github.com/users/sipa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sipa/subscriptions","organizations_url":"https://api.github.com/users/sipa/orgs","repos_url":"https://api.github.com/users/sipa/repos","events_url":"https://api.github.com/users/sipa/events{/privacy}","received_events_url":"https://api.github.com/users/sipa/received_events","type":"User","site_admin":false},"created_at":"2017-07-03T06:07:14Z","updated_at":"2017-07-03T06:07:14Z","body":"I don't have a full explanation for you, but it's very commonly observed that the operations per second is much lower for USB disks (much worse than internal spinning disks). In general, any database system running on USB tends to have terrible performance.\n\nDisks tend to organize things in 512 to 4096 byte sections. Since the pieces of data we need to fetch for validation are only a few hundred bytes, each read tends to result in an independent fetch from disk.\n\nAs for how to move the database without moving blocks: In Linux, you can use a symlink to another directory for the $DATADIR/blocks directory. There is something similar in Windows, but I don't know the details."}}
{"actor":{"display_login":"sipa"}}