Just work at making harvesting a normal part of your play and you’ll never be stuck without build materials.
Every time you drop in, you should be taking some time to harvest wood, stone, and metal as you come across it so it can be added to your inventory. You won’t need to obsess over it, but gathering up some here and some there will ensure that you always have what you need without stressing about it when the time comes. Just work at making harvesting a normal part of your play and you’ll never be stuck without build materials.