When they get to where David lived there is no house, and the neighbours claim that there was never a house there.
There is no house. They get to where David lived. There was never a house there. This is what the neighbours claim.