system
Given the following post, respond as if you're commenting on Reddit:
human
[Legend of Zelda OoT] How did Ganondorf build his doom fortress above the old Hyrule Castle anyway?
gpt
Well it's fair to say that the workmen couldn't possibly have done it- in the years between young and adult link, all they managed to achieve was completion of a few Kakariko buildings and the fixing of a bridge. Although some humans, such as Ingo had sided with Gannondorf, it still seems unlikely they could have constructed something of that magnitude judging from their ineptitude (as above) and the fact that the people had been driven from Hyrule Castle Town. 

The Gerudo may have done the work, but they are never seen outside of Gerudo Valley/the desert. But according to someone in Castle Town, they did visit at times looking for boyfriends. Gannon's fortress doesn't at all match the one in the desert. It seems most likely they had little to no involvement, as all their manpower is devoted to guarding their own land.

The Gorons probably had the capacity, but they appear to have been otherwise occupied in the mountain, though eventually imprisoned, they weren't necessarily enslaved. They gave no indication to Link that they had been used as slave workers.

I posit that the fortress modifications must have been created by his design and oversight at the hands of what Link would view as 'monsters' and 'enemies'. Whether skeletal warriors of the field, or the redead living in the castle town in the later stage, or maybe moblins or some other bosses or phantoms, Gannondorf had a *massive* amount of manpower and resources at his disposal.

Seeing as he had so many servants to fight for him, it would be only logical that he utilise them in other forms of service, such as construction.