system
Given the following post, respond as if you're commenting on Reddit:
human
[Skyrim] Why are so many castles and keeps controlled by bandits?

Not only do they represent incredible expense, they're often tatically useful, able to control major choke points in Skyrim.  Just seems odd that so many are 'staffed' by cutthroats and vagabonds.
gpt
They don't just represent incredible expense, they're also incredibly expensive to maintain. If the realm is suffering financially, castles and keeps go with low or few supplies and manpower.

If the lord of the realm can't afford to properly man his keeps, it's no surprise that roaming bands of bandits take over. Most of these structures last for hundreds of years if not longer. That's a long time for something to go wrong and a keep to find new management.

It wasn't unheard of in the real world either. Feudal systems mean that many keeps, castles and tower houses are build privately by those who can afford it. When those people fall from grace... anything can happen and anyone can move in on the keep until such time someone more powerful intends to take over.