The freedman must not stay in the state longer than twenty years, but like the other alie he must then take all his property and leave, unless he has gained permission from the authorities and his manumittor to remain.