In 1855 Harold's father was made a baronet; but in the 1860s his businesses ran into trouble, so that in 1863 he sold Somerlyton Hall and in 1866 became bankrupt.
V roce 1855 se Haroldův otec stal baronetem, ale v roce 1860 se jeho firmy dostaly do problémů, takže v roce 1863 prodal Somerlyton Hall a v roce 1866 zkrachoval.