Ogilvy was the eldest son of George Ogilvy, 2nd Lord Banff and his second wife, Agnes, the daughter of Sir Alexander Falconer, 1st Lord Falconer of Halkerton.
Ogilvy was the eldest son of George Ogilvy of Sir Alexander Falconer. Ogilvy was the eldest son of 2nd Lord Banff of Sir Alexander Falconer. Ogilvy was the eldest son of his second wife the daughter of Sir Alexander Falconer. Sir Alexander Falconer was 1st Lord Falconer of Halkerton. Agnes was his second wife.