Having first settled in Davenport because he worked for the Ontario, Simcoe and Huron Union Rail-Road, Fleming and his wife moved to Halifax in 1864. At the time, Sandford was working for the Intercolonial Railway.
Five years later, the family moved to Ottawa in the house that was to become their chief residence, Winterholme. Fleming wanted to get closer to the government to which he had to report and justify his decisions. Because he needed to go back and forth between Ottawa and the Maritimes, he kept his house in Halifax.
Later on, the family bought another house in Halifax called The Dingle. In 1908, Fleming donated this property to the citizens of Halifax. He suggested that it be made into a park and that a tower be built to commemorate the 150th anniversary of responsible government in Nova Scotia.
The province had been the first British colony, outside Great Britain, to benefit from this type of government.