Ashleigh was the home of Joseph Hugh Lytle's family from the early 1880s. He was the son of former Lord Mayor John Lytle, and ran the seed and tea business founded by his father. After the Lytles moved out, the house was sold to a school, which became known as Ashleigh House School.
In 1987 Ashleigh House School was amalgamated with Princess Gardens School to become Hunterhouse College.
Ashleigh Manor takes its name from the former Ashleigh House School which no longer exists.