In 1868 and 1870 Ann-Elizabeth Fossett Isaacs‘s daughters Virginia and Maria married former Union army officers James M. Trotter and William H. Dupree and moved to live in Boston. Ann-Elizabeth‘s nephew Frederick Douglass Isaacs also went to Boston, in the early years of the twentieth century.