Gandolfi was born in London in 1933. He has one brother, Antony older by two years. Their father's death in 1938 and being sent for safe keeping to South Africa during World War II were major impacts on his childhood.