Vega was born in Ponce, Puerto Rico, in 1936 and he grew up in Cidra, a small northern town on the island. His family moved to the United States in 1949 when his father became the Baptist minister of a Hispanic congregation in the South Bronx. They moved from the Bronx to El Barrio in 1952 and when he graduated from high school he joined the United States Air Force in 1954. His family encouraged education and artistic expression and he began reading many of the great American authors like Hemingway, Faulkner, Steinbeck and Fitzgerald upon returning home from the Air Force on leave.