The story follows a young teacher, Pat Conroy (played by Jon Voight), in 1969 assigned to isolated "Yamacraw Island" (Daufuskie Island) off the coast of South Carolina and populated mostly by poor black families. He finds out that the children as well as the adults have been isolated from the rest of the world and speak a dialect called Gullah, with "Conrack" of the novel's title being the best they can do to pronounce his last name. The school has only two rooms for all grades combined, with the Principal (school) (Madge Sinclair) teaching grades one through four and Conroy teaching the higher grades. Conroy discovers that the students aren't taught much and will have little hope of making a life in the larger world.