Willette Ballard, a representative for the group, said Caldwell died Tuesday of congestive heart failure at his San Francisco home.