Few ideas...

Georgia Aquarium
IKEA
Outwrite Bookstore (excellent, mostly gay bookstore)
Margaret Mitchell House and Museum/The Center for Southern Literature

Google any of the following for more details:
Underground Atlanta
Stone Mountain
Fox Theatre
High Museum
Lenox Square
Atlanta Botanical Garden
Martin Luther King Center
Michael C. Carlos Museum, Emory University

For more information, you may wish to consult the following online travel guides:
Frommer's
Fodor's