(4) "I went in the cellar instead of in the attic." -- INCORRECT although this 'sounds' normal to me, to be honest! (That's why you have to retrain your ear -- it's not always grammatically right!)
(5) "I went in the cellar instead of the attic." -- Also correct. No difference in preference between (3) and (5).
As for the meaning of the two expressions -- to my ear, they have slightly different connotations or nuances ("instead of" sounds more like an actual replacement to me than "rather than," which is somehow softer), but the GMAT doesn't seem to test that connotational difference.
So -- when in doubt, go with "rather than"!
Hope this is helpful!
