Without the innermost motives of the character written down and set in stone, surely that gives you more room to use that imagination you were going on about.
So you don't have to imagine what the charaters or their surroundings look like, but if that's the limit of what you want to imagine then you aren't that imaginative.