Gandalfs First entry to Dol Guldur

[Tinman]

I have been doing some research on Dol Guldur. I was interested to find out that when gandalf entered it and discovered the map+key, and the true identity of the necromancer, that this was actually his second time entering the tower.

From what i can find, the first time he entered the tower, Sauran fled to remain hidden. He then returned, and spend a few hundred(?) years gathering power again.

Does any one have any further details they can provide on this? I'm wondering:
1.What year did gandalf first enter Dol Guldur?
2. If sauron was not there, what did he find? Did all the orcs and wraiths clear out too? Where did they go?
3. Why did the Wise not tear down or reoccupy Dol Guldur at this time? From a strategic point of view... if there is a potential stronghold that can spell doom for all the surrounding freepeople, and will be extremely taxing to assault if occupied by enemy forces, why would you leave a for rent sign on it?
4. I want as much info on this as possible... so if you have any theories or facts, please let me know (no idea to small!)