Recycle Elvish settlements in 4th Age?

[Tuor of Gondolin]

What do you think happened to the elven enclaves after they left (particularly Rivendell and the Grey Havens). In real earth history places like ancient Rome were mined by later inhabitants while Mayan cities were largely overgrown. And with an expanding mannish population (and for a while hobbits) wouldn't Rivendell especially been an attractive already made and rather comfy place, especially for the northern Dunedain who would obviously have been familiar with it? Or would it have been too steeped with elvish "memories" etc.? I believe there is speculation that Aragorn's rangers and their people had settlements in the angle, fairly close to Rivendell and the line of the Kings was always raised in Rivendell.
The Grey Havens would also seem to be very well situated for a settlement, using the sea for fishing and to avoid going through The Shire would allow quick contact with Gondor and a spur road could be linked to the Greenway.