I think there are always a few people to profit well with a change in urban planning. Usually they can see trends before they happen and have the financial resources to make real estate investments. Sometimes there is some political interference; most of the times it’s just a matter of watching what the politicians are doing.
My stay in London was short, so I can’t comment fully. And a lot has changed. I understand that a lot of central London has empty residences these days.
But Vancouver is interesting. In the 50s and 60s, there was a big round of property development, turning residential neighborhoods into skyrise apartments in the WEST END, which now has one of the highest population densities in the world. That part of Vancouver looks like an architectural nightmare, but it probably answered the call of the left-wing lobbyists to provide affordable housing. But this development just made it more attractive for people to move to Vancouver — which then led to higher rents — and then more left-wing activism to reduce rents. The south side of Vancouver (proper) still has lots of single-home residences. If we just let the free market have its way, south side would be turned into another highrise jungle. I can’t say for sure, but I bet there is a strong political push not do this, which is — once again — interfering with the natural market.
Vancouver — and its surrounding cities — are indeed a haven for immigrants who have lower expectation of individuals having their own space. While that solves some crowding issues, a worker making $40,000 a year can easily rent a one-bedroom apartment in Calgary, but probably not in Vancouver. I don’t think there is much difference in wages between the two cities.
But again, people make choices.