Ed Glaeser has recently written about his views on urban rankings. Permit me to offer my views.
The U.S has roughly 300 major cities. Could all 300 million Americans agree on their rankings from A to Z or would different people have different rankings? Put bluntly, how much would I have to pay the people who choose to live in San Francisco to move to Houston and how much would I have to pay the people who choose to live in Houston to move to San Francisco?
As economists, Glaeser and I would agreed that a market test of the desire to live in a particular city such as San Francisco is to calculate your annual income if you live there and to subtract off your annual rent for a standardized home. If this consumption is large, then an economist would conclude that this isn't a desirable city to live in. After all, suppose there is a great city to live in that offers a great quality of life, high wages and low rents. If such a paradise existed, everyone would try to move there and this would lower market wages at the location and raise the rents at the location until people were just indifferent between living there or not.
So, in an open system of cities where you can migrate across cities --- the differences in a local city quality of life will be reflected in local wages and rents.
Now, I realize that "quality of life" may mean different things to different people. Economists have documented that young, middle aged, and old people have different priorities over what city attributes they care about. For example, older people reveal a taste for warm winters as they migrate to Florida for the winter while young people may care more about bars and nightlife that a specific city offers.
Another demographic determinant of ranking cities is income. New York City is a very different town if you are upper middle class versus poor versus rich. A poor New Yorker is more likely to live in the Bronx or Brooklyn rather than in Manhattan. Yes, these different groups are exposed to roughly the same climate conditions but they will have different access to Opera Tickets, fancy restaurants and stores and good seats at Pro-Sports games. So, the same city can be a very different experience depending where you sit in the income distribution.
Ethnicity is another determinant of city rankings. People tend to self segregate and live with people who look like them. If I am the only person of Chinese heritage in a city, then I am unlikely to find a Chinese newspaper or my hometown's cuisine in an American city. But, if there are 250,000 Chinese immigrants in my city then my U.S city will have the scale to cater to my tastes and I will be able to recreate my "old life" in my U.S city.
Economists use migration as a "market test" of which cities are on the rise. I ask my UCLA students; "how many of you plan to move to Detroit after graduation?" I have never seen a student raise his hand and say "yes" to this question. For Detroit to make a comeback, it must retain its home grown skilled and attract "outsiders" looking for cheap housing and new opportunities.
Ed Glaeser and I have been interested in ranking cities with respect to their carbon footprint. We have written two papers that are available here and here. Such metrics allow for "apples to apples" comparisons that can be made across international borders.
But, let's focus on the real issue here. How will we know when Shanghai is a "better" city than New York City or Los Angeles? Economists have ranked universities by studying where students choose to enroll when they have a choice and when the researcher can see the financial aid packages offered by both competing schools. For example, suppose you know that I have been accepted by Harvard and Yale and that Yale has offered me a $10,000 discount in tuition. If I choose Harvard, then this is strong market evidence that I really prefer Harvard. This paper pursues this strategy.
In the case of Shanghai and NYC, the analogue would be some sort of free agent either in sports or high tech services who can do his job anywhere. If you could observe the salary offers in the two cities and see which choice is accepted (and if you know the rents for housing in the two markets) then you would have the start of a credible ranking index of "International Superstar Cities".
Economists have used such an approach to measure international tax haven effects. See this soccer paper.
There are other "objective criteria" such as murders per-capita, polluted air days, humid day counts, count of highly rated Zagat's restaurants but while each of these could be collected by city it assumes that there is no variation in quality of life within cities and you continue to face the challenge of how you collapse this broad set of data into a single rating for the city as a whole? Why is NYC a better city than Boston? Is it? How do you define your metric for measuring how "far apart" they are? Would Trump ever live in Boston? Why Not?