A closer look at Siemens' green cities rankings

San Francisco won for overall greenest city.Photo: aufmkolk

Cross-posted from the Natural Resources Defense Council.

I may as well start with the caveat that any attempt to measure, score, or rank places with respect to almost anything will be incomplete at best and can be wildly misleading at worst. First, rating systems tend to assign numerical grades to things that are partially or entirely subjective. Which city has the “best” transit service is not just a matter of coverage and service frequency, for example, but also of passenger comfort, convenience for riders’ destinations (which vary from one to another), and whether the door-to-door experience feels safe, among other things. 

Second, even measurements based on quantitative data are complicated. A rating of a city as “highly walkable” because of a large number of conveniences available within a short distance to a large number of people may mask that its sidewalks are actually in poor repair and poorly lit. Settlement in Atlanta spreads far beyond the city limits (in green).Photo: Google EarthSo does one need to calculate measures (or proxy measures) of such factors? And then there’s the whole matter of definition, since a “city” defined by an antiquated municipal boundary won’t be the same as a city defined by actual patterns of settlement and employment (see image of Atlanta, left). And so on.

That said, such ratings and rankings are fun, because they start conversations about what is important. And they can be useful, especially if the authors spend some time describing the particular characteristics that cause a place to be evaluated favorably or unfavorably.

Overall evaluations

So, with that out of the way, let’s get to the findings of a new study of 27 large American and Canadian cities by the Economist Intelligence Unit, conducted for the global corporate giant Siemens. By the Unit’s evaluation, the top cities in their “Green City Index” were:

1. San Francisco
2. Vancouver
3. New York City
4. Seattle
5. Denver

The least green, starting with the lowest rated, were these:

1. Detroit
2. St. Louis
3. Cleveland
4. Phoenix
5. Pittsburgh

The top four certainly offer no surprises; few people would quibble with finding them in a top 10, certainly, in some order or other. I suppose one could quarrel with Denver at No. 5, but it isn’t shocking to see it there. The bottom five already make me wonder about the criteria, though, since four of them are Rust Belt cities that, though economically distressed, may include populations whose living habits produce relatively small environmental footprints compared with those of, say, sprawling Charlotte or Dallas.

Even St. Louis, next-to-last in the overall rankings, has community gardens.Photo: ONSL Restoration Group(In the interest of disclosure, I should note that I am an advisory board member and a longtime content contributor to the Sustainable Cities Collective, which is supported by Siemens. In addition, two of my very good personal and professional friends served on the expert advisory panel for this report. And Natural Resources Defense Council hosts our own evaluation of best city practices on our Smarter Cities website. All that said, I learned about this study and report only through the media.)

Most of the largest cities in the two countries were part of the study but some big names were not: Baltimore, Milwaukee, Salt Lake City, San Diego, Austin, Cincinnati, Kansas City, and New Orleans, for example.

The overall ratings are based on composite numerical scores derived from ratings for the separate categories of carbon dioxide, energy, land use, buildings, transport, water, waste, air, and environmental governance. While Siemens deserves credit for taking on the issue of urban sustainability and studying some very important factors, one can already see some issues: Couldn’t one say that CO2, energy, and buildings all look at the same thing, more or less? Where are health and fitness? Don’t parks deserve their own category (instead of being lumped into land use)? Looking at a report where Charlotte ranks in the top 10 for land use while Pittsburgh is 19th, for example, makes one question the criteria.

Speaking of which, I’ll devote the rest of today’s post to the study’s findings with respect to land use and transportation, categories of particular interest to many readers.

Land use

Here are your top five for land use:

1. New York City
2. Minneapolis
3. Ottawa
4. Boston
5. Vancouver

Bottom five, starting with the worst:

1. Cleveland
2. Detroit
3. Atlanta
4. St. Louis
5. Dallas

Charlotte, Calgary, and Miami significantly outranked Seattle, Chicago, and Toronto for land use. Even Houston ranked in the top half. Really?

Are sidewalk trees, like these in L.A., examples of good land use?Photo: Pieter EdelmanAs it turns out, the scoring of each city was based on four equally weighted factors, two of which were calculated from reported data (percentage of protected green space as a portion of total land area, and population density) — and two of which were scored according to expert judgment (green land-use policies such as tree planting, and policies for containment of sprawl). I suspect that the Rust Belt cities, which have lost inner-city population, may have been disadvantaged in the population density measurement if the researchers looked only within the municipal limits of the central city. That’s unfortunate.  

In any event, there’s a bigger point to be made, though it is relegated to the fine print of the report: “The average density for [the cities studied] is 8,100 people per square mile, which is about 2.5 times less than for Asian cities, at 21,100 people per square mile, and is also less than in Latin America (11,700) and in Europe (10,100).”

According to the report, Minneapolis earned its high ranking by having a very high portion of protected space within its city limits (I wonder how they counted the lakes), and by being evaluated highly in the two subjective measures. Its density actually falls below the median. Charlotte made the top 10 primarily because of the strong expert evaluation given to its policies for green space and revitalization. Among the four factors, I would have given more weight to density. 

Cleveland scored poorly because only 6 percent of its land area is given to protected green space (th
e median was 12 percent) and because its inner-city population density is low. I am shocked that Toronto was apparently downgraded for its failure to address sprawl, given that Ontario’s plan for the “Greater Golden Horseshoe” region around the city is one of the best that I have seen anywhere. Any measure of sprawl containment efforts that looks only within central city limits misses the point, if you ask me.

I wish that the study had examined the mix of land uses (perhaps using a measure similar to Walk Score) and, with respect to parks, had measured access (percentage of population living within a certain distance of a park) rather than gross acreage.  

Transport

Top five:

1. New York City
2. Vancouver
3. San Francisco
4. Montreal
5. Ottawa

Bottom five, starting with the worst:

1. Detroit
2. Phoenix
3. Charlotte
4. Los Angeles
5. St. Louis

Denver, with its light rail system, won praise for transit planning and construction.Photo: Jeffrey BeallFor this category, there are five equally weighted factors. Three are quantitative: share of workers traveling to work by transit, bicycle, or walking; public transport supply, measured by miles of transit network per square mile of area; and average commute time (and this is considered “green” because … ?). Two were qualitatively evaluated by the experts: how extensively the city promotes public transportation and low-carbon travel, and assessment of a city’s efforts to reduce congestion. Assuming the report is referring to motor vehicle congestion, I fail to see what that has to do with the environment (other than contributing somewhat to air pollution, which has its own category separate from transport). I would replace one of those two questionable categories with street connectivity (measured by intersections per square mile), which we know from research is closely associated with rates of walking and vehicle use. And where are vehicle miles traveled per capita, perhaps the most important transportation measure of all?

Charlotte and L.A. were severely undermined by poor performance on the quantitative measures, as one might expect. Personally, though, I might grade them both highly with respect to qualitative policy aspirations, given Charlotte’s superb street-design guidance and its efforts to leverage its light rail system for transit-oriented development, and given L.A.’s aggressive efforts to increase transit supply (which does get a mention). The study also rightly praises Denver’s ambitious program of transit planning and construction, along with Montreal’s bike-sharing program and New York’s new midtown pedestrian zones.

I am very surprised that Washington, D.C. (13th) was not ranked higher for transportation, given that we have the second-busiest rail transit system in the country, a streetcar system in the works, the second highest percentage of commuters walking to work, and an aggressive program to support bicycling (including a great bikeshare program). We were apparently downgraded by a long average commute time, which is dubious as a measure of environmental performance.

Washington, D.C. did not make the top 10 for transportation, despite its good track record for carless commuting.Photo: FK Benfield At any rate, more interesting than the individual city scoring and ranking were some larger findings: The Canadian cities studied, for example, have almost nine times more public transit network per square mile than the U.S. cities (6.2 miles per square mile versus 0.7 mi/sq mi). Wow. In addition, 74 percent of Canadians in the study drive to work, as compared to 90 percent in the U.S. cities. (Both numbers are poor compared to Europe, where the comparable portion is 43 percent.) Unsurprisingly, the higher-density U.S. and Canadian cities as a group outperformed the study’s average with respect to transit usage.

The commitment of Siemens to these issues is laudable. But, as I said at the top of the post, there are going to be problems with almost any methodology. It appears that there are some significant ones with respect to these particular categories. Much more valuable is that they get us talking about what’s really important.

Next, I will look at the rest of the study. Check out the full report [PDF], and extras like the press conference announcing the findings, a slide show, and a video.