With the recent release of the ISO 37120 standard, “Sustainable development of communities – Indicators for city services and quality of life,” participating STAR communities have asked how these new international standards fit in to their work on the US-based STAR Community Rating System (STAR). In early June, STAR Communities conducted a gaps analysis of the new ISO standard (released May 15, 2014) and version 1.1 of the STAR Community Rating System (released in January 2014). Our intent was to explore similarities and differences and inform our process as we continuously improve metrics and reporting for communities engaged in holistically tracking their sustainability progress.
For context, STAR is a national rating system and certification program built by and for local governments and the communities they serve. The intent of STAR is not to rank cities across a standard set of indicators, but rather provide a verified rating of their efforts given a menu of sustainability goals and objectives. These goals and objectives are defined through hundreds of evaluation measures – both quantitative community-level outcome measures and qualitative local action measures. Communities select what they will report on. The ISO Standard establishes data collection requirements that will allow cities to rank each other based on a common indicator. Once certified, a STAR community maintains its certification for three years. The ISO requires annual reporting for compliance.
Here are a few general similarities between STAR and the ISO 37120:
- Both systems divide quantitative metrics into multiple categories covering an array of sustainability topics;
- They allow for cities and counties to participate (counties as individuals in STAR; as aggregates in ISO);
- Each provide methodologies and describe limitations on data reporting;
- They intend to provide communities a standardized means of performance measurement and sharing of best practices; and
- Both offer tools for tracking and monitoring progress.
Before we do a deeper dive into the evaluation, we wanted to highlight that our review focused on STAR’s community level outcome measures, the condition-level indicators that depict a community’s progress toward a preferred state or condition. Represented as trend lines, targets or thresholds in the rating system, they are most similar to the indicators structure of the ISO standard.
Framework and Design
As discussed above, each system has delineated metrics into theme or goal areas based on the aspect of sustainability being addressed. ISO 37120 includes 17 Theme Areas. STAR provides a framework of 7 goal areas, 44 objectives, and an optional innovation category.
|ISO 37120 Theme Areas|
|Environment||Telecommunications and Innovation|
|Fire and Emergency Response||Urban Planning|
|Health||Water and Sanitation|
STAR’s Menu of Goals & Objectives
Since the ISO Theme Areas and STAR’s Goals and Objectives are not exact matches, each STAR outcome measure was evaluated against the ISO Theme Area it most closely matched. Next, the STAR outcome measures were compared to related ISO indicators in terms of the type of data collected, the methodology used, and the constraints on the indicator. The analysis of each Theme Area, provided below, does not delve into minor differences, such as the use of Metric versus US units. Also, a number of the ISO indicators measure on a per capita or per 100,000 population basis. STAR outcome measures use per capita on a very limited basis.
Regarding the indicators, we were surprised to find that only a third of the ISO indicators overlapped with STAR outcome measures. The reasons for this vary, but in many cases it relates to STAR’s use of compliance with a national standard in place of an actual value. For example, if a community is in attainment for air quality, they can provide supporting documentation to that effect. They only need to provide values for PM 2.5, PM 10, and Ozone if they are in non-attainment. Similarly, STAR’s wastewater management outcome is a measure of compliance with EPA effluent standards.
In other cases, ISO indicators focus on topics that represent greater disparities or issues in non-US cities, such as Number of two-wheel motorized vehicles per capita, Percentage of city population living in slums, and Percentage of city population with authorized electrical service.
Finally, a reason for low overlap might be that STAR is based on thresholds and some of the omitted indicators do not have a reputable source for what “doing well” would be, such as Number of firefighters per 100,000 population or Number of police officers per 100,000 population.
There are two Theme Areas where STAR does not have any corresponding outcome measures: Finance and Telecommunications/Innovations. There are 8 STAR outcome measures that did not fit neatly into any of the ISO Theme Areas, including Vulnerability Reduction, Creative Industries, Attendance and Participation in Arts, Social and Cultural Events, Volunteerism, Local Fresh Foods, School Nutrition, and Certified Sustainable Harvests.
Speaking from a STAR perspective, there were several ISO indicators that could be developed into the STAR Community Rating System in the future, such as youth unemployment rate; debt service ratio; number of fire related deaths per 100k population; and number of physicians per 100k population. Most were evaluated during STAR’s development and at the time it was determined that either trend lines, targets and/or thresholds were not in place or that data sets were not consistently available. STAR’s technical and governance committees review metrics and complimentary systems annually to ensure that STAR remains relevant, feasible, timely, useful, systemic, reliable and valid.
Quantified rating systems that are standardized are relatively new for communities. While we discovered that the new ISO standard and STAR do not overlap as much as we thought they might, we found new potential outcomes that may be valuable to include and alternative ways of quantifying metrics. STAR understands that municipalities will want to minimize their data collection efforts and prefer to submit one metric for multiple areas. Over the coming years, we will be working closely with ISO as well as other state and regional programs to determine ways of streamlining the collection process and improving our system.
To see the side-by-side comparison, please download the full report and see the appendices.