Digging into the difference between rankings and ratings
Our nation’s cities and counties are constantly ranked on the Internet, with headlines like ‘Top 10 Foodie Cities’ or ‘Most Dog Friendly U.S. Communities’ popping up every week. Some lists are based on the writer’s own criteria or reader surveys while others are backed by quantitative data. In many cases, data is collected and analyzed by an outside organization making it difficult for communities to know how well the ranking actually reflects current conditions in their community.
Since many of the communities we work with appear on national lists and rankings, we thought it would be interesting to dig in a bit and distinguish between rankings lists and the certified STAR Community Rating® awarded by our organization. We’ll look at 2 key differences between ratings and rankings: demonstrating progress over time versus taking a quick snapshot, and voluntary reporting versus compiling externally derived data.
Interestingly, it was local government leaders that called for the development of the STAR Community Rating System back in 2007-2008. They were seeking a common framework for sustainability and a standard set of metrics to credibly and transparently track their progress. Tired of being externally ranked with little or no input into the process, local leaders developed STAR, a rating system that includes 7 goal areas and hundreds of quantitative and qualitative metrics. STAR’s metrics encompass economic, environmental and social performance areas for local governments and the community at large. The metrics are subject to a continuous improvement process, which recently resulted in STAR Version 2.0.
Demonstrating progress over time versus taking a quick snapshot
For communities to receive a STAR Community Rating, they must collect and submit data for verification for two types of measures—quantitative outcomes and qualitative actions. The combination of these measures allows a community to show progress in community conditions over time while also demonstrating the local actions they are putting into place every day.
In contrast, lists and rankings typically show a community’s metrics for a single year. They provide a snapshot of how a community is performing at a given point in time, but may not tell the whole story. For example, a community may report a high community-wide greenhouse gas emissions level, but community efforts over the past several years have resulted in this number dropping significantly so that the community is now on track to an overall reduction. A snapshot ranking wouldn’t recognize the progress being made. However, in the development of the rating system, it was decided that it is important to reward progress. So meeting the STAR trendline would result in credit awarded for this effort.
Voluntary reporting versus compiling externally derived data
The reporting required by STAR Communities is strictly voluntary. Individual communities choose to participate, choose which metrics to report on, and choose whether to complete the third-party certification process. The process allows communities to capture the entire picture of sustainability in their community. The use of both outcome and action measures encourages communities to make continual progress.
This is in contrast to the methodology used in the creation of many rankings, which often don’t solicit information and feedback from the communities being ranked on the quality of data being used and don’t provide the opportunity for a community to opt out of the rankings.
Often, rankings provide a high level evaluation of how a community is at a point in time, but they do not dial down to the depth that truly shows the efforts and work that is being done in a community. Only through STAR certification does a community establish a comprehensive baseline assessment of their strengths and weaknesses so that they can identify gaps and prioritize investment going forward.
‘Top 10 Most Sustainable Cities’ style rankings will continue to come out every year. Rather than be a passive participant, communities are encouraged to roll up their sleeves and begin actively benchmarking their progress with STAR’s national standards.
For more information on STAR Communities’ annual reporting and certification programs, go to www.STARcommunities.org/getstarted.
Learn about the impact of STAR certification in this recent report that highlights the first 50 STAR Certified Communities.