5 Ecometrics
Central to the work done at BARI is the development and utilization of ecometrics. Ecometrics are a quantitative representation of the physical and social environment. In the Urban Informatics context, ecometrics are created to extract latent constructs—variables that can only be inferred from a dataset—that illustrate some physical or social phenomenon.
To understand this, we need to again contextualize these datasets. They are not created with the intention of beng analyzed or to measure the blight of a neighborhood or the social unrest of a city. The lack of 311 reports may tell a story of an underserved neighborhood. Census measures of income combined with 911 reports of neighborly complaints and domestic disturbance may tell the story a gilded community with beautiful brick façades with next to no collective efficacy. These datasets contain gems—beautifully inelegant snapshots of the societal quotidian. But measuring that? That’s the tough part and that is why we create ecometrics. They provide us with a way to adapt existing data to address new problems.
Of the work that BARI conducts, the production of city-wide ecometrics of social and physical disorder are most emblematic of this hybrid approach. To understand this work we need to venture back to 1982 and an article from the Atlantic called Broken Windows.15
5.1 Broken Windows Theory
During the beginning of the crack cocaine epidemic, criminologists George Kelling and James Wilson wrote a famous article titled “Broken Windows” which outlined a new theory to explain the occurrence of crimes. The premise of this article is that the presence of disorder is more concerning for a neighborhood’s residents than the actual crime that occurs. Further, the “visual cues of disorder … begets predatory crime and neighborhood decline.”16
“Broken Windows” captured the eyes of pundits and policy makers. Its simplicity made it attractive and led to policies such as stop and frisk. Its public support went counter to previous scientific findings. Due the contrast between public policy and scientific findings researchers began to develop ways of quantifying disorder in a city. In a seminal article by Sampson and Raudenbush (1999), the practice of systematic social observation was created.17 This is a process in which photographs of public spaces is taken and coded to identify disorder—e.g. the presence of empty beer cans—which can then be quantitatively analyzed. This is an early example of an ecometric.
5.2 Quantifying Disorder
In 2015, O’Brien and Sampson published the article “Public and Private Spheres of Neighborhood Disorder: Assessing Pathways to Violence Using Large-scale Digital Records.”18 This article epitomizes the hybrid approach to urban studies. In it, O’Brien and Sampson utilize 911 dispatches and 311 call data to create measures of both social and physical disorder. These measures were then used to put Broken Windows theory to the test. The process of using existing administrative datasets as a method of estimating social and physical phenomena illustrates the inductive approach. As there is no explicit measure of public disorder those measures must be inferred. Whereas testing testing the efficacy of Broken Windows is indicative of the more traditional deductive process as it is an explicit testing of theory.
Quantifying disorder is no small task. In their 2015 paper the authors write
Taking up this challenge, O’Brien, Sampson, and Winship (2015) have proposed a methodology for ecometrics in the age of digital data, identifying three main issues with such data and articulating steps for addressing each. These are (1) identifying relevant content, (2) assessing validity, and (3) establishing criteria for reliability.19 20
This is an astute summation of the problems that arise with big data and how they can be overcome. The biggest of concerns, as mentioned in the opening of this section, is the validity of the data we are using.
5.2.1 Defining the phenomenon
The method that they propose requires us to do three main tasks. The first is to clearly define the phenomenon that we are hoping to measure. Following, we must idenify the relevant data. For example, in O’Brien and Sampson (2015), they define five ecometrics as
- Public social disorder, such as panhandlers, drunks, and loud disturbances;
- Public violence that did not involve a gun (e.g., fight);
- Private conflict arising from personal relationships (e.g., domestic violence);
- Prevalence of guns [sic] violence, as indicated by shootings or other incidents involving guns; and
- Alcohol, including public drunkenness or public consumption of alcohol.21"
These definitions provide clear pictures as to what is being measured. The next step is to surf through your data and do your best to match variables or observations to these measures. Then, through some process—usually factor analysis—ensure that these measures are truly relevant.
5.2.2 Validating the measure
Once an ecometric has been defined and properly measured, the next step is to validate it. This process is similar to ground truthing in the geospatial sciences. Often when geographic coordinates are recorded an individual will go to that physical location and ensure that whatever was recorded to be there actually is. This is what we are doing with our ecometrics. We have developed our measures, but we need to compare that to some objective truth.
In Sampson & Raudenbush (1999), they develop measures of physical disorder through their systematic social observation.22 But in order to validate their measures, they compared their results to those of a neighborhood audit.23 This audit served as their ground truth and was used to make any adjustments if needed.
5.2.3 Addressing reliability
This ecometric, like most others, is naturally a time bound snapshot of the social and physical world. These measures will change over time. Because of this it is useful to know how reliable the measure will be for different periods of time24. The authors do with with a bit of statistical finesse—e.g. factor analysis—that is best left to them to explain. But what we are to take away is that ecometrics are often time variant and it is important for us to know how flexible we can be with our time scales.