Digital Opportunities Compass: Metrics to Monitor, Evaluate, and Guide Broadband and Digital Equity Policy

Reposted from the Benton Institute for Broadband & Society Daily Beat, February 28, 2023 (access original post here). © Benton Institute for Broadband & Society 2023.”

The Infrastructure Investment and Jobs Act (IIJA), which includes the Digital Equity Act of 2021 (DEA), establishes a broad framework and significant funding to advance broadband connectivity and digital equity. The law recognizes key factors and populations to address when striving for digital equity. To fully realize the full benefits of digital technology for individuals, communities, and society at large additional insights are needed. The Digital Opportunities Compass is an holistic framework for broadband and digital equity planning, implementation and evaluation.

The DEA identifies the following five measurable objectives: (1) The availability of, and affordability of access to, fixed and wireless broadband technology; (2) the online accessibility and inclusivity of public resources and services; (3) digital literacy; (4) awareness of, and the use of, measures to secure the online privacy of, and cybersecurity with respect to, an individual; and (5) the availability and affordability of consumer devices and technical support for those devices. The DEA also requires that states identify and plan to address the digital equity needs of the following covered populations: low-income individuals; aging individuals; incarcerated individuals, other than those incarcerated in a Federal correctional facility; veterans; individuals with disabilities; individuals with a language barrier; individuals who are members of a racial or ethnic minority group; and individuals who primarily reside in a rural area.

IIJA also requires an assessment of how these five measures impact and interact with the state’s broader outcomes, specifically in the following areas: economic and workforce development goals, plans, and outcomes; educational outcomes; health outcomes; civic and social engagement; and delivery of other essential services.

Effective digital equity efforts will require that states complement the framework established in IIJA with additional insights from what we know about how broadband enables social, community, and economic development. With tremendous effort, states are developing a clear understanding of the current availability, quality, and affordability of broadband. Many realize that pursuing a longer-term digital equity strategy requires going beyond the mapping of availability, access, and access quality. However, there is less clarity on which other factors are relevant and should be considered when seeking to maximize the benefits of high-speed connectivity for community and economic development.

The Digital Opportunities Compass offers a framework to assist in the development of state plans that meet the reporting and assessment requirements of IIJA but go beyond access and affordability to fully harness the benefits of digital technology. As communities and states develop plans to improve digital equity, it is important to establish a shared framework, goals and priorities, to identify opportunities, and monitor progress toward these goals.

The approach presented here is closely tied to the findings of 25 years of research and experience of how broadband and device access, affordability, and digital literacy relate to digital equity and broader social and development outcomes. One clear insight is that achieving such outcomes requires consideration of additional factors beyond the five measurable objectives required in the IIJA.

The Digital Opportunities Compass: An Overview

Building on existing research, frameworks, and measurement tools, the Digital Opportunities Compass includes six components: Contexts, Governance, Connectivity, Skills, Application, and Outcomes. Each of these components includes indicators that have a bearing on the process and outcomes of digital equity initiatives “on the ground.” The indicators under each component allow stakeholders to do an assessment of their overall conditions in order to determine where additional areas of attention may be needed.

Six Components and Indicator Areas

Six components of the Digital Opportunities Compass

#1. Contexts—indicators related to sociodemographic, economic, and community level factors.

#2. Governance—indicators related to local, state, and federal policy, governance, and power.

#3. Connectivity—indicators related to the existence of necessary network infrastructure, as well as the accessibility, affordability, and adoption of internet service and network-enabled devices.

#4. Skills—indicators related to a broad range of activities centered around digital literacy (including secure online practices), training, and skills attainment.

#5. Application—indicators related to the uses and application of digital connectivity and skills, while considering additional sociotechnical contexts.

#6 Outcomes—indicators related to the broader effects of improved digital equity on individuals, communities, and states.

Potential Benefits

The Digital Opportunities Compass has several potential benefits for local, county and state policymakers. It can be used as a vehicle for inter-organizational collaboration and engagement, as well as a holistic framework for broadband and digital equity planning, implementation and evaluation, particularly for communities that may not know where to begin.

The Compass can be used to:

Identify key groups of factors that influence digital equity efforts and outcomes
Measure and assess digital equity efforts and outcomes over time
Utilize a standardized core set of metrics that can be expanded and customized to meet state and community needs
Build, as far as possible, on existing data and indices
Augment existing data with new (qualitative and quantitative) data
Innovatively design infrastructure to help automate data collection (e.g., quality measurement in routers)

Using the Digital Opportunities Compass in Practice

The Digital Opportunities Compass is a holistic set of metrics to monitor, evaluate, and guide state broadband policy. The framework is offered to support state agencies as they develop their digital equity plans this year. The Compass is also a comprehensive framework to ensure that additional factors and contexts are considered that research has shown can influence the outcomes and impacts of digital inclusion and broadband adoption initiatives.

The Digital Opportunities Compass is intended to be a starting point, rather than the final word, regarding the development of a comprehensive assessment framework to monitor, evaluate and guide state broadband and digital equity policy—now and in the future.

In this context, the Compass seeks to be useful both during the early stages of the Broadband Equity, Access, and Deployment (BEAD) and Digital Equity Act (DEA) grant programs this year , as well as during and at the end of the grant programs. The Compass should be useful to the National Telecommunications Administration as the agency seeks to develop a comprehensive and humanistic evaluation framework to assess the outcomes and impacts of these national programs.

Beyond IIJA, we hope that the Digital Opportunities Compass encourages deeper discussion, debate, and reflection not only regarding how to measure digital equity, but also to inspire new directions for research, practice, and policy in an increasingly networked society that demands broadband connectivity.

Download the full working paper, Digital Opportunities Compass: Metrics to Monitor, Evaluate, and Guide Broadband and Digital Equity Policy, from the Quello Center for Media and Information Policy at Michigan State University.

Dr. Colin Rhinesmith moderated a session at Net Inclusion 2023, titled “Building Local Digital Inclusion Ecosystems,” on Wednesday, March 1st at 10:00am (CT) (archived at https://www.youtube.com/watch?v=6YZPe3ZkWJ0, starting at 58:20) and Pierrette Renée Dagg participated in a livestream session, titled “Demonstrating Impact: Strategies and Methods for Evaluating Programs” on March 1st at 2:00pm (CT) (archived at https://www.youtube.com/watch?v=6YZPe3ZkWJ0, starting at 5:28:12).

Colin Rhinesmith (he/him) is the Founder and Director of the Digital Equity Research Center at the Metropolitan New York Library Council, a Research Fellow with the Quello Center for Media and Information Policy at Michigan State University, and a Co-Editor-In-Chief of The Journal of Community Informatics. Rhinesmith’s research examines the role of community informatics projects in creating and sustaining healthy digital equity ecosystems. Previously, Dr. Rhinesmith was an Associate Professor and Director of the Community Informatics Lab in the School of Library and Information Science at Simmons University. He has been a Google Policy Fellow and an Adjunct Research Fellow with New America’s Open Technology Institute, a Senior Fellow with the Benton Institute for Broadband & Society, and a Faculty Associate with the Berkman Klein Center for Internet & Society at Harvard University.

Pierrette Renée Dagg (she/her) is the director of Technology Impact Research for Merit Network at the University of Michigan in Ann Arbor, Michigan. The aim of her work is to bridge the gap between academic scholarship and practical application to positively impact technology and information equity. She is currently a PhD candidate at the University of Toledo in Social and Philosophical Foundations of Education and Education Technology, where she is completing a dissertation on AI in education. She is also a co-instructor at the University of Toledo. Prior to joining Merit, Pierrette held the position of Creative Director at Crain’s Detroit Business magazine. She has been the recipient of multiple awards from the Society of Professional Journalists, PRism, the National Academy of Television Officers and Advisors and was also an Emmy nominee. Pierrette graduated from the University of Toledo in Interdisciplinary Studies and earned the Magna Cum Laude distinction. She was also a member of Phi Kappa Phi and was named the Distinguished Graduating Scholar by the College of Adult and Lifelong Learning. Pierrette completed her Executive MBA from the University of Toledo College of Business and Innovation in September, 2018. She is a member of the International Artificial Intelligence in Education Society, the American Educational Research Association, the Michigan Council for Women in Technology, and the Pop Culture/American Culture Association.

Johannes M. Bauer (he/him) is Professor in the Department of Media and Information and the Director of the Quello Center for Media and Information Policy at Michigan State University. His research explores institutional choice and design problems raised by emerging technologies. Among other topics, he explores how public policy and management affect the benefits and risks of advanced information and communication technologies for individuals, communities, and society. He is currently working on a book on the governance of complex adaptive socio-technical systems.

Greta Byrum (she/they) is a digital equity practitioner and researcher working in the field since the 2010 Broadband Technology Opportunity Program. Over the last decade, she launched the Just Tech Program at the Social Science Research Council; founded Community Tech NY, a non-profit building collaborative community-owned broadband ecosystems with local partners; built the New School’s Digital Equity Lab; founded the Resilient Communities Program at New America, and led the Field Team at New America’s Open Technology Institute. Greta has a background in urban planning, communication technologies, and poetry, and serves on the boards of the Metropolitan New York Library Council, the Southern Connected Communities Project, the Howland Public Library Board of Directors, and the New York State Regents’ Advisory Board on Libraries.

Aaron Schill (he/him) is Director of Research & Programs at the National Digital Inclusion Alliance (NDIA). Aaron joined the NDIA team in January 2022. He is experienced at working with local leaders and elected officials to build consensus and develop place-based solutions to the most pressing community challenges. Throughout his career, Aaron has advocated for data and information access to empower communities to affect positive change. Prior to joining NDIA, Aaron was the director of data and mapping at the Mid-Ohio Regional Planning Commission, where he headed the agency’s work in data analytics, visualization, and GIS. He also established the organization’s strategy and programming for broadband and digital inclusion and co-led the creation of the Franklin County Digital Equity Coalition. Aaron’s career experience includes working as an urban planner and research director primarily in public and nonprofit organizations. He is a two-time graduate of The Ohio State University, with a master’s degree in city and regional planning and a bachelor’s degree in sociology.

The Benton Institute for Broadband & Society is a non-profit organization dedicated to ensuring that all people in the U.S. have access to competitive, High-Performance Broadband regardless of where they live or who they are. We believe communication policy – rooted in the values of access, equity, and diversity – has the power to deliver new opportunities and strengthen communities.

Reposted from the Benton Institute for Broadband & Society Daily Beat, February 28, 2023 (access original post here). © Benton Institute for Broadband & Society 2023.

TAGS: Broadband digital equality

Primary takeaways

Digital inequality shows larger impacts on youth academic performance as compared to time spent on screens.
Digital skills play a significant role in mediating unstructured online engagement (social media use, playing video games, browsing the web) and youth academic, social, and psychosocial development.
Unstructured online engagement and face-to-face social interaction are complementary and continuously interact to create and enhance youth capital outcomes.

A familiar story: concerns of screen time

Today’s discussions of adolescent well-being have coalesced around a clear narrative: teenagers spend too much time online, and their academic performance, mental health, and social lives are deteriorating as a result. A steady stream of academic papers, books, and op-eds, alongside a growing number of policy proposals––school phone bans, age-gated social media use, restrictive screen-time limits––rest on the same underlying claim, aligning with a contemporary, digitized version of the displacement hypothesis:

Screen time, particularly the unstructured, free-time spent on social media, gaming, watching video content, or browsing the web, is said to displace the productive face-to-face activities that build adolescents into capable adults.

The implied and often practiced solution is restriction. In response, this dissertation tested this claim directly, and placed it within the broader context of adolescence.

Across three years, I followed 653 Michigan adolescents from early through late adolescence: in grades 8 or 9 (survey one, 2019) to grades 11 or 12 (survey two, 2022). Notably, these students, studied over time, were part of a broader pooled sample of 5,825 students across the same eighteen highschools. The study window captured the year before and the year after the peak of the COVID-19 pandemic and related lockdown orders, functioning as an unprecedented stress test for theories of adolescent social, academic, and digital life and, importantly, as a benchmark to compare the effects of pandemic-related change and inequality to those effects from screen time alone.

Across four studies of adolescents, consisting of six cross-sectional and longitudinal analyses, findings are not consistent with the displacement narrative, nor the broader concerns about the time youth spend on screens.

Findings are, however, consistent with something the current public and (most) academic discussions have largely overlooked or ignored: the gaps and inequalities that determine whether adolescents can access and use the internet meaningfully in the first place.

What the displacement hypothesis overlooks

Displacement and related research and policy concerning the time young people spend online assumes a “zero-sum” model of adolescent day-to-day time. An hour online is an hour not spent studying, reading, sleeping, or interacting face-to-face (i.e., time spent on more productive or developmentally “better” activity).

Indeed, this makes sense logically. However, as an empirical claim, this model requires time spent online to behave differently from all other ways adolescents allocate time; it must produce uniquely negative outcomes and be inherently harmful across digital contexts, rather than the typical mix of trade-offs corresponding to, and often overlooked among any other social or developmental context.

Yet, online time does not differ from other youth activity. Instead, I find it has a mix of pros, cons, and even some “uniquely digital” benefits which youth utilize for social and academic gains. When I compared unstructured digital media use against traditional face-to-face interaction and activities, both produced similar patterns: some negative associations with academic outcomes, some null, and some positive.

Trade-offs within traditional face-to-face activity (for example, social time with friends and family, or time spent in after-school extracurriculars) are treated as ordinary developmental experiences that must be experienced for the betterment of development. The identical trade-offs involving digital time tend to be overlooked or ignored, and online engagement is perceived as altogether harmful.

A growing body of evidence, including this dissertation, do not support that distinction. Indeed, the developmental context is routinely misread, leaving out the context of the experiences and time spent on digital, as well as face-to-face activities, interactions, existing inequalities, and changes inherent to development. As such, I proposed a novel framework to understand these contexts:

Digital capital exchange

Rather than treating screen time as a unified harm, this dissertation advances an “exchange”-based framework, grounded in James Coleman’s theories of youth capital and digital inequality scholarship, particularly following Eszter Hargittai, Jan van Dijk, and Alexander van Deursen (see this list of all dissertation references for full works).

The core proposition is that adolescents’ online engagement is not an alternative to developmental activity but another, albiet modern domain through which young people accumulate and mobilize online resources––particularly digital skills––that work alongside existing social networks and experiences to be exchanged for human capital (measured as: academic achievement, aspirations, STEM interest) and social capital (peer networks, community participation, extracurricular involvement).

Online time is not the mechanism; instead, it is digital skills that I find to be the most vital component in youth capital exchange and enhancement. Unstructured online engagement contributes to online skills; those skills, accumulated and mobilized alongside existing peer, family, and community networks, translate into the outcomes researchers and parents care about, i.e., academic achievement, aspirations, and face-to-face interaction and social networks.

This digital capital framework treats online and in-person contexts as complementary rather than antagonistic, and it situates adolescents’ digital lives within the structural conditions––connectivity quality, device reliability, autonomy of use––that determine whether exchange can occur at all.

Methods (in brief)

Paper-and-pencil surveys were administered to students in classrooms at two time-points: spring 2019 (N=2,876) and spring 2022 (N=2,949), across the same eighteen predominantly rural Michigan schools, grades 8–12. Official, nationally-ranked standardized reading, writing, and math test scores (PSAT 8/9, PSAT 10, SAT; College Board) were then anonymously linked to students’ survey responses with the help of participating districts.

Cross-sectional path analyses modeled pooled and wave-specific samples (pooled N=5,825); two-wave cross-lagged panel models tested reciprocal, longitudinal relationships on the 653 students who completed both surveys. Multi-group analyses of the cross-lagged panel models compared relationships between girls (N=345) and boys (N=308). All longitudinal models included time-invariant socioeconomic covariates as well as time-varying covariates to reduce omitted-variable bias.

Key findings: an overview

To summarize, to the best of my ability, eight chapters across 376 pages, I present two primary findings:

First: digital inequality predicted larger and more consistent declines in human capital than screen time did.

Unreliable home internet and technology maintenance problems––experiencing and/or dealing with broken or outdated devices and software, restrictive school-issued hardware, issues with connecting to or maintaining internet access––decreased youth GPA and standardized test achievement. And, these effect sizes were substantially larger than any negative direct effect from unstructured digital media use.

Across all four empirical studies, digital inequality emerged as the most substantial predictor of academic and developmental decline.

Second: digital skills mediated the relationship between online time and adolescent academic and social outcomes.

Unstructured digital media use, particularly online gaming and web browsing, predicted higher internet and social media skills for adolescents, which in turn predicted stronger academic achievement and self-efficacy (human capital), and social interaction and extracurricular participation (social capital). The positive indirect effect of screen time through skills offset or exceeded any small negative direct effects across several outcomes (supporting our existing peer-reviewed work: Hales & Hampton, 2025, and which you can read more about here).

These exchange processes were amplified when peer and family networks were modeled alongside digital skills, consistent with the premise that online and offline contexts operate together rather than in competition. The effect was not universal: social media skills amplified rather than offset a negative association with consistency of interest, one of the two subscales of grit. The exchange framework describes a contextual and conditional, domain-specific mechanism, not a blanket defense of time spent online.

Implications

If digital inequality, and not screen time, is the primary predictor of adolescent academic and developmental decline, and still warrants concern regarding access quality and experience even with the broader adoption of digital devices across the United States, the current policy emphasis on restriction is pointed at the wrong target. The evidence supports a different set of priorities.

Stable, reliable home (fast) broadband should be treated as an educational prerequisite rather than a consumer amenity. Unreliable connectivity exerted larger downward pressure on human capital than any measure of screen time, and that pressure intensified during the pandemic-era reliance on digital infrastructure. Technology maintenance, device repair, replacement, technical support, and the flexibility to install software and explore the web autonomously, matters as much as initial access, and school-issued devices that restrict autonomous use appear to hinder skill accumulation rather than support it.

Restrictive parental mediation of internet use was negatively associated with grit and self-efficacy at magnitudes comparable to the positive contributions of face-to-face activity. This challenges the assumption that digital restriction functions protectively. Instructive mediation, teaching adolescents to verify information, navigate platforms critically, and mobilize online resources toward meaningful ends, is the posture the data supports.

Finally, the technical skill-building that occurs through gaming, self-directed exploration, and deep web use is skill-building, not wasted time. Closing the persistent gender gap in these domains likely requires legitimizing technical play for girls, rather than restricting it for everyone.

None of the above is an argument that screen time is benign. It is an argument that screen time is the wrong focus, particularly when studied mostly in isolation. Context matters substantially, whether that is time spent on other activities during adolescence, the period of adolescence itself, digital inequality, resources gained from such online use, and how all such factors interact. The factor that predicts whether a given adolescent can convert online engagement into capital outcomes is structural: access, infrastructure, skills, and the autonomy to use them. These factors are distributed unevenly, and its uneven distribution, not hours logged, is what separates adolescents who thrive from those who fall behind.

The full dissertation is available through Michigan State University’s ProQuest archive, or see the embedded full-text PDF below. I’m happy to share papers, preprints, or the underlying framework with anyone interested and working in this area––don’t hesitate to reach out via my contact form. Thanks for reading.

Tags: Broadband digital equality

Digital Opportunities Compass: Metrics to Monitor, Evaluate, and Guide Broadband and Digital Equity Policy