Penn’s Scientific Method Man

Media Bias Detector

It’s difficult to ignore the division in popular media these days. But why is the division so overwhelming? Watts, after observing the phenomenon, decided to investigate. “Our hypothesis is that this division is exacerbated less by lies or ‘fake news’ than by bias.”

The Lab found a partner in Richard Mack, W’89, who was excited by both the inquiry and one of its outcomes: the Media Bias Detector, which uses large language models (LLMs) to identify media bias in headline news.

Unlike similar tools, the Media Bias Detector relies not on the reputation of publishers but rather on the construction and language of the articles themselves. “We can analyze stories in real time, and our tool is better able to capture the heterogeneity within different news organizations over time,” says Watts. “For example, over President Trump’s first 100 days in office, the Media Bias Detector was able to show what the media covered most heavily and also what was ignored—tariffs and protests, for example—by certain publishers.”

For Mack, one of the Media Bias Detector’s greatest advantages is its location at Penn. “Housing something at Penn, a not-for-profit focused on academic research, is a really powerful model. It’s a model that isn’t advancing a specific point of view or business interest,” he says.

It also ensures that the Media Bias Detector is grounded in a certain academic rigor—something many organizations simply do not have the expertise or resources to achieve. That quality informs everything from how Watts and his team collect data to how they define, identify, and quantify bias on an algorithmic level.

It’s a reputation that earns them points not just with the average consumer who turns to the Media Bias Detector for information, but with industry leaders as well. “We have a number of data partnerships, and we’re forming more,” says Jeanne Ruane. “Having a breadth of data is crucial for the quality of our research, and it benefits not just our own work and that of our partners, but researchers around the globe.”

The Common Sense Project

In 2024, researchers at the CSSLab noticed a lack of common sense—or rather, the absence of any agreement on the definition of what, exactly, common sense is.

As a result, the team launched The Common Sense Project, an international survey designed to define and measure what people around the world consider “common sense.” The international survey is already uncovering how those assumptions differ across groups. Eventually, Watts hopes, the survey data will help AI and machine learning researchers better model common sense.

“Common sense has always been a challenge for AI, and the problem is that we don’t understand how common sense works in humans to begin with,” observes Watts. By registering the differences in how people understand common sense, Watts and his team of researchers hope to identify its underlying principles, which computers could then use to navigate its often-contradictory nature—with applications ranging from agentic AI to decision-making algorithms.

Integrative Experiments

The structure of behavioral science experiments will be familiar to anyone who’s taken high school psychology: first, a question (“Why do people cooperate?”), then a hypothesis, and then, ultimately, an experiment designed to prove the hypothesis either true or false. According to Watts, however, this tried-and-true approach to scientific inquiry has its limitations. “At the end of the day, for one phenomenon we may have dozens or even hundreds of theories,” he says. “The complexity of relevant variables makes it difficult to build cumulative knowledge based on individual findings.”

So CSSLab is redesigning the process, defining every variable that can be manipulated in an experiment and developing predictive AI models that can explain which of these variables contribute—and in what degree—to specific outcomes. “Because we’re developing a general framework, there’s real potential across almost any discipline,” adds Watts. “These studies could inform everything from de-escalation strategies to designing nudges that are actually able to help people take advantage of public services, for example.”

Network for Open Mobility Analysis and Data (NOMAD)

To understand NOMAD, think back to the early days of the COVID-19 pandemic. As the virus spread, private companies opened their troves of geolocation data to academic researchers, hoping to improve developing disease models. It was an altruistic gesture, and at the time, researchers were “very, very excited,” recalls Watts.

Soon, however, it became clear that the data had limitations. For starters, the data is privately owned, so researchers could only access what companies chose to share. There was also little transparency into how the data was collected, which made it difficult to interpret results and build reliable models.

To address that gap, CSSLab launched NOMAD, a National Science Foundation–funded initiative that collects high-quality data and builds algorithms for its analysis. “It’s a one-stop shop for geolocation data, without researchers having to spend years building relationships with data providers, fronting the money, and figuring out everything themselves from scratch,” describes Watts. It’s an obvious benefit for researchers, and one with larger implications down the road. Watts and his team envision a future of improved crisis response, more accurate transportation maps, and policies informed by real population data.