Stats Only Measure What You Ask

The one where I interrogate un/employment statistics


Every other year when I teach the required methods course for one of my departments, I start the quantitative section by explaining “Stats only measure what you ask.” This is why we spend so much time on qualitative methods first, not just because I prefer them or because, as my students fear, quantitative methods are “so hard” I’m trying to “spare them as long as I can”. The fact is: if you don’t know how to ask questions then you will never know how to measure the answers with raw data nor how to interpret someone else’s data sets.

cafe press product for

While it may seem that this information is the exclusive domain of advanced undergraduates or graduate students in certain disciplines and everyone else can simply be thankful they never had to take stats, the fact is we are surrounded by statistical information all the time. (I know I sound like you high school math teacher justifying Geometry, bare with me.) Almost every time you watch national news, several times throughout the year on your local news broadcast, and almost daily on both pop culture shows like The View and news commentary shows like those that run on both Fox and MSNBC, stats play a key part in discussion of our lives. When you don’t know how to ask questions to interpret the data given, you find yourself quoting opinion poll numbers as if they are indisputable or angry later because you find out the information does not measure what you were told it did.

Case in point: The National Un/Employment Statistics

The [U.S.] president has come under fire this week for Bureau of Labor employment data that claimed to have decreased overall unemployment in the U.S. but was actually counting mostly temporary census jobs. THe BLS reported the creation of 431,000 in the last quarter. Both the government and pundits used this number to argue that the economy was getting better and that jobs were on the rise.

No one asked how the BLS measures those jobs, ie what questions it asks to generate its numbers.

CUNY Job Fair/Alvarez/NY Daily News

A huge portion of the jobs created were temporary positions with the Census. The BLS does not exclude part time, temporary, seasonal, etc jobs from its count. Nor do they exclude people who for any reason left (of their own volition, were laid off, or fired) their job and then returned to it. What they count is the number of people reported to be hired in any given quarter.

If you don’t know the questions asked, you cannot understand the data. So because everyone took the numbers released and ran with them, when the 585,729 census workers began to be permanently laid off and the unemployment numbers began to rise again, people who had used the BLS to claim the economy was getting better cried foul. Suddenly there was talk of intentional manipulation of numbers and economic realities to make the President look better.

Enter the next set of misleading numbers released today: Jobless claims at lowest rate since 2008.

According to several agencies charged with measuring unemployment in the U.S., jobless claims have dropped by 3000 people this month. This number measures the number of people either still claiming unemployment or opening new cases for unemployment benefits.

What questions are not being asked?

The decreased number does not include the impact of the 99ers, people for whom all types of extensions have been exhausted and who are therefore kicked off of unemployment. According to HuffPo, the 99ers will likely represent 1 million people this year with no benefits, no incomes, and no statistical representation in the numbers both the government and the media are using to make claims about the health of the economy.

Nor does it count people who have not claimed unemployment because they are steadily burning through their finances or because they have obtained part time, temporary, work that makes them ineligible for benefits but does not constitute actual stable employment. And as always, these numbers only count people in the system not those who for one reason or another have never claimed benefits despite being unemployed.

unemployment office TN/ AP Photo/Josh Anderson

If we ignore these factors, as many of the people reporting on the new numbers have done this am, then we are likely to come to the same erroneous conclusions the BLS numbers caused earlier, ie: the economy is doing better and more people are employed than ever.

The reality is far more bleak. May 2010 saw the start of the 99 week shut off for unemployment benefits across the country (for those states who did not get extensions, these cut offs may have come earlier) and June is predicted to be the largest cut off month in the history of unemployment because of the number of people claiming benefits who have exhausted their allotment.

Add to this the fact that the Labor Department says only 41000 jobs were created in May (AP) and that most Census workers have been or will be permanently laid off and you begin to see a picture that any one of these numbers renders hidden by not asking the right questions.

Presidential Blame?

2004 Unemployment Line Protest outside the RNC held by PFAW/ Dave of

Many are chomping at the bit to blame the discrepancies in the optimistic way the numbers are being reported and discussed and the realities they hide on the President. On the one hand, they are right to note that these numbers are being used to shape policy as well as perception. On the other, 1.2 million jobs were lost in the waning years of the Bush administration and these losses as well as the economic crisis in general all started on Republican watch. Partisans and the past aside, many of the current policies, like the ongoing decision not to extend unemployment benefits any more, are not made by the President but by Congress. Unlike the N. American people, Congress has much more information about how un/employment stats are kept and a greater access to and impetus to review all of the key stats. When they choose not to do that because of erroneous conclusions like “huge spikes in jobs in April and May mean that the economy is bouncing back” or “unemployment claims are lowest they’ve been since 2008 so people have found jobs”, they are guilty of intentional ignorance that ultimately leads to policy decisions that worse the economic lives of many N. Americans living on the edge.


While Congress chooses to look the other way, the mainstream media is leading N. Americans astray.

In reporting un/employment statistics without breaking them down for the average reader and then proclaiming proof that the depression is finally over, something various outlets have done periodically over the last 2 years, the media creates the false sense that the nation is in recovery.  While in other times this might have created a needed moral boost, those times have long gone. Instead, what these declarations do is lead to a continued or increasing sense that unemployed workers are to blame for their circumstances. From employers who dare to question why people have been unemployed for 3 months to 2 years because the media has told them there are 411,000 new jobs out there or families and friends who have managed to hold on to their jobs or because of a lack of specialized skills have managed to find new jobs (albeit at lower rank or pay) vilify or question people with more specialized skills or larger income to debt needs who not only cannot but likely will not be employed by the local grocery store. The media’s misleading conclusions create a vacuum in which the humanity of the unemployed is staunchly sucked out in favor of the false promise that unemployment is a thing of the past and only losers don’t have jobs.


When additional information about the widely reported numbers comes to light, politically motivated pundits jump on it and start a whole new media spin. In this version, the numbers are intentionally manipulated by a corrupt government trying to destroy the [white] working class. Given the kind of mythos the media creates around these numbers, it is not only card carrying supremacists who are inclined to hear this misguided warrior cry. Many people who have sat across the table from a smug employer asking “Well what have you been doing for the last 2 years?” are likely to feel the kind of resentment, dehumanization, and anger that makes blaming the government easy. Filtering those emotions through the lens of race turns that blame into a force similar to those during and immediately after reconstruction or the employment organizing in the 30s that in places like CA, Chicago, and other big cities was predicated on racial supremacy and xenophobia (particularly the vilification of and occasional violence against API immigrants and black people). These sentiments in turn fuel the continued unemployment of people of color which is much higher than that of struggling white communities, while still targeting them as potential threats to the economy.

Getty Image of Unemployment Line date unknown

Resentment amongst communities of color are also on the rise. While partially spurred on by the media myth-making and its consequences, it is also tinged with the failure of the President to implement any programs specifically targeted to racial communities most hit by layoffs and lack of rehire. Despite statistics that show people of color were among the first fired and the last hired and that certain groups’ unemployment rates outpace the national average by almost 5 times as much, neither Congress nor the President have made any move to deal with the intersections of race and class, race-class-gender, or even class and gender during this undeclared recession/depression. And those with long memories, still recall how the President avoided the direct question about how he was helping the 50% unemployed African-American males in NYC during his one televised press conference on unemployment. These issues of course also get us back to other missing factors in BLS data which can and does break things down by both race and gender but whose most cited stats do neither.


What is reported about the economy in the dailies is only part of the picture. Neither the people collecting individual statistics nor the people making huge erroneous leaps from them can be trusted without first remembering: stats only measure what you ask. While the blame lies largely on other shoulders than the President’s it is also imperative that he takes leadership in disseminating clearer information and demanding Congress act on it. We too must take responsibility for being informed and cross-checking any information we receive. In order to be informed advocates for economic renewal and equitable job development, we have to start asking what questions were asked to generate the stats we see and whether those questions justify the conclusions people are making. When we do that, we will not only develop a more accurate analysis of the economy but hopefully defuse some of the anger, resentment, and judgment that is currently permeating our nation and fostering conflict.

You can start by signing the 99er petititon here

2 thoughts on “Stats Only Measure What You Ask

  1. Excellent points. Just like a computer only gives back what one puts in. Which is to say: we learn only what we want to know.

    • good point, that too is part of the knowing how to ask questions puzzle. If you are doing research & you don’t know how or where to ask …

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s