Actual public school middle school math / US Dept. of Education, CC BY 2.0 |

75% of our nation's high school
seniors probably aren’t mathematically incompetent, despite a recent
recurrence of the nearly annual hysteria to the contrary.

**Associated Press, May 7, 2014:**US 12th graders make an "ABYSMAL showing" in math and reading according to the 2013 NAEP results. "Only about one-quarter are performing

__proficiently or better__in math and just 4 in 10 in reading."

**"America's high school seniors lack critical math and reading skills for an increasingly competitive global economy."**(emphases added)

PANIC! CRISIS! Blood in the streets! Dogs and cats getting married ...

BUT ... WAIT a minute. Is this really a
crisis? Are 75% of our 12

^{th}graders honestly mathematical nitwits, as reporting on the NAEP results seems to imply? I wanted to know, so I mucked around the NAEP website.
Based on what I gleaned from the NAEP Test Framework,

**the skills that the****"Basic up to almost Proficient" students generally lack**(compared to the "__Proficient and Above__" students)**are**These skills are necessary in specific STEM-related positions where advanced mathematical concepts are utilized on a daily - or even monthly - basis. "STEM-related" jobs are anticipated to number just 5.6% of all jobs in 2022.*unlikely**to negatively impact*their*general*readiness for college or work.All hand-wringing aside,

**this is not a crisis.**

**Q1: How is "Proficient" defined?**How does it fit into the rubric of the performance levels that the National Center for Ed Statistics uses?

**A1:**The 2013 NAEP Math Assessment Framework defines the levels as follows:

“

__Basic__denotes partial mastery of prerequisite knowledge and skills that are fundamental for proficient work at each grade.__Proficient__represents solid academic performance for each grade assessed. Students reaching this level have demonstrated competency over challenging subject matter, including subject-matter knowledge, application of such knowledge to real-world situations, and appropriate analytical skills.

__Advanced__represents superior performance.

These levels are intended to provide
descriptions of what students

**should**know and be able to do in mathematics.” (Emphasis in original)
As to the last point, it is not clear which of
the first two describes the level at which all or even most students *should*
minimally be performing. The definition of "Basic" is defined
in reference to “Proficient,” which leaves something to be desired.
Similarly Proficient’s “solid academic performance for [the] grade” doesn’t
resonate with meaning for me, either. Maybe it’s a term of art well
understood among educational assessment and statistical professionals, but not
by the likes of me.

**Q2: What exactly does one need to do to demonstrate knowledge and ability at a "Proficient" level for math?**

**A2:**I’m wading through the 88 page Math Assessment Framework, but that’s a post all its own. In brief, the 12

^{th}graders’ assessments cover what they “should know.” The assessment’s drafters note that “what they should know” includes concepts usually introduced in courses

*beyond*the standard, three year high school math sequence of two years of algebra, one year of geometry (Algebra I, Geometry, Advanced Algebra & Trig.) In going through the framework, one can see a significant number of objectives which are asterisked to indicate that these are generally covered in pre-calculus advanced math (standard-track 4

^{th}year high school math), calculus (advanced placement senior math), or beyond.

Also worth noting: the test was administered part-way through the academic year, between January and March 2013.

**The students were tested on a number of concepts that most, even those pursuing a standard**

*four-year*math track, hadn't even encountered yet.**The vast majority of students would do well to master basic concepts through Algebra I and some geometry for work and for life outside of it. An understanding of statistical methods allows students to make some sense of the many statistics thrown around - and often misinterpreted - on a daily basis, and I'm heartened to learn that K-12 students are now learning more about statistical methods than we did 20-some years ago. A perhaps substantial minority would also benefit from a good understanding of some basic calculus concepts (which is generally taught in high school honors classes in the senior year anyway, and may or may not be included in the NAEP.) But trig? Not so much. (*N.2.)**

**What percentages performed at the various thresholds measured?**

Q3: How did 12

Q3: How did 12

^{th}graders perform overall?**A3:**

"Below Basic" level: 36% of student
test-takers

"At or Above Basic": 64%

"At or Above Proficient": 25%, and

"At Advanced": just 2%.

What to take away from these data points depends
on what it means for a 12

^{th}grader to perform in the range between “Basic” and “Proficient.”**But I suspect that rather than panicking about our "At or Above Proficient" showing, it would make more sense to a bit more calmly, and more deliberately, concern ourselves about the 36% who are "Below Basic."**This figure may still be cause for concern (I emphasize, MAY), but it’s certainly not the CRISIS implied by focusing on the fact that 75% of students didn't perform at a level meeting or exceeding “Proficient.”**Q4. Does the difference between the skills and knowledge generally exhibited by "Proficient and Above" students versus the "Basic up to almost Proficient" students actually MATTER**in terms of their readiness for college or work, outside of specific STEM-related fields where advanced mathematical concepts are utilized on a daily - or even monthly - basis?

If not, then the percentage of students scoring
"Proficient or Above" does NOT actually tell us anything meaningful
about the GENERAL college and workplace "readiness" of America's 12th
graders. All it tells us is what percentage are prepared to pursue
further education and training in math-intensive fields. And if that's
the case, 25% "Proficient or Above" is probably just fine, or even
well AHEAD of where we need to be.

Why? Well, the U.S. Bureau of Labor Statistics projects that
"Employment in occupations related to STEM—science, technology,
engineering, and mathematics—is projected to grow [from 8 million] to more than
9 million between 2012 and 2022." The BLS
further projects that total jobs will number around 160.9
million in 2022. In other words, STEM jobs are anticipated make up just a
bit more than FIVE PERCENT of all jobs in 2022 (5.6%, to be precise.)

But isn't the demand/need for STEM workers
growing terribly rapidly? No. STEM jobs are expected to comprise
only a slightly higher portion of new U.S. jobs in 2012-2022: 1 million out of
15.6 million new jobs, or 6.4% of them. (*N.1.)

**A4: Based on what I've gleaned from the Test Framework thus far, I suspect the answer is actually ... NO. Not really.**In 2022, about 94% of workers will be employed in jobs that are NOT STEM-related. Thus, for the overwhelming majority of US workers, the ability to solve quadratic equations, tell the difference between a sine wave and a cosine wave, or use ANY of the knowledge one is supposed to acquire from a high school trigonometry course (usually taken in junior year) is unlikely to have ANY practical use in their work life.

**Q5. What proportion of student test-takers who'd been enrolled in honors-level math performed at the "Proficient" level? How about at the "Advanced" level?**

If "Advanced" is actually meant to
encompass the understanding necessary to pursue further education and a career
in a math-intensive field, which is what it seems like it should mean, then we
would expect the overwhelming majority of honors math students – particularly
those who are studying calculus – to perform at the "Advanced" level
on the NAEP. If the actual results are substantially lower, then either
"Advanced" doesn’t mean what it should, or there are fundamental
problems with the testing methodology (the way it's written and/or scored.)

**A5:**I wasn't able to find a direct answer to this question. But I was able to ascertain that

**a significantly greater share of high school seniors had studied calculus, the typical 12th grade honors-track math course, than performed at the "Advanced" level on the NAEP test.**

**In 2009, the most recent year for which data is available, 16% of students graduating from high school had already taken a calculus course. Granted, not all 12**

^{th}graders actually graduate in a given year, and the NAEP results are supposed to be generalizable to the population of all 12

^{th}graders, not just those who graduate.

Using 12

^{th}grade enrollment and graduation figures for public schools, and assuming that private school seniors aren’t

*more*likely than their public school peers to fail to graduate (which seems a reasonable assumption), we can reach an estimate of the percentage of all seniors who'd studied calculus.

2008-2009 figures here

**and here allow us to estimate that****at least 9.5% of all 12**^{th}graders that year were taking calculus or had previously.**That’s almost five times the 2% rate at which 12**Which leaves me thinking that “Advanced” is likely a much loftier level of achievement than the ordinary reader (this one included) would expect.^{th}grade NAEP test-takers scored at the “Advanced” level in 2013.Amanda Paulson at the Christian Science Monitor reported that 50% of all students scoring in the top quartile had taken calculus, and an additional 34% had taken pre-calculus. Since only 25% scored "Proficient or better," this tells us that

**84% of those scoring at "Proficient" or better had taken (or were in the midst of taking) the fourth-year standard-track**

*or*honors-track math class.**... These are just some of the reasons to be skeptical about the 12th grader readiness "crisis" allegedly indicated by the NAEP math test results.**The sheer number of the advanced mathematical concepts listed in the Testing Framework, to be discussed later, provides further, substantial support for such skepticism.

*N.1. The on-going frenzy over STEM jobs and
STEM-preparedness begs further exploration, and seems to be a topic worthy of
its own series of posts in the future.

*N.2. High school trig teachers: please tell me
if I'm missing something here. I actually did use this knowledge in
studying economics in grad school, but after recently reviewing a comprehensive
trig outline, I am hard pressed to identify any practical uses for this material
outside of "STEM fields."

## No comments:

## Post a Comment