In response to my post "Do
'Alien Species' Exist in the Universe?" Ian Reid wrote:
"Prof Hake . . . . has had a methodological
issue with the mainstream of educational psychology that traces back to a
disciplinary disposition to avoid 'test-retest' data. At least 50 years
ago most in ed psych were taught to avoid 'change scores' because they
have two sources of error and are thus less reliable than single assessment
data."
Unfortunately "what ed psych were taught"
has been shown over and over again to be patently incorrect. The seminal edict "How
we should measure 'change'- or should we?" has been countered in
"Should We Measure Change? YES!" [Hake (2007)] and "The
Impact of Concept Inventories On Physics Education and It's Relevance For
Engineering Education" [Hake (2011a)].
In addition, in a post "Academically
Adrift?" [Hake (2011b)] I wrote [bracketed by lines
"HHHHH. . . . ."; see that post for reverences other than Chronbach
& Furby (1970), Hake (2004a), National Academies (2008), and NCSU (2011).
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
In sharp contrast to the invalid (e.g., Student
Evaluations of Teaching - for a critique see Hake (2005b)), and the indirect
and general-thinking measures listed above is the *direct* measure of students'
higher-level *domain-specific* learning through pre/post testing using (a)
valid and consistently reliable tests *devised by disciplinary experts*, and
(b) traditional courses as controls.
Such pre/post testing, pioneered by economists
(Paden & Moyer 1969) and physicists (Halloun & Hestenes 1985a,b), is
rarely employed in higher education, in part because of the tired old canonical
objections that have prevailed ever since Chronbach & Furby's (1970)
"How we should measure 'change' - or should we?" and were more
recently lodged by Suskie (2004), and countered by Hake (2004b, 2011b) and
Scriven (2004).
Despite the naysayers and its apparent dismissal by
a large segment of the evaluation community [e.g., Shavelson & Huang
(2003), Suskie (2004a,b), Hersh (2005), Shavelson et al. (2008), Shavelson (2009), Benjamin et al.
(2009), Chun (2011), CAE (2011)], formative pre/post testing is gradually
gaining a foothold in introductory astronomy, biology, chemistry, economics,
earth sciences, engineering, math, and physics courses - see, e.g., Hake
(2004a), National Academies (2008), and NCSU (2011) for references.
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
Richard Hake, Emeritus Professor of
Physics, Indiana University; Honorary Member, Curmudgeon Lodge of Deventer, The
Netherlands; President, PEdants for Definitive Academic References which
Recognize the Invention of the Internet (PEDARRII); LINKS TO: Academia
<http://bit.ly/a8ixxm>; Articles <http://bit.ly/a6M5y0>; Blog
<http://bit.ly/9yGsXh>; Facebook <http://on.fb.me/XI7EKm>;
GooglePlus <http://bit.ly/KwZ6mE>; Google Scholar
<http://bit.ly/Wz2FP3>; Linked In <http://linkd.in/14uycpW>;
Research Gate <http://bit.ly/1fJiSwB>; Socratic Dialogue Inducing (SDI)
Labs <http://bit.ly/9nGd3M>; Twitter <http://bit.ly/juvd52>.
REFERENCES [URLs shortened by <http://bit.ly/>
and accessed on 19 Jan 2014.]
Cronbach, L. & L. Furby. 1970. "How we
should measure 'change'- or should we?" Psychological Bulletin 74: 68-80,
an abstract is online at <http://bit.ly/LqCSHy>.
Hake, R.R. 2004a. "Re: Measuring Content
Knowledge," POD posts of 14 &15 Mar 2004, online at <http://bit.ly/hwvI7t
> and <http://bit.ly/eXPBdE>.
Hake, R.R. 2007. “Should We Measure
Change? Yes!” online as a 2.6 MB pdf at <http://bit.ly/d6WVKO>. To appear
as a chapter in "Evaluation of
Teaching and Student Learning in Higher Education" (in preparation).
Hake, R.R. 2011a. "The Impact of Concept
Inventories On Physics Education and It's Relevance For Engineering
Education," invited talk, 8 August, second annual NSF-sponsored
"National Meeting on STEM Concept Inventories," Washington, D.C.,
online as an 8.7 MB pdf at <http://bit.ly/nmPY8F>
or as ref. 64 at <http://bit.ly/a6M5y0>. The abstract and
link were transmitted to various discussion lists and are also on my blog
"Hake'sEdStuff" at <http://bit.ly/pYmIkP>
with a provision for comments.
Hake, R.R. 2011b. "Academically Adrift?"
online on the OPEN! AERA-L archives at <http://bit.ly/gwJD0W>.
Post of 29 Jan 2011 10:00:09-0800 to AERA-L and Net-Gold. The abstract and link to the complete
post were transmitted to various discussion lists are also online on my blog
"Hake'sEdStuff" at <http://bit.ly/hVYzHI> with a provision for
comments.
Hake, R.R. 2014. "Do 'Alien Species' Exist in
the Universe?" online on the OPEN! AERA-L archives at <http://bit.ly/1kHEm0i>.
The abstract and link to the complete post are being transmitted to several
discussion lists and are also on my blog "Hake'sEdStuff" at <http://bit.ly/1dCUNHr>.
Labov, J.B., S.R. Singer, M.D. George, H.A.
Schweingruber, & M.L. Hilton. 2009. "Effective Practices in
Undergraduate STEM Education Part 1: Examining the Evidence," CBE Life Sci
Educ 8(3): 157-161; online at <http://bit.ly/cRc0JC>.
This is a discussion of the "Workshop on Linking Evidence and Promising
Practices in STEM Undergraduate Education" [National Academies (2008)].
National Academies. 2008. Meeting 2 of 13-14
October containing commissioned papers online at <http://bit.ly/Ku9NKD>.
See also the commentary on these workshops by Labov et al. (2009).
NCSU. 2011. "Assessment Instrument Information
Page," Physics Education R & D Group, North Carolina State
University"; online at <http://bit.ly/9gfUpY>.