Fear Not the Oral Exam Kermit Wannabe PhD, Examiners for Harvard Would Pass Thee

Kermit is ready to storm the Ivory Tower armed with a high tech laser light sword to fight off the Committee who would turn him back.

What delightful silliness.

Evidently the committee doesn’t know or credit Kermit with a research proposal and a lit review done under the guidance of a supervisor.  All they know is he wants to be one of them and he’s gonna have to defend against their assault to get a PhD, just like they did.  So defend your work Kermit.

Kermit, like other PhD wannabes before him, doesn’t have experience or preparation with high stakes oral exams. The exam will prime him to deliver dramatics to another hopeful PhD wannabe later in his career.

The committee might let Kermit pass when he doesn’t deserve to pass.  See the tale of Jason Richwine’s Harvard Kennedy School of Public Policy doctoral committee who gave him a pass and later refused responsibility for passing his dissertation linking race and IQ.

“The dissertation was approved, as all dissertations are, by a committee of three. The chair was George Borjas, an conservative economist who writes about immigration for National Review and The Wall Street Journal. Borjas told Slate’s David Weigel, “I have never worked on anything even remotely related to IQ, so don’t really know what to think about the relation between IQ, immigration, etc.… In fact, as I know I told Jason early on since I’ve long believed this, I don’t find the IQ academic work all that interesting.” Not exactly an endorsement of the dissertation.”

Why is Borjas on the committee if he has never worked on anything even remotely related to IQ?  In some oral exams, the chair of the committee is not required to read the dissertation.   Is Borjas not responsible for the exam outcome because he merely chaired the proceedings?  What exactly are the responsibilities of every member of the committee?  Who checks the correctness of the examiners?

“The second person on the committee was Richard Zeckhauser. He studies investing, not immigration, and his Harvard faculty website describes him as “a senior principal at Equity Resource Investments (ERI), a special situations real estate firm.” He said “Jason’s empirical work was careful,” but that he was “too eager to extrapolate his empirical results to inferences for policy.”

So why did Examiner Zeckhauser give Richwine a pass?  Why is he an on the examining committee for Richwine if he identifies himself primarily as a guru of specialty real estate investments?

“The third member of the committee is the big surprise, and the big problem: Christopher Jencks, for decades a leading figure among liberals who did serious research on inequality—a contributor to The New York Review of Books, the author of important books, including Inequality: Who Gets Ahead?, The Homeless and The Black White Test Score Gap. Christopher Jencks knows exactly what’s wrong with the studies purporting to link “race” with “IQ.

Examiner number three, Jencks would not give a public comment when asked.  The italicized text above comes from the article, Why did Harvard Give a Ph D for a Discredited Approach Linking Race and IQ? by Jon Weiner in The Nation in May of 2013.

The writer doesn’t answer the question he asks: Why Did Harvard Give Richwine a PhD?  Did the committee members just phone it in trusting in the shepherding work of Richwine’s Harvard supervisor?  Since the story broke teachers in doctoral programs have examined Richwine’s dissertation and commented that it fails the test for a sound and robust research project.  So why did Richwine’s supervisor advance the process?  Why did none of the checkpoints stop this work from going forward?

There is no academic literature on oral exams, or on the deliberations of examiners or approaches to supervision.  For all the literature on test construction, there is none on oral exams.  Oral exams should adhere to the principles of reliability, validity, inter-rater reliability etc., but they don’t because they are more like rites of passage rooted in the traditions of the middle ages. Vivas lack reliability and validity (Watts, 2012); they are not robust assessment instruments as can be seen by the tale of these three examiners for a Harvard PhD.

If the committee did fail Richwine, Richwine could sue and win.  The oral exam lacks credibility as an assessment instrument. The supervisor approved his methodology in his research proposal and he passed an ethics review.  And the credibility of the Harvard Ph D would unravel.

Kermit take solace in Jason Richwine’s experience.  The camel will get through the eye of the Harvard needle; few oral defenses fail, even when they should.  Oral exams are more akin to a Bar Mitzvah than an exam. No one is going to fail you Kermit.  (You could sue.) There now Kermit wannabe PhD, don’t you feel better?


Watts, Jacqueline. H. (20!2)  Preparing doctoral candidates for the viva: Issues for supervisors and students. Journal of Further and Higher Education. 36(3) pp. 371-381.

Beware of Fakers in Graduate Education Modernization: A Cautionary Tale

The White House Office of Science and Technology Policy (OSTP) sends out Dr. Joan Frey to take up the important cause of graduate education modernization. (Caution: Dr. Frey and the graduate education modernization initiative are not found on the OSTP website, so maybe this is an internet scam or so unimportant that it is not worthy of a link or search result on the OSTP site.  As no great advantage arises from scamming, the latter may be truest.)

Dr. Frey knows the reasons for modernization, rapid technological change in the 21st century, non-academic career paths, 50 000 PhDs conferred yearly in the United States etc.

Dr. Frey also knows that American graduate education is the envy of the world.

She doesn’t refer to or build upon the success of Carnegie Initiative on the Doctorate (CID) or of the work at the University of Minnesota with the Graduate Review and Improvement Process.

She doesn’t know the simple advice from the success of the CID; doctoral programs can undertake modernization via a process of ongoing renewal.

She has not enlisted even American experts in doctoral education.

She doesn’t refer to the scant literature in doctoral education, although she is familiar with the time to completion problem.

She doesn’t want to mess too much with the graduate education for fear of lengthening time to completion.

Why the selective pedagogical amnesia?

American graduate education is the envy of the world.  So OSTP will take a long wind-up approach and organize a committee to strike a committee and meanwhile graduate education modernization can wait.  If it ain’t complete broke down, don’t fix it.

Without graduate education modernization will the American graduate school go the way of the American car industry?  Foreign car makers used American management ideas from business academics to challenge the dominance of the American car industry.  Meanwhile Detroit still suffers from the corporate culture problems American business literature (and the car buying consumer) derides as consumers flock to more ethical and responsive car makers.

Could American graduate education go the way of the American car maker?
Could American graduate education go the way of the American car maker?

Graduate schools compete for students in a graduate education marketplace replete with for-profit universities and open admission policies.  In this context, the graduate education consumer is queen.

With the (non)leadership of the OSTP, the dominance of the American graduate school could go the way of the  American car industry.   A consumer’s choice advantage should flow to a university that modernizes graduate education.  The University of Minnesota is the one to watch.   Via its Graduate Review and Improvement Process (GRIP), University of Minnesota hopes to catapult ahead in reputational rankings and graduate education consumer demand.   OSTP will come following after.  For graduate education modernization leadership look elsewhere.

Ten Years on: How Helpful was the Carnegie Initiative to Rethinking Doctoral Education for the 21st Century?

Unprecedented numbers of doctoral students, new hybrid disciplines and digital investigative technologies in 21st century research cry out for a corresponding catch-up to doctoral education.  Unfortunately, doctoral education escaped the knowledge explosion.  In retrospect, The Carnegie Initiative on the Doctorate (CID) as described in the book The Formation of Scholars: Rethinking Doctoral Education for the 21st Century (2007) just missed the mark in equipping doctoral education to equal the demands of the 21st century.   For example, one miss is the message of the CID that with a 21st century doctoral education a ‘steward of a discipline’ ought to emerge.

The five authors of the book who also facilitated the Carnegie Initiative on the Doctorate.  Chris Golde is in the middle.
The five authors of the book who also facilitated the Carnegie Initiative on the Doctorate. Chris Golde is in the middle.

The Carnegie Foundation for the Advancement of Teaching funds initiatives to advance the performance of education systems by building their capacity to improve.  The CID failed to advance the performance of the system of doctoral education by sticking in outcomes like disciplinary stewardship and by not building systemic capacity for improvement within doctoral programs during the CID.

‘Ongoing renewal’ would be the bumper sticker of the CID.  Judging from the CID Collections website which features submissions from the 80 participating programs in six disciplines, and the book, CID participation built capacity for renewal. (This example from Duke shows the initiatives of a History PhD program).

To support ongoing renewal after the CID, participating programs made up lists of unanswered questions.   In 2005, the mathematics program at the University of Nebraska-Lincoln, posted the following excellent unanswered questions to complete a CID assignment.

Unanswered Questions

  • How can we best poll current students and recent grads about the effectiveness of our initiatives?
  • Have we made sufficient adjustments to our Ph.D. requirements to accommodate students seeking interdisciplinary degrees?
  • What other ways can we include collaborative learning into the program? (I.e., through cross-disciplinary experience, and experiences for those interested in careers in government or industry)

Were these questions answered?

The CID might have required that programs post updates to unanswered questions along with new unanswered questions to instill an incentive for ongoing renewal.  Instead the CID collections website features only work done for the CID.

Better yet the CID might have started-up scholarship of doctoral education with an open-access, (student-edited?) journal like The Journal of Ongoing Renewal of Carnegie Initiative Doctoral Programs. This would also help fill in the vacancies in journals of doctoral education. There is but one journal of doctoral studies.  Good scholarship generates more questions, investigations, sharing, publications and conferences.  Ongoing scholarship of doctoral education spurs ongoing renewal.

The book provides many great examples and insights into renewal yet it fails to complete its primary task: to make the process of rethinking a part of doctoral education.  The book shows how the CID made doctoral education more aligned and responsive to changes of the 21st century, but fails to show doctoral programs how to make renewal automatic and ingrained.

What ingredients for ongoing renewal did emerge from the CID?

1. Initially an expert in doctoral education sets the wheels in motion.  Chris Golde, a leading scholar in doctoral education, set a course for the CID that none of the programs alone or other facilitators could generate.  Without her setting the right tone for the team, the themes and tasks, the CID may have floundered. The book minimizes the role of a knowledge leader in doctoral education to set up renewal efforts, at least initially.

2. Invite doctoral learners and faculty to multidisciplinary conferences.  Without the buzz of fruitful interaction in a much-anticipated, silo-busting context devoted to examining doctoral programs during the CID, can the success of the CID be replicated, even with this book and the great examples?  One powerful way to commit to ongoing renewal would be to bring stakeholders together annually or bi-annually to share results, updates, and future investigations.

3. Involve the learner.  A faculty member participant in the CID described the doctoral student as ‘the secret agent of change’. Doctoral students are the future stewards of research training; to change research training require doctoral students to  critique their own doctoral programs.  This instills a process for ongoing renewal and harnesses the powerful insights of the doctoral student.  Making examination of the doctoral program a graduation requirement equips graduates with insights to doctoral education and makes graduates a vector for change in doctoral education going forward in their careers.

If during graduate school, all doctoral students attended multidisciplinary graduate studies’ conferences, to address unanswered questions about their doctoral program, the will of the CID and Carnegie Foundation would be done.  Sustainability, capacity building, and multidisciplinary exchanges would be brought together to complete a graduation requirement.  (The Graduate Review and Improvement Process (GRIP) program at the University of Minnesota also gets graduate students involved in improvement).

In the 21st century, ongoing renewal makes doctoral training stay a pace of the changes in society and in knowledge production.  Ultimately, a 21st century doctoral education must also produce a disciple of doctoral education even before it produces ‘a steward of a discipline’.  Study/feedback/critique of doctoral training during doctoral education provides for ongoing renewal of doctoral education; a 21st century imperative.

How helpful was the Carnegie Initiative on the Doctorate to rethinking doctoral education for the 21st century?  Ongoing renewal could become the pivotal contribution of the CID to doctoral education for the 21st century.  If hitched to a process like ongoing scholarship and/or made a graduation requirement, ongoing renewal provides the intelligence to keep doctoral education responsive, real, and ready.

Book Highlights: 16 page pdf

A process was needed to get to the unanswered questions when the CID stopped.  Without a new process, a system goes back to its baseline, in this case the baseline is no process for ongoing renewal.  If the CID succeeded in building capacity for ongoing renewal then the unanswered questions generated during the CID a decade ago, should be addressed by now and new unanswered questions would have come up as an ongoing renewal process takes hold.

A decade later, are the 80 CID doctoral programs enacting CID-inspired ongoing renewal?  Has some process for ongoing renewal been created? What happened to the unanswered questions?  How much did participation in the CID spur renewal after the CID finished?  Watch this blog for answers to these questions.

If you would like to help me find out the answers, please contact me.  We can write a paper for the one journal of doctoral studies.  If you participated in the CID as a student, faculty member or facilitator, I welcome contact/correction.