The problem with business school is that until recently it has not done enough to teach how to succeed in business. In the 1970s and 1980s, when I attended UCLA's business school, there were two models for MBA programs. One, the "quantitative" approach, presented MBA education in accordance with the claim that business and management are scientific problems that can be solved through optimization and modern financial models. By that time, of course, David Halberstam had written "The Best and the Brightest" which in chapters 12 and 13 included a scathing critique of Robert McNamara's management science, which discarded common sense in favor of statistical modeling. Moreover, it is absurd to claim that even a modest percentage of the challenges business executives face involve problem solving. This is a common claim, but most anyone who's labored in the corporate world knows that interpersonal and political skills are far more important to success than problem solving. But business school had been doing next to nothing to develop such skills. The second approach was the case study method, which is somewhat more practical than teaching business students how to do regression analysis, but frequently covers irrelevant material and concepts and also does little to develop interpersonal skills. Two graduates of the Harvard Business School (most closely associated with the case study method), Jeff Skilling and Rebecca Marks, did well with the case study method in school but lacked elementary business competencies (see The Smartest Guys in the Room, by McLean and Elkind)
I thought about this for years in my twenties and thirties (in the 1980s) as I realized that I had learned next to nothing of practical value in business school, with the exception of a course called "Nucleus" that was taught by Professor Eric Flamholtz, who was a pioneer in the competency-based approach. Flamholtz's exercises and insights showed me that teaching business competencies was a yet-unrealized possibility. (Which isn't to say that I didn't have many other excellent academic experiences at UCLA, starting with my chief academic inspiration, Professor Dominique Hanssens in the marketing department there. But there was little that could be applied in most real world business settings as opposed to statistical modeling and hypothesis testing.)
As I thought about how to make business school more practical and of use to students, I realized that the key issues for junior executives involve developing interpersonal skills, politics and power. I suggested to other academics that this be taught, but I wasn't clear how to do so. Finally, I learned about competency-based education through a colleague at Iona, Ted Schwartz.
The idea of competency-based education is that skills are identified and targeted. Students assess themselves with respect to the skills (objective assessment not being valid or even available). Then, they read about how to improve with respect to the skill. The skill can be cognitive or acognitive. For instance, the skill could involve technical knowlege, but more importantly it can involve self-awareness, emotional intelligence, communication, interpersonal skills, conflict resolution and use of power. After learning about how to improve with respect to the skill the student then applies the skill in a real-life setting (skill application) and writes about why they chose to work on the particular skill in the particular setting. David Whetten and Kim Cameron have developed this model in their textbook Developing Management Skills.
NCATE claims that education schools ought to apply "skills assessment" as part of their accreditation program. I have no problem with developing skills. My problem is the "assessment" part. A competency-based approach is effective for teaching purposes as it focuses on teaching and improving with respect to a targeted skill set, and skills that students should be learning in applied programs like business or education can be targeted. This is still done only to a limited degree in business schools, which is unfortunate.
But one thing that I never heard from any of the folks involved in the competency movement is that anyone should ever be penalized or judged for having or not having a competency or be "assessed" in a punitive way. I really don't think anyone ever suggested that and if they did they would have been wrong. The idea is much more subtle than that. Competencies interact with the work environment, so there is no one right competency in the sense that mathematics is right. Of course, there are general competencies that are beneficial across a wide spectrum of occupations. For example, good interpersonal skills, understanding how one's emotions influence one's judgment, etc. are good competencies to have, along with math, reading and writing skills. I'm all for business schools and education schools teaching things like that.
To assess such competencies is another story. It is much more difficult to assess than to describe or teach about a competency like interpersonal skills, communication, how to gain power, etc. Even if they can be tested students can fake their responses once they realize what the test is for. Also, development of tests is extremely hard to do. So there shouldn't be any assessment or testing unless you are willing to invest in a "live" assessment center approach involving structured exercises and hundreds or thousands of dollars per assessee. On the other hand, using these concepts in structuring education programs is something I support as a component of education. Students need to develop interpersonal and self awareness skills as much as writing and math skills. All of the above should be part of a professional education program.
As far as NCATE, they did not describe any methodology about training or teaching the competencies such as social justice. They did not define social justice. They just told education schools that they should assess students using social justice dispositions, which is nonsensical. NCATE slipped on an ideological banana peel. First you have to define what the dispositions mean, then develop measures, etc.
So in short I object to the "assessment" but not to the "dispositions".
Wednesday, May 23, 2007
More on NCATE and Teacher Dispositions
While presenting at the US Department of Education Office of Postsecondary Education National Advisory Committee on Institutional Quality and Integrity on Monday, June 5, in Arlington, Va. I mentioned that there is a well-developed body of knowledge about managerial competencies, which is the same idea as NCATE's "dispositions," and the Hay-McBer consulting firm, founded by the well-known psychologist David McClelland, has done considerable work in developing competency measures and validating them. But NCATE has not toiled in this way and so its claim to use dispositions in assessing prospective teachers is spurious. Last year, NCATE failed to respond to my repeated inquiries as to what measures they use to assess dispositions in students that they claim to evaluate using "dispositional assessment", and whether and how they have validated such measures. Since NCATE has no realistic measures and no validated measures, their claim that they use dispositions to evaluate students is nonsense. The chief reason for NCATE's making a misleading claim that I can think of is the possibility that NCATE wants to encourage harassment of some categories of students, such as conservatives who disagree with its political ideology. Hence, I presented at the meeting that the entire use of dispositions is inappropriate and in violation of judicial statutory interpretation that prohibits governmental use of ideological litmus tests. Contrast NCATE's fly-by-night claims to be using "dispositional assessment" with how it has been done in business schools.
Boyatzis* defines competencies as "an underlying characteristic of a person which results in effective and/or superior performance in a job." Each competency has two dimensions. The first identifies the various competencies. The second involves three competency levels: motives (unconscious), self image (conscious) and skills (behavioral). "Each level may vary in its impact on the disposition of the person to use the competency." Here we see the infamous word "disposition" that NCATE throws around.
Boyatzis distinguishes between motives and traits. "A motive is a recurrent concern for a goal state...A trait is a dispositional or characteristic way in which the person responds." Boyatzis argues that there are dynamic interactions among (from most unconscious) traits and motives, self-image, skills, the person, job demands and the organizational environment.
Boyatzis's book uses data from 12 organizations and 2,000 people in 41 job titles. He uses "the job comptence assessment method" which involves analyzing each job, scoring interviews of job incumbents ("behavioral event interviewing"), development and application of objective tests to measure the competencies, and correlation of the interview and objective test scores to job performance measures, i.e., validation.
In 2000 the Hay-McBer firm with which Boyatzis is associated did a study of teaching competencies in the UK. They found the following competencies:
Challenge and Support
Confidence
Creating Trust
Respect for Others
Analytical Thinking
Conceptual Thinking
Drive for Improvement
Information Seeking
Initiative
Flexibility
Holding People Accountable
Managing Pupils
Passion for Learning
Impact and Influence
Teamworking
Understanding Others
These competencies explain 30% of the variance of an outcomes measure. Note that Hay-McBer does not claim that it can assess teachers or prospective teachers along these dimensions. The only competency-related measures that can do so are behavioral assessment centers that involve multiple reviewers who anonymously evaluate performance on structured exercises. The reason is that written assessment measures can be gamed, and so often do not have validity in prediction of performance. The chief exceptions are integrity tests and the conscientiousness measure of the "Big-5" personality inventory (for example the NEO-AC instrument).
Note that "social justice disposition" is nowhere to be seen in any of these discussions. Also note that none of the experts who have studied competencies claim that even objective test measures can validly predict future job performance. Moreover, none of the experts who have studied competencies and dispositions has come anywhere close to asserting that a particular professor can assess competencies in a particular student, particularly when such a professor dislikes the student's politics.
That NCATE advocates or accepts such procedures is evidence of incompetence.
*Richard E. Boyatzis, The Competent Manager: A Model for Effective Performance, New York: John Wiley and Sons, 1982.
**self awareness, conceptualization, concern with relationships, concern with impact, developing others, diagnostic uses of concepts, efficiency orientation, logcial thought, managing group processes, memory, perceptual objectivity, positive regard, proactivity, stamina and adaptation, use or oral presentations, use of socialized power, use of unilateral power.
Boyatzis* defines competencies as "an underlying characteristic of a person which results in effective and/or superior performance in a job." Each competency has two dimensions. The first identifies the various competencies. The second involves three competency levels: motives (unconscious), self image (conscious) and skills (behavioral). "Each level may vary in its impact on the disposition of the person to use the competency." Here we see the infamous word "disposition" that NCATE throws around.
Boyatzis distinguishes between motives and traits. "A motive is a recurrent concern for a goal state...A trait is a dispositional or characteristic way in which the person responds." Boyatzis argues that there are dynamic interactions among (from most unconscious) traits and motives, self-image, skills, the person, job demands and the organizational environment.
Boyatzis's book uses data from 12 organizations and 2,000 people in 41 job titles. He uses "the job comptence assessment method" which involves analyzing each job, scoring interviews of job incumbents ("behavioral event interviewing"), development and application of objective tests to measure the competencies, and correlation of the interview and objective test scores to job performance measures, i.e., validation.
In 2000 the Hay-McBer firm with which Boyatzis is associated did a study of teaching competencies in the UK. They found the following competencies:
Challenge and Support
Confidence
Creating Trust
Respect for Others
Analytical Thinking
Conceptual Thinking
Drive for Improvement
Information Seeking
Initiative
Flexibility
Holding People Accountable
Managing Pupils
Passion for Learning
Impact and Influence
Teamworking
Understanding Others
These competencies explain 30% of the variance of an outcomes measure. Note that Hay-McBer does not claim that it can assess teachers or prospective teachers along these dimensions. The only competency-related measures that can do so are behavioral assessment centers that involve multiple reviewers who anonymously evaluate performance on structured exercises. The reason is that written assessment measures can be gamed, and so often do not have validity in prediction of performance. The chief exceptions are integrity tests and the conscientiousness measure of the "Big-5" personality inventory (for example the NEO-AC instrument).
Note that "social justice disposition" is nowhere to be seen in any of these discussions. Also note that none of the experts who have studied competencies claim that even objective test measures can validly predict future job performance. Moreover, none of the experts who have studied competencies and dispositions has come anywhere close to asserting that a particular professor can assess competencies in a particular student, particularly when such a professor dislikes the student's politics.
That NCATE advocates or accepts such procedures is evidence of incompetence.
*Richard E. Boyatzis, The Competent Manager: A Model for Effective Performance, New York: John Wiley and Sons, 1982.
**self awareness, conceptualization, concern with relationships, concern with impact, developing others, diagnostic uses of concepts, efficiency orientation, logcial thought, managing group processes, memory, perceptual objectivity, positive regard, proactivity, stamina and adaptation, use or oral presentations, use of socialized power, use of unilateral power.
NCATE Ends Its Advocacy of Social Justice Dispositional Assessment
I took the Amtrak to Washington to attend a meeting of the National Advisory Committee on Institutional Quality and Integrity. The Advisory Committee was reviewing petitions to extend recognition of various accreditation associations such as the American Dental Association Commission on Dental Association, the Oklahoma State Regents for Higher Education and the National Accrediting Commission of Cosmetology Arts and Sciences.
Among the accrediting organizations requesting extension was the National Council for the Accreditation of Teacher Education (NCATE). NCATE submitted a petition for renewal of recognitioin and expansion of the scope of its recognition so that it could accredit distance learning programs.
Last spring (2005) there was a controversy at Brooklyn College concerning NCATE's approach to dispositional assessment concerning a student named Goldwyn and his professor, Priyar Parmar. In addition, Steven Head of San Jose State has filed suit at San Jose State University concerning SJSU's treatment of his candidacy in its teacher education program because of NCATE's and SJSU's approach to dispositional assessment.
At the Advisory Board meeting Arthur Wise, head of NCATE, indicated that NCATE has dropped social justice from its accrediting criteria. Naturally, Steve Balch, head of NAS, Anne Neal, head of ACTA and Greg Lukianoff, head of FIRE as well as myself were delighted.
In my remarks to the committee I described Steven Head's case and the fact that he said that all conservatives had been driven out of SJSU's teacher ed program. I indicated that NCATE's entire approach to using dispositions is inappropriate because the dispositions that they use have not been validated and that they sent me on a wild goose chase when I asked for evidence that their approach had been validated. Dispositions are too easily used as pretexts for politically motivated retaliation to be used in assessing students unless there are objective instruments and measures. I also said that NCATE has overseen the decline of American education, that students with median SAT scores cannot do basic math or write because of teaching approaches that NCATE advocates and that NCATE is the nexus of educational decline, that they are a bunch of losers and should be declined accreditation recognition altogether.
Among the accrediting organizations requesting extension was the National Council for the Accreditation of Teacher Education (NCATE). NCATE submitted a petition for renewal of recognitioin and expansion of the scope of its recognition so that it could accredit distance learning programs.
Last spring (2005) there was a controversy at Brooklyn College concerning NCATE's approach to dispositional assessment concerning a student named Goldwyn and his professor, Priyar Parmar. In addition, Steven Head of San Jose State has filed suit at San Jose State University concerning SJSU's treatment of his candidacy in its teacher education program because of NCATE's and SJSU's approach to dispositional assessment.
At the Advisory Board meeting Arthur Wise, head of NCATE, indicated that NCATE has dropped social justice from its accrediting criteria. Naturally, Steve Balch, head of NAS, Anne Neal, head of ACTA and Greg Lukianoff, head of FIRE as well as myself were delighted.
In my remarks to the committee I described Steven Head's case and the fact that he said that all conservatives had been driven out of SJSU's teacher ed program. I indicated that NCATE's entire approach to using dispositions is inappropriate because the dispositions that they use have not been validated and that they sent me on a wild goose chase when I asked for evidence that their approach had been validated. Dispositions are too easily used as pretexts for politically motivated retaliation to be used in assessing students unless there are objective instruments and measures. I also said that NCATE has overseen the decline of American education, that students with median SAT scores cannot do basic math or write because of teaching approaches that NCATE advocates and that NCATE is the nexus of educational decline, that they are a bunch of losers and should be declined accreditation recognition altogether.
Quote of the Day
From Bruce R. Hopkins, The Law of Tax Exempt Organizations (NY: John Wiley and Sons, 2003), p.192.
"'The exposition of propositions the correctness of which is readily demonstrable is doubtless educational. As the truth of the view asserted becomes less and less demonstrable, however, 'instruction' or 'education' must, we think, require more than mere assertion and repetition.' (National Alliance v. United States, 710 F.2d 868 (DC Cir. 1983)) ...Thus, the federal tax law does not contain a threshold, generic definition of the term educational, but rests on the concept that subjects spoken or written about must be objectively developed or founded...Inherent in the concept of educational is the principle that an organization is not educational in nature where it zealously propagates particular ideas or doctrines without presentation of them in any reasonably objective or balanced manner. The point is reflected in the income tax regulations that define the term educational, where it is stated 'An organization may be educational even though it advocates a particular position or viewpoint so long as it presents a sufficiently full and fair exposition of the pertinent facts as to permit an individual or the public to form an independent opinion or conclusion. On the other hand, an organization is not educational if its principal function is the mere presentation of unsupported opinion.'"
"'The exposition of propositions the correctness of which is readily demonstrable is doubtless educational. As the truth of the view asserted becomes less and less demonstrable, however, 'instruction' or 'education' must, we think, require more than mere assertion and repetition.' (National Alliance v. United States, 710 F.2d 868 (DC Cir. 1983)) ...Thus, the federal tax law does not contain a threshold, generic definition of the term educational, but rests on the concept that subjects spoken or written about must be objectively developed or founded...Inherent in the concept of educational is the principle that an organization is not educational in nature where it zealously propagates particular ideas or doctrines without presentation of them in any reasonably objective or balanced manner. The point is reflected in the income tax regulations that define the term educational, where it is stated 'An organization may be educational even though it advocates a particular position or viewpoint so long as it presents a sufficiently full and fair exposition of the pertinent facts as to permit an individual or the public to form an independent opinion or conclusion. On the other hand, an organization is not educational if its principal function is the mere presentation of unsupported opinion.'"
Subscribe to:
Comments (Atom)
