DSM: COLTS on Assessment Methodology.

Arie Dirkzwager (aried@xs4all.nl)
Wed, 21 Oct 1998 12:11:38 +0000

Dear colleguas.
        This is an invitation to participate in an experience regarding a
COLTS (Cooperative Learning and Teaching System) I plan to manage in the
form of an e-mail discussion. As an important side effect I'll collect exact
data on this experience and report on them eventually to evaluate this
"COLTS" experimentally. The topic taught and studied will be "Assessment
Methodology" as assessment is the backbone of a COLTS as I see it.
        In a COLTS every participant is sometimes a "teacher" giving
explanations in short statements and sometimes a "student" asking critical
questions, also in the format of short statements figuring as "hypotheses".
A "lesson" is a collection of such statements with the assignment to judge
for each statement its truth-value expressed as the (personal) probability
that it is true according to the following directions:
        For each given statement think up as many arguments pro and contra that
statement and then give your judgement about each of the twenty statements
as an estimate of its p(true) according to your knowledge about what they
are about, of course when you think a statement is most likely false your
reported p(true) should be less than .50.

        If you are interested to participate in this experience please read
on. Underneath is the first "lesson" and when you reply to it you'll get the
second and following ones everytime you do the "assignment". It will take
you very little time so please participate so I'll get a lot of data to
evaluate this system.

Arie Dirkzwager.

Lesson 1.

        (1) The main goal of assessment is to predict and may be influence future
behavior of people.
        p(true)= %
        (2) The main goal of assessment is to see if certain specified goals
are attained.
        p(true)= %
        (3) The main goal of assessment is to provide a solid (scientific?)
basis for decision making and management.
        p(true)= %
        (4) To think is an important human behavior, it shows in all
physical behavior (speaking, writing, acting) and the products of that
behavior (answers to questions and assignments, performances on a specified
task) are the basic information for assessment.
        p(true)= %
        (5) The information needed to assess is to be found in past and present
behavior of people.
        p(true)= %
        (6) Assessment methodology is about methods to elicite and evaluate the
information about or from people which best predicts their future behavior
and its effects on themselves and on their environment.
        p(true)= %
        (7) Any assessment results in an evaluation that can be expressed as a
grade (A, B, C, D, or E), or as a (real) number, the larger this number the
better the performance.
        p(true)= %
        (8) Grades can be assigned the numbers 5, ,4, 3, 2, and 1 on an ordered
scale, adding or multiplying them (e.g. to compute average grades) generally
has no meaning. With real numbers those operations might have meaning,
depending on what they stand for.
        p(true)= %
        (9) Psychometrics is the main science to study assessment methodology, so
this course is a course in psychometrics.
        p(true)= %
        (10) This is a collaborative course, every participant is sometimes the
teacher giving (critical) explanations and sometimes the student asking
(critical) questions. They help each other to realistically self-assess
their knowledge and understanding.
        p(true)= %
        (11) The backbone of a collaborative course are explanations in the form of
statements and continuous assessment by evaluating those statements.
        p(true)= %
        (12) Given statements are either assumed to be either true or false or to
be proven to be true or false in the future.
        p(true)= %
        (13) Any person has an opinion on each statement that can be expressed as
his personal probability p(true) that the statement is true. His personal
probability that it is false is equal to p(false) = 1 - p(true).
        p(true)= %
        (14) A judgement is correct when either the statement judged is true and
p(true)>=.50 or the statement is false and p(false)>=.50
        p(true)= %
        (15) The probability p(correct) should be equal to p(true) when the
statement is true and equal to p(false) when the statement is false.
        p(true)= %
        (16) A person is realistic in his /her judgement when in the long run the
proportion of correct judgements with a certain value p(correct) is about
equal to this p(correct).
        p(true)= %
        (17) Multiple Choice is the best assessment method.
        p(true)= %
        (18) Multiple Evaluation is the best assessment method.
        p(true)= %
        (19) Any person exposed to Multiple Evaluation can learn to be realistic in
his judgements.
        p(true)= %
        (20) It is not possible to reach consensus on these twenty statements as an
outcome of this course.
        p(true)= %
         Reply by giving your judgements by filling in your personal
probabilities and send them to my private address "probabilities and send them to my private address "aried@xs4all,nl" (NOT to
the whole list quoting this "lesson"). Especially when you are a specialist
in assessment or psychometrics disagreeing with my quite strong opinions
(only the last statement is probably false in my opinion) I would appreciate
your reply very much and we may discuss it.
Thanks for your cooperation,
Kind regards,



Educational Instrumentation Technology,
Computers in Education.
Huizerweg 62,
1402 AE Bussum,
The Netherlands.
voice: x31-35-6981676
FAX: x31-35-6930762
E-mail: aried@xs4all.nl

When reading the works of an important thinker, look first for the
apparent absurdities in the text and ask yourself how a sensible person
could have written them." T. S. Kuhn, The Essential Tension (1977).
Accept that some days you are the statue, and some days you are the bird.

This archive was generated by hypermail 2.0b3 on Thu Dec 23 1999 - 09:01:53 EST