Can Learning Be Assessed?
"The only source of knowledge is experience."
n other articles, we've explored a number of ideas related to the topic of assessment, as well as why the standardized tests used in education today are theoretically flawed, based on false premises, and, to the extent that they have any value at all, measure only the lowest forms of learning. So what are the options?
Let's assume that the purpose of assessment in education is to evaluate or estimate the nature, quality, or ability of learning. We've largely determined that there is no universally accepted, or adequately comprehensive, definition of learning. But we can certainly broaden the idea of the "acquisition of knowledge" by recognizing that there are many different kinds of knowledge. Only by recognizing and identifying what, specifically, we want to "assess" can we begin to explore valid methods of assessment.
Kinds of Knowledge
earning a skill requires a different thought process than learning the formula for salt or the definition of alliteration. Using alliteration in writing requires a different thought process than simply defining the word. There is, however, a tendency to group these processes under two headings—declarative and procedural knowledge. Simplistically, knowing the rules of volleyball is declarative knowledge and being able to spike a ball is procedural.
Similarly, educators often divide knowledge into content and process. Content is the information that learners are expected to acquire. "Process is all the cognitive skills that the curriculum activities are intended to develop that are supposed to enable the student to do something with the content."(1) Although arguments abound over whether content or process is more important (another limiting dichotomy), theorists are fairly careless about distinguishing between them. For example, many so-called "content" standards are stated as processes—something a student must do to demonstrate "possession" of the content.
The knowledge categories of content and process are too broad to be particularly useful. Dr. Carl Bereiter suggests a potentially more useful breakdown of knowledge.
- Statable knowledge—This is knowledge the knower can pass on to others in some way, be it sentences, diagrams, formulas, or stories. Such knowledge can be observed by others, evaluated, and compared. This is the primary type of knowledge included in standards and "measured" by "high-stakes" tests. However, this type of assessment is further limited by the form of the tests in which students must select one "correct" answer from a list of choices. Notice that the other ways in which a knower can pass on statable knowledge (writing, diagrams, stories) are largely eliminated in high-stakes tests because they don't lend themselves to single correct answers that can be machine-graded! Even when students are asked to write a short answer, the question itself and the grading rubric specify the points that the answer must include. If those points are there, the answer is correct…if not, it's incorrect…despite the depth of thinking or lack thereof included in the answer.
- Implicit understanding—Implicit knowledge refers to our relationships with things, people, or situations in the world. Understanding implies a deep enough relationship to produce intelligent action on the part of the knower. In Bereiter's words, "Implicit understanding is more like perception than like having propositions in the head."(2) The center fielder who, at the instant the bat strikes the baseball, begins to run to the exact spot where the ball will land has implicit understanding. This is very different from someone who knows and/or can use the equation for calculating projectile motion.
- Episodic knowledge—This type of knowledge is related to people's memories of things that happened to them personally. Episodic knowledge is something a person learned through experience that may then become useful in some other context. Walking past a high-rise building under construction may remind a person of having a branch fall on his head in the woods. The recalled knowledge influences the person's behavior. While this might be classified as "prior knowledge," it is only "selected" from a person's body of knowledge when an appropriate context arises, or when a person is asked to "predict" what might happen in a given context.
- Impressionistic knowledge—Bereiter asserts that all knowledge has an affective, or emotional, component. In fact, several neuroscientists propose that emotion is a necessary characteristic of human thought.(3) Neuroscientist Antonio Damasio(4) suggests that, in the absence of emotions, higher thought processes are impaired. In the case of impressionistic knowledge, the feeling is the knowledge. It is the hunch, the intuitive sense, the gut feeling that may not be accessible, nor expressible, at a conscious level, but that influences our behavior nonetheless.
- Skill—Bereiter suggests there are two parts to any skill: a cognitive part—the "knowing how"—and a second part that can improve with practice. In school, much of the focus is on the first part, knowing how. Students are taught "how" to do many different tasks, from solving a particular type of math problem to using reference materials. Unfortunately, with the proliferation of "hows" that students are now required to learn, there is little time for the practice that will improve those skills. Skills that remain at the level of "knowing how" are less likely to be used than skills in which the person feels some competence. For example, I may "know how" to play bridge, but I rarely do it because I have had little practice and am painfully aware of how poorly I play.
- Regulative knowledge—In the acquisition and use of knowledge, the issue of "self" as learner is typically ignored in education. In my experience, students become instantly alert when a teacher begins discussing how the brain/mind works. Suddenly, the information relates to them! They come to recognize that the more they know about how they process information, the more effectively they can monitor and assess the products of their own thinking. Reflection on the beliefs that influence one's life, or the biases that limit perceptions, are just as valuable for students as they are for teachers. Developed at an early age, self-reflection may make a profound difference in a person's development.
Bereiter points out that schools place the greatest emphasis on statable knowledge and the "knowing how" portion of skills—the only two forms of knowledge that can be "explained" and directly conveyed from one person to another. Neglect of the others comes partly from their variability from person to person. When educators are caught in the trap of group-think, there is no place for "knowledge" that varies among students, can't be easily transmitted to an entire group, and can't be objectively assessed. Yet knowing what we now know about the jaggedness profile of individuals, as well as how people's behavior changes from one context to another, ignoring variability becomes equivalent to throwing out any type of fruit that doesn't looking like an apple, or ignoring any word that doesn't have six letters and begin with the letter m!
Beyond statable knowledge and "knowing how," the other types of knowledge and/or understanding have not been adequately explored or even recognized as critical components of learning in public education. However, they are all present in what most of us recognize as intelligent behavior. "Competence in any domain will likely involve all six kinds of knowledge."(5)
Breaking knowledge down into its components is useful to educators to the extent that it helps to identify areas neglected in the teaching/learning environment. It is not useful if each component is broken off into a domain of its own and addressed as a separate part of instruction, as is often done with so-called thinking skills. In real life—in authentic learning—these forms of knowledge develop and function in parallel. They are interrelated parts of the system that encompasses the knower and the known. Dealing with the parts in isolation is likely to result in even more inert knowledge that is useful only in the context in which it is learned. Even now, with greater recognition of the importance of social and emotional learning, many are obsessed with finding ways to standardize and measure achievement in these areas.
One need only look at programs such as the "seven domains" approach used at Rainbow Community School in Asheville, NC, to see a learning environment in which all of these types of learning are not only recognized, but fully incorporated in an integrated way and with equal emphasis. This is holistic education.
Assessment: What Doesn't Work
ere's what we know:
- The concept of an average (or standard) person is the result of dangerously flawed interpretations of data from the 18th and 19th century. Despite compelling evidence that any system designed to fit the "average" actually fits no one, public education still maintains that learners are best served by one-size-fits-all standards and assessment based on a mythical average.
- Comparing individuals based on their "possession" of a given set of knowledge objects (see the article on metaphors) is meaningless because it gives little or no information about the ability of any learner to use the information. In addition, these tests, by their very design, strengthen and maintain social and economic classes because the major factors to which the tests actually correlate are the annual income and level of education of families.
- The only types of knowledge that lend themselves to the collection of numerical data are simplistic facts that can be easily separated from other facts to yield one correct answer.
- Learning is much more than the acquisition of statable knowledge and the "knowing how" component of skills. Given the complexity of the components of learning, as well as the complexity and variation in the way each human processes those components, learning cannot be "measured." That does not mean it can't be assessed.(6)
- Standardized tests are built on the premise that learning can be measured, and that those measurements can then be used to rank the abilities of, not only learners, but also teachers and schools. Since any meaningful types of learning cannot be measured, this premise is fatally flawed and any conclusions draw from it are meaningless. Using scores on standards tests to predict "success" is just digging deeper into the rabbit hole of error.
Quantitative vs. Qualitative Assessment
- Quantitative: relating to, measuring, or measured by the quantity of something rather than its quality.
- Qualitative: relating to, measuring, or measured by the quality of something rather than its quantity.
- Objective: not influenced by personal feelings or opinions in considering and representing facts.
- Subjective: based on or influenced by personal feelings, tastes, or opinions.
Recall that the word assessment means "the evaluation or estimation of the nature, quality, or ability of someone or something." And then notice that the word "quantitative" related to the quanity, not the quality of something.
Many people, particularly in the United States, have become obsessed with numbers. They have come to believe that attaching numerical data to any statement make it "more true." Any statement without such data is questionable. Thus the current attachment to quantitative assessment. For some reason, the ability to reduce a person to a number is given credence far beyond its actual value.
A qualitative evaluation, such as "The man is tall" is fuzzy. And worse, it's subjective. Your idea of "tall" may differ from mine. In this case, it certainly appears that the quantitative measurement is "more true," not to mention more useful, than the qualitative assessment/evaluation. But is that always the case?
Here's another comparison of a decision based on objective and subjective factors.
Is the decision made based on objective probability any more "true" than the one made on the basis of subjective probability? And let's face it…how many people (other than meteorological statisticians) would take the time to base everyday decisions such as this on quantitative data.
Quantitative measurement is valuable when assessing discrete, observable objects. If, for example, you are choosing furniture for a room, it can certainly be useful. But given a more complex situation, such as the probability of rain, which depends on a number of largely unpredictable (or unobservable) factors, quantitative measurement loses any advantage.
Another way to define objective and subjective is that the objective refers to tangible things while subjective refers to intangible things. Tangible refers to things that have a physical presence that can be verified, generally by touch or other senses. Given those definitions, the question becomes "Are tangible things necessarily more important or valuable than intangible things?"
From the descriptions of knowledge earlier in this article, almost all forms of learning are intangible. Even the possession of specific knowledge objects is only indirectly observable. I can tell you what I know. I can show you what I know by selecting a correct answer on a test. In neither of these cases is the knowledge itself tangible. You may also observe my behavior and infer that I know something, although this is certainly subjective because it is based on your own knowledge and beliefs.
We saw in an earlier article how the ability to collect and store data about people rose exponentially in the early 19th century. The problem was that no one knew what to do with the numbers. Enter Quetelet and Francis Galton, who introduced the use of probability and statistics to the social sciences. While the manipulation of the numbers…the data itself…may be tangible, any conclusions based on that manipulation must be subjective. We saw this in the errors Quetelet and Galton made in their own interpretations of average.
Assuming that we can't measure learning, but still want to assess it, let's begin by discussing the elements of authentic assessment.
- Bereiter, C. (2000). Education and Mind in the Knowledge Age, Taylor and Francis, Inc. 2. In addition, the entire book can be read and downloaded at URL: http://www.cocon.com/observetory/carlbereiter/ [Note: the spelling of "observetory" is the way it is spelled in the URL]
- Edelman, G. (1992) Bright Air, Brilliant Fire. New York: Basic Books.
- Damasio, A R. (1995) Descartes' Error: Emotion, Reason, and the Human Brain. New York: Avon Books, 43-45.
- Bereiter, Education and Mind…, p 19.
- Tienken, Christopher, et al (Dec. 2016)Predicting Middle Level State Standardized Test Results Using Family and Community. URL: http://www.tandfonline.com/doi/full/10.1080/19404476.2016.1252304
Do you like what you find here? Are you intrigued? Please take the opportunity to share this page on your favorite social media site. It helps raise awareness and starts or adds to dialogue. Take a moment to share this page.