Michael F. Shaughnessy
Eastern New Mexico University
Portales , New Mexico

For many years, the identification of children with learning difficulties or learning disabilities has been fraught with peril. There have been issues of statistically significant discrepancy, issues of co-normed versus non co-normed tests, issues of reliability and validity and various other issues of whether a “ statistically significant discrepancy “ is one standard deviation, one and a half standard deviations or two standard deviations. “ Response to Intervention “ is a relatively new concept that will be integrated into some schools over the next year or two. It is imperative that administrators, counselors, teachers, and parents know about this new approach to identifying students with learning disabilities, as this will affect future generations of students. In this interview Gary Germann discusses Response to Intervention and his progress Monitoring System entitled AIMSweb.

1) Increasingly, school systems are moving toward computer based data management systems. Why is this occurring?

For a number of reasons: First, the technology is now in place and available for the task. Schools have traditionally used desktop computer technology for two purposes - the provision of instruction and as a communication tool (Internet and email). These uses justified a build up of technology and created the technological infrastructure necessary for web based assessment and data management.

Second, technology is now capable of performing the task. The power of the processor, the capacity for storage and increased bandwidth created the possibility and opportunity to manage and report student assessment data.

Third, the need for efficient management and reporting of student assessment data created a problem for schools. When problems are presented to the market place in the form of legal mandates and the consumer has been provided the money to pay for solutions, capitalism works its magic – products are provided that solve the problem and sometimes, even at reasonable costs i.e. AIMSweb.

Finally, No Child Left Behind brought new requirements for additional assessment, progress monitoring and reporting. These new accountability requirements created a demand for data management and reporting only provided through the use of technology.

In my opinion 1-3 of the above are necessary for major system movement, but the last is essential. Major educational reform that requires change in institutional behavior too often occurs only when the change is mandated with the threat of fiscal sanction. I think this is probably true of most monopolies.

2) You have been quoted that most educational decisions are made because of the P's. What exactly do you mean by that and why does it happen this way?

When policy makers are mandated to solve problems and the problems are presented with an absence of data, then the rationale for decisions are based on reasons other than empirical data. I have facetiously suggested that those reasons are too often personalities, power, pennies, politics, and parochialism or the “P's.” Any one that has sat through a 90 minute IEP meeting or a four hour school board meeting can relate to this. When decision makers are required to make decisions without supporting data, valuable time is used in finding the rationale necessary to justify the decision. If given a choice, decision makers are more comfortable making data based decisions.

So, decision makers make decisions in an effort to improve instruction and increase achievement without good data. Why does this happen? To make a long story short, no valid and reliable student progress data that has been collected directly, consistently, frequently and continuously over time has been available. Schools report data to students using end of unit mastery tests that are often neither valid nor reliable. They report data to parents using Report Cards that have no basis what-so-ever in measurement science and they report data to policy makers via results on high stakes tests that provide too little data, too late. And of course, special education reports assessment data designed to identify pathology and allocate fiscal and human resources, not make instructional decisions. So different consumers get different data, from different sources, at different times and often data that is not designed for making the required decisions. In this environment, the P's find fertile ground.

3) CBM is getting a lot of attention recently. What exactly is CBM and how did this measurement system start and what contributed to its growth and acceptance?

I'll try my best to define CBM consistent with the Godfather himself, Stan Deno, University of Minnesota, the developer of CBM: I think Stan would say that CBM (curriculum based measures) are a set of valid and reliable, short, easy to administer, sensitive and time efficient measures of reading, spelling, written expression and math computation designed and used for educational decision making. These characteristics lend themselves to direct, frequent and continuous measurement of student progress over time. They are essentially the “vital signs” of educational health and are used in much the same way as medicine's vital signs (temperature, blood pressure, etc.).

Early in his training career Stan was struck with the fact that he lacked a valid way to measure the impact of his practicum student's consultation with teachers. If students in training worked with teachers to increase the achievement of certain students in specific academic and social skills, what data supported these program modifications and guided future modifications? He discovered there were no independent variables that could be measured and charted over time that were technically adequate, sensitive to change, were cost efficient, short in duration and could be administered frequently and continuously. At about that time the Institute for Research on Learning Disabilities (IRLD) was funded at the University of MN . This provided Stan with a source of funds and a group of exceptional graduate students to begin the development of a measurement system that eventually became CBM. I was director of a special education cooperative at the time and was independently engaged in developing a data based problem solving system. We discovered each other and a terrific relationship combining training, research and practice emerged.

What contributed to its growth and acceptance? Well it sure wasn't the good research and data provided by the IRLD. Those were largely ignored for twenty five years (no man is a prophet in his own land or time).

I think that it takes time for the law to catch up with good practice. That's why people on the “leading edge” of innovation often find themselves on the “bleeding edge.” The IRLD gave growth to a talented group of graduate student's. Lynn Fuchs, Karen Wesson King, Doug Marston, Mark Shinn and Jerry Tindal to name the ones I was most associated with. These “disciples” of Stan Deno went to deliver the gospel at other universities and began training students. Over the years hundreds of teachers and school psychologists were trained and sent into the schools, with direct, frequent and continuous measurement in their minds and CBM in their back packs. Now a third generation of trainers, trained in formative assessment and problem solving are using CBM and continue to build an empirical and operational foundation for its application. AIMSweb is a 30 year development of a set of tools found to be necessary to deliver the system.

Two additional things brought CBM into the mainstream. First, the colossal failure of the LD construct created the opportunity to use an educational eligibility model. Second, NCLB and IDEA came into the picture and both called for direct, frequent and continuous measurement of student's progress. What better than CBM, a tool that was designed for just such a purpose? What better than AIMSweb, a tool designed to implement an educational problem solving model?

4) Does (Response to Intervention) RTI offer a better solution to identifying Learning Disabilities and what are the origins of this idea?

Well, let me be clear about several things. First, there aren't any better psychometric solutions when trying to identify LD. Trying to identify LD as an internal, child centered disability, is the problem, not the solution. We need to stop trying to answer political resource allocation/entitlement questions using psychometric models. Who is LD is not a question answered by educators, it is one answered by politicians. If the purpose of determining eligibility is to improve instruction and increase achievement, then academic and social skills and the response to instruction is best measured, not pathology. If its purpose is to justify entitlement, it is a question of resource allocation and best answered by politicians. Eligibility for LD is not, has never been, nor will ever be a question best answered with psychometrics.

Second, measuring the existence of LD and measuring the sharpness of a Unicorn's horn are equally difficult and people shouldn't spend too much time at either. Speaking of horses, this is one that we should stop beating.

Third, Stan Deno often says “there is no right way to do a wrong thing”. Wiser men are few. Justifying the allocation of public funds to better teach children with instructional problems on the basis of presumed pathology is a wrong thing and there is no way to make it right. Medicine should not make treatment to patients with cancer available, or not available, based on whether the patient's cancer is primarily caused by genetic or environmental reasons. In the same way educators should not make instructional treatment available to students based on the presumed etiology of the problem. A child reading 12 words correct in first grade materials at the end of grade one is “educationally handicapped” to the same degree regardless of what is causing the discrepancy. The handicap is never the disability; it is always the discrepancy between the student's actual performance and the expected performance. It is this discrepancy that is the basis of the teacher's referral, it is this discrepancy that is assessed, it is this discrepancy that is targeted for reduction or elimination in the IEP and it is the reduction and/or elimination of this discrepancy that gets a child out of special education.

What a shame it will be and what an opportunity lost, if RTI becomes another way to define LD or a “pre-referral” intervention strategy! The hope for RTI, and children, is that education will now embrace the scientific method, so important to scientific progress, and that RTI represents. RTI is a new way of doing business, not a new way of identifying LD. The basic problem that educators face each day is the natural diversity that is found in a classroom. When public policy requires the identification of pathology to justify program modification it lessens general educations needs to create instructional programs designed to deal with natural diversity for all students.

5) When children don't respond to intervention could it be due to perhaps a bad home environment, lack of motivation, or borderline mental retardation?

No. That is way too easy! I like to think that educator's control the variables necessary for success. I guess I would rather put my money on things like poor pacing, lack of practice, poor scope, wrong sequence, poor curriculum, inadequate time on task, lack of fidelity in the delivery of the curriculum, poor management of the contingencies of reinforcement, etc. etc. Good instruction can overcome the worst home environment, motivate the most difficult student and surpass achievement expectations regardless of the limitations.

Ultimately, from an educator's perspective, students' learning problems require instructional solutions. As educators our focus should not be on student learning problems. They should be on adult instructional solutions. The effects of a poor home can be overcome with good instructional models. Lack of motivation is not pathological in nature. It is completely an instructional problem. Borderline or on the line, or below the line mental retardation is not a curable medical disorder at this point in time. However, quality instruction can reduce the handicapping affects and lessen the handicapping impact of the condition.

Students are taught skills at different rates. At some point, those who are taught, for whatever reason, at a rate that causes a discrepancy between what they do and what the expectations are, become handicapped in that instructional environment. When their needs are more than general education can provide it triggers a referral into an instructional environment more suited for their needs. The responsibility for student learning rests with the people responsible for instruction. It serves little purpose to put the responsibility for lack of learning on the home, readiness, economically disadvantaged, the child or the disability. All problems of learning require instructional solutions.

6) What is this DIBELS thing and why is it important?

Mark Shinn graduated from the University of MN and did research as part of the team for the IRLD at the UM. He was hired by the University of Oregon to be head of their school psychology program. In his position he tried to create a training program that brought a higher degree of relevance to the field of school psychology. He trained students to become experts in problem solving, progress monitoring and emphasized CBM and formative assessment of instruction. He hired Roland Good, mentored his development, supported his research and provided the basic knowledge base that eventually lead to the development of the early literacy measures, with the exception of CBM Reading, which was developed by Stan Deno.

The first time I actually heard the term DIBS (Dynamic Indicators of Basic Skills) was in 1984 when a group of us from the IRLD were in Iowa training them on CBM and problem solving. We were searching for something other than the term CBM and DIBS was one of the suggestions, but CBM won out for various reasons.

Roland and Ruth did research on several skills that precede and predict reading and tied them to Reading CBM and called them collectively DIBELS. It wasn't the measures that moved DIBELS to the front. I had actually developed Continuous Assessment Programs for the Apple IIe in the early 80's that created random lists of non-sense words, letters of the alphabet for letter naming and letter sound fluency and of course the basic research had already validated R-CBM. We also developed a software program that reported progress on these measures. Several things, in my opinion, created the DIBELS phenomenon. First, it was called and marketed as a test. Education love tests. Second, its development coincided with NCLB perfectly. It was at the right place, with a solution to the mandate, at the right time. Third, it had friends with the advantage of the “bully pulpit.” Fourth, it was listed as an approved test for Reading First Schools, by the “Assessment Committee” whose membership, strangely, included Roland Good. Fifth, since it was developed using public monies it was possible to give the system away for free, using web based technology. Finally, the feedback and information it provided via its web based data management system solved mandated reporting problems for its customers.

Why is DIBELS important? It gave credibility to CBM as THE best over all indicator of reading proficiency and it suggested skills that could be used to predict future reading success. Up until this time, no one had successfully done that via the political process. All efforts had been research driven.

7) What exactly is AIMSweb? How does it captures what is happening in education at this time?

AIMSweb is a 3-tiered progress monitoring system, based on the science of CBM, and delivered via the technology of the Internet. That is the short answer. AIMSweb is so much more.

AIMSweb is a manifestation of a set of ideas that myself and others have held for many years. To name a few:

The critical assessment question is not “ Did the child learn.” Rather it is: “ Is the child learning?” AIMSweb provides the tools to answer that question by providing a “formative” assessment solution.

The purpose of assessment is not to determine “apriori” the instructional needs of the child. It is to provide a data base of student progress data, so that instruction can be formed by the assessment data, not predicted prior to the data.

The allocation of educational services based on a medical classification system is wrong. The disability is never, never, never, never, the educational problem requiring instructional solution. This perspective results in the wrong labels, identified with the wrong instruments, given for the wrong purposes, by people trained in the wrong methodology. It creates an administrative bureaucracy (special education) that is separate from the general education system in terms of its funding, administration, staffing, licensing, rules, curriculum, locations, etc. It focuses attention on process, not progress, to the detriment of all. It perpetuates a model that blames the child for instructional failure, with the unintended consequence of delaying instructional improvements in the general education programs. The problem is always the discrepancy between a set of expectations for progress/performance and the child's actual progress/performance. AIMSweb identifies this discrepancy as the problem, it measures and quantifies it and provides a complete system for measuring the response to instruction intended to reduce or eliminate the discrepancy.

Good decisions are made when good data is provided in a timely manner and in an understandable format to appropriate decision makers. AIMSweb uses web-based technology to manage and report student progress data to all decision makers in a format useful at all levels of decision making.

Finally, the assessment of skills can become un-necessarily complicated. Valid and reliable research based methods for measuring skills have been developed and are currently available via the science of CBM. AIMSweb brings that science to the schools in a meaningful and manageable set of tools.

8) Is all of this computerized, web-based assessment going to help or overwhelm the average classroom teacher?

Well, first let's differentiate web-based assessment from web based progress monitoring systems like AIMSweb. AIMSweb is NOT a web based assessment system. To me, that means any number of “for purchase” assessment systems that are administered to the student on the computer. All AIMSweb assessments required administration by a trained human and are typically paper and pencil from the perspective of the student.

AIMSweb is a web based data management system. It manages and reports progress data to decision makers at all levels, not for the purpose of contributing to their plight but to assist them to become more efficient and effective.

Average classroom teachers are overwhelmed with bad data. Average classroom teachers want assessment data that informs and directs their instruction. When it is provided they are better able to do what they do best, teach!

9) Is it fair to ask classroom teachers to be responsible for Response to Intervention? Seems like we are adding another thing to an already over flowing plate of duties and responsibilities?

I think this question is: Is it right to ask teachers to be responsible for learning? In the absence of response to instruction, can we assume learning took place? Would a boss accept this from a sales person: “I sold them the product, they just didn't buy it!” Should an administrator accept this from a teacher: “I taught them the skill, they just didn't learn it!” Or how about this: Should doctors be responsible for their patient's response to their interventions? Should car mechanics be responsible for their interventions?

AIMSweb assumes that schools have three primary responsibilities and together they represent the meat, potatoes and vegetables on every educator's plate of responsibilities: We are responsible to increase achievement, to improve instruction and to report our success. How do we best achieve those responsibilities? You use a system like AIMSweb to assess frequently, directly and continuously where the student is at any given time relative to his norm group and relative to a criterion and in terms of his/her rate of progress. A team of people examines the data and determines if the current intervention is generating the appropriate progress. If the student is falling behind in achievement, we attempt to respond by improving our instruction. Finally, we report our progress to relevant stakeholders. If educators did this well, I think they would be surprised at how many things they currently are doing and contribute to the sense of being over whelmed would go away.

10) Where can teachers and parents get more information about AIMS WEB and Response to Intervention? www.aimsweb.com