Data driven decision making

Citation metadata

Author: Marlow Ediger
Date: Mar. 2003
From: College Student Journal(Vol. 37, Issue 1)
Publisher: Project Innovation (Alabama)
Document Type: Article
Length: 2,721 words

Document controls

Main content

Full Text: 

Information for making curricular decisions come from many sources. One model being that measurable data might secure needed content for determining sequence in the curriculum. Wide access to computer technology and within a testing philosophy, information might well be incorporated from printouts to determine future courses of action. Individual differences need to be provided for in designing the curriculum. The question arises, "Should information come completely from objective sources in making these decisions?" What about daily classroom work of pupils? Daily work of pupils reveals subjective information from the following sources: written work, oral experiences including discussions, quality projects completed, attitudes indicated by pupils such as volunteering to do extra work, and being exceptionally good listeners, among others. There, no doubt, could be a harmony between the objective and subjective facets to make quality decisions pertaining to pupil progress.

**********

Much is written on state standards and high expectations for pupils to achieve. The major curriculum area to receive attention here is reading, since all pupils on the third grade level are to be able to read 3rd grade materials, according to the Elementary and Secondary Education (ESEA) act of 2002. The objectives are to be determined on the state level such as under the supervision of the state department of education. With all pupils in a state being tested in grades three through eight, beginning with the 2005-2006 school year, according to the Elementary and Secondary Education Act, machine scoring is necessary. Data from computer point outs may then provide information for decision making pertaining to teaching pupils. Data driven decisions will be compared with pupils/teachers deciding the scope and sequence of reading instruction, as indicated in portfolio development (See Education Week, January 10, 2002).

Scope in the Teaching of Reading

Scope answers the question, "What should be taught in reading?" The written objectives on the state level then answers the question pertaining to what is to be taught in the reading curriculum. The results pertain to objectives which are available to classroom teachers and provide the scope or breadth of content/skills to be taught. Tests to measure pupil achievement are also written on the state level. To be of good quality, tests need to be valid. Teachers then need to teach what is written in the state mandated set of objectives. Little leeway is provided for teacher selection of objectives if the state mandated ends are numerous in number and are required teaching for pupil attainment.

In addition to validity, the tests need to measure consistently or in a reliable manner. In pilot studies made of the state mandated tests, reliability figures need to be high, be it test/retest, split half, and/or alternative forms. If pupil A for example, measures on the seventieth percentile the first time and the twentieth percentile the second time on the same state mandated test, confusing results are in evidence! The reader of the test data from the printout has little or nothing to go by when attempting to notice a pupil's achievement. Certainly, on a test, there should be consistent test results from pupil A and others when test/retest reliability is in evidence. If data from tests are to be used in decision making, then valid and reliable tests need to be written and in the offing (Ediger, 2002, 44-49).

Which decisions will need to be made from data driven test results? If pupils are tested annually in grades 3-8, then data from grade three will determine if the pupil is to be promoted to grade four, and for successive grades. The test then is to measure if the pupil in grade three, for example, has met grade level requirements. If not, he/she may be required to be tutored before/after school, attend summer school, and/or repeat the grade. So far, the mandates have not been that specific, but the chances are there will be tightening up of state mandated standards for pupils to achieve. The state mandated standards are conceived to be absolutes and measure what is salient in pupil's learning to read. Hopefully, each state will have excellent information in feedback results to help the teacher decide what should be taught in reading so that the pupil might achieve more optimally test wise in reading. Thus, the printout needs to have information on what the pupil has missed in word recognition techniques such as

* phonics and syllabication.

* use of context clues, and in comprehension of content read as in

* reading critical and creatively.

* reading to solve problems.

* reading to follow directions.

* reading to predict, infer, and obtain a main idea.

* reading to secure needed facts.

* reading to get subordinate ideas as related to the main idea.

* reading to write an outline (Ediger and Rao, 2000, Chapter

From tested word recognition skills, the results need to transfer to every day reading activities of learners. Thus, a diagnosed needed skill may be phonics and, more specifically sound/symbol relationship for the letter "M" The pupil then needs to make the proper associations in phoneme/grapheme relationships when engaged in reading. Phonics instruction should assist the learner to become a more proficient reader. Phonics is a tool to increase reading ability. Phonics is not to be learned for the sake of doing so. The point being that test taking is not reading such as taking a test on grapheme and phoneme relationships. There, to be sure, are elements of reading in the phonics section when taking of tests. But if reading for comprehension purposes is the major objective of instruction, then pupils need to read and understand sequential ideas. Phonics may then be applied in the actual act of reading.

There are plethora of limitations for gauging pupil achievement based on testing and include the following:

* it is a one shot approach in determining achievement since state mandated tests are given once a year at the most and by 2005-2006, in grades 3-6 as mandated by the ESEA federal mandate.

* it measures on demand achievement on a specific testing day rather than evaluating daily progress in the classroom setting.

* it provides numerical results such as a percentile instead of what is more subjective and needs to be assessed such as student achievement in written work.

* it measures segments of subject matter for pupils to read on a test, not a continuous sequential story or expository writing.

* it can be machine scored, making it possible for large numbers of pupil's tests to be scored in a very short period of time. However, selected important facets of a pupil's achievement cannot be machine scored such as a learner's ability to communicate well orally.

* it is a one way street of communication on pupil achievement and progress, since questions from learners cannot be raised about the clarity or meaning of test items.

* it may not be possible to see pupil's test results to compare them with the key used in scoring. Computer glitches are definitely possible. Further questions involve pupils and parents seeing the test taken by pupils to notice clarity of items written as well as if there is more than one correct res rise. Machine scoring may permit one correct response only.

* it does not measure pupil purposes in reading such as a learner self selecting a library book to read.

* it does not measure pupil achievement in identifying and solving a problem pertaining to content read. The purposes for testing were developed by the writers of the state mandated test.

* it does not measure pupil achievement in attitudes toward reading.

* it does not permit parents to raise questions pertaining to helping their offspring in reading.

* it does not measure pupil's reading voluntarily to themselves when free time is available (See Valtin and Naegele (2001).

This does not emphasize that testing and measuring should be minimized. Rather, the weaknesses of state mandated tests need to be taken out. This will take time and money. Perhaps one answer is to have the portfolio approach to assess pupil reading achievement along with testing and measuring procedures. The portfolio approach stresses evaluating the every day reading tasks which pupils complete. A random sampling of these may be placed into the pupil's personal portfolio. Pupils with teacher guidance may place items such as the following into a personal portfolio:

* cassette recordings of oral reading.

* a video tape of the pupil involved in discussing content from reading, as well as participating in a reader's theater.

* self evaluation of progress made in reading when comparing an earlier with a later evaluation.

* a record kept of library books read with a short written evaluation of the contents of each book.

* diary entries written and dated pertaining to impressions gained from the basal reader used in the curriculum.

* logs which summarize recorded diary entries.

* illustrations drawn to reveal understandings gained from reading a library book.

* a model made to indicate comprehension in reading a library book. A snap shot may need to be incorporated due to the bulkiness of the constructed item.

* a mural completed to illustrate selected concepts read.

* journal writing to write main ideas gained in reading from the basal as well as from library books (See Ediger and Rao, 2001, Chapter Seven).

A Comparisons of Philosophies in Reading

Data driven instruction stresses selected philosophical strands in reading instruction. First, it believes strongly in whatever exists, exists in some amount, and then can be measured. Pupil achievement in reading can be discussed, identified, and then measured such as in percentile results. Second, what exists can be stated in precise terms to be used as objectives for pupil attainment. Precision in stating objectives pertaining to state mandated standards is salient--These objectives need to be precise enough so that teachers may use then in teaching students. Third, all students need to experience the same objectives so that comparisons may be made among schools and school systems within a state. In the comparisons made, poorly performing schools may be sorted out from those doing well. Poorly performing schools need help to overcome weaknesses, be taken over by the state, or make it possible for pupils to transfer to a well performing school. Report cards published in the media are to indicate how well pupils are achieving when comparisons are made among schools and school districts. Fourth, data from computerized scoring of pupil's tests may then be used to make curricular decisions. Decision making may especially be relevant when the teacher receives adequate feedback such as diagnostic and remediation information for each pupil, based on the data obtained. Scientifically orientated information from objective test results, it is assumed, provides reliable information in improving reading instruction. Fifth, weak teachers and schools may be pinpointed from low scoring data. Inservice education may then be provided to up test scores. Sixth, alignment of the local reading curriculum with the state mandated objectives is necessary for pupils to achieve well in reading. Teachers then need to use the state mandated objectives in developing learning opportunities for classroom instruction. Test scores might well go downhill unless the local reading curriculum Is aligned with the state mandated objectives of instruction. Seventh, decision making by reading teachers is minimized as the objectives are chosen by the state, and learning opportunities are to be aligned with these ends. The teacher then becomes less important in determining the scope and sequence in the reading curriculum. Eighth, the pupil is provided with a local interpretation free curriculum in that the objectives, learning opportunities, and evaluation procedures have been predetermined for the learner. Active pupil and teacher involvement in helping to determine the ongoing curriculum in the teaching of reading is not in evidence. The pupil's purposes in the reading curriculum have largely been decided upon by the state. Ninth, with state mandated tests, the temptation is for teachers to drill pupils involving large amounts of time on taking the annual test. Time taken for teaching test taking robs pupils of instructional activities in aiding pupils to achieve more optimally in reading. A quality reading program is sequential for learners and builds upon what has been learned previously. Tenth, pupils need to perceive relationships of knowledge acquired rather than content being perceived as being isolated. In and on tests, the items do not relate to each other in reading. No test item then is to provide background information or clues for succeeding responses to be made. In daily reading, pupils are to perceive ideas as being interrelated so that transfers in learning may be made from one situation to the next which is opposite of mandated tests and testing (See Ediger, 1971, p. 9).

Portfolio development, also called constructivism as a philosophy of education, stresses including selected pupil products and processes from every day classroom experiences. A random selection needs to be chosen so that the portfolio does not become too bulky for responsible evaluators to view, including parents. The pupil is then heavily involved, with teacher guidance, in selecting portfolio items. A table of contents should be developed for the portfolio to itemize the kinds of materials contained therein, such as "poems written." Viewers may then notice and compare earlier with later poems written by the pupil.

The portfolio should stress products and processes therein which harmonize with the statement of objectives, located next in sequence, page wise, to the table of contents. A wide variety of items need to be contained in the portfolio to notice pupil achievement in different curriculum areas. Pupils need to reveal growth and achievement in all curriculum areas and this is a part of the accountability movement. State mandated test results may be included in the portfolio; however, test results are not authentic measurements and evaluations of a pupil's products and processes completed in the classroom.

Selected, qualified appraisers need to be hired to assess each portfolio. Subjectivity will be involved in the evaluation endeavors. Developing a quality rubric to be used in the appraisal of the portfolio will assist in coming up with increased reliability assessment. Thus, two appraisers of a portfolio should then come up with somewhat consistent results when using rubric guidelines to evaluate. Interscorer reliability is then in evidence if the two agree entirely or almost entirely on the rating to be given each portfolio. Consistency of portfolio rating results should then be in evidence (Ediger, 1997, Chapter Thirteen).

Comparing Measurement with Portfolio Philosophy

Precise measurement of achievement, such as in terms of percentiles, is important in data driven instruction. Portfolio philosophy does not believe that achievement can be measured precisely, but can be noted in degrees. Data driven instruction emphasizes that teaching is a science and can produce objective results from pupils after being tested on what has been learned. Portfolio philosophy advocates that teaching and learning is more of an art with creativity being involved. Data driven instruction relies on computer use and scoring to secure objective results in the assessment process. Portfolio philosophy stresses the use of raters to score the products and processes included therein. It emphasizes showing to the public where each pupil is in achievement on a continuum. The portfolio is ongoing and does not indicate at a precise time with a specific percentile what a pupil has learned.

Educators need to continually study diverse philosophies of assessment and come up with improved approaches to determine what pupils have learned (See Allen, 2001).

References

Allen, Rick (2001), "Technology and Learning," Curriculum Update. Published by the Association for Supervision and Curriculum Development, 1-8.

Education Week (January 10, 2002), Quality Counts. Pew Charitable Trust Report on Education in the Fifty States.

Ediger, Marlow, and D. Bhaskara Rao (2000), Teaching Reading Successfully. New Delhi, India: Discovery Publishing House, Chapter Eight.

Ediger, Marlow (2002), "Scope in the Reading Curriculum," Experiments in Education, 30 (3), 44-49.

Ediger, Marlow, and D. Bhaskara Rao (2001), Teaching Social Studies Successfully. New Delhi, India: Discovery Publishing House, Chapter Seven.

Ediger, Marlow (1971), "Developing the Unit: Use Variety of Activities," Wyoming Education News, 37 (8), 9.

Ediger, Marlow (1997), "Portfolios and The Pupil," Teaching Reading and the Language Arts. Kirksville, Missouri: Simpson Publishing Company, Chapter Nineteen.

Valtin, Renate, and Ingrid M. Naegele (2001), "Correcting Reading and Spelling Difficulties: A Balanced Model for Remedial Reading," The Reading Teacher, 55 (1), p 41.

DR. MARLOW EDIGER
Truman State University
North Newton, KS 67117

Source Citation

Source Citation   

Gale Document Number: GALE|A99816474