EFFECTIVENESS AND FEASIBILITY OF ONLINE ASSESSMENTS

1












THE EFFECTIVENESS AND FEASIBILITY OF ONLINE ASSESSMENTS:
A LITERATURE REVIEW
LACIE J. OLIVER
TROY A. WININGER
NICHOLLS STATE UNIVERSITY
EFFECTIVENESS AND FEASIBILITY OF ONLINE ASSESSMENTS

2
A majority of states moved to online assessments ahead of mandates for Common Core
State Standards (CCSS), which increased the prevalence of online testing (Davis, 2014). States
either employ Smarter Balanced Assessment Consortium (Smarter Balanced) or Partnership for
Assessment of Readiness for College and Careers (PARCC) to measure students attainment of
CCSS (Herman, Linn, & National Center for Research on Evaluation, S., 2013). Both online
assessments are based on CCSS, designed to test the higher-order thinking skills of students, and
possess the rigor needed to demonstrate whether or not students have met these standards. The
2014-15 school year will start soon and many educational leaders question whether schools are
ready to implement online assessments of this nature (Fletcher, 2012). Nevertheless, PARCC and
Smarter Balanced are not the only online assessments that schools need to prepare for. Teachers
are also utilizing other online assessments within their classrooms to take the place of traditional
paper/pencil assessments (De Souza-Hart, 2010). However, a review of literature does not
unanimously advocate online assessments. Some literature favors online assessments for its
efficiencies and advantages while other literature supports traditional testing environments
because of complications associated with technology. Without thorough prudence, educational
authorities are transitioning to online assessments. Uncertainty exists among researchers about
the effectiveness and feasibility of using online assessments.
Current research mentions many advantages of online assessments. Rastgoo and Namvar
(2010) believe that in the 21
st
Century traditional assessments do not sufficiently measure
students ability. They describe the importance of assessment within the learning process. Not
only does assessment give students quality feedback, but it can also improve the quality of
teaching. Assessments should consist of multiple methods of measurements with diverse tasks.
EFFECTIVENESS AND FEASIBILITY OF ONLINE ASSESSMENTS

3
When used correctly, online blogs develop students reading comprehension, writing
skills, and critical thinking. While blogs also seek to assess crucial 21
st
century skills found in
CCSS like the PARCC and Smarter Balanced assessments, they provide an easy and more
controllable method for individual teachers to assess their own students knowledge and skills
(De Souza-Hart, 2010). Crisp (2010) believes technology provides easy accessibility with the
ability to assess higher-level skills, which is a trend in the list of benefits for online assessment
over traditional assessments. Online assessments offer options to teachers available in the new
capabilities of Assessment 2.0, which is using technology for assessment that enables teachers to
set tasks for students in immersive, interactive, and dynamic ways. These new methods create
environments for exploring and creating responses and more sophisticated assessment tasks as
compared to traditional assessments. Assessments like sophisticated role-playing scenarios better
reflect the complexities of the real world. Teachers can begin using interactive assessment tools
in place of familiar question types available in common learning management systems. New
learning management systems are using innovative e-assessment formats such as drag-and-drop,
hotspot, matrix or extended matching questions, voice responses, and certainty-based marking to
better assess students level of understanding. A new value on contestable knowledge requires
sophisticated assessment tasks that can assess higher order thinking such as providing additional
digital tools within the questions to provide teachers with the opportunity to ask more
sophisticated questions and students with the opportunity to engage in higher order development.
Sophisticated e-assessments can also contain media-rich stimulus material and require the test
taker to interact with the stimulus material in a variety of ways. Many sophisticated e-assessment
tasks can generate a relatively large quantity of data about what the student did during the task.
For example, providing students with tools that they can use to construct non-text responses,
EFFECTIVENESS AND FEASIBILITY OF ONLINE ASSESSMENTS

4
analyze data, interact with digital objects, or interrogate objects within an assessment task assess
students at a higher depth of knowledge. Allowing students to manipulate data, to examine the
consequences of their responses, and to make informed decisions about potential solutions
constructs items that assess advanced skill development in students. Crisps journal article
(2010) provides additional evidence on the benefits of online assessments, particularly in new
question types that can assess higher order levels. This supports Scalise and Giffords (2006)
notion that question types are crucial to the effectiveness of online assessment and can be
advantageous as long as questions are written appropriately. Technology creates new
opportunities for assessment with strong scoring ability and providing real-time feedback.
Previous state assessments have grossly understated Webbs depth of knowledge levels 3 and 4-
again, needed for 21
st
century student success-and PARCC and Smarter Balanced assessments
can assess them more effectively (Herman et al., 2013). These high-level DOK performance
tasks connect to Scalise and Giffords premise about the importance of question forms on
computer-based assessments in order to take advantage of the technology embedded in the tests
and address higher rigor.
Another strength of online assessments over traditional assessments is high-quality, real-
time feedback. Feedback is critical to both learners and instructors. Also, the ability to provide
real-time feedback in situational context back to teachers and students is a hallmark of successful
computer based learning (Ong, 2007). Online assessments have the advantage of providing
teachers with nearly immediate feedback, giving them the opportunity to utilize the data to
benefit students (Fletcher, 2012; and Stowel, 2013).

The real-time feedback of online assessments lends to another beneficial feature-data use.
Online assessments make acquiring data much quicker and easier. Through online assessment,
EFFECTIVENESS AND FEASIBILITY OF ONLINE ASSESSMENTS

5
educators can evaluate and monitor student progress more efficiently (Mill, 2008). Research
results indicate a modest positive relationship between general data-use practices and student
achievement in certain grades and subjects. Teachers and principals that reviewed and analyzed
student data and used the information to make instructional decisions had higher increases in
their students achievement in some grades and subjects. Additionally, more principals reported
having support in the form of data infrastructure, adequate time for review and discussion of
data, professional development, and the appropriate human resources, increased their students
achievement. Some research provides insight into the positive relationship between data-use and
student achievement. The use of learning analytics and assessment data by teachers and
principals in different grades and subjects may increase student achievement, even if only
modestly (Faria et al., 2012). Moreover, assessments are meaningful when all students are
assessed and all teachers are involved in grading. Student attainment can be measured over time
through assessment analytics and can be compared with peers, benchmarks or standards (Ellis,
2013). Mill (2008) explains that through online assessment, acquiring data is quicker, and
teachers can evaluate and monitor student progress more efficiently.
Hewson (2012) and Stowell (2013) also mention secondary benefits such as saving on
printing costs and saving time due to automated delivery.
Some question the reliability and validity of online assessments. Hewson (2012) found
that performance on online and offline assessments did not differ regardless of preferred mode,
therefore, supporting the validity of online assessments. Additionally, Ricketts and Wilks (2002)
found that once the interface was updated for online assessments (ex. eliminating scrolling),
students performed better and were more positive toward online assessments. They cite the
improvement in performance to support the validity of online assessments. Stowell (2013)
EFFECTIVENESS AND FEASIBILITY OF ONLINE ASSESSMENTS

6
contradicts this view in his study of student emotions on traditional and online assessments. His
hypothesis was not supported when the study showed that students with high text anxiety in a
traditional environment still experienced high test anxiety in the online testing environment. He
also found the performance on the online quizzes did not differ from the classroom quizzes,
which was supported by other research studies. Previous studies comparing paper and online
tests found that some online tests to be more difficult, and others claiming paper tests to be more
difficult. Also, other studies found paper and online test comparable, and other studies observed
mixed results. Keng, McClarty, and Davis (2008), analyzed online and paper versions of the
Texas Assessment of Knowledge and Skills (TAKS), which is a statewide, high-stakes
assessment. They found significant differences favoring the paper test for certain grades and
subjects, and other items types favoring the online version. For example, Reading/ELA items
that were longer in passage length and math items requiring graphing and geometric
manipulations or involved scrolling in the online administration tended to favor the paper test.
However, this article does not take into account the possibility that the ability to draw on items
could be added, as advised by Crisp about including tools for manipulation within items.
Other literature opposes online assessments in favor of traditional tests. Crisp (2010)
claims that traditional testing tasks, such as multiple choice questions or short answer questions,
are favored because they are quick and easy to write and score. Also, the traditional questions
types were known as valid and reliable. Technical problems can also plague online assessment
and negate their advantages. Consecutive years of computer malfunctions during testing caused
distrust of online assessments, and some districts reverted back to traditional paper-and-pencil
tests to avoid problems. Examples of computer glitches include slow loading times, unwanted
boots from the testing system, and even attacks from unknown hackers. Executives of both
EFFECTIVENESS AND FEASIBILITY OF ONLINE ASSESSMENTS

7
Smarter Balanced and PARCC assessments touting benefits about online testing mentioned in
other literature, but the executives display an obvious bias for online testing (Davis, 2014).
Although online assessments can create savings from printing costs, Northcote (2002), Mill
(2008), and Fletcher (2012) agree that technology for online assessments is not available to some
schools due to tight budgets.
Additional research cautions against a reckless movement toward electronic assessments.
Cheating is a major issue in a traditional classroom, and it is magnified in an online setting.
Assessment cannot have meaning unless the safeguards are put into place to reduce the cheating
taking place (Bothel, 2002). Others feel plagiarism voids the efficiency benefits of online
assessments (Northcote, 2002). Computer and telecommunication skills can also complicate
assessment. Some tests do not assess a students computer skills before taking the actual
assessment. Assessment does not have to be done online or paper pencil for it to be authentic.
Assessment is authentic if it measures student growth, knowledge and skills (Bothel, 2002).
Another issue for online assessments is students inexperience with technology. Both PARCC
and the Smarter Balanced Assessment, like Fletcher mentions, connect to Common Core State
Standards, and students will need to retrieve knowledge under pressure. This task can be difficult
if students lack sufficient experience with the actual technology they are using. Students may be
familiar with certain devices at home, but it can be troublesome for them to transfer their skills to
a different device (Gullen, 2014).
Online assessments also carry legal liabilities both similar and different from traditional
assessments. Legal risks involved in online testing-such as accommodating special education
students-could lead to possible litigation from injustices that take place in the highly-magnified
and scrutinized high-stakes tests. Both school districts and testing companies can be liable. For
EFFECTIVENESS AND FEASIBILITY OF ONLINE ASSESSMENTS

8
example, lawsuits can even come from private citizens such as angered parents upset over
technical problems that may have caused students to fail high-stakes tests, and thus prevent
graduation. Further, special education is the most litigious part of education (based on number
of lawsuits) resulting from the foundational guarantee of the Individuals with Disabilities
Education Act (IDEA), which requires that students be provided an IEP (Individualized
Education Plan) that reflects the students' unique needs. National online tests will struggle to
offer flexibility for this as the tests must meet norm-referenced standardization, and it will be
difficult to modify the delivery options of testing in an online environment as it relates to time of
delivery, pacing, and question interface. Special education law does not traditionally allow
procedural flexibility even if a district claims their network overloads (Bathon, 2012).
Many contend online assessments will inevitably and expectedly cause some problems.
However, a few authors offer advice on how to avoid potential pitfalls of online testing. Gullen
(2014) makes a good point in stating that teachers need to start teaching these small skills such as
highlighting text, scrolling, using a mouse, and using an online calculator before the students will
be ready to take an online assessment. She suggests educations need to do the following to
prepare students for online assessments: enhance digital skills among students, assign cross
curricular tasks, promote self-sufficiency, provide practice, try out technology, and debrief
students. Securing student competency of technology skills prior to testing will combat
opinions, such as Keng, that argue against the validity of online assessments. Legal experts also
advise educational leaders to review contracts with wireless and bandwidth providers, review
district filtering software, and consult with districts legal counsels about potential litigation
(Bathon, 2012). A stress-test in which thousands of students access a sample test simultaneously
to test a networks infrastructure and bandwidth before testing can highlight potential problems
EFFECTIVENESS AND FEASIBILITY OF ONLINE ASSESSMENTS

9
regarding the technological aspect of online assessments. Using a wired connection over
wireless if possible creates a more stable and secure connection. Also, districts should limit
high-bandwidth applications during testing such as streaming videos (Flanigan, 2013).
Flanigans guidance can help districts combat problems with online testing, particularly Daviss
examples.
Overall, the adoption of CCSS by a majority of states commenced the widespread use of
online assessments that are rigorous enough to assess thinking skills that 21
st
century students
must possess. Dynamic question types and computer adaptive testing present new opportunities
to assess standards in ways that are impractical for traditional assessments. Further, fast delivery
and scoring allows for real-time feedback and promotes the use of assessment data for
instruction, which is increasingly popular in a data-driven age. While technical problems and
technological inequity hinder complete acceptance of online assessments, savings from printing
and copying costs will quiet those concerns especially as technoloy evolves. Legal
complications, plaigairism and cheating also present drawbacks, but could be resolved with
advancements in technology and other safeguards. Validity and reliabilty of online assessemnts
are inconclusive and debated in a similar manner for all standardized tests-even paper versions.
In sum, most literature emphatically promotes the use of online assessments due to its irrefutable
technological advantages while a less significant amount of research points out solvable
problems. Efficient online assessments provide the capability to better assess students
understanding of mandated CCSS and at a higher depth of knowledge.



EFFECTIVENESS AND FEASIBILITY OF ONLINE ASSESSMENTS

10
REFERENCES CITED
Bathon, J. (2013). For Districts, Online Testing Has Legal Liabilities. T H E Journal, 40(7),
17-20.

Bothel, R. (2002). Epilogue: A Cautionary Note About On-Line Assessment. New Directions
For Teaching & Learning, 2002(91), 99.

Crisp, G. (2010). Interactive E-Assessment--Practical Approaches to Constructing More
Sophisticated Online Tasks. Journal Of Learning Design, 3(3), 1-10.

Davis, M. R. (2014). Online Testing Glitches Causing Distrust in Technology. Education Week,
33(30), 20.

De Souza-Hart, J. A. (2010). Biology Blogs: An Online Journal Club & Assessment Tool.
American Biology Teacher, 72(3), 149.

Ellis, C. (2013). Broadening the scope and increasing the usefulness of learning analytics:
The case for assessment analytics. British Journal Of Educational Technology, 44(4),
662-664.

Faria, A., Heppen, J., Li, Y., Stachel, S., Jones, W., Sawyer, K., & ... Council of the Great
City, S. (2012). Charting Success: Data Use and Student Achievement in Urban
Schools. Executive Summary. Council Of The Great City Schools,

Flanigan, R. L. (2013). Technology Readiness For Online Testing. Education Week, S22.

Fletcher, G. H. (2012). The New Online Assessments: What We Know So Far. T H E
Journal, 39(5), 20-22.

Gullen, K. (2014). Are Our Kids Ready for Computerized Tests? Educational Leadership,
71(6), 68-71.

Herman, J., Linn, R., & National Center for Research on Evaluation, S. (2013). On the Road to
Assessing Deeper Learning: The Status of Smarter Balanced and PARCC Assessment
Consortia. CRESST Report 823. National Center For Research On Evaluation,
Standards, And Student Testing (CRESST),

Hewson, C. C. (2012). Can Online Course-Based Assessment Methods Be Fair and
Equitable? Relationships between Students' Preferences and Performance within Online
and Offline Assessments. Journal Of Computer Assisted Learning, 28(5), 488-498.

Keng, L., McClarty, K., & Davis, L. (2008). Item-Level Comparative Analysis of Online and
Paper Administrations of the Texas Assessment of Knowledge and Skills. Applied
Measurement In Education, 21(3), 207-226.

EFFECTIVENESS AND FEASIBILITY OF ONLINE ASSESSMENTS

11
Mill, L. B. (2008, April). Online Assessment: Put Down Your Pencils. School Administrator.
p. 8.

Northcote, M. M. (2002). Online assessment: friend, foe or fix?. British Journal Of
Educational Technology, 33(5), 623-625.

Ong, J. (2007). Automated Performance Assessment and Feedback for Free-Play Simulation-
Based Training. Performance Improvement, 46(10-), 24-31.

Rastgoo, A., & Namvar, Y. (2010). Assessment Approaches in Virtual Learning. Turkish
Online Journal Of Distance Education, 11(1), 42-48.

Ricketts, C. C., & Wilks, S. J. (2002). Improving Student Performance through Computer-
Based Assessment: Insights from Recent Research. Assessment & Evaluation In Higher
Education, 27(5), 475-79.

Scalise, K., & Gifford, B. (2006). Computer-Based Assessment in E-Learning: A Framework
for Constructing "Intermediate Constraint" Questions and Tasks for Technology
Platforms. Journal Of Technology, Learning, And Assessment, 4(6),

Stowell, J. R. (2012). Emotions Experienced by Students Taking Online and Classroom
Quizzes. Journal Of Educational Computing Research, 47(1), 93-106.