Introduction

COOL4Ed (the California Open Online Library for Education) was created so that faculty can easily find, adopt, utilize, review and/or modify free and open etextbooks for little or no cost. The COOL4Ed accessibility open textbook evaluations can inform faculty, staff, and students how the free and open etextbooks meet 15 accessibility “checkpoints” that could impact the learning of learners with a range of disabilities.

The California State University and MERLOT has designed a framework that will provide the accessibility community a systematic and easy-to-use method to evaluate the accessibility of COOL4Ed resources.   This framework was designed to enable the accessibility community to be able to quickly ascertain useful and quality information concerning digital course materials so they will usable by all students irrespective of disability status.  These methods can be used by publishers to improve the accessibility of their e-textbooks, and by faculty and students in their selection of e-textbooks.

Project Goals

The goals of the accessibility project were to (a) develop methods for assessing the accessibility of e-textbooks in different formats (e.g., E-Pub, HTML, PDF), (b) develop a scoring system for rating the accessibility of these e-textbooks, and (c) use the methodology and scoring system to evaluate the e-textbooks that are posted on the COOL4ed.org site.

The Accessibility Evaluation Methodology

The Summary Evaluation: Each “Accessibility Evaluation” begins with a summary and score (out of 10) for the overall etextbook accessibility score.  The 10-point scale (with a score of 1 being the lowest score possible and a score of 10 being the highest) was used to provide users a snapshot of the accessibility evaluation.

The Detailed Report with Two Evaluation Formats (Assistive and Non-Assistive Technologies):  The results of applying two evaluation methodologies (Assistive and Non-Assistive) are presented so that the user can determine the etextbook’s accessibility from the varied technologies available to users to navigate their digital textbooks. 

• The “Non-Assistive Technologies (NAT) Evaluations” address readily available tools such as the keyboard and navigation typically found on our devices and are typically available to the general public
• The “Assistive Technologies (AT) Evaluations” address specialized tools such as Kurzweil and NVDA which are not typically available to the general public. Assistive Technologies includes but is not limited to the following:
  • Accessibility features of desktop operating systems (e.g. high-contrast display themes, settings from the Keyboard and Mouse control panels)
  • Accessibility-related software included with desktop operating systems (e.g. VoiceOver, Microsoft Narrator)
  • Third-party accessibility software and hardware:
  • Screen readers (e.g. JAWS, Window Eyes)
  • Magnification software (e.g. ZoomText Magnifier/Reader, MAGIC Pro with Speech)
  • Reading software for users with learning disabilities (e.g. Read and Write Gold, Kurzweil 3000)
  • Refreshable Braille displays

The Accessibility Checkpoints

The following 15 checkpoints used in the accessibility evaluations are built upon the Section 508 technical standards and reviewed/approved by CAST, a nationally recognized organization with expertise in accessibility and UDL.  The checkpoints have been organized and tailored to the unique characteristics of digital resources used in higher education courses.  The scoring metric is a “pass/fail” qualifier, with a score of 7 and above being a “pass” and a score below 7 being a “fail”.

1. Accessibility Documentation
2. Text Access
3. Text Adjustment
4. Reading Layout
5. Reading Order
6. Structural Markup/Navigation
7. Tables
8. Hyperlinks
9. Color and Contrast
10. Language
11. Images
12. Multimedia
13. Flickering
14. Science, Technology, Engineering and Math (STEM)
15. Interactive Elements

Scoring the Textbooks

The evaluators developed and tested three different scoring techniques, and they determined that in the Summary section they would use a 10-point scale (with a score of 1 being the lowest score possible and a score of 10 being the highest) because it would allow users to quickly understand the results of the accessibility evaluation. They also determined a “pass/fail” qualifier for the Checkpoints section, with a score of 7 and above being a “pass” and a score below 7 being a “fail.”

Accessibility Evaluation Manuals

The evaluation framework was designed to provide the accessibility community a systematic and easy-to-use method to add their comments and evaluations concerning the accessibility of COOL4Ed resources.  We evaluated the quality of the etextbooks with the intent of addressing the most effective tools, both in terms of capability and costs.

The following accessibility evaluation manuals address four different e-textbook formats (EPUB, HTML, PDF, and Word).  Each section of the manual contains a short description of the checkpoint, what components of the e-textbook the evaluators should focus on, and how the evaluators should use the tools and materials provided to evaluate each checkpoint. The manuals were tested to ensure that they are understandable and easy to follow by novice users.

• Assistive Technologies (AT) Evaluation Manuals
  • Evaluation Methods for Word formats (link to PDF)
  • Evaluation Methods for PDF formats (link to PDF)
  • Evaluation Methods for HTML formats  (link to PDF)
  • Evaluation Methods for EPUB3 formats  (link to PDF)

• Non-Assistive Technologies (NAT) Evaluation Manuals
  • Methods for Word formats  (link to PDF)
  • Methods for PDF formats   (link to PDF)
  • Methods for HTML formats  (link to PDF)
  • Methods for EPUB3 formats  (link to PDF)

 

Evaluators

The accessibility evaluations were performed by the Center for Usability in Design and Accessibility (CUDA) at California State University, Long Beach; faculty and graduate students with expertise in human factors, usability, and accessibility performed the evaluations of over 150 free and open etextbooks.   Project oversight by the director of CUDA, Gerry Hanley, Ph.D.

COOL4Ed.org has published the accessibility evaluation rubric and provides a detailed description of the methodology used to evaluate the accessibility of the etextbooks in COOL4Ed.

For More Information Contact:
Gerry Hanley, Assistant Vice Chancellor, Academic Technology Services, California State University, Office of the Chancellor (562) 951-4259 or lkennedy@calstate.edu.

---

---