In late 2018, WPCampus released a request for proposals to conduct an accessibility audit of the WordPress block editor, also known as Gutenberg. In early 2019, we announced our selection of Tenon, LLC to conduct the audit, at a cost of $31,200.
Today, we’re excited to release Tenon’s report on the accessibility of the new editor.
This report was made possible by the extraordinary generosity of more than 100 members of the WordPress community, who collectively contributed $10,264 to fund this project. We’re also grateful to Matt Mullenweg and Automattic, who pledged to ensure that the audit would be fully funded and are covering the balance of the cost of this effort.
In addition, this audit would not be possible without the members of our vendor selection committee. These amazing and talented people donated their time and energy to review proposals, ask smart questions, and ensure that the report we’re sharing today would be valuable to our community.
Why WPCampus commissioned a Gutenberg audit
WPCampus values accessibility. Our participants are particularly sensitive to their institutions’ legal obligations to provide accessible technologies.
Beyond those legal obligations, colleges and universities worldwide have committed to providing their communities an accessible digital experience. The uncertainty surrounding the accessibility of Gutenberg makes it difficult for institutions using WordPress to plan their transition to the new editor.
With that in mind, WPCampus commissioned an accessibility audit to provide our participating institutions (and the broader WordPress community) with an independent assessment. This audit is a resource to help make informed decisions.
What was tested
Tenon began their testing of the editor in January 2019 on the most current version of WordPress at that time, version 5.0.3.
WordPress versions 5.1 and 5.1.1 (released while Tenon was testing) contained updates made in Gutenberg version 4.8. These updates included four resolved accessibility issues.
At the time of this writing, an additional 116 issues related to accessibility have been closed thanks to the efforts of the WordPress Accessibility Team and Gutenberg contributors. These changes are targeted for inclusion in WordPress version 5.2 in early May.
Some of Tenon’s findings may overlap with many of these closed issues. This report is best understood as a snapshot of the state of the editor in early 2019, and as a reference document for measuring continuing progress toward accessibility.
How to read this report
As we prepared to publish the audit results, we had an internal discussion about how this report might be received by the broader WordPress community. We’d like to address that here.
An inaccessible editing experience has real and significant consequences for many people in our community. The findings provided by Tenon are serious, and deserve to be considered with thought and care.
These findings should not be used as fodder for out-of-context quotes on social media or in blog comments. Please refrain from using this report as a means to attack others.
Please use this report as what it is intended to be: constructive feedback in support of the WordPress project. We hope this report generates discussion about accessibility, excitement about inclusive design, and action toward improving the editing experience.
This report is valuable for anyone working on a complex user interface. Please use this report as an educational resource to help guide and assess your future projects.
Results of the accessibility audit
Overview
Tenon has provided WPCampus with the following documents:
- A 34-page Executive Summary
- Tenon’s testing plan, which groups features of the editor into testing components
- A 329-page long-form technical report describing the logged issues in detail
- A summary document describing Tenon’s usability testing
- A CSV file containing the issues logged during the technical audit
- A guide to importing the CSV file into issue tracking tools
To help facilitate improvements to the editor based on these findings, Tenon has created a collection of issues on GitHub.
Recommendation for site administrators (and general WordPress users)
The Executive Summary document contains the most immediately relevant information for site administrators. That document describes the findings of the audit and the potential impact of the new editor for content creators.
The user-based testing report is also recommended reading. This document provides insight into the particular challenges of using the editor for some in our community. We believe this is important context for understanding the necessity of making the editor accessible.
Recommendation for WordPress core contributors and web developers
As with site administrators, we recommend web developers begin with the Executive Summary and the report from Tenon’s user-based testing. In addition, we recommend developers prioritize the long-form technical report.
We encourage WordPress core contributors to review all of the audit documents provided by Tenon. This is a substantial time commitment, but will be useful in implementing the changes needed to make the editor more accessible.
Further, we encourage any web developer to take the time to read and review all of the information provided in this audit. That includes developers not directly involved in the WordPress project. Tenon has compiled a wealth of information, and we’re excited to share it. Examining this report is a tremendous opportunity for professional development, and sets a standard every developer building a complex user interface should aspire to.
Recommendation for higher education institutions that use WordPress
WPCampus commissioned this audit to provide an independent assessment to institutions that use WordPress, and not to provide any specific guidance or legal advice. This information may be used to determine potential legal risk, challenges, or impact faced by using the Gutenberg editor.
We recommend that your web team review the report. We also recommend that you share it with your IT accessibility office and/or your disability services office.
The WPCampus Slack community is always available as a place to share ideas, concerns, and strategies. If you’re involved in higher ed and you’re not already on our Slack, visit our Get Involved page to create your account.
And don’t forget that the WPCampus 2019 Call for Speakers is open until Sunday, May 5!