What accessibility testing is possible
We are going to look at each Section 508 provision in turn and discuss the algorithmic and human judgment aspects involved in testing for compliance with the provision. This will be a fairly open-ended discussion. In the algorithmic section we will talk about what parts of the accessibility testing could feasibly be automated. We will not be looking at particular tools, but what tools could do. The points labeled here as judgment calls are things that you, as experts, can evaluate and say whether or not a given page passes or fails.
When there are specific tests in our set of tests for reviewing the tools, I will list them in the section on algorithmic testing. For example, the Algorithmic Testing section for Text Equivalents immediately below refers to two tests (#1, #4) relating to the presence of alt attributes.
Text Equivalents
§1194.22(a). A text equivalent for every non-text element shall be provided (for example, via “alt”, “longdesc”, or in element content).
Algorithmic Testing
A computer program can verify the presence of alt attributes on elements where they are required, including img, area and input with type="image" (#1,
#4). You can algorithmically check whether the alt attribute is actually present and whether it consists
of a text string (#2). This, of course, includes the null string alt="", which is desirable on formatting
images and never valid on an input element with type="image" or on any link where
there is no text as part of the anchor (#22, #23, #24). These restrictions can also be checked.
Common errors or violations could also be detected, such as the use of filenames for alt-text. Words like ‘ spacer ’, ‘ go ’, ‘ click ’ or ‘ null ’, and an alt attribute consisting of a space are also errors which software
could detect or highlight for examination. Several of the tools that we will study in the second section of this chapter
have places where you can add suspicious words or phrases and other characteristics of the alt-text that should raise errors
or warnings (#2, #5).
I think tools should distinguish between the kinds of alt-text failure. There are active images which need alt-text,
like image links, image buttons and image-map areas (#6). When these are missing they are very serious errors. On the other
end of the spectrum of concern, formatting images, ones which are less than 10 pixels in either dimension, though they
need alt-text, if they don’t have it, it is just not a big deal. These images are required to have an alt attribute
to be valid XHTML and should have alt="" which tells a screen reader to ignore the image. But ignore
the image is exactly what the screen readers do if the alt-text is missing. The only tool to make distinctions between
the types of images is the LIFT machine. Unfortunately they have about eleven different types of images including Banner
Images, Bullet Images, Composition Images, Decorative Images, Thumbnail Images, Repetitive images and five more.
It is certainly possible for a software program to raise the question of whether or not a longdesc attribute belongs
on an image for, say, images larger than a predetermined size. This doesn’t seem very helpful to me, but many software
tools do raise this question for any non-trivial image. Furthermore, the presence of certain file types should trigger
questions about the existence of text equivalents. These include (but are certainly not limited to) the various media file
types such as .wav, .ra, .rm or .ram files.
Judgment
Judgment comes in when evaluating the quality of the text equivalents, that is, “Does a text string serve
as a textual equivalent to the non-text element?” More specifically, does the alt-text convey the function of the
image when the image is active? Does the alt-text convey the information of the image in the case the image is not active
and when the image is conveying information? You should use alt="" for images that do not convey
information, like formatting images.
As I have said in earlier chapters, there is some disagreement about the finer details. For example, I would say that
graphical “bullets” in front of items in a list should have alt="" while others might argue for alt="bullet".
Of course such lists should be coded as HTML unordered lists (<ul> ) with CSS specifying a list-style-image
referencing those graphical “bullets”. However, the basic rules still apply and either choice for alt-text complies with §1194.22(a)
regardless of which side of this particular argument you take.
The existence of text equivalents for audio and multimedia will also need to be determined by visual inspection. For instance, whether or not a transcript of a speech (or other audio communication) is accurate, it will have to be checked by having someone actually read through the text and checking that it does match the audio presentation.
If alt-text or object content is used for applets or objects (#3), that text must be compared with the applet or object
for equivalence. Consider an applet that acts as a ticker application for displaying stock prices. Alt-text or object content
that says, Applet displays current stock prices, (as suggested by the Access Board
guide to the 508 Standards http://www.access-board.gov/sec508/guide/1194.22.htm ) is not actually an equivalent,
since it does not enable the user to obtain the current stock prices. Some more imaginative way of presenting the equivalent
data must be found, and to decide whether that equivalence is achieved requires human judgment.
Judgment is also required in determining which images require long or in-line descriptions, like charts and graphs, as we discussed in Chapter 6, and whether a long description is adequate in conveying the information of the image.
Synchronized Multimedia
§1194.22(b) Equivalent alternatives for any multimedia presentation shall be synchronized with the presentation.
Algorithmic
The existence of multimedia content can be detected by file extension in an anchor or contents of an object element.
For some multimedia formats (.smi for example) it may be possible to determine algorithmically whether or
not captioning is included. I think the most important role for software testing relative to this provision of the 508
Standards, is to highlight the places where multimedia appears, so that humans can examine the accessibility accommodations
of that content.
Judgment
It is not possible to determine the existence of captions for all media formats. Therefore, human evaluation is required just to determine whether the synchronized equivalents are present. Then there is the question of the accuracy of the captions. Is an audio description necessaryand if it is necessary, are the audio descriptions adequate? These questions have to be answered to determine compliance with this provision of the Section 508 Standards on Web Accessibility.
Color Coding
§1194.22(c). Web pages shall be designed so that all information conveyed with color is also available without color, for example from context or markup.
Algorithmic
Automated software could be used to check for color specifications in the stylesheet and corresponding elements in the web page—and deduce any color changes in the page that way. If there are no color changes through HTML or CSS on a web page then the page automatically complies with §1194.22(c) as far as text is concerned. However, images on the page could still be color-coded (images used as links, for instance). If the image color conveys important information, then it raises accessibility issues.
Moreover, if there were color changes in the page then it would be difficult to detect algorithmically whether information was being conveyed by color alone. For example, a simple search for ‘color words’ in a page’s source code might well turn up phrases from the page content such as the books in green are available for checkout —which might suggest non-compliance, but still requires a human evaluation.
Judgment
A general evaluation of the page is necessary to determine if information is conveyed by color alone. Most sources that discuss testing for this provision recommend that human testing be done by viewing the page in black and white. I think that is wrong. It is much easier to check for the use of color conveying information when color is available to the person addressing the question.
Stylesheets
§1194.22(d). Documents shall be organized so they are readable without requiring an associated stylesheet.
Algorithmic
An algorithm can detect the presence of stylesheets, style elements or style attributes.
If they don’t exist then the page complies with §1194.22(d) vacuously. A testing tool could determine if stylesheets are
present and if there is no CSS positioning in those styles. In that case also, this provision is probably met. Beyond these
boundary conditions, human judgment is needed to study how the document looks with style sheets disabled. And this is important
because generally screen readers operate on a page as if style sheets were disabled.
Judgment
Pages should be viewed using a browser with stylesheets turned off to determine whether or not the reading order of the page makes sense. It is also a judgment call whether there are structural elements being presented just with stylistic changes instead of using HTML markup, like lists or headings.
Server-Side Maps 1
§1194.22(e). Redundant text links shall be provided for each active region of a server-side image map.
Algorithmic
If no server-side maps are present (<img> with the ismap attribute) then the page obviously
passes this provision (#7). If a server-side map is found (<img> with the ismap attribute
and no valid usemap attribute) then the question must be addressed as to whether or not redundant text links
are available. A program could be written to click on every pixel of the map, record the new URL, and compare it with all href values
on the page. That sort of overkill would be kind of silly, though!
Software can at least detect the presence of server-side maps and flag them up for subsequent human evaluation.
Judgment
If a server-side map is found then it must be evaluated for the availability of text links for all the active regions of the map.
Server-Side Maps 2
§1194.22(f). Client-side image maps shall be provided instead of server-side image maps except where the regions cannot be defined with an available geometric shape.
Algorithmic
If there is no server-side map on the page then it obviously complies with §1194.22(f) (#7). If there is one, then see §1194.22(e).
Since all regions can actually be defined with polygons, the page should fail if a server-side map is used. All areas
of the client-side map must have a valid alt attribute (#6). The ismap attribute should never be
used on an input element with type="image" (an “image button”).
Judgment
My interpretation of this provision is that the page does not comply with §1194.22(f) when a server-side map is found. The absence of a server-side map means that the page is compliant. Since this is an either/or requirement, no human judgment is required.
Table Headers
§1194.22(g). Row and column headers shall be identified for data tables.
Algorithmic
If a page has no tables at all, then this provision is vacuously satisfied. I believe it is possible to devise sophisticated heuristic algorithms to separate data tables from layout tables with adequate certainty (#32). Since almost all tables are layout tables, and since many web sites still use layout tables, it would be especially advantageous if software could successfully separate out tables that are data tables. There are lots of issues involved in achieving this. The use of images in cells would suggest a layout table, whereas uniformity of cell content suggests a data table, and so on. Nested tables are almost certainly not data tables. Once an accessibility checker has determined that a table is probably a data table, then the question of compliance comes down to whether or not row and column headers have been identified.
Having detected a data table, the tool could assert compliance if all row and column heading cells (column 1 and row
1) are marked with th elements or if only the first column or the first row of cells is marked with th.
The tool’s capacity to identify data tables should extend to a determination that accessible table markup (th, headers, summary, scope, caption )
is being used when it should not be used (#33).
Judgment
No matter how sophisticated the algorithm to detect data tables is, every page must be checked for their presence because the heuristics cannot be guaranteed to find data tables in every case. Once a table is determined to be a data table, the validity of its headings specification must be verified by examining the HTML source code for headings markup.
Complex Tables
§1194.22(h). Markup shall be used to associate data cells and header cells for data tables that have two or more logical levels of row or column headers.
Algorithmic
Although one could probably determine which tables on a site are data tables (albeit with some degree of uncertainty), I cannot imagine any way of algorithmically determining whether or not a given table is complex, in the sense that it has two or more logical levels of row or column headers. However, if all data cells in the table have valid header attributes then the table conforms to this provision—whether complex or not.
Judgment
Again, the situation is similar to that for the previous provision. Firstly, any page must be searched for data tables. When one is found it requires human judgment to determine whether or not there are two or more logical levels of row or column headers. If there is at most one row heading and at most one column heading for each data cell, and these are in the same row and column as the cell, respectively, then the table is not complex. Judgment is required to determine whether or not the headers markup that is required by this provision is adequate. This can be verified by human testing with assistive technology or careful analysis of the HTML code.
Frames
§1194.22(i). Frames shall be titled with text that facilitates frame identification and navigation.
Algorithmic
Software can check whether the frame elements in a frameset have valid title attributes
(#8, #34, #40). Exception can be taken in some cases when title attributes are inadequate, similar to considerations
of suspicious alt-text (#9).
Judgment
Human judgment will determine whether or not the frame title attributes are useful for the purposes of
identification and navigation. For example, title="frame10078" and title="frame10077" are
not useful ways of identifying a frame to show what its purpose is.
Flicker Rate
§1194.22(j). Pages shall be designed to avoid causing the screen to flicker with a frequency greater than 2 Hz and lower than 55 Hz.
Algorithmic
Software can detect whether or not there are technologies present on a web page that might cause flickering
in the hazardous range. Java, Flash or JavaScript all qualify as potential culprits. Furthermore, any animated gif on the
page could cause flickering. Animation and the frequency of frame transitions also could be detected algorithmically, but
a frame rate in the hazardous range does not necessarily mean that the screen will be flickering at this rate; it is the
change from light to dark that causes flickering. Taking the requirement beyond specific flicker rates, software can certainly
flag moving text with the deprecated blink and marquee elements (#35, #36).
Judgment
We must inspect any page that has potential “flickering technology” on it, to check whether or not flickering is occurring. If so, we need to determine whether the frequency is in the range disallowed by this provision and rectify matters if this is the case.
Text-only Page
§1194.22(k). A text-only page, with equivalent information or functionality, shall be provided to make a web site comply with the provisions of this part, when compliance cannot be accomplished in any other way. The content of the text-only page shall be updated whenever the primary page changes.
Algorithmic
While it is possible to detect algorithmically some references to text-only pages, it is impossible to evaluate the equivalence of content or frequency of update of a text-only page. This provision mandates the use of a text-only page only in circumstances where compliance cannot be accomplished in any other way. The fact that a page cannot be made accessible in the first place obviously cannot be determined algorithmically.
Judgment
There are two parts to evaluating a web page for compliance with this provision. Firstly, is it really impossible to make the page accessible in some other way? Secondly (having determined that this is the case), is there a text-only page readily available and, if so, is it accessible? Does it meet all the other Section 508 provisions for web access and does the text-only version offer equivalent functionality and information to the original? Finally, is it kept up to date with the same frequency as the original page? These are all difficult questions to answer, making this one of the most daunting of the 508 provisions for compliance testing.
Scripting
§1194.22(l). When pages utilize scripting languages to display content, or to create interface elements, the information provided by the script shall be identified with functional text that can be read by assistive technology.
Algorithmic
If a page uses scripts to modify attributes for visual effect then this is fine, from an accessibility standpoint, since it does not affect the user’s ability to interact with the site. In some cases this could be determined with a software accessibility checker since (in principle) an algorithm could be devised to check if the ‘fly-over submenu’ links are available on the page opened by the main menu item link. However, this is a pretty complex task: it is, in effect, one program trying to evaluate another.
Judgment
The criterion here should simply be whether someone using assistive technology can handle the page and its interactions. This requires testing using the screen readers or a talking browser like HPR. Without access to assistive technology, you can evaluate items such as fly-over menus by testing whether there are alternative text links for all submenu links readily available, that is, is the enhancement provided by the script simply visual?
Applets and Plug-Ins
§1194.22(m). When a web page requires that an applet, plug-ins, or other application be present on the client system to interpret page content, the page must provide a link to a plug-in or applet that complies with §1194.22(a) through (l).
Algorithmic
Software accessibility checkers that can detect the existence of applets, plug-ins, or other applications, are required to interact with a page or are embedded into that page.
Judgment
Human involvement is required in determining whether or not a link is present to open or obtain the plug-in, applet, or application, and whether or not that plug-in, applet, or application meets the Section 508 Software standards §1194.21 (a)–(l).
Online Forms
§1194.22(n). When electronic forms are designed to be completed online, the form shall allow people using assistive technology to access the information, field elements, and functionality required for completion and submission of the form, including all directions and cues.
Algorithmic
A testing tool can check that every input element of type text, password, checkbox, radio and file has
an id attribute matching the for attribute of a label element enclosing text,
the prompt for that input element (#10, #12-16). If there is no such label then the tool can
check if there is a valid title attribute on the form control (#11). The same checking applies to select menus
and textarea elements (#17, #18).
There can be algorithmic checks on the quality of both the title and the label, for example
that neither be empty (#20).
In addition, input elements of type image must have a valid alt attribute,
but that is an alt-text problem, not a form problem.
Judgment
Human judgment comes in verifying that the prompts (labels) and titles are appropriate. Keyboard access should be checked as well.
Skip Navigation
§1194.22(o). A method shall be provided that permits users to skip repetitive navigation links.
Algorithmic
If there were a reliable algorithm to determine the position of the main content, and what to skip over, then screen reader developers would include that in their products, and so no ‘skip navigation link’ would be needed! In fact, screen readers include jumps like this, but they are still heuristic and quite likely to miss the mark. For each large block of links (greater than, say, five), software could check if there is a link to a local anchor that skips over those links (#29). In a similar way, a tool could check for the presence of headings markup that could serve instead of skip links (#30). If either of these tests turn out positive then the page complies with §1194.22(o). The page probably also complies if there is no ‘large block’ of links. The page also complies with this provision if it uses frames that comply with 1194.22(i), and which separate main content from navigation.
Judgment
Examine the page to determine whether there are large bkocks of “navigation links” and use assistive technology to verify that a ‘skip navigation link’ works in helping users bypass those links.
Timed Responses
§1194.22(p). When a timed response is required, the user shall be alerted and given sufficient time to indicate more time is required.
Algorithmic
If there is no META-Refresh, there are no forms, or if forms are present and there is no scripting, then there are no timed responses. There may be heuristics to help decide if a timed response is required, but it is difficult to imagine an algorithm that could further decide whether or not sufficient time has been allowed.
Judgment
Check all form submissions to determine if timed responses are required and if it is possible to request more time if they are. If the META-Refresh is used, is the time allowed sufficient or is the user given the option of requesting more time.