Evaluating website accessibility part 1, Background and Preparation

In more and more countries across the world it is becoming required for government and other public service websites to be based on standards and follow accessibility guidelines. That in turn is making it necessary for the people involved in building and maintaining these sites to be able to build accessible websites and evaluate website accessibility.

Many people, web developers as well as website owners, are new to website accessibility and find it difficult to evaluate. This three-part article series is intended to make it easier for non-experts to perform a basic accessibility check. I hope it will be helpful enough to make at least a few websites more accessible.

The series consists of the following articles:

  1. Evaluating website accessibility part 1, Background and Preparation (this article) provides some background and suggests some useful tools for the evaluation process.
  2. Evaluating website accessibility part 2, Basic Checkpoints explains accessibility aspects that can be tested with automated tools as well as some relatively easy manual checks.
  3. Evaluating website accessibility part 3, Digging Deeper takes a look at things that are difficult to test with automated tools and require more time and/or experience to evaluate manually.

Note that running through the checkpoints described in this article series does not make testing with actual disabled persons using screen readers and other assistive devices unnecessary. But if you do not have the means to do that, going through the checkpoints described in this article series will give you a very good indication of the accessibility status of the website you are evaluating.


A common definition of web accessibility is access to the Web by everyone, regardless of disability. That definition implies that accessibility is only about people with disabilities. It may or may not be the official definition of accessibility, but either way I find it way too narrow, and I’m not the only one. Robert Nyman, for instance, ponders the subject in What is Accessibility?.

When I mention accessibility in this article (and elsewhere), I mean accessibility in a wider sense that also includes device independence – universal access, regardless of disability, user agent or platform. The web should be accessible to all.

A quick look around at various public service websites here in Sweden (and in most of the rest of the world) reveals a sad fact. The developers who have built many of those sites have failed to ensure that their sites are based on web standards and are accessible to all. Even very recently redesigned and rebuilt sites fail miserably at this. If you didn’t know any better, you could think the year was 1997 again by looking at the markup used on many of these sites. That includes sites redesigned in 2006. Pathetic.

For more info on the current state of web standards usage of public service websites around the world, I suggest reading the articles Government web standards usage: USA, Government Web Standards Usage: New Zealand, Government Web Standards Usage: People’s Republic of China, and Mätning av grundläggande tillgänglighet (in Swedish).

Web standards do not equal accessibility, but using web standards is an integral part of building an accessible website, and the validation results usually indicate if any effort at all has been spent on making a site accessible.

When you look at the results from the mass validations of Swedish public sector websites performed by Verva (Swedish Administrative Development Agency), an unfortunate side effect of the validation effort can be seen. Since the first round of validation several sites have tweaked the markup of their front page to make it valid, but keep using presentational markup, massive amounts of inline CSS, layout tables, and spacer GIFs with useless alt texts. This kind of lip service disgusts me, and I have more respect for those who do nothing than for those who cheat their way to validation just for the sake of having no errors in Verva’s mass validation.

The low accessibility and general low quality of most public sector websites is clearly not acceptable. To some extent it is understandable – unless you have some knowledge of website accessibility it is difficult to determine whether a site is accessible or not. Many public sector organisations hire a web agency to build their websites because the organisations themselves do not have any in-house knowledge of HTML. This makes it very hard for those responsible to determine if their website is accessible and if the company they have hired to build it are doing their job properly.

Unfortunately, many web agencies do not seem to be interested in building high-quality websites, using best practices, or making sure the sites they build are accessible to all. Why this is, I do not know. It is easy to assume that people working in the web industry take pride in doing a good job and always strive to do their best. This is clearly far from always the case.

Another major problem is the marketing used by certain CMS vendors. Marketing in which they falsely and knowingly claim that sites based upon their software are fully accessible and adhere to web standards, when all they are really interested in is wowing prospective buyers with WYSIWYG editors, fly-out menus, drag-and-drop integration with Microsoft Office and other bling-bling. As long as CMS vendors are allowed to market their products with false information I’m afraid we won’t see a whole lot of improvement, since there are very few web agencies that will ensure that the CMS they are using produces valid, semantic, and accessible code.

Ok, now that I’m done ranting about that I’ll get to the point of this article: to explain how you can do a basic website accessibility evaluation.

Preparation: install some tools

First of all I recommend downloading and installing some very useful tools that will make accessibility evaluation much easier:

Many of you will already have one or more of these tools installed, but if you don’t, now is the time to download and install them.

The checkpoints

The checkpoints in these articles cover a lot of areas that typically cause accessibility problems. Some of these are more commonly encountered than others. There are 18 checkpoints in total:

Basic checkpoints

  1. Validate HTML and CSS
  2. No frames, please
  3. Automated accessibility checking tools
  4. Images and alternative text
  5. Make sure that JavaScript is unobtrusive
  6. Increase text size
  7. Look for semantic markup
  8. Disable CSS
  9. Use Fangs to emulate a screen reader

Digging deeper

  1. Colour contrast
  2. Document titles
  3. Link text
  4. Non HTML formats
  5. Platform discrimination
  6. Keyboard navigation
  7. Data tables
  8. Form controls and labels
  9. Use a screen reader
  10. Don’t overlook the content
  11. Further reading

Evaluating website accessibility part 2, Basic Checkpoints will be out soon. Until then maybe we could discuss ways of convincing CMS developers to improve the accessibility and web standards compliance of their products. Anyone have any bright ideas?


This article has been translated into the following languages:

Posted on March 6, 2006 in Accessibility, Web Standards


  1. You missed the best (bar none) validation tool out there, Tawdis download the “standalone tool” it will validate your entire site and also provide pointers to the manual checks to do. Seriously its the best thing since Chris Pederic Perhaps using it to spider a cms’d site and provide them (the cms author) with a breakdown of their errors could help?

  2. March 6, 2006 by Scott

    I have a hunch that many CMS developers don’t know any better. IMHO we should start contacting them and find out if they are receptive to assistance in improving the accessibility of their product. Those that are should be a piece of cake, and those that aren’t will take some time.

    From my experience, getting over that initial hurdle of the other person understanding the importance of standards is the hardest to overcome. The biggest problem is not the people who don’t any better, but the people who don’t care and are satisfied with “good enough”.

  3. I must say that back in 2001 I used inline CSS purely for the sake of validation. But on the other hand, I want just new to the internet and HTML in general and I thought CSS looked sort of interesting. I did far from calling myself a professional, nor did I design sites which were critical for citizens.

  4. Whilst evaluation tools have their uses, far too much emphasis is placed on them in my opinion. All too often, they are seen as the “be all and end all” by corporate managers taken in by salesmen or charlatan web design companies.

    Not knocking the article though - breaking the problem down into a manageable process educates and minimises the trauma of “where do I start?”

  5. incidentally, i did a live 2 hour presentation based on my articles on evaluating web sites for accessibility with firefox and accessibility testing and reporting with TAW3.

    this series looks to be a great addition along the same lines. nice one.

  6. There are 16 checkpoints in total:


  7. Thanks for the tip on HTML Validator Roger, that’s one I wasn’t aware of. Very handy, especially when validator.w3.org is bogged down or if you’re behind a firewall/intranet.

  8. Another fantastic article — some really great tips and/or reminders for every reader. I’ll be looking forward to reading subsequent parts 2 and 3.

  9. March 7, 2006 by Tommy Olsson

    Another useful tool for basic accessibility checking is the Opera browser. It provides easy toggling of images, CSS, JavaScript, frames, plugins, cookies, referrer logging, etc.

  10. March 7, 2006 by Bertrand Denis

    Maybe someone knows the Opquast project ? This is a system with checklist of good practises for websites… A kind of evaluation. An interesting tool !

  11. March 7, 2006 by Normand Lamoureux

    Thanks for this usefull article.

    I am sorry to say that, but frames are not an unaccessible facteur in themselves.

    With JAWS, for example, users can press Insert-F9 to get the full list of frames in the page, and then, reach the one they want.

    The main problem with frames is that they are often whithout a significant name.

    On the other hand, most part of the time using frames still a bad web development idea.

    (Sorry for my bad English.)

  12. March 7, 2006 by Justin

    Accessibility isn’t always dismissed by content management systems (CMS), though they are often potrayed as the bad guys of markup and accessibility. You probably guess then I am employed by a company with our own CMS, and as a HTML and CSS coder I try to do my very best to keep up with “standards”. In fact, if anyone has time to evaluate a site of ours (http://www.burnesscommunications.com/ or maybe http://www.ensuringsolutions.org/) and give us feed back on how to make it better, or maybe even just a 1-10 rating it would be greatly appreciated.

    Also, I try to use Opera’s “text browser” emulation to review our sites, how accurate is that?

  13. Very nice article - it revealed quite a few nice tools I was completely unaware of to help in the development of accessible sites. Thanks so much.

  14. Justin, I’m happy to run a Taw report for you on either of these sites (and indeed have) but its not in an easily sendable form. You would be better downloading and running the tool and indeed the other tools mentioned here yourself including the html validator plugin for Firefox (errors from the front page http://www.ensuringsolutions.org/ listed below using this tool) and doing manual checks for instance its not possible to use the menu items by using the keyboard… you can contact me through my website if you need some links or help

    line 522 column 173 - Warning: <a> attribute "target" lacks value
    line 1007 column 1 - Warning:  inserting "type" attribute
    line 1040 column 48 - Warning:  inserting "type" attribute
    line 1041 column 17 - Warning:  inserting "type" attribute
    line 1042 column 35 - Warning:  inserting "type" attribute
    line 171 column 4 - Warning:  id and name attribute value mismatch

  15. March 7, 2006 by Roger Johansson (Author comment)

    Scrambled: Thanks for the tip about Tawdis. It looks like a useful tool.

    Scott: I have tried approaching a couple of CMS developers, but at least those who create commercial CMS products seem a lot more interested in adding useless features.

    Karl: I agree, and I will go into that aspect of automated tools in the next article in this series.

    Patrick: Ah, I missed your article on TAW3. Thanks for mentioning it.

    Ryan: Ooops. Thanks for catching that typo. There used to be 16 checkpoints until a few days ago.

    Tommy: Yep, Opera is very useful. I didn’t mention it to avoid information overload, and because I do use Firefox (with the Web Developer Extension) more.

    Normand: You are correct that frames are not necessarily inaccessible. On the other hand they do nothing to increase accessibility. More on that in the next article.

    Justin: Nice to see a CMS developer taking interest in accessibility. I don’t have the time right now to do an in-depth evaluation of the sites you mention, but after a quick (really quick) look I’d say you’re doing a whole lot better than most CM systems. But do try to move that Web side story code into the body element ;-).

    Opera’s text browser emulation is reasonably accurate, but not as good as the real thing in my opinion.

  16. And by all means: Do not forget reading «Don’t Make Me Think!».

  17. You definitely bring up an excellent point about the CMS software. I was evaluating one earlier today, so I’ll give an example. They were using great and new technology behind this new CMS, obviously had some very good programmers and designers working on it (from the demo I tried out) … but there was a statement that said (paraphrasing here) something to the effect of… “You can write table-based tag-soup just as easily as you can write perfectly standards based code using our system.”

    That gave me a little pause. Seriously, if you can create such an well made system, why not absolutely force coding compliance, or parse the code to convert it to compliant code before it reaches the publishing point? I’d really like to see -new- software come out that only allows standard code. It will just make it easier in the long run. I definitely like the idea of contacting some of these software companies and helping them convert what they have.

  18. March 8, 2006 by Brian

    Just to back up what Justin says, not all CMS people are mindlessly ploughing away with tables and rubbish code in their systems. I spend a lot of time trying to convince clients that Accessibility IS important and they should pay attention to it and using the CMS system that we give them properly can enable them to have nice friendly sites that are producing decent code.

    In reply to Nicole, the problem is that you can’t remove the ability to put in tables, as that would be as bad as removing header tags or some other perfectly valid bit of HTML. I’m also not sure it would ever be possible to automatically assess if the data in the tables would be tabular and thus valid.

    As Spider-Man’s uncle Ben said, with great power comes great responsibility. If you want a fully featured CMS, the people using it have to take responsibility not to abuse its features to churn out rubbish. That said clearly as a group of products most CMS’s could do more to limit the ability to get it wrong, and a lot of them could start by at least trying to produce valid code.

  19. One important factor is of course the content itself. Even if you pass the triple-A squared check and use valid HTML it will be irrelevant if your content is difficult to understand.

    A lot of accessibility work lies in helping editors adopt a common writing style that has its foundation in the needs of the target audience. You can do a lot for accessibility by just working with content.

    In my experience this area is often overlooked.

    1. 2006-03-08, 15.06 by Peter Krantz One important factor is of course the content itself. Even if you pass the triple-A squared check and use valid HTML it will be irrelevant if your content is difficult to understand.

    I guess that may be covered by Roger in

    Evaluating website accessibility Part 3, Digging Deeper takes a look at things that are difficult to test with automated tools and require more time and/or experience to evaluate manually. ?

  20. Re: Peters comment - and hopefully (Roger) you will cover what you think about

    Even if you pass the triple-A squared check

    actually passing the Triple-A Checkpoint as I have being having a lot of discussions at work lately about the possibility of being able to pass this Checkpoint? any thoughts Peter, Roger?

  21. It’s a theoretical judgment call and most that do label Triple-A do’t really understand the subject in any great depth.

  22. Thanks for the article Roger. You are correct in many of your points.

    What you say about the mass validation checklist is true. Most sites do only fix the startpage nothing else, but is really accessibility about DOCTYPE and Dublin Core? I only see one good point with this. You can tell the boss, -“Hey, we suck at this validation test that Verva does. Can I get 500 hours to fix this?”.

    Most sites today uses a CMS system. So to get better quality in those sites we have to look at the people who is building these systems, the CMS vendors.

    The thing with CMS vendors is that they only see accessibility as one part in building the system, they are also interested in system integration, usability, content conversion, system management, security etc. etc. etc. From their point of view accessibility is only -one- part of the system.

    So when they build their system their customers ask for different requirements, sometimes accessibility is one of the requirement sometimes there is others.

    The good thing in Sweden is that Verva has a checklist for CMS systems, the clients do not need to know anything about accessibility, just send the vendors the checklist to see what state the CMS system is in. The thing is that -not- everyone knows about this checklist.

    Peter Krantz also has a great point, the content itself. When we educate new content editors we educate them in three parts, the CMS system, how to write good content and last but not least accessibility and usability. A great CMS system can never write good headings, understandable links and well written content.

    I also want to point out one thing that is a -problem- for all developers who is working in a large company or goverment. The computers we work with are many times ‘locked’ by IT so no FF, no plugins, no screen reader and no text based browser, I know, this sucks.

    I also think that your checklist should be divided into three groups, CMS vendors, web agencies and web developers and website clients and managers.

    And last, to convince CMS vendors to start build accessible and standard products, they only listen to one thing, -money-. If clients say that the want it and they can pay for it, vendors will build it. -“Money talks”.

    Again, thanks for a great article and hopefully even better follow-ups.

  23. March 8, 2006 by Roger Johansson (Author comment)

    Henrik: Yeah, that is one excellent book!

    Nicole: I don’t see get it either. These are people with excellent programming skills that can get their incredibly (to me) complex code to compile, yet producing valid HTML seems too difficult.

    Brian: Agreed. A lot of the responsibility for making a site accessible is on the person using a CMS. That doesn’t excuse CMS developers from doing their part though.

    Peter: Absolutely. I’m bringing that up in the next two articles. The importance of working with content is why at my dayjob we always teach this to our clients’ editors. How well they pick it up varies a lot, but at least they don’t come out of the course knowing less than they did before.

    Scrambled: You guessed right - I will be covering content later.

    About passing triple-A, well I’d say it is next to impossible to do.

    Robert: I’d say that goes for most that display single- and double-A labels as well.

    Jens: Accessibility is obviously about much much more than that, which is also stated in Verva’s document about the validation.

    Your point about computers in large organisations being locked is sad, but unfortunately true. It continues to amaze me how many IT departments are allowed to make it impossible for web developers to do their jobs properly. They should all be fired.

  24. We’re using Microsoft CMS with Telerik’s RAD editor.

    I have been responsible for producing the prototype pages and I have gone out of my way to follow all the best practices I could find. I am confident that the code I have produces is valid, accessible and has goo usability.

    However, I’m not so confident that this will be the case once we have finished implementing the prototypes as real CMS templates.

    The RAD editor produces some interesting code, visual studio (not 2005) produces even more interesting code, and users will put in exceptionally interesting content.

    I’m going to keep on the backs of the developers working on this CMS, but it seems that it is very difficult to convince these systems to produce anything like valid and accessible code.

  25. March 14, 2006 by Marc Luzietti

    Not all of the CMS stuff out there is created by idiot corporations. Netui, for example, while originally created by BEA, is controlled now by the Apache Foundation. To my ever-lasting disappointment, I can’t get a valid ID out of netui if we need to use a java bean as a data source, as it will autopopulate both id and name with the name of the bean (which always has curly braces).

  26. Nice article. I hadn’t come across fangs before so this is going to improve my coding!

  27. After reading Patricks comment about TAWS and parts of his article about it, I got the app (Java) for OSX. Installation changed my dock (removing apps changing order and configuration). The app was spanish only. Trying to change to english did not work.

    I do not know, what to think about an app for testing web accessibility, if it does not honour the accessibility of my computer. Sad experience.

  28. March 29, 2006 by Marlon

    This article is really useful. I work for a company whose chief product is a CMS system. As much our client base is local governemnet, public service and NHS accessibility is a must.

    We use css and write standards compliant code and are continuing to empahasize the importance of accessibility with our latest release. Check out the website at www.eibs.co.uk. If anyone has any tips on integrating accessible designs/styles with a CMS it would be appreciated.

    P.S. Firefox browser has some really good extensions to aid accessibility testing.

  29. Roger, Just re-read your series - thankyou again.

    However, I noted that you didn’t mention the AIS Web Accessibility Toolbar for IE (www.visionaustralia.org.au/ais/toolbar) which is available in English plus 11 other languages.

Comments are disabled for this post (read why), but if you have spotted an error or have additional info that you think should be in this post, feel free to contact me.