Understanding Issue Reports In Tenon.io API Response

Tenon.io's primary goal is simple: deliver accurate and relevant accessibility test results in a way that is useful and clear. We hope that the users of our API can immediately glance at results and know what is wrong, where, and why. We hope the guidance we provide gives API users the information they need to be able to fix the issues we uncover.

The Tenon API returns a JSON string which contains a resultSet object. That object will contain an array of issues. Each issue in the Tenon JSON response will contain the following keys.

Complete issue example

All issues returned in the Tenon API response have the following structure.

{
        "bpID": 1,
        "certainty": 100,
        "errorDescription": "All images must have an alt attribute. Not supplying an alt attribute will mean that users who
            cannot see the image will not understand what the image conveys. ",
        "errorSnippet": "<img src=\"/images/foo.jpg\">",
        "errorTitle": "Image missing alt attribute",
        "issueID": "f85bbbdd7c6815cecfd3ebdc3394e417",
        "position": {
            "column": 0,
            "line": 1765
        },
        "viewPortLocation": {
            "bottom-right": {
                "x": 25,
                "y": 36
            },
            "height": 30,
            "top-left": {
                "x": 0,
                "y": 6
            },
            "width": 25
        },
        "priority": 80,
        "ref": "https://tenon.io/bestpractice.php?bpID=1&tID=9",
        "resultTitle": "Provide alt attributes for all img elements",
        "signature": "d020c1b6fd8de65565b32c02e6da4295",
        "standards": [
            "Web Content Accessibility Guidelines (WCAG) 2.0, Level A: 1.1.1 Non-text Content"
        ],
        "tID": 9,
        "xpath": "/html/body/div[4]/div[1]/div[3]/div[1]/div[1]/div[1]/a[1]/div[1]/div[3]/div[1]/img[1]"
    }

Let's break this down into its individual pieces:

bpID (Best Practice ID)

Example: bpID: 118

tID (Test ID)

Example: tID: 3

certainty

  • 0
  • 20
  • 40
  • 60
  • 80
  • 100

Example: certainty: 100

priority

Example: priority: 73

errorDescription

Example: errorDescription: "Language of the document has not been set"

errorSnippet

Example: errorSnippet: "<html class="no-js"><!--<![endif]--><head><meta charset="utf-8"> <title></title> <style type="text/css"> body { width: 1600px; margin: 0; padd",

errorTitle

Example: errorTitle: "Language not set"

issueID

Example: issueID: "96c6741ea5e491b9597a96089dce60c1"

position.column

Example: position: { column: 24, line: 3 }

position.line

Example: position: { column: 24, line: 3 }

viewPortLocation

Example: "viewPortLocation": { "bottom-right": { "x": 25, "y": 96 }, "height": 30, "top-left": { "x": 0, "y": 66 }, "width": 25 }

ref

Example: ref: "https://tenon.io/bestpractice.php?tID3&bpID=118"

resultTitle

Example: errorDescription: "Language of the document has not been set"

signature

Example: signature: "44a94237c000e850648a666c8f85d30d"

standards

Example: standards: [ "Web Content Accessibility Guidelines (WCAG) 2.0, Level A: 3.1.1 Language of Page" ]

xPath

Example: xpath: "/html/body/table[1]/tr[1]/td[1]/div[4]/img[1]"

What do I do if I disagree with a test result?

All tests for all accessibility tools are based on the tool creators' interpretation of the relevant standards and what it means to comply with those standards. Tenon is no different. There are two reasons why you might disagree with a test result:

  • You disagree with our interpretation of the standard.
  • We have a buggy test.

You have two courses of action: filter it out or tell us. We actually prefer the latter.

Filtering it out

It is impossible for us to meet the unique needs of all users and all organizations - particularly those which have created their own internal standards and defined their own risk tolerance. In such cases you may wish to filter out those results. The best and quickest way to do that is to find the Test ID (tID) or even the Best Practice ID (bpID) of the results you disagree with and filter out of future results.

We strongly recommend against sorting/ filtering of results based on any value that contains an alphanumeric string, such as errorDescription. This value is very likely to change because best practice and test information is not tied to any release cycle but is rather likely to evolve as we see need and opportunity for the information to become more clear. We provide a handful of numeric items which should be used for filtering and ordering instead, such as the Best Practice ID or TestID.

Before you do that, please consider that test content should be regarded as 'fluid' in nature. Our overall goal in testing is to provide a high degree of accuracy and efficiency in testing. In general, we'd prefer that you not filter your results based specifically on the Test ID (tID) or Best Practice ID (bpID). Instead, we'd rather you contact us to talk about what you disagree with and why.

Tell us

Quality is hugely important to us. If we have a bug we need to know about it. It benefits everyone - you, us, and other users - to tell us when you disagree with a test result. Here are the things we hope you can provide to make the conversation most useful:

  • Report Date (responseTime)
  • Report ID (reportID)
  • Test ID (tID)
  • Tested URL or document source, if possible
  • Specific issue source (errorSnippet)
  • Explanation of specifically what you disagree with and why

What happens next:

After we get your message, we may need a while to digest it and in a lot of cases, we will subject it to our internal unit testing process. Improving and correcting our tests takes time. But you can be sure we are listening and will you will receive a response within a few days.

If you find a bug in our tests, we will likely make an immediate change to our test's logic to ensure similar errant results are no longer returned. Our tests may be modified quickly allowing Tenon to stay in step with changes to browsers, techniques and new specifications. In some cases, the issue raised may be more a matter of opinion than a clear cut bug. Our goal, as a company, is to deliver the best possible results to all of our customers so we may choose to stand by our interpretation of the standards and make no change. In which case we will inform you of the reasoning behind our decision and if you still disagree with us, that's fine. In such cases, we encourage you to filter out that specific tID from your results.