Understanding Issue Reports In Tenon.Io API Response

Tenon.io's primary goal is simple: deliver accurate and relevant accessibility test results in a way that is useful and clear. We hope that the users of our API can immediately glance at results and know what is wrong, where, and why. We hope the guidance we provide gives API users the information they need to be able to fix the issues we uncover.

The Tenon API returns a JSON string which contains a resultSet object. That object will contain an array of issues. Each issue in the Tenon JSON response will contain the following keys.

Complete issue example

All issues returned in the Tenon API response have the following structure.

Let's break this down into its individual pieces:

bpID (Best Practice ID)

Numeric string or integer. The bpID maps to the specific unique ID for the best practice that was violated. Each best practice may have more than one applicable test.

Example: bpID: 118

tID (Test ID)

Numeric string or integer. The tID maps to the specific unique ID for the test that was failed.

Example: tID: 3

certainty

Numeric string or integer. This represents how confident we are that we've found an issue. This score will be one of the following:

You should expect to find very few results from Tenon which have a low certainty, but it is possible.

Example: certainty: 100

priority

Numeric string or integer. This represents the calculated priority for this specific issue. The priority for an issue is calculated based on a number of factors relating to user impact, ease & speed of repair, issue location, etc. We supply this value to assist you in determining the order in which repairs are recommended.

Example: priority: 73

errorDescription

Alphanumeric string. This is a description of the error and will often contain remediation guidance.

Note: We denote code samples using backtick characters (`) following [Markdown conventions for code samples](https://help.github.com/articles/basic-writing-and-formatting-syntax/#quoting-code). HTML elements & attributes, CSS properties & declarations, or JavaScript samples will follow this convention, allowing you to more clearly understand the content.

Example: errorDescription: "Language of the document has not been set"

errorSnippet

Alphanumeric string. This is the snippet of code found to be in violation. It is limited to 255 characters and will often contain only the HTML element with the issue. This snippet of code will be encoded into appropriate HTML Entities.

Example: errorSnippet: "<html class="no-js"><head> <meta charset="utf-8"> <title></title> <style type="text/css"> body { width: 1600px; margin: 0; padd",

errorTitle

Alphanumeric string. This is a concise title for the issue.

Example: errorTitle: "Language not set"

issueID

Alphanumeric string. This is a unique ID value for this specific issue.

Example: errorSnippet: issueID: "96c6741ea5e491b9597a96089dce60c1"

position.column

Numeric string or integer. This is the column of the first character of the violating code, relative to the entire document source as tested. It does not correlate to your actual file's source code but rather the final rendered document's source.

Example: position: { column: 24, line: 3 }

position.line

Numeric string or integer. This is the line of the first character of the violating code, relative to the entire document source as tested. It does not correlate to your actual file's source code but rather the final rendered document's source.

Note: in very rare edge cases, such as when comments are placed above the document's DOCTYPE, this number may be off by a line or two.

Example: position: { column: 24, line: 3 }

viewPortLocation

Object containing physical, on-screen location and size of the node where the issue exists.

Example: "viewPortLocation": &#123 "bottom-right": &#123 "x": 25, "y": 96 }, "height": 30, "top-left": &#123 "x": 0, "y": 66 }, "width": 25 }

ref

Alphanumeric string. If your request's 'ref' parameter was set to '1', you will see a fully qualified URL here for a more lengthy discussion of the error and more informative remediation guidance.

Example: ref: "https://tenon.io/bestpractice.php?tID3&bpID=118"

resultTitle

Alphanumeric string. This is the specific title of the Best Practice that was violated.

Example: errorDescription: "Language of the document has not been set"

signature

Alphanumeric string. This is an MD5 hash of the error, to ensure uniqueness.

Example: signature: "44a94237c000e850648a666c8f85d30d"

standards

This is an array of WCAG 2.0 Success Criteria that were violated in this issue. Each array item will be an alphanumeric string.

Example: standards: [ "Web Content Accessibility Guidelines (WCAG) 2.0, Level A: 3.1.1 Language of Page" ]

xPath

Alphanumeric string. This is the full xPath for the issue. As is the case with position, it does not correlate to your actual file's source code but rather the final rendered document's source.

Example: xpath: "/html/body/table[1]/tr[1]/td[1]/div[4]/img[1]"

What do I do if I disagree with a test result?

All tests for all accessibility tools are based on the tool creators' interpretation of the relevant standards and what it means to comply with those standards. Tenon is no different. There are two reasons why you might disagree with a test result:

You disagree with our interpretation of the standard.
We have a buggy test.

You have two courses of action: filter it out or tell us. We actually prefer the latter.

Filtering it out

It is impossible for us to meet the unique needs of all users and all organizations - particularly those which have created their own internal standards and defined their own risk tolerance. In such cases you may wish to filter out those results. The best and quickest way to do that is to find the Test ID (tID) or even the Best Practice ID (bpID) of the results you disagree with and filter out of future results.

We strongly recommend against sorting/ filtering of results based on any value that contains an alphanumeric string, such as errorDescription. This value is very likely to change because best practice and test information is not tied to any release cycle but is rather likely to evolve as we see need and opportunity for the information to become more clear. We provide a handful of numeric items which should be used for filtering and ordering instead, such as the Best Practice ID or TestID.

Before you do that, please consider that test content should be regarded as 'fluid' in nature. Our overall goal in testing is to provide a high degree of accuracy and efficiency in testing. In general, we'd prefer that you not filter your results based specifically on the Test ID (tID) or Best Practice ID (bpID). Instead, we'd rather you contact us to talk about what you disagree with and why.

Tell us

Quality is hugely important to us. If we have a bug we need to know about it. It benefits everyone - you, us, and other users - to tell us when you disagree with a test result. Here are the things we hope you can provide to make the conversation most useful:

Report Date (responseTime)
Report ID (reportID)
Test ID (tID)
Tested URL or document source, if possible
Specific issue source (errorSnippet)
Explanation of specifically what you disagree with and why

What happens next:

After we get your message, we may need a while to digest it and in a lot of cases, we will subject it to our internal unit testing process. Improving and correcting our tests takes time. But you can be sure we are listening and will you will receive a response within a few days.

If you find a bug in our tests, we will likely make an immediate change to our test's logic to ensure similar errant results are no longer returned. Our tests may be modified quickly allowing Tenon to stay in step with changes to browsers, techniques and new specifications. In some cases, the issue raised may be more a matter of opinion than a clear cut bug. Our goal, as a company, is to deliver the best possible results to all of our customers so we may choose to stand by our interpretation of the standards and make no change. In which case we will inform you of the reasoning behind our decision and if you still disagree with us, that's fine. In such cases, we encourage you to filter out that specific tID from your results.