The Conformance Committee is focused on promoting the adoption and utilization of VoiceXML by establishing a common understanding of the specification and its dependencies. This work may require test suite creation, certification programs, technical clarifications where the specifications are vague, and other forms of communication.

1 Summary

End Date: 31 August 2011
Chair: Ken Rehor and Ian Sutherland
IP Policy: RAND

2 Background

The VoiceXML Forum’s Platform Certification Program certifies specific functionality of VoiceXML implementation platforms. The program includes test suites that define required and optional VoiceXML functionality, a test harness to run the tests, and independent testing. The test suite is based on the W3C Recommendations for VoiceXML 2.0, VoiceXML 2.1, and SRGS 1.0, and their corresponding Implementation Report test suites. The program goals include: 

  • interoperability of platforms, voice service providers, tools, and applications
  • clarification of vendor implementation of language features and compatibility
  • a common test suite available to the entire industry
  • Third-party independent certification testing

Platform vendors can test their platforms against the publicly available test suite. VoiceXML Forum Sponsor and Promoter members have access to a test harness to assist in local execution of the tests.

The certification program is managed and developed by VoiceXML Forum members and provides impartiality through the use of an independent third-party test laboratory. The VoiceXML Forum Certification Program is open to all vendors who wish to have their products tested. 

3 Scope

The Conformance Committee is chartered with the following tasks:

3.1 Test Suite

Define, develop, and update tests for relevant standards. Develop a test execution environment and supporting documentation to enable vendors to run the tests in their own labs, as well as prepare for platform certification.

3.2 Platform Certification Program

Develop and manage a program to certify vendor platforms with the Forum’s test suites. An independent, third-party testing laboratory performs testing on behalf of the VoiceXML Forum to ensure neutrality.

3.3 Industry Relationships

The Conformance Committee works with the W3C Voice Browser Working Group (VBWG) to improve the test suites and relevant specifications. The Committee refines and augments the test suites originally supplied by the W3C as “Implementation Report” tests. Suggested changes to the tests and relevant specifications are provided back to the VBWG and are published on the Forum’s website.

4 Deliverables

There are several types of deliverables from the committee:

  • Software
    • Test scripts that accurately reflect the desired behavior relevant specifications
    • Execution environment and supporting tools
    • Documentation

     

  • Certification Program
    • Process and procedures for independent testing services
    • Contracts for vendor testing and testing laboratory
    • Program management

     

Deliverable:  Conformance Test Suite Modules

New modules of the conformance test suite covering the following:

  • SISR 1.0
  • SSML 1.0
  • PLS 1.0
  • RFC 5552

The modules for SSML 1.0 and PLS 1.0 will not attempt to cover the subjective aspects of the Recommendations, but will focus on measurable, automatable aspects, such as timing and acceptance of correct syntax.

Deliverable: Test Execution Environment

Any expansions or modifications necessary to the following:

  • TXML-to-VXML conversion and macro expansion tools
  • ICS, CTR generation tool, and documentation
  • Test harness
  • Documentation for vendors and testing lab

Deliverable: Platform Certification Program

  • Updates to process documents and participant agreement
  • Offer packaging and pricing structure
  • Input to Marketing and PR plan, including identification and commitment of vendors for testing program
  • Pilot testing phase
    • 3 to 6 platforms certified in pilot phase
    • Every test assertion passed by at least one vendor.

     

Deliverable: Test Laboratory Updates

These updates will be performed by the test laboratory.

  • Updates to test execution environment to cover new modules
  • Updates to documentation

5 Dependencies

On 27 September 2001, the VoiceXML Forum signed a Memorandum of Understanding with the W3C. This document remains in effect and gives the W3C ‘sole control of the definition and evolution of the dialog markup language’ known as VoiceXML. It further specifies that:

The VoiceXML Forum will coordinate the creation of test suites and conformance evaluation with the W3C. The VoiceXML Forum will provide specification clarification requests to the W3C through the channels provided by the W3C for this purpose.

The effect of this agreement is that clarifications concerning the substance or intent of the VoiceXML specification must be referred by the W3C Liaison to the Voice Browser Working Group of the W3C. The Conformance Committee has considerable leeway in determining the scope of its certification program and in defining and improving tests of language features. Where coordination is needed, the W3C Liaison will be the preferred channel.

W3C test suites are licensed under the W3C Document License. The Committee will need to understand the requirements of that license for any W3C test suites that are redistributed or modified.

6 Communications

This committee meets for one hour per week on a teleconference bridge and carries out much of its work by email. There are additional resources necessary for developing and maintaining the test suites.

  • Conference calls: weekly with meeting minutes posted to the internal mailing list
  • Internal mailing list intended primarily for committee members but readable by all Forum members eligible for participation
  • Internal web site: the repository for internal documents, committee, email archive, calendar, conference call details, files, CVS, etc.
  • Public web page: describing the activities of the working group

7 Participant Requirements and Commitment 

Conformance Committee members are expected to commit to attending the majority of regular conference calls, following discussion on the lists, and contributing to the development and maintenance of the conformance test suites. This should average out to approximately four hours per week. 

The successful development of a VoiceXML certification program will benefits the entire VoiceXML ecosystem. The participation on this committee is expected to include a broad spectrum of companies. Specifically, this committee should have active representatives from platform providers, tool vendors, and application developers. 

Copyright © 2010, VoiceXML Forum. All rights reserved.