Sign-Speak

Welcome

API Specifications

RESTful ASL Recognition

RESTful English Speech Recognition

RESTful ASL Production

RESTful English Speech Generation

WebSockets Sign Language Recognition

Feedback

Private API Requests

SDK Specifications

Sample Sentences

Sign-Speak API

Improve your accessibility like never before with the Sign-Speak API

Production

<p class="slate-p " >Welcome to Sign-Speak&apos;s API and SDK docs! Sign-Speak provides Sign Language Recognition and Avatar technology with just one API call! Our models have high quantitative benchmarks (BLEU=55.1), and our avatars have similarly been rated by the Deaf community as understandable and accurate. In addition to our Sign Language offerings, we additionally provide a suite of other services (such as text-to-speech and speech-to-text) at similarly high levels of accuracy. Sign-Speak is your one-stop shop for building services for anyone in the Deaf and Hard of Hearing space!</p><p class="slate-p " >If you are interested, you can sign up at our <a href="client.sign-speak.com" target="">development portal</a>. If you have any questions, reach out to us at <a href="mailto:management@sign-speak.com" target="">management@sign-speak.com</a>.</p><p class="slate-p " ></p><h2 class="slate-h2 " >Offerings</h2><p class="slate-p " >Sign-Speak&apos;s services are available in four levels of granularity:</p><ul class="slate-ul " ><li class="slate-li " ><div style="position:relative"><strong class="slate-bold">Little customization requirements - Sign-Speak&apos;s Automated Interpreting System (AIS): </strong>Access our all-inclusive platform that leverages AI and human interpreters to provide 24/7 access to your customers. Provide more access for less.</div></li><li class="slate-li " ><div style="position:relative"><strong class="slate-bold">Moderate Customization Requirements - SDK and Sign-Speak Component Library:</strong> Easily request and surface results via the Sign-Speak Component Library. You can customize the elements via tailwind. This is currently only provided in React, but we are looking at expanding.</div></li><li class="slate-li " ><div style="position:relative"><strong class="slate-bold">High Customization Requirements - SDK Access:</strong> Access the Sign-Speak API via the SDK in select languages. This allows you to easily submit translation requests and get back the result easily.</div></li><li class="slate-li " ><div style="position:relative"><strong class="slate-bold">Highest Customization Requirements - API Access:</strong> Access the Sign-Speak API from any programming language via a RESTful and Websockets interface. This provides you with the most flexibility, though may require more code.</div></li></ul><p class="slate-p " >Note that customization requirements is tied to how much customization you need from an off-the-shelf solution. <strong class="slate-bold">We recommend you to start by trying our AIS system to see our API in action.</strong></p><p class="slate-p " ></p><h2 class="slate-h2 " >Request Types</h2><p class="slate-p " >Sign-Speak&apos;s API is setup to be flexible to your use case. There are various endpoints to interact with the system in a realtime or batch manner. Requests are either inference-based requests (e.g. sign language recognition, sign language avatar), in which Sign-Speak&apos;s AI is used, or administrative requests (this is when no AI is engaged, such as for key management). All administrative requests operate via a RESTful API. However, inference requests fall into three types:</p><p class="slate-p " ><code class="slate-code"><strong class="slate-bold">BLOCKING</strong></code><strong class="slate-bold"> (RESTful):</strong> <code class="slate-code">BLOCKING</code> requests attempt to execute a request synchronously. These requests will keep the connection open and blocked for 30 seconds. If our AI models finish processing your request by that time, the request will be returned synchronously. In a vast majority of cases, this occurs. However, if your request takes longer than 30 seconds, it will undergo a <strong class="slate-bold">batch-downgrade. </strong>In batch-downgrades, the request will return a <code class="slate-code">request_id</code> (under a HTTP code of <code class="slate-code">202</code>). This request will then become a <code class="slate-code">BATCH</code> request (and can be querried as such).</p><p class="slate-p " ><code class="slate-code"><strong class="slate-bold">BATCH</strong></code><strong class="slate-bold"> (RESTful):</strong> <code class="slate-code">BATCH</code> requests are useful for fire-and-forget jobs, or jobs in batch pipelines. Batch requests return a</p><p class="slate-p " ><code class="slate-code">request_id</code> that identify that request. You can then query via a <code class="slate-code">GET</code> to see if that request has been completed (and if so, what the result is). <code class="slate-code">BATCH</code> requests may be longrunning.</p><p class="slate-p " ><strong class="slate-bold">Websocket request: </strong>Websocket requests are the preferred mechanism to interact with our system when integrating into realtime applications. Websocket requests process the request <em class="slate-italic"><strong class="slate-bold">while</strong></em><strong class="slate-bold"> a video is being recorded</strong>. This allows us to up-front some of the processing time prior to the user pressing the <em class="slate-italic">stop recording </em>button (allowing for lower latency).</p><h2 class="slate-h2 " >Payment Mechanism</h2><p class="slate-p " >Inference API requests are billed at a per-second rate (per-second of input). This per-second rate can be viewed by logging into your portal. <strong class="slate-bold">Note that the per-second rate does not accumulate or charge you until after your trial period</strong>. Requests are additionally rate-limited and quota-limited. </p><p class="slate-p " ><strong class="slate-bold">Account-wide quota: </strong>Your account has a monthly quota which dictates the maximum amount of minutes processed. We automatically set this behind the scenes and adjust it to ensure flexibility, and prevent abuse. You can view this via the administrative RESTful API. Reach out to use if you&apos;d like this to be increased.</p><p class="slate-p " ><strong class="slate-bold">Key-level quota: </strong>You can set quotas per key to allow for individual key management. You can manage this via the administrative RESTful API.</p><p class="slate-p " ><strong class="slate-bold">Account-wide rate-limits: </strong>We automatically impose reasonable rate-limits to each account to prevent abuse. These rate-limits restrict the number of requests you can issue in a minute time-frame. Reach out to use if you&apos;d like this to be increased.</p><h2 class="slate-h2 " >Keys</h2><p class="slate-p " >Our API access are built around <strong class="slate-bold">API keys</strong>. You should be able to see your APIs key by logging into your <a href="client.sign-speak.com" target="">development portal</a>. Our API keys come in three flavors:</p><p class="slate-p " ><code class="slate-code"><strong class="slate-bold">PRIVATE</strong></code><strong class="slate-bold"> Keys:</strong> These keys can manage your entire account. For example, they can mint new keys, set quotas on keys, delete and promote keys, and view usage. Private keys can additionally view requests and results from all keys, as well as using the inference APIs. Only use these keys <strong class="slate-bold">on the server.</strong> Never deploy a private key to your front end code.</p><p class="slate-p " ><code class="slate-code"><strong class="slate-bold">ROOT</strong></code><strong class="slate-bold"> key:</strong> These are a special type of private key which cannot be delted. <strong class="slate-bold">Only use this key to mint other keys</strong>. This key exists to ensure that your account always has an API key.</p><p class="slate-p " ><code class="slate-code"><strong class="slate-bold">PUBLIC</strong></code><strong class="slate-bold"> Keys:</strong> Public keys can only execute inference-based requests, and query their own requests (during <code class="slate-code">BATCH</code> mode). Each <code class="slate-code">PUBLIC</code> key is sandboxed from eachother.</p><p class="slate-p " ></p><h2 class="slate-h2 " >Limited offerings</h2><p class="slate-p " >Sign-Speak has several features which it only offers to partners and clients on a <em class="slate-italic">case-by-case</em> basis.</p><p class="slate-p " ><strong class="slate-bold">Personalized Avatar:</strong> Our avatar has the ability to mimic a person&apos;s likeness dynamically.</p><p class="slate-p " ><strong class="slate-bold">Streaming Mode:</strong> When streaming mode is enabled and using websockets, our model will produce<em class="slate-italic"> Interim Results.</em></p><p class="slate-p " ><strong class="slate-bold">Zero-Input Mode: </strong>When using zero-input mode, our models do not require the user to press a “start” or “stop” button. Rather, they can continuously sign into the model and it will predict.</p><p class="slate-p " ><strong class="slate-bold">Fine-Tuning: </strong>Fine-tune our general models to match your use-case. Note that this is unnessasary in most everyday use cases, but can be helpful if you have unique terminology.</p><p class="slate-p " ><strong class="slate-bold">Avatar Signing Style: </strong>Allow your users to set the avatar&apos;s signing style between SEE, PSE, ASL, Classifiers, and more!</p><p class="slate-p " ></p><p class="slate-p " >Welcome to Sign-Speak&apos;s API and SDK docs! Sign-Speak provides Sign Language Recognition and Avatar technology with just one API call! Our models have high quantitative benchmarks (BLEU=55.1), and our avatars have similarly been rated by the Deaf community as understandable and accurate. In addition to our Sign Language offerings, we additionally provide a suite of other services (such as text-to-speech and speech-to-text) at similarly high levels of accuracy. Sign-Speak is your one-stop shop for building services for anyone in the Deaf and Hard of Hearing space!</p><p class="slate-p " >If you are interested, you can sign up at our <a href="client.sign-speak.com" target="">development portal</a>. If you have any questions, reach out to us at <a href="mailto:management@sign-speak.com" target="">management@sign-speak.com</a>.</p><p class="slate-p " ></p><h2 class="slate-h2 " >Offerings</h2><p class="slate-p " >Sign-Speak&apos;s services are available in four levels of granularity:</p><ul class="slate-ul " ><li class="slate-li " ><div style="position:relative"><strong class="slate-bold">Little customization requirements - Sign-Speak&apos;s Automated Interpreting System (AIS): </strong>Access our all-inclusive platform that leverages AI and human interpreters to provide 24/7 access to your customers. Provide more access for less.</div></li><li class="slate-li " ><div style="position:relative"><strong class="slate-bold">Moderate Customization Requirements - SDK and Sign-Speak Component Library:</strong> Easily request and surface results via the Sign-Speak Component Library. You can customize the elements via tailwind. This is currently only provided in React, but we are looking at expanding.</div></li><li class="slate-li " ><div style="position:relative"><strong class="slate-bold">High Customization Requirements - SDK Access:</strong> Access the Sign-Speak API via the SDK in select languages. This allows you to easily submit translation requests and get back the result easily.</div></li><li class="slate-li " ><div style="position:relative"><strong class="slate-bold">Highest Customization Requirements - API Access:</strong> Access the Sign-Speak API from any programming language via a RESTful and Websockets interface. This provides you with the most flexibility, though may require more code.</div></li></ul><p class="slate-p " >Note that customization requirements is tied to how much customization you need from an off-the-shelf solution. <strong class="slate-bold">We recommend you to start by trying our AIS system to see our API in action.</strong></p><p class="slate-p " ></p><h2 class="slate-h2 " >Request Types</h2><p class="slate-p " >Sign-Speak&apos;s API is setup to be flexible to your use case. There are various endpoints to interact with the system in a realtime or batch manner. Requests are either inference-based requests (e.g. sign language recognition, sign language avatar), in which Sign-Speak&apos;s AI is used, or administrative requests (this is when no AI is engaged, such as for key management). All administrative requests operate via a RESTful API. However, inference requests fall into three types:</p><p class="slate-p " ><code class="slate-code"><strong class="slate-bold">BLOCKING</strong></code><strong class="slate-bold"> (RESTful):</strong> <code class="slate-code">BLOCKING</code> requests attempt to execute a request synchronously. These requests will keep the connection open and blocked for 30 seconds. If our AI models finish processing your request by that time, the request will be returned synchronously. In a vast majority of cases, this occurs. However, if your request takes longer than 30 seconds, it will undergo a <strong class="slate-bold">batch-downgrade. </strong>In batch-downgrades, the request will return a <code class="slate-code">request_id</code> (under a HTTP code of <code class="slate-code">202</code>). This request will then become a <code class="slate-code">BATCH</code> request (and can be querried as such).</p><p class="slate-p " ><code class="slate-code"><strong class="slate-bold">BATCH</strong></code><strong class="slate-bold"> (RESTful):</strong> <code class="slate-code">BATCH</code> requests are useful for fire-and-forget jobs, or jobs in batch pipelines. Batch requests return a</p><p class="slate-p " ><code class="slate-code">request_id</code> that identify that request. You can then query via a <code class="slate-code">GET</code> to see if that request has been completed (and if so, what the result is). <code class="slate-code">BATCH</code> requests may be longrunning.</p><p class="slate-p " ><strong class="slate-bold">Websocket request: </strong>Websocket requests are the preferred mechanism to interact with our system when integrating into realtime applications. Websocket requests process the request <em class="slate-italic"><strong class="slate-bold">while</strong></em><strong class="slate-bold"> a video is being recorded</strong>. This allows us to up-front some of the processing time prior to the user pressing the <em class="slate-italic">stop recording </em>button (allowing for lower latency).</p><h2 class="slate-h2 " >Payment Mechanism</h2><p class="slate-p " >Inference API requests are billed at a per-second rate (per-second of input). This per-second rate can be viewed by logging into your portal. <strong class="slate-bold">Note that the per-second rate does not accumulate or charge you until after your trial period</strong>. Requests are additionally rate-limited and quota-limited. </p><p class="slate-p " ><strong class="slate-bold">Account-wide quota: </strong>Your account has a monthly quota which dictates the maximum amount of minutes processed. We automatically set this behind the scenes and adjust it to ensure flexibility, and prevent abuse. You can view this via the administrative RESTful API. Reach out to use if you&apos;d like this to be increased.</p><p class="slate-p " ><strong class="slate-bold">Key-level quota: </strong>You can set quotas per key to allow for individual key management. You can manage this via the administrative RESTful API.</p><p class="slate-p " ><strong class="slate-bold">Account-wide rate-limits: </strong>We automatically impose reasonable rate-limits to each account to prevent abuse. These rate-limits restrict the number of requests you can issue in a minute time-frame. Reach out to use if you&apos;d like this to be increased.</p><h2 class="slate-h2 " >Keys</h2><p class="slate-p " >Our API access are built around <strong class="slate-bold">API keys</strong>. You should be able to see your APIs key by logging into your <a href="client.sign-speak.com" target="">development portal</a>. Our API keys come in three flavors:</p><p class="slate-p " ><code class="slate-code"><strong class="slate-bold">PRIVATE</strong></code><strong class="slate-bold"> Keys:</strong> These keys can manage your entire account. For example, they can mint new keys, set quotas on keys, delete and promote keys, and view usage. Private keys can additionally view requests and results from all keys, as well as using the inference APIs. Only use these keys <strong class="slate-bold">on the server.</strong> Never deploy a private key to your front end code.</p><p class="slate-p " ><code class="slate-code"><strong class="slate-bold">ROOT</strong></code><strong class="slate-bold"> key:</strong> These are a special type of private key which cannot be delted. <strong class="slate-bold">Only use this key to mint other keys</strong>. This key exists to ensure that your account always has an API key.</p><p class="slate-p " ><code class="slate-code"><strong class="slate-bold">PUBLIC</strong></code><strong class="slate-bold"> Keys:</strong> Public keys can only execute inference-base</p><p class="slate-p " >d requests, and query their own requests (during <code class="slate-code">BATCH</code> mode). Each <code class="slate-code">PUBLIC</code> key is sandboxed from eachother.</p><h2 class="slate-h2 " >Need help testing? We can help. </h2><p class="slate-p " >We understand that you may need to test your integration without prior ASL knowledge. To help you get started quickly, we&apos;ve provided some sample videos available in the “Sample Sentences” section below.</p><p class="slate-p " ></p><p class="slate-p " ></p>

Welcome to Sign-Speak's API and SDK docs! Sign-Speak provides Sign Language Recognition and Avatar technology with just one API call! Our models have high quantitative benchmarks (BLEU=55.1), and our avatars have similarly been rated by the Deaf community as understandable and accurate. In addition to our Sign Language offerings, we additionally provide a suite of other services (such as text-to-speech and speech-to-text) at similarly high levels of accuracy. Sign-Speak is your one-stop shop for building services for anyone in the Deaf and Hard of Hearing space!

If you are interested, you can sign up at our 

. If you have any questions, reach out to us at 

Sign-Speak's services are available in four levels of granularity:

Note that customization requirements is tied to how much customization you need from an off-the-shelf solution. 

We recommend you to start by trying our AIS system to see our API in action.

Sign-Speak's API is setup to be flexible to your use case. There are various endpoints to interact with the system in a realtime or batch manner. Requests are either inference-based requests (e.g. sign language recognition, sign language avatar), in which Sign-Speak's AI is used, or administrative requests (this is when no AI is engaged, such as for key management). All administrative requests operate via a RESTful API. However, inference requests fall into three types:

 requests attempt to execute a request synchronously. These requests will keep the connection open and blocked for 30 seconds. If our AI models finish processing your request by that time, the request will be returned synchronously. In a vast majority of cases, this occurs. However, if your request takes longer than 30 seconds, it will undergo a 

In batch-downgrades, the request will return a 

 requests are useful for fire-and-forget jobs, or jobs in batch pipelines. Batch requests return a

 that identify that request. You can then query via a 

 to see if that request has been completed (and if so, what the result is). 

Websocket requests are the preferred mechanism to interact with our system when integrating into realtime applications. Websocket requests process the request 

. This allows us to up-front some of the processing time prior to the user pressing the 

Inference API requests are billed at a per-second rate (per-second of input). This per-second rate can be viewed by logging into your portal. 

Note that the per-second rate does not accumulate or charge you until after your trial period

. Requests are additionally rate-limited and quota-limited. 

Your account has a monthly quota which dictates the maximum amount of minutes processed. We automatically set this behind the scenes and adjust it to ensure flexibility, and prevent abuse. You can view this via the administrative RESTful API. Reach out to use if you'd like this to be increased.

You can set quotas per key to allow for individual key management. You can manage this via the administrative RESTful API.

We automatically impose reasonable rate-limits to each account to prevent abuse. These rate-limits restrict the number of requests you can issue in a minute time-frame. Reach out to use if you'd like this to be increased.

. You should be able to see your APIs key by logging into your 

 These keys can manage your entire account. For example, they can mint new keys, set quotas on keys, delete and promote keys, and view usage. Private keys can additionally view requests and results from all keys, as well as using the inference APIs. Only use these keys 

 Never deploy a private key to your front end code.

 These are a special type of private key which cannot be delted. 

. This key exists to ensure that your account always has an API key.

 Public keys can only execute inference-based requests, and query their own requests (during 

Sign-Speak has several features which it only offers to partners and clients on a 

 Our avatar has the ability to mimic a person's likeness dynamically.

 When streaming mode is enabled and using websockets, our model will produce

When using zero-input mode, our models do not require the user to press a “start” or “stop” button. Rather, they can continuously sign into the model and it will predict.

Fine-tune our general models to match your use-case. Note that this is unnessasary in most everyday use cases, but can be helpful if you have unique terminology.

Allow your users to set the avatar's signing style between SEE, PSE, ASL, Classifiers, and more!

 Public keys can only execute inference-base

d requests, and query their own requests (during 

We understand that you may need to test your integration without prior ASL knowledge. To help you get started quickly, we've provided some sample videos available in the “Sample Sentences” section below.

<p class="slate-p " >This section documents our RESTful and Websockets API. Note that there are some undocumentd features (in particular surrounding the “Limited offerings” listed above.</p><p class="slate-p " ></p>

This section documents our RESTful and Websockets API. Note that there are some undocumentd features (in particular surrounding the “Limited offerings” listed above.

<p class="slate-p slate-indent-0" style="0px">The American Sign Language (ASL) Recognition section of the Sign-Speak API allows users to submit videos for ASL recognition. For best results</p><ol class="slate-ol " ><li class="slate-li " ><div style="position:relative"><strong class="slate-bold">Use conversational ASL: </strong>Frequently, learning (or ASL instructional) videos employ slow signing or not frequently used conversational constructions. We recommend using conversational ASL signed at a <strong class="slate-bold">normal speed. </strong>Our models have been trained on thousands of Deaf signers, and thus work best on those who&apos;s primary language is ASL. Our models operate best when individual&apos;s use their normal signing style rather than trying to sign slow or “code switch” (in fact, signing slowly, or trying to “over-enunciate signs” decreases accuracy).</div></li><li class="slate-li " ><div style="position:relative"><strong class="slate-bold">Proper positioning: </strong>Try to position your users as shown below. For best accuracy, have users sign at about this distance, and use a landscape video. Our models can ingest all types of videos (including one-handed signing on vertical videos!), but these do the best.</div></li><li class="slate-li " ><div style="position:relative"><strong class="slate-bold">Start Small: </strong>Start with one to two sentences before graduating up to more complex samples. This will ensure that your setup is good before moving to more complex input.</div></li><li class="slate-li " ><div style="position:relative"><strong class="slate-bold">Use Standard Signs: </strong>Our model does not currently work well with personal sign names. Our model has, however, been trained on a wide variety of dialects and regional signs across America.</div></li></ol><div style="text-align:left"><img src="https://theneo-prod-public.s3.us-east-1.amazonaws.com/0deb6287-e534-4dc0-8759-f0db0cbe8fcb.png" style="
              width:967px;
	            height:649px;
	            aspect-ratio:967 / 649"
         /></div><p class="slate-p " >This guidance extends to all sign language recognition use.</p><p class="slate-p " ><strong class="slate-bold">Available Models:</strong></p><p class="slate-p " >We currently provide models of varying complexity of models. The modeling version follows SLR.[model version].[model size]. These models include SLR.2.xs, SLR.2.sm, SLR.2.md, and SLR.3.lg. Note that SLR.2.md and SLR.3.lg are currently only offered to select customers.</p><p class="slate-p " ><strong class="slate-bold">Additional Pointers and notes:</strong></p><ol class="slate-ol " ><li class="slate-li " ><div style="position:relative">For the RESTful API, the video MUST be base64encoded. We do not currently support uploading links or such. Our websocket API allows for raw binary.</div></li><li class="slate-li " ><div style="position:relative">Our model can be configured to output <em class="slate-italic">caption regions </em>(this is similar to the concept of a single subtitle within an SRT file)<em class="slate-italic">. </em>These correspond to each individual sentence the user signed, and presents a start and stop timecode, a prediction, and a log-confidence.</div></li><li class="slate-li " ><div style="position:relative">Our models output confidence scores in a log base-e format. To recover the 0-1 log, simply compute <code class="slate-code">EXP(confidence)</code>.</div></li><li class="slate-li " ><div style="position:relative">You can provide feedback to our models to improve (or alternatively personalize your models depending on what featureset you have enabled).</div></li></ol><p class="slate-p " ></p>

The American Sign Language (ASL) Recognition section of the Sign-Speak API allows users to submit videos for ASL recognition. For best results

This guidance extends to all sign language recognition use.

We currently provide models of varying complexity of models. The modeling version follows SLR.[model version].[model size]. These models include SLR.2.xs, SLR.2.sm, SLR.2.md, and SLR.3.lg. Note that SLR.2.md and SLR.3.lg are currently only offered to select customers.

The base64 encoded video to be recognized.

payload

Enabling this flag enables 'single_recognition_mode', which is suitable when the video contains only one to two signed statements (e.g., for command driven applications).

single_recognition_mode

Specifies the request class as either BATCH or BLOCKING. BLOCKING requests wait until the recognition is completed before returning a result. BATCH requests return immediately with a request_id that can be used to check the recognition status. However, BLOCKING requests may be downgraded to BATCH if they take too long. 

request_class

The specific model to use for ASL recognition. If you are in SoleTenancy and have a fine-tuned model, you can use the corresponding ID.

model_version

This can be used to lightly hint our model with additional contextual information. Future model updates will improve the steerability of our models.

hint

X-api-key

An ID that you can pass into the feedback system for Human in the loop Learning

feedback_id

The recognized English text generated by the ASL recognition system.

prediction

batch_id

BATCH Request Submitted (or batch-downgrade has occured)

Request was malformed. For example, you tried to access a model you didn't have access to. See the response for details

Unauthorized (typically means no API key, or an invalid API key was provided)

Quota Reached, or Rate-Limit reached. If this is a rate-limit issue, you will also get a Retry-After in the header.

Internal Sign-Speak error. Provide your API key and request to Sign-Speak to debug your failure. Note that no debugging can occur if your retention policy is configured as RETAIN_NONE.

<p class="slate-p " >English Speech Recognition is a section of the Sign-Speak API that allows users to perform speech recognition on English audio files. Note that our speech recognition models have been tuned to be used in real-world scenerios. Currently, our models filter for near-filed voices specifically (so that they do not capture background noise).</p>

English Speech Recognition is a section of the Sign-Speak API that allows users to perform speech recognition on English audio files. Note that our speech recognition models have been tuned to be used in real-world scenerios. Currently, our models filter for near-filed voices specifically (so that they do not capture background noise).

The payload parameter should contain a base64 encoded audio file that needs to be recognized.

Enabling the single_recognition_mode flag indicates that the file should not be automatically segmented before recognition. This mode is suitable for scenarios where only one to two statements are being spoken, such as a voice assistant program.

The hint parameter is optional and can be used to provide additional information or context to the recognition model. This parameter is entirely optional but can help improve recognition accuracy.

The request_class parameter specifies the type of request to be made. It can have two possible values: BATCH or BLOCKING.BATCH requests return immediately with a request_id that can be used to check on the recognition status later. BLOCKING requests wait until the recognition is completed to return a result, but they may experience a batch-downgrade if they take too long.

The model parameter specifies the specific model to be used for recognition. For English recognition, only the MAIN model is supported.

model

The prediction parameter contains the recognized English text from the audio or video input.

Request was malformed. See the response for details

Internal Sign-Speak error. A request_id will also be provided to you, which you can give to Sign-Speak to debug your failure. Note that no debugging can occur if your retention policy is configured as RETAIN_NONE.

<p class="slate-p " >Retrieves The Result Of A Batch Recognition Request For English Speech Using The Provided Id.</p><p class="slate-p " ></p><div class="slate-div"></div><p class="slate-p " ></p><p class="slate-p " ></p><div class="slate-div"></div><p class="slate-p " ></p><p class="slate-p " ></p>

Retrieves The Result Of A Batch Recognition Request For English Speech Using The Provided Id.

Retrieve Batch Result

The identifier associated with the feedback provided for the RecognitionRegion in the English speech recognition process.

The recognized English text from the speech input.

The BATCH Recognition Request for English Speech has been completed and the result is available.

Unauthorized (typically means no API key, or an invalid API key was provided).

Forbidden. This API key does not have authorization to access this request. Note that this can also occur if a request was generated with a different Public API key (all Public API keys only have access to their own requests, but Private API keys have access to all requests)

The BATCH Recognition Request with the given ID was not found.

The rate limit has been exceeded. A Retry-After will be provided in the header.

An internal error occurred while processing the request. A request_id will be provided for debugging purposes.

<p class="slate-p " >The Delete Batch Result section allows users to remove the result of a batch recognition request for English speech using a provided ID.</p><p class="slate-p " ></p><div class="slate-div"></div><p class="slate-p " ></p><p class="slate-p " ></p><div class="slate-div"></div><p class="slate-p " ></p><p class="slate-p " ></p>

The Delete Batch Result section allows users to remove the result of a batch recognition request for English speech using a provided ID.

Delete Batch Result

The id path parameter specifies the ID of the BATCH Recognition Request for English Speech that needs to be deleted. This ID uniquely identifies the specific batch result to be removed from the system.

The BATCH Recognition Request for English Speech result has been successfully deleted.

The BATCH Recognition Request with the given ID was not found or the result has already been deleted.

An internal error occurred while attempting to delete the request result. A request_id will be provided for debugging purposes.

<p class="slate-p " >Our ASL avatars look just like a human (see below). You can use these avatars to sign a message in ASL. These models have been widely rated by the Deaf community as accurate. We recommend you start with our MALE avatar, and primarily use grammatically correct sentences. Note that as well as raw text, you can also input time-coded text (e.g. from an SRT) if you&apos;d like to align the signing with some external source (such as a video). Make sure you <strong class="slate-bold">only either provide English or the Timestamped English.</strong></p><p class="slate-p " >Additionally, note that we have different avatar models providing different quality of signs. The modeling naming follows SLP.[model version].[model size]. These models include SLP.2.xs, SLP.2.sm, SLP.2.md, and SLP.3.lg. Note that SLP.2.md and SLP.3.lg are currently only available to select partners due to capacity.</p><div style="text-align:center"><iframe
                        title="embed"
                        frameBorder="0"
                        allowFullScreen=""
                        src="https://www.youtube.com/embed/5_XcZeF94ZY"
                        id="5_XcZeF94ZY"
                        provider="youtube"
                        class="slate-MediaEmbedElement-iframe"
                        width="100%"
                    ></iframe></div><p class="slate-p " ></p>

Our ASL avatars look just like a human (see below). You can use these avatars to sign a message in ASL. These models have been widely rated by the Deaf community as accurate. We recommend you start with our MALE avatar, and primarily use grammatically correct sentences. Note that as well as raw text, you can also input time-coded text (e.g. from an SRT) if you'd like to align the signing with some external source (such as a video). Make sure you 

only either provide English or the Timestamped English.

Additionally, note that we have different avatar models providing different quality of signs. The modeling naming follows SLP.[model version].[model size]. These models include SLP.2.xs, SLP.2.sm, SLP.2.md, and SLP.3.lg. Note that SLP.2.md and SLP.3.lg are currently only available to select partners due to capacity.

The english parameter represents the text or words that need to be translated into American Sign Language (ASL). It can be provided as a string value. Provide this OR the timestamped_english

english

The request_class parameter specifies the type of request to be made. It can be set as either BATCH or BLOCKING.For BATCH requests, the API will return immediately with a request_id that can be used to check the status of the recognition later. These requests are suitable for scenarios where immediate response is not required.On the other hand, BLOCKING requests will wait until the recognition process is completed before returning a result. However, if BLOCKING requests take too long, they may be downgraded to BATCH requests. BLOCKING requests are useful when real-time response is necessary.

identity

timestamped_english

The Authorization header parameter is used to provide credentials for authentication. It should contain the authentication token or API key required to access the ASL Production endpoint.

BLOCKING Request Completed. This is just mp4 bytes of the generated video.

<p class="slate-p " >The English Speech Generation section of the Sign-Speak API enables users to convert text into spoken English. By utilizing this functionality, users can generate MP3 audio files of the spoken text.  We use neural voices to sound lifelike. Note that you can only either provide the english or the timestampped english.</p>

The English Speech Generation section of the Sign-Speak API enables users to convert text into spoken English. By utilizing this functionality, users can generate MP3 audio files of the spoken text.  We use neural voices to sound lifelike. Note that you can only either provide the english or the timestampped english.

The Authorization header parameter is used for authentication and authorization purposes. It typically contains a token or credentials to verify the authenticity and permissions of the request.

File

<p class="slate-p " >Performing WebSockets based Sign Language Recognition involves several steps. We recommend using WebSockets within a Javascript context as illustrated below:</p><ol class="slate-ol " ><li class="slate-li " ><div style="position:relative">Start a video stream.</div></li></ol><div class="code-block-wrapper"  style="width:100%;min-width:100%" ><div class="code-block-header">JavaScript</div><pre class="slate-CodeBlockElement slate-code_block"><textarea id="publishedPageCodeblock" style="display:none !important; visibility: hidden; opacity: 0; width: 1px; height: 1px;" type="hidden"> </textarea><code class="language-javascript keep-markup drop-tokens"><div class="slate-code_line"><div class="slate-code_line">let options: MediaRecorderOptions;
let isChromeBrowser = navigator.userAgent.includes(&apos;chrome&apos;) &amp;&amp; !navigator.userAgent.includes(&apos;edge&apos;) &amp;&amp; !navigator.userAgent.includes(&apos;opr&apos;) &amp;&amp; !navigator.userAgent.includes(&apos;crios&apos;);
if (isChromeBrowser) {
    options = { bitsPerSecond: 1000000, mimeType: &quot;video/webm;codecs=VP9&quot; };
} else {
    options = { bitsPerSecond: 1000000 };</div><div class="slate-code_line">}</div><div class="slate-code_line"><div class="slate-code_line">let mediaRecorder = new MediaRecorder(mediaStream, options);</div></div></div></code></pre></div><p class="slate-p " >2. Initiate a socket connection</p><div class="code-block-wrapper"  style="width:100%;min-width:100%" ><div class="code-block-header">JavaScript</div><pre class="slate-CodeBlockElement slate-code_block"><textarea id="publishedPageCodeblock" style="display:none !important; visibility: hidden; opacity: 0; width: 1px; height: 1px;" type="hidden"> </textarea><code class="language-javascript keep-markup drop-tokens"><div class="slate-code_line">let configPacket = {</div><div class="slate-code_line">    api_key: “API_KEY”,</div><div class="slate-code_line">    slice_length: 500, // milliseconds</div><div class="slate-code_line">    single_recognition_mode: false,</div><div class="slate-code_line">    … // all other configuration details you would like to specify</div><div class="slate-code_line">}</div><div class="slate-code_line">socket = new WebSocket(&quot;wss://api.sign-speak.com/stream-recognize-sign&quot;);</div><div class="slate-code_line">await new Promise((resolve, reject) =&gt; {</div><div class="slate-code_line">    socket.addEventListener(&quot;open&quot;, () =&gt; {</div><div class="slate-code_line">        socket.send(JSON.stringify(configPacket));</div><div class="slate-code_line">        resolve(null)</div><div class="slate-code_line">    })</div><div class="slate-code_line">})</div></code></pre></div><p class="slate-p " >3. Setup the socket Listener</p><div class="code-block-wrapper"  style="width:100%;min-width:100%" ><div class="code-block-header">JavaScript</div><pre class="slate-CodeBlockElement slate-code_block"><textarea id="publishedPageCodeblock" style="display:none !important; visibility: hidden; opacity: 0; width: 1px; height: 1px;" type="hidden"> </textarea><code class="language-javascript keep-markup drop-tokens"><div class="slate-code_line">socket.addEventListener(&quot;message&quot;, (event) =&gt; {</div><div class="slate-code_line">    // Parse JSON</div><div class="slate-code_line">    let data;</div><div class="slate-code_line">    try {</div><div class="slate-code_line">        data = JSON.parse(event.data);</div><div class="slate-code_line">    } catch (err) {</div><div class="slate-code_line">        console.warn(&quot;Received non-JSON from recognition socket:&quot;, event.data);</div><div class="slate-code_line">        return;</div><div class="slate-code_line">    }</div><div class="slate-code_line">    if (data === &quot;OUT_OF_QUOTA&quot;) {</div><div class="slate-code_line">        // this means you have no more quota</div><div class="slate-code_line">    }</div><div class="slate-code_line">    for (const pred of data.prediction || []) {
        const { finished, prediction, confidence, feedback_id } = pred;</div><div class="slate-code_line">    }</div><div class="slate-code_line">})</div><div class="slate-code_line">        </div></code></pre></div><p class="slate-p " >4. Send video data (ideally as it is being captured)</p><div class="code-block-wrapper"  style="width:100%;min-width:100%" ><div class="code-block-header">JavaScript</div><pre class="slate-CodeBlockElement slate-code_block"><textarea id="publishedPageCodeblock" style="display:none !important; visibility: hidden; opacity: 0; width: 1px; height: 1px;" type="hidden"> </textarea><code class="language-javascript keep-markup drop-tokens"><div class="slate-code_line"><div class="slate-code_line">mediaRecorder.ondataavailable = (e) =&gt; {
    const blob = new Blob([e.data], { type: &quot;video/mp4&quot; });
    let reader = new FileReader();
    reader.readAsArrayBuffer(blob);
    reader.onloadend = () =&gt; {
        socket.send(reader.result as ArrayBuffer);
    }
};</div><div class="slate-code_line">mediaRecorder.start(500); // this 500 must match the slice_length</div></div></code></pre></div><p class="slate-p " >5. When the user presses the stop button, send the done signal</p><div class="code-block-wrapper"  style="width:100%;min-width:100%" ><div class="code-block-header">Plain text</div><pre class="slate-CodeBlockElement slate-code_block"><textarea id="publishedPageCodeblock" style="display:none !important; visibility: hidden; opacity: 0; width: 1px; height: 1px;" type="hidden">socket.send("NEXT");
// or
socket.send("DONE") </textarea><code class="language-plainText keep-markup drop-tokens"></code></pre></div><p class="slate-p " >You can send <code class="slate-code">DONE</code> if you&apos;d like to close the websocket after receiving the output. If you send <code class="slate-code">NEXT</code>, you will be able to start sending another set of inputs corresponding to another session to the same websocket.</p>

Performing WebSockets based Sign Language Recognition involves several steps. We recommend using WebSockets within a Javascript context as illustrated below:

4. Send video data (ideally as it is being captured)

5. When the user presses the stop button, send the done signal

 if you'd like to close the websocket after receiving the output. If you send 

, you will be able to start sending another set of inputs corresponding to another session to the same websocket.

<p class="slate-p " >The Feedback section of the Sign-Speak API allows users to provide feedback for a prediction made by the API. With various feedback mechanisms available, users have the flexibility to provide feedback on specific parts or the entire recognition. This section enables users to improve the accuracy and performance of the Sign-Speak API based on their input. Different feedback mechanisms are available.</p><p class="slate-p " ></p><div class="slate-div"></div><p class="slate-p " ></p><p class="slate-p " ></p><div class="slate-div"></div><p class="slate-p " ></p><p class="slate-p " ></p>

The Feedback section of the Sign-Speak API allows users to provide feedback for a prediction made by the API. With various feedback mechanisms available, users have the flexibility to provide feedback on specific parts or the entire recognition. This section enables users to improve the accuracy and performance of the Sign-Speak API based on their input. Different feedback mechanisms are available.

The rating signifies if a given translation or recognition was accurate. This is used via RLHF to continually improve our models

rating

The correction body parameter is used to provide any corrections or amendments related to the feedback. It allows users to submit additional information or clarify any inaccuracies in the original feedback. This parameter should be included in the request body.

correction

The Authorization header parameter is used to authenticate the request. It should contain the authorization token or credentials required for the user to access the feedback resource.

The id path parameter represents the unique identifier of the feedback. It is a required parameter that specifies which feedback resource the request is targeting. The value for this parameter should be provided in the endpoint URL.

The feedback has been successfully submitted

Unauthorized. API key is missing or invalid.

Forbidden. This API key did not generate this request.

<p class="slate-p " >The Private API Requests section of the Sign-Speak API enables users to securely authenticate their requests and access protected resources. By generating a private API key, users can ensure the confidentiality and integrity of their data while interacting with the API.</p><p class="slate-p " ></p><p class="slate-p " ></p>

The Private API Requests section of the Sign-Speak API enables users to securely authenticate their requests and access protected resources. By generating a private API key, users can ensure the confidentiality and integrity of their data while interacting with the API.

<p class="slate-p " >The Sign-Speak JavaScript SDK enables seamless integration of advanced American Sign Language (ASL) and speech recognition technologies into your applications, promoting inclusive user experiences.</p><p class="slate-p " >Install the SDK using NPM or Yarn</p><div class="code-block-wrapper"  style="width:100%;min-width:100%" ><div class="code-block-header">Plain text</div><pre class="slate-CodeBlockElement slate-code_block"><textarea id="publishedPageCodeblock" style="display:none !important; visibility: hidden; opacity: 0; width: 1px; height: 1px;" type="hidden">npm install 
@sign-speak/react-sdk
# -or-
yarn add @sign-speak/react-sdk </textarea><code class="language-plainText keep-markup drop-tokens"></code></pre></div><p class="slate-p " >Then, set your API key, either by setting the env variable SIGN_SPEAK_API_KEY or via javascript</p><div class="code-block-wrapper"  style="width:100%;min-width:100%" ><div class="code-block-header">Plain text</div><pre class="slate-CodeBlockElement slate-code_block"><textarea id="publishedPageCodeblock" style="display:none !important; visibility: hidden; opacity: 0; width: 1px; height: 1px;" type="hidden">import { setKey } from'sign-speak-sdk/network/key';
// Set the API Key
setKey('YOUR_API_KEY'); </textarea><code class="language-plainText keep-markup drop-tokens"></code></pre></div><p class="slate-p " >You can view more information on the SDK at <a href="https://www.npmjs.com/package/@sign-speak/react-sdk" target="">NPM</a>, so here, we will only briefly go through the various aspects of the SDK. Our SDK exposes three levels of access:</p><ul class="slate-ul " ><li class="slate-li " ><div style="position:relative"><strong class="slate-bold">Network Calls </strong>- You can use the network calls component of the SDK to directly invoke various functionality of the API. Use this when you want the most control over the functionality. Example:</div></li></ul><div class="code-block-wrapper"  style="width:100%;min-width:100%" ><div class="code-block-header">Plain text</div><pre class="slate-CodeBlockElement slate-code_block"><textarea id="publishedPageCodeblock" style="display:none !important; visibility: hidden; opacity: 0; width: 1px; height: 1px;" type="hidden">import { produceSign } from'sign-speak-sdk/network/rest';
const response = await produceSign({
  english: "Welcome to Sign-Speak!"
}, {,
  model: "MALE",
  request_class: "BATCH"
}); </textarea><code class="language-plainText keep-markup drop-tokens"></code></pre></div><ul class="slate-ul " ><li class="slate-li " ><div style="position:relative"><strong class="slate-bold">React Hooks </strong>- You can use the react hooks in a large number of use cases when you want to manage the display of the components, but want to delegate the actual state management to the SDK.</div></li></ul><div class="code-block-wrapper"  style="width:100%;min-width:100%" ><div class="code-block-header">Plain text</div><pre class="slate-CodeBlockElement slate-code_block"><textarea id="publishedPageCodeblock" style="display:none !important; visibility: hidden; opacity: 0; width: 1px; height: 1px;" type="hidden">import React, { useEffect } from 'react';
import { useSignLanguageRecognition } from 'sign-speak-sdk';
function ASLHookExample() {
  const { prediction, startRecognition, stopRecognition } = useSignLanguageRecognition();
  useEffect(() => {
    startRecognition();
    return () => stopRecognition();
  }, []);
  return <p>ASL Output: {prediction}</p>;
} </textarea><code class="language-plainText keep-markup drop-tokens"></code></pre></div><ul class="slate-ul " ><li class="slate-li " ><div style="position:relative"><strong class="slate-bold">Component Library - </strong>The component library allows for one-line integration. Our component library follows the model of ShadCN, in that we recommend you to copy the actual component source code into your project and use it as a starting point. You do have limited customization options within the actual component library if you wish</div></li></ul><div class="code-block-wrapper"  style="width:100%;min-width:100%" ><div class="code-block-header">Plain text</div><pre class="slate-CodeBlockElement slate-code_block"><textarea id="publishedPageCodeblock" style="display:none !important; visibility: hidden; opacity: 0; width: 1px; height: 1px;" type="hidden">import { SignRecognition } from 'sign-speak-sdk';
const MyASLRecognition = () => (
  <div>
    <h2>ASL Recognition</h2>
    <SignRecognition />
  </div>
); </textarea><code class="language-plainText keep-markup drop-tokens"></code></pre></div><p class="slate-p " >You can easily mix-and-match these to get your desired level of customization. As a quick example, below is an example of how to use the Sign Language Recognition and Sign Language Production components:</p><p class="slate-p " ></p><div class="code-block-wrapper"  style="width:100%;min-width:100%" ><div class="code-block-header">Plain text</div><pre class="slate-CodeBlockElement slate-code_block"><textarea id="publishedPageCodeblock" style="display:none !important; visibility: hidden; opacity: 0; width: 1px; height: 1px;" type="hidden">import { useEffect, useState } from "react";
import { RecognitionResult, setKey, SignProduction, SignRecognition, SpeechProduction } from "@sign-speak/react-sdk"
export default function Demo({
    apiKey
}: {
    apiKey: string
}) {
    const [directionToASL, setDirectionToASL] = useState(false)
    const [engText, setEngText] = useState("Nice to meet you.")
    const [submittedEngText, setSubmittedEngText] = useState("Nice to meet you.")
    const [recognizedASL, setRecognizedASL] = useState("")
    const [show, setShow] = useState(false)
    useEffect(() => {
        setKey(apiKey)
        setShow(true)
    }, [apiKey])
    const parseSLRResult = (result: RecognitionResult | null) => {
        if (result) {
            setRecognizedASL(result.prediction.filter(x => x.confidence > Math.log(0.5)).map(x => x.prediction).join(" "))
        }
    }
    if (!show) {
        return null
    }
    return <div className="m-5 w-full">
      <div className="flex flex-row items-center">
        <h1 className="font-semibold text-2xl mr-4">Translation Demo</h1>
        <button onClick={() => setDirectionToASL(!directionToASL)} className="btn bg-base-100">{
          directionToASL ? <>
            English to ASL
          </> : <>
            ASL to English
          </>
        } </button>
      </div>
      <div className="flex flex-row justify-center gap-8 w-full mt-4">
        {
          directionToASL ? <>
            <div className="card flex-1">
              <div className="card bg-base-100 w-full p-6">
                <h1 className="font-semibold">English</h1>
                <textarea
                  value={engText}
                  onChange={e => setEngText(e.target.value)}
                  placeholder="Enter Text in English"
                  className="bg-base-200 rounded-lg mt-4 p-2 textarea resize-none"
                />
                <div>
                  <button
                    onClick={() => setSubmittedEngText(engText)}
                    className="btn btn-primary mt-2"
                  >
                    Translate
                  </button>
                </div>
              </div>
            </div>
            <div className="card flex-1">
              <div className="card bg-base-100 p-6 w-full">
                <h1 className="font-semibold mb-4">ASL Translation</h1>
                <SignProduction
                  key="production"
                  videoContainerClassName="w-full relative aspect-video"
                  loadingClassName="w-full h-full absolute left-1/2 -translate-x-1/2 top-0 skeleton"
                  videoClassName="w-full h-full"
                  text={submittedEngText}
                />
              </div>
            </div>
          </> : <>
            <div className="card flex-1">
              <div className="card bg-base-100 w-full p-6">
                <h1 className="font-semibold mb-2">ASL</h1>
                <SignRecognition interpretationClassName="hidden" containerClassName="relative" loadingClassName="absolute w-full h-full absolute top-0 skeleton !rounded-none" buttonClassName={"btn btn-primary"} onResult={parseSLRResult} />
              </div>
            </div>
            <div className="card flex-1">
              <div className="card bg-base-100 p-6 w-full">
                <h1 className="font-semibold mb-4">English Translation</h1>
                <SpeechProduction text={recognizedASL} />
                <p>{recognizedASL}</p>
              </div>
            </div>
          </>
        }
    </div>
  </div>
} </textarea><code class="language-plainText keep-markup drop-tokens"></code></pre></div><p class="slate-p " ></p>

The Sign-Speak JavaScript SDK enables seamless integration of advanced American Sign Language (ASL) and speech recognition technologies into your applications, promoting inclusive user experiences.

Then, set your API key, either by setting the env variable SIGN_SPEAK_API_KEY or via javascript

You can view more information on the SDK at 

, so here, we will only briefly go through the various aspects of the SDK. Our SDK exposes three levels of access:

You can easily mix-and-match these to get your desired level of customization. As a quick example, below is an example of how to use the Sign Language Recognition and Sign Language Production components:

<p class="slate-p " >We&apos;ve provided sample sentences, including easy, medium, and advanced examples, for your reference:</p><p class="slate-p " ><a href="https://drive.google.com/drive/folders/17_FPHjlYEye-0pmKA9HebckynAaXhCyj?usp=sharing" target="_self">https://drive.google.com/drive/folders/17_FPHjlYEye-0pmKA9HebckynAaXhCyj?usp=sharing</a></p><p class="slate-p " ></p>

We've provided sample sentences, including easy, medium, and advanced examples, for your reference:

Sections