Saymotion REST API

Revisions

Beta v1.0.0

Initial rest APIs

Beta v1.1.0

/account/creditBalance (introducing feature limits)

/job/process (changes due to variant & inpainting generation)

/job/list (changes due to variant & inpainting generation and also multi mp4 downloads)

/job/download (changes due to variant & inpainting generation)

Beta v1.1.1

/job/process (Added skipFBX parameter)

Beta v1.2.0

/job/process (Added merging parameter)

The SayMotion REST API lets you convert text prompts into 3D animations without having to use the Saymotion Web Portal. It can be used from web, mobile or desktop apps.

Authentication

The SayMotion REST API uses basic HTTP Authentication to keep your requests and data secure. To use the API you will need a Client ID and a **Client Secret **which are provided by DeepMotion. If you do not have these please contact DeepMotion Support or your sales representative.

To retrieve your API access token you need to add the following Authorization header to your token request:

Authorization: Basic Base64(<clientId>:<clientSecret>)

where the value of <clientId>:<clientSecret> is base 64 encoded. For Example, if your Client ID is 1a2b and your client Secret is 3c4d then your authorization header should look like this:

Authorization: Basic MWEyYjozYzRk

where MWEyYjozYzRk is the base64 encoded value of 1a2b:3c4d.

API Endpoints

All SayMotion API requests must be made against the following base URL using the HTTPS protocol and port:

Production Environment: 	(Contact DeepMotion)

For using our API from browser javascript locally (to avoid CORS error), please send request from the origin below:

http://localhost

For production deployment, please let us know your production url (scheme, host, port), so that we can configure our CORS setting accordingly.

API Reference

Account APIs

API 1: Authentication & getting Access Token

Desc	Authenticate client credentials and returns a time limited session cookie to be used in the subsequent REST API calls. After the session expires, this API needs to be called again to get a new session cookie. Please use this api from the server side to stay secure for web applications and you can use the returned session cookie in other API calls even from the client side.
Method + URI	GET {host}/account/v1/auth
Header(s)	Authorization: Basic Base64(<clientId>:<clientSecret>)
Request
Response	Sample Response Header: set-cookie: dmsess=s%3AEsF23MoyDEq7tTWQM8KfA_wjKkSrOFwU.2fjJTfDP%2FT2BeA5DFenwOH4t8XzqZsbSc6M2mZwS%2BWg; Domain=.deepmotion.com; Path=/; Expires=Mon, 03 Aug 2020 13:36:26 GMT; HttpOnly (Note: dmsess is the session cookie. This cookie needs to be sent in all subsequent REST API calls. Sample Request Header for other API calls: cookie:dmsess=s%3AEsF23MoyDEq7tTWQM8KfA_wjKkSrOFwU.2fjJTfDP%2FT2BeA5DFenwOH4t8XzqZsbSc6M2mZwS%2BWg)

API 2: Credit Balance

Desc	Retrieves Credit Balance for an user
Method + URI	GET {host}/account/v1/creditBalance
Header(s)	cookie:dmsess=<cookie-value-returned-from-authentication-api>
Request	n/a
Response	JSON object: {"credits":<value>,"subscription":{"name":<value>,"credits":<value>,"featureLimits":{"maxVariantsGeneration":<value>},"currentPeriod":{"start":<value>,"end":<value>}}}

Job APIs

API 1: Start Processing

Desc	Start processing text to motion OR rendering a previously generated animation
Method + URI	POST {host}/job/v1/process/processor
Header(s)	cookie:dmsess=<cookie-value-returned-from-authentication-api>
Request	POST body should include a JSON object: { “params": [<params>, ...] }. <processor> specifies which processor to use to process the job, must be one of the following: Processor Id Description text2motion textPrompt to animation generator render Renders an animation to video <params> specifies additional parameters that will be passed to the specified processor. For the **text2motion **processor, here are the parameters for a regular job. "params": [ “prompt=<value>”, "model=<value>”, “dis=<value>”, “footLockingMode=<value>”, “poseFilteringStrength=<value>”, “rootAtOrigin=<value>”, “skipFBX=<value>” ] And here are the common parameters for an inpainting or merging job ‘t2m_rid=<value>’ ‘variant_id=<value>’ ‘editRequest={"numTrimLeft" : <value>, "numTrimRight" : <value>}’ Inpainting exclusive parameter: ‘inPaintingRequest={ “prompt” :<value>, “intervals” : [ { “start” :<value>, “end”:<value> } ] }’ Merging exclusive parameter:: ‘mergingRequest={ “t2m_rid”:<value>, “variant_id”:<value>, “editRequest”:{"numTrimLeft" : <value>, "numTrimRight" : <value>}, “prompt”: <value>, “blendDuration”: <value> }’ prompt A detailed text prompt to generate motion with. For inpainting or merging jobs, the prompt parameter should be included inside the inPaintingRequest or mergingRequest parameter, instead of as a standalone parameter in a regular job. model The 3d model used for showcasing the generated motion/animation dis (optional) This parameter influences motion generation to improve it in some cases like inter body parts penetration etc, but may cause side effects in animation quality sometimes. By default simulation is turned on. Use dis=s to turn off the simulation. footLockingMode (optional) This parameter value can be one of the below: auto : default mode, automatic switching between locking and gliding modes of the foot, recommended for general cases always : forced foot locking all the time. only used when Auto mode can not remove all the foot gliding unsired never : forced to disable foot locking and character grounding. used when the motion is completely in the air or in the water and therefore neither foot locking nor character grounding is needed. grounding : forced disabling foot locking, however character is still grounded. Only used when Auto mode prevents the desired foot gliding (i.e. during shuffling dances) in the motion or locks the foot for too long on the ground during fast and short foot/ground contacts (i.e. during sprints or jumps.) poseFilteringStrength (optional) Applies an advanced AI filter that helps remove jitter and produce smoother animations though may result in lower animation accuracy for certain frames or sequences Default value is 0.0 and range is 0.0 - 1.0 rootAtOrigin (optional) Place a root joint at the origin of the output character. This is helpful in some cases, for example, for UE4 retargeting. Default value is 0 and value can be either 0 or 1 skipFBX (optional) Skips FBX format generation. Set skipFBX=1 t2m_rid It is a previous text2motion job request id from which to generate inpainting/merging jobs variant_id It is a specific variant animation id from the above previously generated text2motion job **blendDuration : **duration of the merge prompt generated frames editRequest A json string, containing simple edit requests to the inpainting/merging region such as trimming it from left (integer value) or right (integer value). The values represent frame counts and are always positive. inPaintingRequest A json string, containing inpainting prompt and intervals in frame numbers. mergingRequest A json string, contains merging related information such as the id & variant id of the to be merged animation job. For the **render **processor, here are the parameters. Only two required parameters are t2m_rid which is a previous text2motion job request id & variant_id which is a specific variant animation from the above previously generated text2motion job. Other optional parameters are below: *To replace the default background with a solid color (for green screening etc.) bgColor=0,177,64,0 (RGBA color code in the range of 0-255 for each channel, please note, the last channel (alpha) value is not in effect ) *To set a studio like 3d background with a solid color tint backdrop=studio **bgColor=0,177,64,0 ** Also, List of supported 2D backdrops: Checker_Cloudscape Golden_Age_Glitz Inferno_Stage Jungle_Grove Mystic_Brick Mystic_Concrete Rainbow_Spotlight Red_Carpet Retro_Revue Sakura_Dreamscape Spotlit_Stage Timbered_Retreat Urban_Rooftop *To enable character shadow **shadow=1 ** (not applicable for 2D backdrop) *camMode values are below. Default is 0 0 (Cinematic) The character is kept in the center of the frame 1 (Fixed) Camera will stay fixed relative to the background 2 (Face) Camera keeps the torso and face in the center of frame *camHorizontalAngle Camera horizontal angle in degrees, where zero means forward camera Default value is 0 and range is -90 - +90
Response	JSON object: { "rid": <request id> }

API 2: Poll for Job Status

Desc	Polls for real-time status of a given processing job
Method + URI	GET {host}/job/v1/status/rid GET {host}/job/v1/status/rid1,rid2,..,rid
Header(s)	cookie:dmsess=<cookie-value-returned-from-authentication-api>
Request	Clients can request the current status of previously submitted processing requests (API3). Use comma (‘,’) to separate multiple request ids if retrieving status for more than 1 request.
Response	JSON object: { "count": <number of records in status array>, "status": [ <status>, … … ] } Each element in status array is a JSON object: { "rid": <request id>, "status": <status name>, "details": <status details, see below>, “positionInQueue”: <position in the queue for only PROGRESS status> } <status name> is one of the following case sensitive values: Status Name Description PROGRESS Request is still being processed SUCCESS Request is processed successfully FAILURE Request has failed <status details> for PROGRESS: { “step”: <current step>, “total”: <expected total number of steps> } <status details> for SUCCESS: { “In”: <original video file>, “out”: <processed video file> } <status details> for RETRY and FAILURE include last error message. Currently the format is: { “exc_message”: <exception message, if any>, “exc_type”: <exception type, if any> } But please note the format may change if we decide to mask error information (or pass more information) to client applications.

API 3: Get Download URLs

Desc	Get download URLs for the specified request ids
Method + URI	GET {host}/job/v1/download/rid?variant_id=1 GET {host}/job/v1//download/rid1,rid2,...,rid?variant_id=2,3,...,n
Header(s)	cookie:dmsess=<cookie-value-returned-from-authentication-api>
Request	Clients can request download URLs for finished processing requests. Use comma (‘,’) to separate request ids if retrieving download URLs for multiple processing requests. A query parameter called variant_id to be able to download specific variant animation related resources. Use comma (‘,’) to separate variant_id values which correspond to individual rid of multiple job requests
Response	JSON object: { “count”: <number of records in links array>, “links”: [ <link>, … ... ] } Each element in links array is a JSON object: { “rid”: <request id>, “parameters”: <input params of the job>, “renderJobList” (available only for text2motion jobs):<value> “variantDownloadStatus”: <boolean flag, false when variant id is wrong or user doesn’t has sufficient balance to download this variant data> “urls”: [ { “name”: <name of the downloadable item> “files”: <links of the files by extension> [ { <file type>: <URL to download the corresponding file>}, {<file type>: <URL to download the corresponding file>} ] } ] } Please note that if the specified request has not finished yet or has failed, the response will not include any download urls, and the link object will look like: { “rid”: “1234567890” }

API 4: List All Video Processing requests by Status & processor

Desc	List past and current request ids Note: failed jobs and old jobs may be removed by system after a predefined retention period
Method + URI	GET {host}/job/v1/list GET {host}/job/v1/list/status1,...,status/processor
Header(s)	cookie:dmsess=<cookie-value-returned-from-authentication-api>
Request	Client can request to get list of existing request ids of current user Client can specify one or multiple status value(s) along with processor id to retrieve only request ids with the same status value(s). For example, GET /list/PROGRESS/text2motion will only return list of text2motion requests that are still being processed
Response	JSON object: { "count": <number of records in the rids array>, "list": [ { "rid": <job request id>, "processingInfo":<detail timing of the processing>, "processor":<processor id>, "parameters":<job input parameters>, “variants” (available only for text2motion jobs): <{“variant_id”:{“download”:<download flag>}}> “chargedAmount”: <credits used for this job> "status": <status of the job>, "ctime": <creation time,milliseconds since epoch>, "mtime": <last modification time, milliseconds since epoch> }, ... ... ] }

API 5: Cancel progressing job

Desc	Cancel job for the specified request id
Method + URI	GET {host}/job/v1/cancel/rid
Header(s)	cookie:dmsess=<cookie-value-returned-from-authentication-api>
Request	Clients can cancel in progress request/job.
Response	JSON object: { “result”: true }

Custom Character APIs

API 1: Model Upload Url

Desc	Retrieves signed urls to upload 3d model data(fbx, glb, gltf, or vrm format) and thumbnail(preferably png format)
Method + URI	GET {host}/character/v1/getModelUploadUrl
Header(s)	cookie:dmsess=<cookie-value-returned-from-authentication-api>
Request	Query parameters: <name>: base name of the files (without extension) (optional) <modelExt>: file extension of the model file. Example: fbx (optional) <thumbExt>: file extension of the thumb file. Example: jpg (optional) <resumable>: 0 or 1(default) returns resumable or regular signed url (optional)
Response	JSON object: { “modelUrl”: signed url “thumbUrl”: signed url } After retrieving the urls, actual model & thumbnail upload are required to that storage urls. If ’resumable’ option is set in the request, we need one POST and one subsequent PUT request for each signed url, otherwise a single PUT request will do the job per url. POST request to url: <x-goog-resumable>: start (set in the request header) <location>: resumable url (set in the response header by server) PUT request to resumable url location/url: attach raw bytes of the model or thumbnail file in the request body.

API 2: Store Model

Desc	Store the asset paths returned from getModelUploadUrl in database
Method + URI	POST {host}/character/v1/storeModel
Header(s)	cookie:dmsess=<cookie-value-returned-from-authentication-api>
Request	Body parameters: <modelUrl>: model url returned from API 1 (optional if <modelId> is provided) <modelName>: model name (optional) <thumbUrl>: thumbnail url returned from API 1 (optional) <modelId>: model id to update existing model info (name or thumb) (optional if <modelUrl> is provided) <createThumb>: 0 (default) or 1, indicate if the thumbnail of the model needs to be generated (optional)
Response	JSON object: { “modelId”: <Unique model id that can be passed to text2motion process API>, “faceDataType”:<if the model supports facial rig>, “handDataType”:<if the model supports hand/finger rig> }

API 3: List Models

Desc	List models based on specific query or without
Method + URI	GET {host}/character/v1/listModels
Header(s)	cookie:dmsess=<cookie-value-returned-from-authentication-api>
Request	Query parameters: <modelId>: existing model id (optional) <searchToken>: for example search by model name (optional) <stockModel>: = When this parameter is supplied, all stock models (including deepmotion & roblox) will return in api response along with the account's custom models. Beside that, each model details now include a platform field which can be one of the below values : custom, deepmotion, roblox . (optional)
Response	JSON object: [ { “Id: Unique model id that can be passed to video process API “name”: name of the model “thumb”: url of the thumbnail if exist “glb”: url of glb format of the character “rigId”: rigTemplate id with which this model is associated with “ctime”: creation timestamp “mtime”: modification timestamp “platform”: platform of the model } ]

API 4: Delete Model

Desc	Delete model with specific model ID
Method + URI	DELETE {host}/character/v1/deleteModel/<model ID>
Header(s)	cookie:dmsess=<cookie-value-returned-from-authentication-api>
Request
Response	JSON object: { “count”: number of models that have been deleted. Currently only one model can be deleted at a time. }

Saymotion Restful API Error Codes (Updates as we add more features)

Error Code	Meaning
101	Not enough credit
498	Unknown pipeline error
494	Invalid pipeline Input
501	Error while generating motion
502	Error parsing motion generation parameters
599	Motion generation timeout
603	Error processing pose tracking parameters
604	Error loading animation data for pose tracking
605	Physics Filter is incompatible with used custom character
607	Error while processing the body tracking
610	Error saving pose tracking intermediate results
699	Pose tracking timeout
703	Error processing pose correction parameters
704	Error loading the character animation assets for pose corrections
710	Error saving pose correction intermediate results
799	Pose correction timeout
803	Error processing bvh exporter parameters
804	Error loading the character animation assets for bvh exporting
810	Error saving bvh results
899	Bvh exporting timeout
901	Error loading the mesh of the custom character
902	Error loading the BVH custom character
903	Error copying animations onto the custom character
904	Error exporting animations for the custom character
905	Custom character doesn’t include skinned mesh information
906	More than half of the required blendshapes are missing
907	Error loading facial definition for the custom character
908	Error loading facial tracking data
909	Error loading the metadata of the custom character
999	Animation baking timeout
1101	Process stuck
1102	Invalid input parameter
1105	Failed to load input character
1106	Failed to attach animation to character
1107	Failed to configure backdrop