Single-camera API overview

Our single camera API takes a video file in .mp4 format and returns animation data in the .fbx .usdc and .usdz formats.

The best quality output is obtained when the following steps are all included in the video:

1. A human is fully visible at the start of the video
2. The camera is static for the duration of the capture
3. The entire actor is completely visible in every frame of the capture
4. There are no other people visible at any time during the capture
5. The actor holds an “A pose” such as [ref fig1] for 1 second at the start of the video
6. Clothing worn by the actor has good contrast to their background
7. The actor is well lit

In theory #1 is the only hard requirement for your capture to be processed by the API. Failure to meet the other criteria will degrade the output quality to varying degrees.

Existing applications

We recommend using one of our existing application, built on the same APIs, to experiment with inputs and outputs.

Move One iOS app

Our Move One app is the best place to test the process of capturing some motion and getting output data back. It uses the same API and Swift SDK. Our knowledge base provides further details on how to get good quality single person capture outputs.

Move One Bot

For tests that use a pre-existing video, consider the Move One Bot available on our Discord server. This will take any video up to 10s in length at at least FHD resolution and return the output data in direct messages.

Authentication

All calls to our API require an API key to be provided in the Authorization header of the request. At this stage, while we are in an early access period, individual API keys will be issued via a one-time secret link. If you require multiple API keys (e.g. for development environments) please request these from your Move AI contact.

Data model

Files

All inputs and outputs from the system are files and have a guid.

Takes

A take is a video input file and the optional additional files that can be used by the system to improve the quality of the output.

Jobs

Takes that are currently processing or have been processed. The take is referenced by the job. The output files are attached to the job. The state of the job tells the user what stage of the processing cycle the job is currently in.

Giving feedback

You will be provided with an API key by a member of staff at Move AI. Please feel free to reach out to them with any questions or feature requests. We can also set up shared slack channels to provide more support to early adopters of the API.

Developer notes

Job state

The lifecycle of a job is:

• NOT_STARTED - submitted but not started
• STARTED - has been sent to a server for processing
• RUNNING - is running on the server
• FINISHED - has produced some outputs (this has no relation to the quality of the output, just that some output was generated)
• FAILED - we couldn’t process the output

Processing time

Three main factors drive the time it takes for a take to process:

• Duration of video
• Resolution/framerate
• Availability of processing servers

For 10s FHD at 60fps with a server immediately available the processing should be complete with 5 minutes. If there isn't a server available then the time may be as high as 30 minutes for the same video. We make efforts to ensure that this happens as rarely as possible but at certain times, especially as we release updates to the processing engine, these may be more common. This is part of the reason we advise you avoid polling in production and use webhooks.

Webhooks

We provide an interface to be able to subscribe a webhook to events associated with Jobs. Details about the events are documented here:

Webhook Events | move.ai · Svix

The GraphQL mutation required to specify your endpoints is:

mutation CreateWebhook {
    webhook: upsertWebhookEndpoint(
        url: "https://your-webhook-url.com",
        description: "This webhook does awesome things.",
        secret: "your-secret",
        events: ["ugc.job.state.completed"],
        uid: "your-unique-id-for-webhook",
        metadata: "{\"key\": \"value\"}"
    ) {
        uid
        secret
        description
        url
        events
        metadata
    }
}

Complete JavaScript example

A simple JavaScript example app is available on github here: https://github.com/move-ai/move_js_sample.

Complete Python example

This uses our Python SDK.

pip install move-ugc-python

import os
import time
from datetime import datetime

import requests
from move_ugc import MoveUgc

class MoveAPI:
    def __init__(self, api_key, endpoint_url=None):
        self.api_key = api_key
        if endpoint_url is None:
            endpoint_url = 'https://api.move.ai/ugc/graphql'
        self.endpoint_url = endpoint_url
        self.client = MoveUgc(api_key=api_key, endpoint_url=endpoint_url)

    def create_files(self, video_path):
        video_file = self.client.files.create(file_type="mp4")

        with open(video_path, 'rb') as f:
            requests.put(video_file.presigned_url, data=f.read())

        return video_file.id

    def create_take(self, video_file_id,  metadata=None):
        if metadata is None:
            metadata = {"test": "test"}
        take = self.client.takes.create(
            video_file_id=video_file_id,
            metadata=metadata,
        )
        return take

    def get_take(self, take_id):
        take = self.client.takes.retrieve(id=take_id)
        return take

    def create_job(self, take_id):
        job = self.client.jobs.create(take_id=take_id, metadata={"test": "test_job"})
        return job

    def get_job(self, job_id, expand=False):
        # Get a job using the Move One Public API
        # Implement job retrieval logic using move_ugc_python SDK
        if expand is False:
            job = self.client.jobs.retrieve(id=job_id)
        else:
            job = self.client.jobs.retrieve(
                id=job_id, expand=["take", "outputs", "client"]
            )
        return job

    def download_outputs(self, job_id, output_dir, output_name):
        # make output dir if it doesn't exist
        if not os.path.exists(output_dir):
            os.makedirs(output_dir)
        # get job
        job = self.get_job(job_id, expand=True)
        # for each output download the file in the output_dir
        output_paths = []
        for output in job.outputs:
            output_file_name = f"{output_name}{output.file.type}"
            output_path = os.path.join(output_dir, output_file_name)
            with open(output_path, 'wb') as f:
                response = requests.get(output.file.presigned_url)
                f.write(response.content)
            output_paths.append(output_path)
        return output_paths

if __name__ == '__main__':

    move = MoveAPI(os.getenv("MOVE_API_KEY"))
    filename = 'portrait.mp4'
    video_file_id = move.create_files(filename)
    take = move.create_take(video_file_id)
    job = move.create_job(take.id)
    attempts = 0
    while attempts < 100:
        job = move.get_job(job.id)
        update_str = f"[{datetime.now().isoformat()} | {attempts}] Job {job.id} is {job.state}"
        print(update_str)
        if job.state == 'FINISHED':
            outputs = move.download_outputs(job.id, 'outputs', filename.split('.')[0])
            print(outputs)
            break
        else:
            time.sleep(30)
            attempts += 1