Media

Transcode and inspect audio/video from a single API call — no media-processing account, no per-minute plan. Your agent posts a file (inline or by storage reference) and gets back a re-encoded clip, an extracted or transcoded audio track, a loudness-normalized mix, a sped-up cut, a concatenated reel, a watermarked or captioned video, an animated GIF, a single frame, an extracted subtitle file, or a metadata report. Backed by ffmpeg/ffprobe on a dedicated arm64 media worker; priced per call, not per month.

curl -X POST https://api.relaystation.ai/v1/media/thumbnail \
  -H 'Authorization: Bearer rs_live_<key>' \
  -H 'Idempotency-Key: thumb-promo-20260611' \
  -H 'Content-Type: application/json' \
  -d '{ "file": { "inline": "<base64 video>" }, "timestamp": 3, "format": "png" }'

The same call works on the lodestone path — no account, just a signed x402 payment instead of an API key:

curl -X POST https://api.relaystation.ai/v1/media/thumbnail \
  -H 'X-Payment: <base64 EIP-3009 authorization>' \
  -H 'Idempotency-Key: thumb-promo-20260611' \
  -H 'Content-Type: application/json' \
  -d '{ "file": { "inline": "<base64 video>" }, "timestamp": 3, "format": "png" }'

The operations

19 ops, three billing shapes. The transform ops bill per MB of input (min 1 MiB); frames bills per frame; the read/grab/visualize ops are flat per call. Some ops take a second input (concat, overlay, subtitle-burn, audio-mix) alongside the primary file (see the second input below). Three ops return JSON instead of a file: probe, silence (in detect mode), and metadata (in read mode); frames returns a manifest of presigned image refs.

Inputs, caps, and the billing grain

Files ride the shared input convention: { "inline": "<base64>" } up to 4 MB, or { "inputKey": "..." } minted from POST /v1/cputools/upload-url up to 50 MB — when to use which, how outputs come back, and how to chain are all in Passing & receiving files.

The synchronous window sets caps, all operator-tunable: inputs ≤ 50 MB (cputools.media.max_mb → 413 INPUT_TOO_LARGE) and ≤ 300 s for the audio + speed ops (cputools.media.max_seconds → 422 DURATION_TOO_LONG); convert is tighter (≤ 60 s, downscaled to 720p — the re-encode is the expensive op); gif is ≤ 15 s and ≤ 640 px wide (GIFs blow up fast). The ffmpeg run itself is killed at 25 s (cputools.media.timeout_ms) inside the ~29 s sync window. Charge-on-attempt with a safety net: an undeliverable transform (a timeout kill, an unconvertible input) throws and the wrapper reverses the charge — you’re never billed for work that didn’t ship. Every cap check above runs before the charge, so a rejected request is free.

The second input

concat, overlay, subtitle-burn, and audio-mix take a second input next to the primary file. That second input is itself an input-source object — exactly the shape the primary uses:

• concat — a files array (2–10), each item { "inline": "<base64>" } (≤ 4 MB) or { "inputKey": "..." } (≤ 50 MB). The whole array is the input; you bill on the summed size.
• overlay — an overlay field carrying the watermark image ({ inline } or { inputKey }).
• subtitle-burn — a subtitle field carrying the SRT/WebVTT file ({ inline } or { inputKey }).
• audio-mix — an inputs array of additional tracks alongside the primary file; each item is an input-source object. The total (primary + additional) is capped at cputools.media.audio-mix.max_inputs (default 8 → up to 7 additional). You bill on the primary file’s size.

Each secondary input is resolved and ownership-checked the same way as the primary; mint large secondaries through POST /v1/cputools/upload-url first — see Passing & receiving files. Outputs always come back as a single file in the uniform envelope.

Input formats — sniffed, not trusted

The input format is detected by ffprobe (the container’s format_name), never by file extension. The allowlist is the operator-tunable cputools.media.input_formats; an input whose sniffed format isn’t on it returns a free 422 UNSUPPORTED_FORMAT before any charge, and a file that isn’t probeable media at all returns a free 422 MEDIA_PARSE_FAILED. The default allowlist:

mp4 · mov · m4a · m4v · 3gp · 3g2 · mj2 · matroska (mkv) · webm · avi · flv · mpegts · mpeg · mpegvideo · asf · wmv · ogg · mp3 · wav · flac · aac · gif

(Matching is by ffprobe format-name token — e.g. a .mkv file sniffs as matroska,webm, and the QuickTime family sniffs as the combined mov,mp4,m4a,3gp,3g2,mj2.)

Each op also has an operator kill-switch (cputools.media.<op>.enabled); a disabled op returns a free 422 OP_DISABLED.

probe — the full response

{ "file": { ... } } is the whole request. The response carries everything ffprobe reports, under a probe key:

{
  "probe": {
    "formatName": "mov,mp4,m4a,3gp,3g2,mj2",
    "durationSeconds": 12.43,
    "sizeBytes": 1048576,
    "bitRate": 674812,
    "hasVideo": true,
    "hasAudio": true,
    "width": 1280,
    "height": 720,
    "videoCodec": "h264",
    "audioCodec": "aac",
    "streams": [
      { "index": 0, "type": "video", "codec": "h264" },
      { "index": 1, "type": "audio", "codec": "aac" }
    ]
  }
}

formatName is ffprobe’s container short-name list. durationSeconds, sizeBytes, and bitRate are null when the container doesn’t report them; width / height / videoCodec are null for audio-only inputs (and audioCodec for silent video). streams[] lists every stream with its index, type (video / audio / subtitle / …), and codec (null when unknown).

trim — options

Cuts by stream-copy: codecs are preserved, no re-encode, fast. Input duration ≤ 300 s.

convert — options

The full re-encode. Video targets come back H.264 + AAC, downscaled to ≤ 720 px height (cputools.media.convert.max_height, aspect preserved), encoded with the x264 veryfast preset (cputools.media.convert.preset). Audio targets drop the video track. Input duration ≤ 60 s (cputools.media.convert.max_seconds).

The optional scale/crop/rotate transforms compose with the default downscale; absent, convert behaves exactly as before (aspect-preserving downscale to the height cap).

thumbnail — options

Needs a video stream (422 NO_VIDEO_STREAM).

audio-extract — options

Needs an audio stream (422 NO_AUDIO_STREAM). Input duration ≤ 300 s.

gif — options

Needs a video stream (422 NO_VIDEO_STREAM). Single-pass palettegen/paletteuse for quality; lanczos scaling; the GIF loops.

audio-convert — options

A standalone audio→audio re-encode — change the format, bitrate, or sample rate of an audio file. (Use audio-extract instead when you want to pull the audio off a video.) Needs an audio stream (422 NO_AUDIO_STREAM). Input duration ≤ 300 s.

loudnorm — options

EBU R128 loudness normalization (ffmpeg loudnorm) — bring a clip to a consistent perceived level for podcasts, voiceover, or music. The output is audio (the normalized audio stream, re-encoded); any video track is dropped. Needs an audio stream (422 NO_AUDIO_STREAM). Input duration ≤ 300 s.

speed — options

Change playback speed. Pitch is preserved by default (audio is time-stretched, not pitch-shifted); set pitchCorrect: false for a tape-style speed-and-pitch change.

subtitle-extract — options

Pull an embedded subtitle/caption track out of a container to a text sidecar. If the requested subtitle stream doesn’t exist, you get a free 422 NO_SUBTITLE_STREAM. Flat-priced.

concat — options

Join 2–10 clips end-to-end with the ffmpeg concat demuxer — a fast stream-copy (no re-encode). The inputs must share the same codec and container; a mismatched set is a 422 CONVERT_FAILED (re-encode each via convert to a common format first). Billed per MB of the summed input size.

overlay — options

Composite an image (a logo or watermark) onto a video. The primary file must be a video (422 NO_VIDEO_STREAM); the overlay is a second input carrying the image (see the second input). The video is re-encoded. Billed per MB of the video. Input duration ≤ 300 s.

subtitle-burn — options

Render a subtitle file into the video frames with libass. The primary file must be a video; the subtitle is a second input carrying the SRT/WebVTT file (see the second input). The video is re-encoded. Billed per MB of the video. Input duration ≤ 300 s.

Hard subs, not soft. The captions are burned (rasterized) into the picture, not added as a separate, toggleable subtitle track. The result has no soft subtitle stream and the captions cannot be turned off. If you want a removable track, ship the SRT/WebVTT alongside the video instead. Rendering uses libass with its default font.

waveform — options

Render an audio file as a waveform or spectrogram PNG (ffmpeg showwavespic / showspectrumpic) — a quick visual of a track for thumbnails, previews, or QA. Needs an audio stream (422 NO_AUDIO_STREAM). Input duration ≤ cputools.media.waveform.max_seconds (default 600 s). Flat-priced; returns the PNG in the uniform output envelope.

silence — options

Detect or remove silence with ffmpeg silencedetect / silenceremove. Needs an audio stream. Input duration ≤ cputools.media.silence.max_seconds (default 1800 s). Flat-priced.

• action: "detect" (default) returns JSON, not a file: { "silence": [{ "start", "end", "duration" }], "count": N } (an interval still open at end-of-file has end/duration null).
• action: "trim" returns the silence-removed audio in the uniform output envelope.

storyboard — options

Build a frame-sampled thumbnail sprite sheet (a contact sheet) from a video — cols×rows tiles sampled evenly across the clip (ffmpeg fps sample + tile). Needs a video stream (422 NO_VIDEO_STREAM). Input duration ≤ cputools.media.storyboard.max_seconds (default 3600 s). Flat-priced; returns one PNG. By default the sampling rate is computed so cols×rows frames span the whole clip (the grid fills regardless of length); pass intervalSeconds to sample at a fixed cadence instead.

metadata — options

Read or write container metadata tags via ffprobe / ffmpeg -metadata (stream-copy — no re-encode). Flat-priced.

• Read (no set): returns JSON { "metadata": { ...tags } } — the container’s format tags.
• Write (set present): writes the allowlisted tags and returns the re-tagged file in the uniform output envelope (the source container is preserved). Unknown tag keys are ignored.

frames — options

Extract N evenly-sampled frames from a video → a manifest of presigned image refs (the same shape as POST /v1/pdf/render). Needs a video stream. Input duration ≤ cputools.media.frames.max_seconds (default 3600 s). Billed per frame. Returns { "frames": [{ "index", "outputKey", "outputUrl" }], "count": N } — fetch each via its presigned outputUrl (~1 h) or chain outputKey as another op’s inputKey.

MCP tools

The same surface is callable over MCP at https://api.relaystation.ai/mcp: media_probe, media_metadata, media_thumbnail, media_subtitle_extract, media_waveform, media_silence, media_storyboard, media_trim, media_audio_extract, media_audio_convert, media_loudnorm, media_speed, media_convert, media_audio_mix, media_concat, media_overlay, media_subtitle_burn, media_gif, and media_frames. Same auth, same prices as the HTTP routes. (media_concat, media_overlay, media_subtitle_burn, and media_audio_mix carry the second input as the files / inputs array / overlay / subtitle field, same input-source shape as file.)

Next

Quickstart · Authentication · x402 wire format · Passing & receiving files · Document conversion · API reference · more image + file tools at cputools.relaystation.ai