feat(sume): serve the installed video and image models on Sume (#485)

`@hypit/provider-sume` submits one asynchronous job to `https://api.sume.com`
with a Bearer API key, polls `GET /v1/jobs/{id}/status` until it is terminal,
reads the artifacts from `GET /v1/jobs/{id}/result` and stores them in the
current Build. It maps twelve models the Distribution already describes: the
four Seedance variants, MiniMax H3, Grok Imagine 1.5, PixVerse V6 and C1, GPT
Image 2, Nano Banana 2 and Pro, and Seedream 5.0 lite. It adds no model package.

Video jobs post to `/v1/video-router/generate`, whose flat `image_url`,
`end_image_url` and `reference_*_urls` fields take the mapping as data.
`/v1/videos` rejects `aspect_ratio: adaptive` for Seedance, which the router
accepts. Image jobs post to `/v1/images` with `mode: "async"`, and routes.ts
builds `input_references` from the reference URLs. Sume serves Nano Banana 2 as
Nano Banana 2.1, so `nano-banana-2` runs on `google/nano-banana-2.1`.

References are uploaded through `POST /v1/files/generate-upload-url`: the bytes
are PUT to the presigned URL and the returned `media.sume.com` URL travels in
the job. Service limits are reported as unsupported before any upload: Seedance
web search and automatic 2.5 length, MiniMax H3 under 5 seconds, Grok 1.5 with
more than one image, PixVerse frames, reference videos, seed and multi-clip,
GPT Image 2 background, ratios a catalog row does not list, more than ten
image references, and Seedream quality above basic or with its content check
off. A failed job keeps Sume's code, message and the upstream provider message.
`defaultConcurrency` is 2, the number of jobs Sume's free plan runs at once.

Live runs through the Provider's start/poll/collect against a free-plan key all
completed: text-to-image and reference editing on every image model,
text-to-video, first frame, first and last frame, reference images, and
reference video with reference image and audio on Seedance 2.5, Seedance 2 mini
and MiniMax H3. A `hypit build` chaining Seedream into Nano Banana Pro through
a Runtime Profile completed as well. Total spend was about $3.98.
This commit is contained in:
Sharky Wang
2026-11-09 20:19:42 +08:00
committed by GitHub
parent 2d31e88765
commit 2ca4e5d2ea
15 changed files with 909 additions and 0 deletions

View File

@@ -25,6 +25,7 @@ npm install @hypit/provider-tokendance
npm install @hypit/provider-hiapi
npm install @hypit/provider-pollo
npm install @hypit/provider-beatapi
npm install @hypit/provider-sume
npm install @hypit/provider-monid
```
@@ -64,6 +65,14 @@ and uploads reference media through BeatAPI's file endpoint. Its
[video](https://docs.beatapi.io/video-api) and [image](https://docs.beatapi.io/image-api) model
indexes list the rest.
### Sume
[Sume](https://www.sume.com) offers video and image models through one job API.
[`@hypit/provider-sume`](https://github.com/hypit-ai/hypit/blob/main/packages/provider-sume/README.md)
serves the Seedance 2.0 and 2.5 models, MiniMax H3, Grok Imagine 1.5, PixVerse V6 and C1, GPT Image
2, Nano Banana 2.1 and Pro and Seedream 5.0 lite, and uploads reference media through Sume's upload
URL. Its [developer documentation](https://docs.sume.com/llms.txt) and live model catalog list the rest.
### Monid
[Monid](https://monid.ai) is a Hypit service partner.

View File

@@ -22,6 +22,7 @@ npm install @hypit/provider-tokendance
npm install @hypit/provider-hiapi
npm install @hypit/provider-pollo
npm install @hypit/provider-beatapi
npm install @hypit/provider-sume
npm install @hypit/provider-monid
```
@@ -56,6 +57,14 @@ Pollo 只接受公网 URL 形式的参考素材,带参考素材的请求需要
并通过 BeatAPI 的文件接口上传参考素材。
其余模型见它的[视频](https://docs.beatapi.io/video-api)与[图片](https://docs.beatapi.io/image-api)模型索引。
### Sume
[Sume](https://www.sume.com) 通过一个任务接口提供视频和图片模型。
[`@hypit/provider-sume`](https://github.com/hypit-ai/hypit/blob/main/packages/provider-sume/README.md)
提供 Seedance 2.0、2.5 系列、MiniMax H3、Grok Imagine 1.5、PixVerse V6 与 C1、GPT Image 2、
Nano Banana 2.1 与 Pro、Seedream 5.0 lite,并通过 Sume 的上传地址上传参考素材。
其余模型见它的[开发者文档](https://docs.sume.com/llms.txt)与实时模型目录。
### Monid
[Monid](https://monid.ai) 是 Hypit 的服务合作方。

View File

@@ -284,6 +284,7 @@
"@hypit/media-local": "workspace:*",
"@hypit/provider-monid": "workspace:*",
"@hypit/provider-pollo": "workspace:*",
"@hypit/provider-sume": "workspace:*",
"@hypit/provider-tokendance": "workspace:*",
"@hypit/provider-whisperx-local": "workspace:*",
"@hypit/ranking": "workspace:*",

View File

@@ -0,0 +1,106 @@
# `@hypit/provider-sume`
Optional Provider, not installed with Hypit. Install in the video project:
```bash
npm install @hypit/provider-sume
```
Hypit Runtime Provider for a [Sume](https://docs.sume.com/public-api) API key. Each capability
submits one asynchronous job to `https://api.sume.com` with a `Bearer` API key, polls
`GET /v1/jobs/{id}/status` until the job is terminal, reads the generated files from
`GET /v1/jobs/{id}/result` and stores them in the current Build.
| Capability | Sume model | Submit path |
| --- | --- | --- |
| `@hypit/seedance@1#seedance-2` | `seedance-2` | `/v1/video-router/generate` |
| `@hypit/seedance@1#seedance-2-fast` | `seedance-2-fast` | `/v1/video-router/generate` |
| `@hypit/seedance@1#seedance-2-mini` | `seedance-2-mini` | `/v1/video-router/generate` |
| `@hypit/seedance@1#seedance-2.5` | `seedance-2.5` | `/v1/video-router/generate` |
| `@hypit/minimax-h3@1#minimax-h3` | `minimax-h3` | `/v1/video-router/generate` |
| `@hypit/grok-imagine@1#grok-imagine-video-1.5-preview` | `grok-imagine-video-1.5` | `/v1/video-router/generate` |
| `@hypit/pixverse@1#pixverse-v6` | `pixverse-v6` | `/v1/video-router/generate` |
| `@hypit/pixverse@1#pixverse-c1` | `pixverse-c1` | `/v1/video-router/generate` |
| `@hypit/gpt-image@1#gpt-image-2` | `openai/gpt-image-2` | `/v1/images` |
| `@hypit/nano-banana@1#nano-banana-2` | `google/nano-banana-2.1` | `/v1/images` |
| `@hypit/nano-banana@1#nano-banana-pro` | `google/nano-banana-pro` | `/v1/images` |
| `@hypit/seedream@1#seedream-5-lite` | `bytedance-seed/seedream-5-lite` | `/v1/images` |
Sume's [video](https://docs.sume.com/models/videos.md) and [image](https://docs.sume.com/models/images.md)
catalogs list further models; this Provider maps only the models the Distribution already describes.
Sume serves Nano Banana 2 as its successor, Nano Banana 2.1, so a `nano-banana-2` request renders on
`google/nano-banana-2.1`.
Video jobs carry the opening and closing frames as `image_url` and `end_image_url`, and references as
`reference_image_urls`, `reference_video_urls` and `reference_audio_urls`. Image jobs carry their
references as `input_references`. Every submission uses `mode: "async"`. `duration`, `aspect_ratio`,
`resolution` and `generate_audio` keep their authored values, except that MiniMax H3's `768P` is sent
as `768p`, PixVerse's `quality` travels as `resolution`, and Nano Banana's `jpg` output format is sent
as `jpeg`. A Grok Imagine 1.5 image travels as the first frame, and the first frame then sets the
shape, so the authored aspect ratio is not sent with it.
Service limits this Provider reports as unsupported before uploading any reference:
- Seedance has no `web_search` field. An authored `false` is dropped; an authored `true` is refused.
- Seedance 2.5 takes 4 to 30 seconds, so `duration="-1"` for an automatic length is refused.
- MiniMax H3 takes 5 to 15 seconds.
- Grok Imagine 1.5 takes a prompt alone or one first-frame image, does not render `2:3` or `3:2`, and
needs an explicit aspect ratio without an image.
- PixVerse V6 and C1 take a prompt or reference images, without frames, reference videos, a seed or
multi-clip, for 2 to 15 seconds, and do not render `2:3`, `3:2` or `auto`.
- GPT Image 2 has no `background` field, accepts up to ten reference images and does not render
`3:2`, `2:3`, `2:1`, `1:2`, `3:1`, `1:3`, `21:9` or `9:21`.
- Nano Banana 2.1 accepts up to ten reference images.
- Seedream 5.0 lite renders at 2K, which is `quality="basic"`; accepts up to ten reference images;
does not render `21:9`; and has no switch to turn its content check off, so `nsfw-check="false"` is
refused.
Seedance visual references carry `person-reference`; the Provider accepts the declaration and
transmits nothing for it, since Sume's request body has no field for it.
Referenced Resources are uploaded through `POST /v1/files/generate-upload-url`: the Provider PUTs the
bytes to the returned presigned URL and sends the returned public `media.sume.com` URL in the job. One
Resource is uploaded once per Runtime operation. Embedded callers may replace that transport with
`publicAssetUrl(artifact, resources, fields)`, which must return a public HTTPS URL Sume can fetch.
Runtime Profile example:
```json
{
"format": "hypit.runtime-local@1",
"dataRoot": ".hypit/runtimes/local",
"credentials": {
"local": {
"use": "@hypit/credential-store-local"
}
},
"endpoints": {
"sume.default": {
"use": "@hypit/provider-sume",
"pool": "sume.default",
"config": {
"apiKey": { "store": "local", "key": "sume.api-key" },
"defaultConcurrency": 2,
"pollIntervalMs": 5000
}
}
},
"bindings": {}
}
```
`baseUrl` defaults to `https://api.sume.com`. Store the API key with
`hypit auth login sume.default --runtime hypit.runtime.json`. `defaultConcurrency` defaults to 2,
which is the number of jobs Sume's free plan runs at once; raise it to the workspace's plan limit.
Optional `requestTimeoutMs`, `operationTimeoutMs` and `actionLimits` bound single HTTP calls, the
whole remote job and action concurrency. Each submission sends the Runtime operation as its
`Idempotency-Key`, so a retried submission of the same body resolves to the same job and is charged
once.
HTTP failures keep Sume's `error.code`, message, `details`, `request_id` and any
`retry_after_seconds`; a failed or canceled job keeps its error code, message and the upstream
provider's message. Any URL inside a surfaced reason is redacted, since an upload or result URL is an
access capability rather than diagnostic content. Polling transport failures and HTTP 429/5xx
preserve the same job. Sume's `retry_after_seconds` takes precedence over the standard `Retry-After`
header; either wait is capped at the original operation deadline, where the next poll concludes
timeout without querying early.

View File

@@ -0,0 +1,39 @@
{
"name": "@hypit/provider-sume",
"version": "0.1.0",
"license": "SEE LICENSE IN LICENSE",
"private": true,
"type": "module",
"exports": {
".": "./src/index.ts"
},
"files": [
"dist",
"README.md"
],
"hypit": {
"activation": "./src/activation.ts"
},
"peerDependencies": {
"@hypit/hypit": "workspace:^0.3.1",
"@hypit/runtime-local": "workspace:^0.1.1"
},
"devDependencies": {
"@hypit/endpoint": "workspace:*",
"@hypit/generation": "workspace:*",
"@hypit/gpt-image": "workspace:*",
"@hypit/grok-imagine": "workspace:*",
"@hypit/hypit": "workspace:*",
"@hypit/minimax-h3": "workspace:*",
"@hypit/nano-banana": "workspace:*",
"@hypit/pixverse": "workspace:*",
"@hypit/protocol": "workspace:*",
"@hypit/runtime": "workspace:*",
"@hypit/runtime-local": "workspace:*",
"@hypit/seedance": "workspace:*",
"@hypit/seedream": "workspace:*"
},
"dependencies": {
"@hypit/generation": "workspace:^0.1.0"
}
}

View File

@@ -0,0 +1,62 @@
import {
createRuntimeEndpointAdapterFacet,
runtimeConfigCredentialRef,
runtimeConfigActionLimits,
runtimeConfigExact,
runtimeConfigObject,
runtimeConfigPositiveInteger,
runtimeConfigString,
} from "@hypit/runtime-local/extension";
import { createSumeProvider } from "./provider.js";
const adapter = createRuntimeEndpointAdapterFacet({
use: "@hypit/provider-sume",
activate(context) {
if (context.pool === undefined) throw new Error("Sume Provider Pool is required");
const config = runtimeConfigObject(context.config, "Sume");
runtimeConfigExact(config, [
"baseUrl",
"apiKey",
"defaultConcurrency",
"actionLimits",
"pollIntervalMs",
"requestTimeoutMs",
"operationTimeoutMs",
], "Sume");
const baseUrl = runtimeConfigString(config.baseUrl, "Sume baseUrl");
if (baseUrl !== undefined) {
const url = new URL(baseUrl);
if (url.protocol !== "https:" && url.hostname !== "localhost" && url.hostname !== "127.0.0.1") {
throw new Error("Sume baseUrl must use HTTPS or loopback");
}
}
const apiKey = runtimeConfigCredentialRef(config.apiKey, "Sume apiKey");
if (apiKey === undefined) throw new Error("Sume apiKey CredentialRef is required");
const actionLimits = runtimeConfigActionLimits(config.actionLimits);
const defaultConcurrency = runtimeConfigPositiveInteger(config.defaultConcurrency, "Sume defaultConcurrency");
const pollIntervalMs = runtimeConfigPositiveInteger(config.pollIntervalMs, "Sume pollIntervalMs");
const requestTimeoutMs = runtimeConfigPositiveInteger(config.requestTimeoutMs, "Sume requestTimeoutMs");
const operationTimeoutMs = runtimeConfigPositiveInteger(config.operationTimeoutMs, "Sume operationTimeoutMs");
return {
endpoint: createSumeProvider({
instance: context.instance,
pool: context.pool,
...(baseUrl === undefined ? {} : { baseUrl }),
apiKey,
...(defaultConcurrency === undefined ? {} : { defaultConcurrency }),
...(actionLimits === undefined ? {} : { actionLimits }),
...(pollIntervalMs === undefined ? {} : { pollIntervalMs }),
...(requestTimeoutMs === undefined ? {} : { requestTimeoutMs }),
...(operationTimeoutMs === undefined ? {} : { operationTimeoutMs }),
}),
};
},
});
export const hypitPackage = {
format: "hypit.package@1" as const,
facets: [adapter],
};
export default hypitPackage;

View File

@@ -0,0 +1,53 @@
import { EndpointServiceError } from "@hypit/hypit/endpoint";
import { EndpointHttpError, retryAfterMs } from "@hypit/hypit/endpoint/http";
/** Sume's `{ error: { code, message, request_id, retry_after_seconds, details } }` envelope and failed jobs. */
function record(value: unknown): Record<string, unknown> | undefined {
return value !== null && typeof value === "object" && !Array.isArray(value)
? value as Record<string, unknown> : undefined;
}
function text(value: unknown): string | undefined {
return typeof value === "string" && value.trim().length > 0 ? value.trim() : undefined;
}
// Responses may carry a signed upload or result URL. Keep the reason, not its access capability.
export function safeSumeReason(value: string): string {
return value.replace(/https?:\/\/\S+/giu, "[redacted-url]");
}
export class SumeServiceError extends EndpointServiceError {}
export class SumeHttpError extends EndpointHttpError {
constructor(status: number, response: { readonly headers: Headers }, bodyText: string,
request: { readonly method: string; readonly path: string; readonly model?: string }) {
let body: Record<string, unknown> | undefined;
try { body = record(JSON.parse(bodyText)); } catch { /* Non-JSON gateway failures still have HTTP evidence. */ }
const error = record(body?.error);
const code = text(error?.code) ?? "SUME_HTTP_ERROR";
const reason = text(error?.message) ?? (error === undefined ? text(bodyText.slice(0, 2000)) : undefined);
const details = error?.details === undefined ? undefined : JSON.stringify(error.details).slice(0, 1000);
const requestId = text(error?.request_id) ?? text(response.headers.get("x-sume-request-id"));
const serviceRetryAfter = typeof error?.retry_after_seconds === "number" ? Math.round(error.retry_after_seconds * 1000) : undefined;
const retryAfter = serviceRetryAfter ?? retryAfterMs(response.headers);
const facts = [
`Sume HTTP ${status}`, code, `${request.method} ${request.path}`,
...(request.model === undefined ? [] : [`model=${request.model}`]),
...(requestId === undefined ? [] : [`request=${requestId}`]),
...(retryAfter === undefined ? [] : [`retry-after=${retryAfter}ms`]),
];
const message = [reason, details].filter((item) => item !== undefined).join(" ");
super(code, `${facts.join("; ")}${message.length === 0 ? "" : `: ${safeSumeReason(message)}`}`,
status, retryAfter);
}
}
/** A terminal `failed` or `canceled` job; `undefined` otherwise. */
export function sumeJobFailure(status: string, job: Record<string, unknown> | undefined, id: string): SumeServiceError | undefined {
if (status !== "failed" && status !== "canceled") return undefined;
const error = record(job?.error);
const code = text(error?.code) ?? text(error?.public_reason) ?? (status === "canceled" ? "SUME_JOB_CANCELED" : "SUME_JOB_FAILED");
const upstream = text(record(error?.details)?.provider_error_message);
const reason = [text(error?.message) ?? text(job?.error), upstream].filter((item) => item !== undefined).join(" ") || undefined;
return new SumeServiceError(code,
`Sume job ${id} ${status}; ${code}${reason === undefined ? "" : `: ${safeSumeReason(reason)}`}`);
}

View File

@@ -0,0 +1,2 @@
export { createSumeProvider, sumeProviderModuleRef } from "./provider.js";
export type { CreateSumeProviderOptions } from "./provider.js";

View File

@@ -0,0 +1,127 @@
import type { ModuleRef } from "@hypit/hypit/protocol";
import type { GenerationWireMapping } from "@hypit/generation";
const SEEDANCE: ModuleRef = { name: "@hypit/seedance", version: "1" };
const MINIMAX: ModuleRef = { name: "@hypit/minimax-h3", version: "1" };
const GROK: ModuleRef = { name: "@hypit/grok-imagine", version: "1" };
const PIXVERSE: ModuleRef = { name: "@hypit/pixverse", version: "1" };
const GPT_IMAGE: ModuleRef = { name: "@hypit/gpt-image", version: "1" };
const NANO_BANANA: ModuleRef = { name: "@hypit/nano-banana", version: "1" };
const SEEDREAM: ModuleRef = { name: "@hypit/seedream", version: "1" };
/**
* Sume catalog ids and the body fields each one documents.
*
* Video models post to `POST /v1/video-router/generate`, whose body carries every reference as a
* flat URL field. Image models post to `POST /v1/images`; its `input_references` array of
* `{ type, image_url: { url } }` objects is built in routes.ts from the `images` URLs written here.
* Ports the service has no field for are written under their own names so routes.ts can drop an
* authored "off" value and refuse an authored "on" value before anything is uploaded.
*/
const seedanceFields = {
prompt: { as: "value", field: "prompt" },
firstFrame: { as: "url", field: "image_url", resourceFields: ["personReference"] },
lastFrame: { as: "url", field: "end_image_url", resourceFields: ["personReference"] },
referenceImage: { as: "urlArray", field: "reference_image_urls", resourceFields: ["personReference"] },
referenceVideo: { as: "urlArray", field: "reference_video_urls", resourceFields: ["personReference"] },
referenceAudio: { as: "urlArray", field: "reference_audio_urls" },
duration: { as: "value", field: "duration" },
aspectRatio: { as: "value", field: "aspect_ratio" },
resolution: { as: "value", field: "resolution" },
generateAudio: { as: "value", field: "generate_audio" },
webSearch: { as: "value", field: "web_search" },
} as const satisfies GenerationWireMapping["fields"];
const seedance = (name: string): GenerationWireMapping => ({
capability: { module: SEEDANCE, name }, result: "video", routes: [{ model: name }], fields: seedanceFields,
});
const pixverse = (name: "pixverse-v6" | "pixverse-c1"): GenerationWireMapping => ({
capability: { module: PIXVERSE, name }, result: "video", routes: [{ model: name }],
fields: {
prompt: { as: "value", field: "prompt" },
firstFrame: { as: "url", field: "image_url" },
lastFrame: { as: "url", field: "end_image_url" },
referenceImage: { as: "urlArray", field: "reference_image_urls" },
...(name === "pixverse-v6" ? {
referenceVideo: { as: "urlArray" as const, field: "reference_video_urls" },
multiClip: { as: "value" as const, field: "multi_clip" },
seed: { as: "value" as const, field: "seed" },
} : {}),
duration: { as: "value", field: "duration" },
// PixVerse's quality band is Sume's resolution.
quality: { as: "value", field: "resolution" },
aspectRatio: { as: "value", field: "aspect_ratio" },
generateAudio: { as: "value", field: "generate_audio" },
},
});
export const sumeMappings: readonly GenerationWireMapping[] = [
seedance("seedance-2"),
seedance("seedance-2-fast"),
seedance("seedance-2-mini"),
seedance("seedance-2.5"),
{
capability: { module: MINIMAX, name: "minimax-h3" }, result: "video",
routes: [{ model: "minimax-h3" }],
fields: {
prompt: { as: "value", field: "prompt" },
firstFrame: { as: "url", field: "image_url" },
lastFrame: { as: "url", field: "end_image_url" },
referenceImage: { as: "urlArray", field: "reference_image_urls" },
referenceVideo: { as: "urlArray", field: "reference_video_urls" },
referenceAudio: { as: "urlArray", field: "reference_audio_urls" },
duration: { as: "value", field: "duration" },
aspectRatio: { as: "value", field: "aspect_ratio" },
resolution: { as: "value", field: "resolution" },
},
},
{
capability: { module: GROK, name: "grok-imagine-video-1.5-preview" }, result: "video",
routes: [{ model: "grok-imagine-video-1.5" }],
fields: {
prompt: { as: "value", field: "prompt" },
images: { as: "urlArray", field: "images" },
duration: { as: "value", field: "duration" },
aspectRatio: { as: "value", field: "aspect_ratio" },
resolution: { as: "value", field: "resolution" },
},
},
pixverse("pixverse-v6"),
pixverse("pixverse-c1"),
{
capability: { module: GPT_IMAGE, name: "gpt-image-2" }, result: "image",
routes: [{ model: "openai/gpt-image-2" }],
fields: {
prompt: { as: "value", field: "prompt" },
images: { as: "urlArray", field: "images" },
aspectRatio: { as: "value", field: "aspect_ratio" },
resolution: { as: "value", field: "resolution" },
background: { as: "value", field: "background" },
},
},
// Sume serves Nano Banana 2 as its successor, Nano Banana 2.1.
...([["nano-banana-2", "google/nano-banana-2.1"], ["nano-banana-pro", "google/nano-banana-pro"]] as const).map(([name, model]) => ({
capability: { module: NANO_BANANA, name }, result: "image" as const,
routes: [{ model }],
fields: {
prompt: { as: "value" as const, field: "prompt" },
images: { as: "urlArray" as const, field: "images" },
aspectRatio: { as: "value" as const, field: "aspect_ratio" },
resolution: { as: "value" as const, field: "resolution" },
outputFormat: { as: "value" as const, field: "output_format" },
},
})),
{
capability: { module: SEEDREAM, name: "seedream-5-lite" }, result: "image",
routes: [{ model: "bytedance-seed/seedream-5-lite" }],
fields: {
prompt: { as: "value", field: "prompt" },
images: { as: "urlArray", field: "images" },
aspectRatio: { as: "value", field: "aspect_ratio" },
quality: { as: "value", field: "quality" },
outputFormat: { as: "value", field: "output_format" },
nsfwCheck: { as: "value", field: "nsfw_check" },
},
},
];

View File

@@ -0,0 +1,264 @@
import type { AsyncEndpoint, EndpointCredential, EndpointInvocationContext, EndpointOutcome } from "@hypit/hypit/endpoint";
import { EndpointServiceError, credentialRef, defineEndpoint, wakeAfter } from "@hypit/hypit/endpoint";
import type { CredentialRef, EndpointActionLimits, ResourceStore } from "@hypit/hypit/endpoint";
import { EndpointHttpError, EndpointResponseError, EndpointTransportError, withRequestDeadline } from "@hypit/hypit/endpoint/http";
import type { GenerationArtifactUrlResolver } from "@hypit/generation";
import { canonicalize } from "@hypit/hypit/protocol";
import type { BlobRef, CanonicalValue } from "@hypit/hypit/protocol";
import { sumeRouteForCapability, sumeRoutes } from "./routes.js";
import { SumeHttpError, SumeServiceError, sumeJobFailure } from "./errors.js";
export const sumeProviderModuleRef = { name: "@hypit/provider-sume", version: "1" } as const;
export type CreateSumeProviderOptions = {
readonly instance?: string;
readonly pool?: string;
readonly baseUrl?: string;
readonly apiKey?: CredentialRef;
readonly defaultConcurrency?: number;
readonly actionLimits?: EndpointActionLimits;
readonly pollIntervalMs?: number;
readonly requestTimeoutMs?: number;
readonly operationTimeoutMs?: number;
readonly fetch?: typeof globalThis.fetch;
/** Publish a referenced Resource at a URL the service can fetch; replaces Sume's upload URL. */
readonly publicAssetUrl?: (artifact: BlobRef, artifacts: ResourceStore, fields?: Readonly<Record<string, string | number | boolean>>) => Promise<string>;
};
type Handle = {
readonly contract: "hypit.sume-operation@1";
readonly jobId: string;
readonly route: string;
readonly startedAt: number;
readonly urls?: readonly string[];
};
const extensions: Readonly<Record<string, string>> = {
"image/png": "png", "image/jpeg": "jpg", "image/webp": "webp",
"audio/mpeg": "mp3", "audio/wav": "wav", "audio/x-wav": "wav", "audio/aac": "aac", "audio/mp4": "m4a",
"video/mp4": "mp4", "video/quicktime": "mov", "video/webm": "webm",
};
function assert(condition: unknown, message: string): asserts condition { if (!condition) throw new Error(message); }
function object(value: unknown, subject: string): Record<string, unknown> {
assert(value !== null && typeof value === "object" && !Array.isArray(value), `${subject} must be an object`);
return value as Record<string, unknown>;
}
function apiBaseUrl(value: string): string {
let trimmed = value.trim();
while (trimmed.endsWith("/")) trimmed = trimmed.slice(0, -1);
assert(trimmed.length > 0, "Sume base URL is empty");
return trimmed;
}
function apiKey(credentials: Readonly<Record<string, EndpointCredential>>): string {
const value = credentials.apiKey?.secret;
if (typeof value !== "string" || value.length === 0) {
throw new SumeServiceError("SUME_CREDENTIAL_UNAVAILABLE",
"Sume apiKey credential is unavailable; store a Sume API key for this Endpoint");
}
return value;
}
function httpsUrl(value: unknown, subject: string): string {
assert(typeof value === "string" && /^https:\/\//u.test(value), `${subject} has no HTTPS URL`);
return value;
}
function failure(error: unknown): EndpointOutcome {
if (!(error instanceof EndpointServiceError || error instanceof EndpointHttpError
|| error instanceof EndpointTransportError || error instanceof EndpointResponseError)) throw error;
const code = error instanceof EndpointServiceError || error instanceof EndpointHttpError ? error.code : "SUME_ERROR";
return { status: "failed", failure: { code, message: error.message } };
}
function retrying(error: unknown, handle: CanonicalValue, pollIntervalMs: number, deadlineAt?: number): EndpointOutcome | undefined {
const delay = error instanceof EndpointTransportError ? pollIntervalMs
: error instanceof EndpointHttpError && (error.status === 429 || error.status >= 500)
? error.retryAfterMs ?? pollIntervalMs : undefined;
if (delay === undefined) return undefined;
return { status: "pending", handle, wakeAt: Math.min(Date.now() + delay, deadlineAt ?? Number.MAX_SAFE_INTEGER), progress: { phase: "retrying" } };
}
function unknownSubmission(error: unknown): EndpointOutcome | undefined {
return error instanceof EndpointTransportError ? { status: "failed", failure: {
code: "SUME_SUBMISSION_UNKNOWN",
message: `Sume submission transport failed; remote outcome is unknown: ${error.message}`,
} } : undefined;
}
class SumeClient {
constructor(readonly baseUrl: string, readonly timeout: number, readonly fetcher: typeof globalThis.fetch) {}
/** One authenticated call; returns the `data` member of Sume's envelope. */
async json(path: string, key: string, init: RequestInit = {}): Promise<Record<string, unknown>> {
return await withRequestDeadline(this.timeout, async ({ signal, wait }) => {
const response = await wait(this.fetcher(`${this.baseUrl}${path}`, {
...init, signal, headers: { authorization: `Bearer ${key}`, ...(init.headers ?? {}) },
}));
const text = await wait(response.text());
if (!response.ok) {
const input = typeof init.body === "string" ? JSON.parse(init.body) as Record<string, unknown> : undefined;
throw new SumeHttpError(response.status, response, text, {
method: init.method ?? "GET", path, ...(typeof input?.model === "string" ? { model: input.model } : {}),
});
}
let body: unknown;
try { body = text.length === 0 ? {} : JSON.parse(text); } catch { throw new EndpointResponseError(`Sume returned invalid JSON (${response.status})`); }
return object(object(body, "Sume response").data, "Sume response data");
}, () => new EndpointTransportError("Sume request timed out", { timeout: true }));
}
/** Ask for an upload URL, PUT the Resource there and return the public HTTPS URL a job reads. */
async upload(artifact: BlobRef, resources: ResourceStore, key: string): Promise<string> {
const kind = artifact.mediaType.split("/", 1)[0] ?? "";
assert(kind === "image" || kind === "video" || kind === "audio",
`Sume accepts image, audio and video references, not ${artifact.mediaType}`);
const bytes = await resources.get(artifact.resource);
assert(bytes !== undefined && bytes.byteLength === artifact.size, `Reference Resource ${artifact.resource} is unavailable or has changed`);
const slot = await this.json("/v1/files/generate-upload-url", key, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({
content_type: artifact.mediaType,
size_bytes: artifact.size,
filename: `${artifact.resource}.${extensions[artifact.mediaType] ?? "bin"}`,
}),
});
const uploadUrl = httpsUrl(slot.upload_url, "Sume upload slot");
const headers = slot.upload_headers === undefined ? {} : object(slot.upload_headers, "Sume upload headers") as Record<string, string>;
await withRequestDeadline(this.timeout, async ({ signal, wait }) => {
const response = await wait(this.fetcher(uploadUrl, {
method: "PUT", signal, headers: { "content-type": artifact.mediaType, ...headers }, body: new Uint8Array(bytes),
}));
// The upload URL is presigned; its path is a credential and stays out of the error.
if (!response.ok) throw new SumeHttpError(response.status, response, "", { method: "PUT", path: "[upload-url]" });
await wait(response.arrayBuffer());
}, () => new EndpointTransportError("Sume upload timed out", { timeout: true }));
return httpsUrl(slot.public_url, "Sume upload slot");
}
async download(url: string): Promise<{ readonly bytes: Uint8Array; readonly mediaType: string }> {
return await withRequestDeadline(this.timeout, async ({ signal, wait }) => {
const response = await wait(this.fetcher(url, { signal }));
if (!response.ok) throw new SumeHttpError(response.status, response, "", { method: "GET", path: "[result-url]" });
return { bytes: new Uint8Array(await wait(response.arrayBuffer())), mediaType: response.headers.get("content-type")?.split(";", 1)[0] ?? "application/octet-stream" };
}, () => new EndpointTransportError("Sume download timed out", { timeout: true }));
}
}
function resolverFor(client: SumeClient, context: EndpointInvocationContext, publicAssetUrl: CreateSumeProviderOptions["publicAssetUrl"]): GenerationArtifactUrlResolver {
const resolved = new Map<string, Promise<string>>();
return (artifact, fields) => {
const existing = resolved.get(artifact.resource);
if (existing !== undefined) return existing;
const promise = publicAssetUrl === undefined
? client.upload(artifact, context.resources, apiKey(context.credentials))
: publicAssetUrl(artifact, context.resources, fields);
resolved.set(artifact.resource, promise);
return promise;
};
}
/** Job statuses before a terminal one. */
const pendingStatuses: readonly string[] = ["queued", "processing"];
function endpoint(client: SumeClient, pollIntervalMs: number, maxOperationMs: number, publicAssetUrl: CreateSumeProviderOptions["publicAssetUrl"]): AsyncEndpoint {
return {
async start(context) {
try {
const route = sumeRouteForCapability(context.need.capability);
assert(route !== undefined, "Sume does not implement this exact capability");
const request = route.prepare(context.need.constraints);
await context.reportProgress?.({ phase: `Preparing Sume request: ${request.model}` });
const body = await request.compile(resolverFor(client, context, publicAssetUrl));
await context.reportProgress?.({ phase: `Submitting Sume request: ${request.model}` });
let response: Record<string, unknown>;
try {
response = await client.json(request.media === "image" ? "/v1/images" : "/v1/video-router/generate", apiKey(context.credentials), {
method: "POST",
headers: { "content-type": "application/json", "idempotency-key": context.operation },
body: JSON.stringify(body),
});
} catch (error) {
const unknown = unknownSubmission(error);
if (unknown !== undefined) return unknown;
throw error;
}
const job = response.job === undefined ? undefined : object(response.job, "Sume job");
const jobId = job?.id ?? response.request_id;
if (typeof jobId !== "string" || jobId.length === 0) throw new EndpointResponseError("Sume response has no job id");
const handle: Handle = { contract: "hypit.sume-operation@1", jobId, route: route.key, startedAt: Date.now() };
const receipt = { id: jobId };
await context.checkpoint?.({ handle: canonicalize(handle), receipt });
return { ...wakeAfter(canonicalize(handle), pollIntervalMs, Date.now(), { phase: "submitted" }), receipt };
} catch (error) {
return failure(error);
}
},
async poll(context) {
let operationDeadlineAt: number | undefined;
try {
const handle = object(context.handle, "Sume handle") as unknown as Handle;
const route = sumeRouteForCapability(context.need.capability);
assert(route !== undefined && handle.contract === "hypit.sume-operation@1" && handle.route === route.key, "Sume handle is invalid");
const receipt = { id: handle.jobId };
operationDeadlineAt = handle.startedAt + maxOperationMs;
if (Date.now() >= operationDeadlineAt) {
return { status: "failed", receipt, failure: { code: "SUME_OPERATION_TIMEOUT", message: `Sume job ${handle.jobId} exceeded this Provider's operationTimeoutMs (${maxOperationMs}); remote outcome is unknown` } };
}
const key = apiKey(context.credentials);
const id = encodeURIComponent(handle.jobId);
const state = await client.json(`/v1/jobs/${id}/status`, key);
const status = String(state.sume_status);
if (pendingStatuses.includes(status)) {
return { ...wakeAfter(canonicalize(handle), pollIntervalMs, Date.now(), { phase: status }), receipt };
}
const job = state.job === undefined || state.job === null ? undefined : object(state.job, "Sume job");
const rejected = sumeJobFailure(status, job, handle.jobId);
if (rejected !== undefined) return { ...failure(rejected), receipt };
if (status !== "completed") throw new EndpointResponseError(`Sume returned unknown job status ${status}`);
const result = object((await client.json(`/v1/jobs/${id}/result`, key)).result, "Sume job result");
const artifacts = Array.isArray(result.artifacts) ? result.artifacts.map((item, index) => object(item, `Sume artifact ${index + 1}`)) : [];
const media = artifacts.filter((item) => item.type === route.result);
if (media.length === 0) throw new EndpointResponseError(`Sume job completed without ${route.result} artifacts`);
const urls = media.map((item, index) => httpsUrl(item.url, `Sume ${route.result} artifact ${index + 1}`));
return { status: "ready", handle: canonicalize({ ...handle, urls }), receipt };
} catch (error) {
return retrying(error, context.handle, pollIntervalMs, operationDeadlineAt) ?? failure(error);
}
},
async collect(context) {
try {
const handle = object(context.handle, "Sume handle") as unknown as Handle;
const route = sumeRouteForCapability(context.need.capability);
assert(route !== undefined && handle.route === route.key && Array.isArray(handle.urls), "Sume collection route differs");
await context.reportProgress?.({ phase: "Receiving generated files" });
const blobs: BlobRef[] = [];
for (const url of handle.urls) {
const downloaded = await client.download(url);
blobs.push(await context.resources.put(downloaded.bytes, downloaded.mediaType));
}
return { status: "completed", result: { value: route.packageResult(blobs) }, receipt: { id: handle.jobId } };
} catch (error) {
return retrying(error, context.handle, pollIntervalMs) ?? failure(error);
}
},
};
}
export function createSumeProvider(options: CreateSumeProviderOptions = {}) {
const requestTimeoutMs = options.requestTimeoutMs ?? 300_000;
const operationTimeoutMs = options.operationTimeoutMs ?? 30 * 60_000;
for (const [name, value] of Object.entries({ requestTimeoutMs, operationTimeoutMs })) {
assert(Number.isSafeInteger(value) && value > 0, `Sume ${name} must be a positive integer`);
}
const client = new SumeClient(apiBaseUrl(options.baseUrl ?? "https://api.sume.com"), requestTimeoutMs, options.fetch ?? globalThis.fetch);
const asyncEndpoint = endpoint(client, options.pollIntervalMs ?? 5_000, operationTimeoutMs, options.publicAssetUrl);
return defineEndpoint({
instance: options.instance ?? "sume.default", pool: options.pool ?? options.instance ?? "sume.default",
pricing: { kind: "page", url: "https://www.sume.com/pricing/api" },
credentials: { apiKey: options.apiKey ?? credentialRef("local", "sume.api-key") },
credentialInputs: { apiKey: { label: "Sume API key" } },
defaultConcurrency: options.defaultConcurrency ?? 2,
...(options.actionLimits === undefined ? {} : { actionLimits: options.actionLimits }),
capabilities: sumeRoutes.map((route) => ({
capability: route.capability, returns: route.returns, lifecycle: "asynchronous" as const, endpoint: asyncEndpoint, capacity: route.capability.name, supports: route.supports,
})),
});
}

View File

@@ -0,0 +1,166 @@
import {
compileWireRequest,
generationTypes,
sealGeneratedImageSet,
sealGeneratedVideoSet,
selectWireModelForRequest,
} from "@hypit/generation";
import type { GenerationArtifactUrlResolver, GenerationRequest, GenerationWireMapping } from "@hypit/generation";
import { canonicalize } from "@hypit/hypit/protocol";
import type { BlobRef, CanonicalValue, CapabilityRef, StoredValue, TypeRef } from "@hypit/hypit/protocol";
import type { EndpointRequest, EndpointSupport } from "@hypit/hypit/endpoint";
import { sumeMappings } from "./mapping.js";
export type SumePreparedRequest = {
readonly model: string;
readonly media: "image" | "video";
readonly compile: (resolve: GenerationArtifactUrlResolver) => Promise<Record<string, unknown>>;
};
export type SumeRoute = GenerationWireMapping & {
readonly key: string;
readonly returns: TypeRef;
readonly supports: (request: EndpointRequest) => EndpointSupport;
readonly prepare: (constraints: CanonicalValue) => SumePreparedRequest;
readonly packageResult: (artifacts: readonly BlobRef[]) => StoredValue;
};
function scalar(request: GenerationRequest, port: string): string | number | boolean | undefined {
const value = request.ports[port]?.[0];
return typeof value === "string" || typeof value === "number" || typeof value === "boolean" ? value : undefined;
}
function count(request: GenerationRequest, port: string): number {
return request.ports[port]?.length ?? 0;
}
/** Ratios the model packages offer that the Sume catalog row for the model does not list. */
const UNLISTED_RATIOS: Readonly<Record<string, readonly string[]>> = {
"grok-imagine-video-1.5-preview": ["2:3", "3:2"],
"pixverse-v6": ["2:3", "3:2", "auto"],
"pixverse-c1": ["2:3", "3:2"],
"gpt-image-2": ["3:2", "2:3", "2:1", "1:2", "3:1", "1:3", "21:9", "9:21"],
"seedream-5-lite": ["21:9"],
};
/** Sume's per-model input ranges, checked before any reference is uploaded. */
function rejection(mapping: GenerationWireMapping, request: GenerationRequest): string | undefined {
const { name } = mapping.capability;
const model = selectWireModelForRequest(mapping, request);
const ratio = scalar(request, "aspectRatio");
if (ratio !== undefined && UNLISTED_RATIOS[name]?.includes(String(ratio)) === true) {
return `Sume ${model} does not render aspect ratio ${String(ratio)}`;
}
if (mapping.capability.module.name === "@hypit/seedance") {
if (scalar(request, "webSearch") === true) return `Sume ${model} has no web_search field`;
if (name === "seedance-2.5" && scalar(request, "duration") === -1) {
return "Sume seedance-2.5 takes a duration of 4 to 30 seconds, not automatic";
}
}
if (name === "minimax-h3" && scalar(request, "duration") === 4) {
return "Sume minimax-h3 takes a duration of 5 to 15 seconds";
}
if (name === "grok-imagine-video-1.5-preview") {
if (count(request, "images") > 1) return "Sume grok-imagine-video-1.5 takes a prompt alone or one first-frame image";
if (ratio === "auto" && count(request, "images") === 0) {
return "Sume grok-imagine-video-1.5 needs an explicit aspect ratio without a first frame";
}
}
if (name === "pixverse-v6" || name === "pixverse-c1") {
if (count(request, "firstFrame") > 0 || count(request, "lastFrame") > 0) {
return `Sume ${model} takes a prompt or reference images, not frames`;
}
if (count(request, "referenceVideo") > 0) return `Sume ${model} takes no reference videos`;
if (scalar(request, "multiClip") === true) return `Sume ${model} has no multi-clip field`;
if (scalar(request, "seed") !== undefined) return `Sume ${model} takes no seed`;
if (scalar(request, "duration") === 1) return `Sume ${model} takes a duration of 2 to 15 seconds`;
}
if (name === "gpt-image-2") {
if (scalar(request, "background") !== undefined) return "Sume openai/gpt-image-2 has no background field";
if (count(request, "images") > 10) return "Sume openai/gpt-image-2 accepts up to ten reference images";
}
if (name === "nano-banana-2" && count(request, "images") > 10) {
return "Sume google/nano-banana-2.1 accepts up to ten reference images";
}
if (name === "seedream-5-lite") {
if (scalar(request, "quality") !== "basic") {
return "Sume bytedance-seed/seedream-5-lite renders at 2K only, which is quality basic";
}
if (scalar(request, "nsfwCheck") === false) {
return "Sume bytedance-seed/seedream-5-lite has no switch to turn its content check off";
}
if (count(request, "images") > 10) return "Sume bytedance-seed/seedream-5-lite accepts up to ten reference images";
}
return undefined;
}
/** Fields the model package states that Sume names differently, spells differently or omits. */
function normalize(mapping: GenerationWireMapping, input: Record<string, unknown>): Record<string, unknown> {
const { name } = mapping.capability;
if (input.web_search === false) delete input.web_search;
// MiniMax spells its native tier 768P; Sume's catalog lists 768p.
if (name === "minimax-h3" && input.resolution === "768P") input.resolution = "768p";
if (name === "grok-imagine-video-1.5-preview") {
const images = input.images as readonly string[] | undefined;
delete input.images;
// A first frame keeps its own shape, and Sume applies aspect_ratio to text-to-video only.
if (images !== undefined && images.length > 0) {
input.image_url = images[0];
delete input.aspect_ratio;
}
}
if (name === "pixverse-v6" && input.multi_clip === false) delete input.multi_clip;
if (mapping.result === "image") {
const images = input.images as readonly string[] | undefined;
delete input.images;
if (images !== undefined && images.length > 0) {
input.input_references = images.map((url) => ({ type: "image_url", image_url: { url } }));
}
}
if (mapping.capability.module.name === "@hypit/nano-banana" && input.output_format === "jpg") input.output_format = "jpeg";
if (name === "seedream-5-lite") {
delete input.quality;
delete input.nsfw_check;
}
return input;
}
function capabilityKey(capability: CapabilityRef): string {
return `${capability.module.name}@${capability.module.version}#${capability.name}`;
}
export const sumeRoutes: readonly SumeRoute[] = sumeMappings.map((mapping) => ({
...mapping,
key: capabilityKey(mapping.capability),
returns: mapping.result === "image" ? generationTypes.imageSet : generationTypes.videoSet,
supports: (request) => {
const reason = rejection(mapping, request.constraints as unknown as GenerationRequest);
return reason === undefined ? { status: "supported" } : { status: "unsupported", reason };
},
prepare: (constraints) => {
const request = constraints as unknown as GenerationRequest;
const reason = rejection(mapping, request);
if (reason !== undefined) throw new Error(reason);
const model = selectWireModelForRequest(mapping, request);
return {
model,
media: mapping.result === "image" ? "image" : "video",
compile: async (resolve) => ({
model,
...normalize(mapping, (await compileWireRequest(mapping, request, resolve)).input as Record<string, unknown>),
mode: "async",
}),
};
},
packageResult: (artifacts) => ({
kind: "inline",
value: canonicalize(mapping.result === "image"
? sealGeneratedImageSet({ images: artifacts })
: sealGeneratedVideoSet({ videos: artifacts })),
}),
}));
const byCapability = new Map(sumeRoutes.map((route) => [route.key, route]));
export function sumeRouteForCapability(capability: CapabilityRef): SumeRoute | undefined {
return byCapability.get(capabilityKey(capability));
}

View File

@@ -0,0 +1,22 @@
import test from "node:test";
import { assertMappingCoversPorts } from "@hypit/generation";
import { gptImage2Ports } from "@hypit/gpt-image";
import { grokImaginePorts } from "@hypit/grok-imagine";
import { minimaxH3Ports } from "@hypit/minimax-h3";
import { nanoBananaPorts } from "@hypit/nano-banana";
import { pixversePorts } from "@hypit/pixverse";
import { seedancePorts } from "@hypit/seedance";
import { seedream5LitePorts } from "@hypit/seedream";
import { sumeMappings } from "../src/mapping.js";
// A port added to one of these model packages must be mapped here before Sume can serve it.
test("every Sume mapping covers its model's ports", () => {
const tables = {
...seedancePorts, ...grokImaginePorts, ...nanoBananaPorts, ...pixversePorts,
"minimax-h3": minimaxH3Ports, "gpt-image-2": gptImage2Ports, "seedream-5-lite": seedream5LitePorts,
};
for (const mapping of sumeMappings) {
assertMappingCoversPorts(tables[mapping.capability.name as keyof typeof tables], mapping);
}
});

46
pnpm-lock.yaml generated
View File

@@ -201,6 +201,9 @@ importers:
'@hypit/provider-pollo':
specifier: workspace:*
version: link:packages/provider-pollo
'@hypit/provider-sume':
specifier: workspace:*
version: link:packages/provider-sume
'@hypit/provider-tokendance':
specifier: workspace:*
version: link:packages/provider-tokendance
@@ -2111,6 +2114,49 @@ importers:
specifier: workspace:*
version: link:../runtime-local
packages/provider-sume:
dependencies:
'@hypit/generation':
specifier: workspace:^0.1.0
version: link:../generation
devDependencies:
'@hypit/endpoint':
specifier: workspace:*
version: link:../endpoint
'@hypit/gpt-image':
specifier: workspace:*
version: link:../gpt-image
'@hypit/grok-imagine':
specifier: workspace:*
version: link:../grok-imagine
'@hypit/hypit':
specifier: workspace:*
version: link:../..
'@hypit/minimax-h3':
specifier: workspace:*
version: link:../minimax-h3
'@hypit/nano-banana':
specifier: workspace:*
version: link:../nano-banana
'@hypit/pixverse':
specifier: workspace:*
version: link:../pixverse
'@hypit/protocol':
specifier: workspace:*
version: link:../protocol
'@hypit/runtime':
specifier: workspace:*
version: link:../runtime
'@hypit/runtime-local':
specifier: workspace:*
version: link:../runtime-local
'@hypit/seedance':
specifier: workspace:*
version: link:../seedance
'@hypit/seedream':
specifier: workspace:*
version: link:../seedream
packages/provider-tokendance:
dependencies:
'@hypit/generation':

View File

@@ -29,6 +29,7 @@ const packageDirectories = [
"provider-hiapi",
"provider-monid",
"provider-pollo",
"provider-sume",
"provider-tokendance",
];
const packageNames = packageDirectories.map((name) => `@hypit/${name}`);
@@ -45,6 +46,7 @@ const optionalPackages = [
"@hypit/provider-hiapi",
"@hypit/provider-monid",
"@hypit/provider-pollo",
"@hypit/provider-sume",
"@hypit/provider-tokendance",
];

View File

@@ -56,6 +56,7 @@ npm install @hypit/provider-tokendance
npm install @hypit/provider-hiapi
npm install @hypit/provider-pollo
npm install @hypit/provider-beatapi
npm install @hypit/provider-sume
npm install @hypit/provider-monid
```