mirror of
https://github.com/hypit-ai/hypit.git
synced 2026-11-16 14:50:55 +08:00
feat(sume): serve the installed video and image models on Sume (#485)
`@hypit/provider-sume` submits one asynchronous job to `https://api.sume.com` with a Bearer API key, polls `GET /v1/jobs/{id}/status` until it is terminal, reads the artifacts from `GET /v1/jobs/{id}/result` and stores them in the current Build. It maps twelve models the Distribution already describes: the four Seedance variants, MiniMax H3, Grok Imagine 1.5, PixVerse V6 and C1, GPT Image 2, Nano Banana 2 and Pro, and Seedream 5.0 lite. It adds no model package. Video jobs post to `/v1/video-router/generate`, whose flat `image_url`, `end_image_url` and `reference_*_urls` fields take the mapping as data. `/v1/videos` rejects `aspect_ratio: adaptive` for Seedance, which the router accepts. Image jobs post to `/v1/images` with `mode: "async"`, and routes.ts builds `input_references` from the reference URLs. Sume serves Nano Banana 2 as Nano Banana 2.1, so `nano-banana-2` runs on `google/nano-banana-2.1`. References are uploaded through `POST /v1/files/generate-upload-url`: the bytes are PUT to the presigned URL and the returned `media.sume.com` URL travels in the job. Service limits are reported as unsupported before any upload: Seedance web search and automatic 2.5 length, MiniMax H3 under 5 seconds, Grok 1.5 with more than one image, PixVerse frames, reference videos, seed and multi-clip, GPT Image 2 background, ratios a catalog row does not list, more than ten image references, and Seedream quality above basic or with its content check off. A failed job keeps Sume's code, message and the upstream provider message. `defaultConcurrency` is 2, the number of jobs Sume's free plan runs at once. Live runs through the Provider's start/poll/collect against a free-plan key all completed: text-to-image and reference editing on every image model, text-to-video, first frame, first and last frame, reference images, and reference video with reference image and audio on Seedance 2.5, Seedance 2 mini and MiniMax H3. A `hypit build` chaining Seedream into Nano Banana Pro through a Runtime Profile completed as well. Total spend was about $3.98.
This commit is contained in:
@@ -25,6 +25,7 @@ npm install @hypit/provider-tokendance
|
||||
npm install @hypit/provider-hiapi
|
||||
npm install @hypit/provider-pollo
|
||||
npm install @hypit/provider-beatapi
|
||||
npm install @hypit/provider-sume
|
||||
npm install @hypit/provider-monid
|
||||
```
|
||||
|
||||
@@ -64,6 +65,14 @@ and uploads reference media through BeatAPI's file endpoint. Its
|
||||
[video](https://docs.beatapi.io/video-api) and [image](https://docs.beatapi.io/image-api) model
|
||||
indexes list the rest.
|
||||
|
||||
### Sume
|
||||
|
||||
[Sume](https://www.sume.com) offers video and image models through one job API.
|
||||
[`@hypit/provider-sume`](https://github.com/hypit-ai/hypit/blob/main/packages/provider-sume/README.md)
|
||||
serves the Seedance 2.0 and 2.5 models, MiniMax H3, Grok Imagine 1.5, PixVerse V6 and C1, GPT Image
|
||||
2, Nano Banana 2.1 and Pro and Seedream 5.0 lite, and uploads reference media through Sume's upload
|
||||
URL. Its [developer documentation](https://docs.sume.com/llms.txt) and live model catalog list the rest.
|
||||
|
||||
### Monid
|
||||
|
||||
[Monid](https://monid.ai) is a Hypit service partner.
|
||||
|
||||
@@ -22,6 +22,7 @@ npm install @hypit/provider-tokendance
|
||||
npm install @hypit/provider-hiapi
|
||||
npm install @hypit/provider-pollo
|
||||
npm install @hypit/provider-beatapi
|
||||
npm install @hypit/provider-sume
|
||||
npm install @hypit/provider-monid
|
||||
```
|
||||
|
||||
@@ -56,6 +57,14 @@ Pollo 只接受公网 URL 形式的参考素材,带参考素材的请求需要
|
||||
并通过 BeatAPI 的文件接口上传参考素材。
|
||||
其余模型见它的[视频](https://docs.beatapi.io/video-api)与[图片](https://docs.beatapi.io/image-api)模型索引。
|
||||
|
||||
### Sume
|
||||
|
||||
[Sume](https://www.sume.com) 通过一个任务接口提供视频和图片模型。
|
||||
[`@hypit/provider-sume`](https://github.com/hypit-ai/hypit/blob/main/packages/provider-sume/README.md)
|
||||
提供 Seedance 2.0、2.5 系列、MiniMax H3、Grok Imagine 1.5、PixVerse V6 与 C1、GPT Image 2、
|
||||
Nano Banana 2.1 与 Pro、Seedream 5.0 lite,并通过 Sume 的上传地址上传参考素材。
|
||||
其余模型见它的[开发者文档](https://docs.sume.com/llms.txt)与实时模型目录。
|
||||
|
||||
### Monid
|
||||
|
||||
[Monid](https://monid.ai) 是 Hypit 的服务合作方。
|
||||
|
||||
@@ -284,6 +284,7 @@
|
||||
"@hypit/media-local": "workspace:*",
|
||||
"@hypit/provider-monid": "workspace:*",
|
||||
"@hypit/provider-pollo": "workspace:*",
|
||||
"@hypit/provider-sume": "workspace:*",
|
||||
"@hypit/provider-tokendance": "workspace:*",
|
||||
"@hypit/provider-whisperx-local": "workspace:*",
|
||||
"@hypit/ranking": "workspace:*",
|
||||
|
||||
106
packages/provider-sume/README.md
Normal file
106
packages/provider-sume/README.md
Normal file
@@ -0,0 +1,106 @@
|
||||
# `@hypit/provider-sume`
|
||||
|
||||
Optional Provider, not installed with Hypit. Install in the video project:
|
||||
|
||||
```bash
|
||||
npm install @hypit/provider-sume
|
||||
```
|
||||
|
||||
Hypit Runtime Provider for a [Sume](https://docs.sume.com/public-api) API key. Each capability
|
||||
submits one asynchronous job to `https://api.sume.com` with a `Bearer` API key, polls
|
||||
`GET /v1/jobs/{id}/status` until the job is terminal, reads the generated files from
|
||||
`GET /v1/jobs/{id}/result` and stores them in the current Build.
|
||||
|
||||
| Capability | Sume model | Submit path |
|
||||
| --- | --- | --- |
|
||||
| `@hypit/seedance@1#seedance-2` | `seedance-2` | `/v1/video-router/generate` |
|
||||
| `@hypit/seedance@1#seedance-2-fast` | `seedance-2-fast` | `/v1/video-router/generate` |
|
||||
| `@hypit/seedance@1#seedance-2-mini` | `seedance-2-mini` | `/v1/video-router/generate` |
|
||||
| `@hypit/seedance@1#seedance-2.5` | `seedance-2.5` | `/v1/video-router/generate` |
|
||||
| `@hypit/minimax-h3@1#minimax-h3` | `minimax-h3` | `/v1/video-router/generate` |
|
||||
| `@hypit/grok-imagine@1#grok-imagine-video-1.5-preview` | `grok-imagine-video-1.5` | `/v1/video-router/generate` |
|
||||
| `@hypit/pixverse@1#pixverse-v6` | `pixverse-v6` | `/v1/video-router/generate` |
|
||||
| `@hypit/pixverse@1#pixverse-c1` | `pixverse-c1` | `/v1/video-router/generate` |
|
||||
| `@hypit/gpt-image@1#gpt-image-2` | `openai/gpt-image-2` | `/v1/images` |
|
||||
| `@hypit/nano-banana@1#nano-banana-2` | `google/nano-banana-2.1` | `/v1/images` |
|
||||
| `@hypit/nano-banana@1#nano-banana-pro` | `google/nano-banana-pro` | `/v1/images` |
|
||||
| `@hypit/seedream@1#seedream-5-lite` | `bytedance-seed/seedream-5-lite` | `/v1/images` |
|
||||
|
||||
Sume's [video](https://docs.sume.com/models/videos.md) and [image](https://docs.sume.com/models/images.md)
|
||||
catalogs list further models; this Provider maps only the models the Distribution already describes.
|
||||
Sume serves Nano Banana 2 as its successor, Nano Banana 2.1, so a `nano-banana-2` request renders on
|
||||
`google/nano-banana-2.1`.
|
||||
|
||||
Video jobs carry the opening and closing frames as `image_url` and `end_image_url`, and references as
|
||||
`reference_image_urls`, `reference_video_urls` and `reference_audio_urls`. Image jobs carry their
|
||||
references as `input_references`. Every submission uses `mode: "async"`. `duration`, `aspect_ratio`,
|
||||
`resolution` and `generate_audio` keep their authored values, except that MiniMax H3's `768P` is sent
|
||||
as `768p`, PixVerse's `quality` travels as `resolution`, and Nano Banana's `jpg` output format is sent
|
||||
as `jpeg`. A Grok Imagine 1.5 image travels as the first frame, and the first frame then sets the
|
||||
shape, so the authored aspect ratio is not sent with it.
|
||||
|
||||
Service limits this Provider reports as unsupported before uploading any reference:
|
||||
|
||||
- Seedance has no `web_search` field. An authored `false` is dropped; an authored `true` is refused.
|
||||
- Seedance 2.5 takes 4 to 30 seconds, so `duration="-1"` for an automatic length is refused.
|
||||
- MiniMax H3 takes 5 to 15 seconds.
|
||||
- Grok Imagine 1.5 takes a prompt alone or one first-frame image, does not render `2:3` or `3:2`, and
|
||||
needs an explicit aspect ratio without an image.
|
||||
- PixVerse V6 and C1 take a prompt or reference images, without frames, reference videos, a seed or
|
||||
multi-clip, for 2 to 15 seconds, and do not render `2:3`, `3:2` or `auto`.
|
||||
- GPT Image 2 has no `background` field, accepts up to ten reference images and does not render
|
||||
`3:2`, `2:3`, `2:1`, `1:2`, `3:1`, `1:3`, `21:9` or `9:21`.
|
||||
- Nano Banana 2.1 accepts up to ten reference images.
|
||||
- Seedream 5.0 lite renders at 2K, which is `quality="basic"`; accepts up to ten reference images;
|
||||
does not render `21:9`; and has no switch to turn its content check off, so `nsfw-check="false"` is
|
||||
refused.
|
||||
|
||||
Seedance visual references carry `person-reference`; the Provider accepts the declaration and
|
||||
transmits nothing for it, since Sume's request body has no field for it.
|
||||
|
||||
Referenced Resources are uploaded through `POST /v1/files/generate-upload-url`: the Provider PUTs the
|
||||
bytes to the returned presigned URL and sends the returned public `media.sume.com` URL in the job. One
|
||||
Resource is uploaded once per Runtime operation. Embedded callers may replace that transport with
|
||||
`publicAssetUrl(artifact, resources, fields)`, which must return a public HTTPS URL Sume can fetch.
|
||||
|
||||
Runtime Profile example:
|
||||
|
||||
```json
|
||||
{
|
||||
"format": "hypit.runtime-local@1",
|
||||
"dataRoot": ".hypit/runtimes/local",
|
||||
"credentials": {
|
||||
"local": {
|
||||
"use": "@hypit/credential-store-local"
|
||||
}
|
||||
},
|
||||
"endpoints": {
|
||||
"sume.default": {
|
||||
"use": "@hypit/provider-sume",
|
||||
"pool": "sume.default",
|
||||
"config": {
|
||||
"apiKey": { "store": "local", "key": "sume.api-key" },
|
||||
"defaultConcurrency": 2,
|
||||
"pollIntervalMs": 5000
|
||||
}
|
||||
}
|
||||
},
|
||||
"bindings": {}
|
||||
}
|
||||
```
|
||||
|
||||
`baseUrl` defaults to `https://api.sume.com`. Store the API key with
|
||||
`hypit auth login sume.default --runtime hypit.runtime.json`. `defaultConcurrency` defaults to 2,
|
||||
which is the number of jobs Sume's free plan runs at once; raise it to the workspace's plan limit.
|
||||
Optional `requestTimeoutMs`, `operationTimeoutMs` and `actionLimits` bound single HTTP calls, the
|
||||
whole remote job and action concurrency. Each submission sends the Runtime operation as its
|
||||
`Idempotency-Key`, so a retried submission of the same body resolves to the same job and is charged
|
||||
once.
|
||||
|
||||
HTTP failures keep Sume's `error.code`, message, `details`, `request_id` and any
|
||||
`retry_after_seconds`; a failed or canceled job keeps its error code, message and the upstream
|
||||
provider's message. Any URL inside a surfaced reason is redacted, since an upload or result URL is an
|
||||
access capability rather than diagnostic content. Polling transport failures and HTTP 429/5xx
|
||||
preserve the same job. Sume's `retry_after_seconds` takes precedence over the standard `Retry-After`
|
||||
header; either wait is capped at the original operation deadline, where the next poll concludes
|
||||
timeout without querying early.
|
||||
39
packages/provider-sume/package.json
Normal file
39
packages/provider-sume/package.json
Normal file
@@ -0,0 +1,39 @@
|
||||
{
|
||||
"name": "@hypit/provider-sume",
|
||||
"version": "0.1.0",
|
||||
"license": "SEE LICENSE IN LICENSE",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"exports": {
|
||||
".": "./src/index.ts"
|
||||
},
|
||||
"files": [
|
||||
"dist",
|
||||
"README.md"
|
||||
],
|
||||
"hypit": {
|
||||
"activation": "./src/activation.ts"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@hypit/hypit": "workspace:^0.3.1",
|
||||
"@hypit/runtime-local": "workspace:^0.1.1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@hypit/endpoint": "workspace:*",
|
||||
"@hypit/generation": "workspace:*",
|
||||
"@hypit/gpt-image": "workspace:*",
|
||||
"@hypit/grok-imagine": "workspace:*",
|
||||
"@hypit/hypit": "workspace:*",
|
||||
"@hypit/minimax-h3": "workspace:*",
|
||||
"@hypit/nano-banana": "workspace:*",
|
||||
"@hypit/pixverse": "workspace:*",
|
||||
"@hypit/protocol": "workspace:*",
|
||||
"@hypit/runtime": "workspace:*",
|
||||
"@hypit/runtime-local": "workspace:*",
|
||||
"@hypit/seedance": "workspace:*",
|
||||
"@hypit/seedream": "workspace:*"
|
||||
},
|
||||
"dependencies": {
|
||||
"@hypit/generation": "workspace:^0.1.0"
|
||||
}
|
||||
}
|
||||
62
packages/provider-sume/src/activation.ts
Normal file
62
packages/provider-sume/src/activation.ts
Normal file
@@ -0,0 +1,62 @@
|
||||
import {
|
||||
createRuntimeEndpointAdapterFacet,
|
||||
runtimeConfigCredentialRef,
|
||||
runtimeConfigActionLimits,
|
||||
runtimeConfigExact,
|
||||
runtimeConfigObject,
|
||||
runtimeConfigPositiveInteger,
|
||||
runtimeConfigString,
|
||||
} from "@hypit/runtime-local/extension";
|
||||
|
||||
import { createSumeProvider } from "./provider.js";
|
||||
|
||||
const adapter = createRuntimeEndpointAdapterFacet({
|
||||
use: "@hypit/provider-sume",
|
||||
activate(context) {
|
||||
if (context.pool === undefined) throw new Error("Sume Provider Pool is required");
|
||||
const config = runtimeConfigObject(context.config, "Sume");
|
||||
runtimeConfigExact(config, [
|
||||
"baseUrl",
|
||||
"apiKey",
|
||||
"defaultConcurrency",
|
||||
"actionLimits",
|
||||
"pollIntervalMs",
|
||||
"requestTimeoutMs",
|
||||
"operationTimeoutMs",
|
||||
], "Sume");
|
||||
const baseUrl = runtimeConfigString(config.baseUrl, "Sume baseUrl");
|
||||
if (baseUrl !== undefined) {
|
||||
const url = new URL(baseUrl);
|
||||
if (url.protocol !== "https:" && url.hostname !== "localhost" && url.hostname !== "127.0.0.1") {
|
||||
throw new Error("Sume baseUrl must use HTTPS or loopback");
|
||||
}
|
||||
}
|
||||
const apiKey = runtimeConfigCredentialRef(config.apiKey, "Sume apiKey");
|
||||
if (apiKey === undefined) throw new Error("Sume apiKey CredentialRef is required");
|
||||
const actionLimits = runtimeConfigActionLimits(config.actionLimits);
|
||||
const defaultConcurrency = runtimeConfigPositiveInteger(config.defaultConcurrency, "Sume defaultConcurrency");
|
||||
const pollIntervalMs = runtimeConfigPositiveInteger(config.pollIntervalMs, "Sume pollIntervalMs");
|
||||
const requestTimeoutMs = runtimeConfigPositiveInteger(config.requestTimeoutMs, "Sume requestTimeoutMs");
|
||||
const operationTimeoutMs = runtimeConfigPositiveInteger(config.operationTimeoutMs, "Sume operationTimeoutMs");
|
||||
return {
|
||||
endpoint: createSumeProvider({
|
||||
instance: context.instance,
|
||||
pool: context.pool,
|
||||
...(baseUrl === undefined ? {} : { baseUrl }),
|
||||
apiKey,
|
||||
...(defaultConcurrency === undefined ? {} : { defaultConcurrency }),
|
||||
...(actionLimits === undefined ? {} : { actionLimits }),
|
||||
...(pollIntervalMs === undefined ? {} : { pollIntervalMs }),
|
||||
...(requestTimeoutMs === undefined ? {} : { requestTimeoutMs }),
|
||||
...(operationTimeoutMs === undefined ? {} : { operationTimeoutMs }),
|
||||
}),
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
export const hypitPackage = {
|
||||
format: "hypit.package@1" as const,
|
||||
facets: [adapter],
|
||||
};
|
||||
|
||||
export default hypitPackage;
|
||||
53
packages/provider-sume/src/errors.ts
Normal file
53
packages/provider-sume/src/errors.ts
Normal file
@@ -0,0 +1,53 @@
|
||||
import { EndpointServiceError } from "@hypit/hypit/endpoint";
|
||||
import { EndpointHttpError, retryAfterMs } from "@hypit/hypit/endpoint/http";
|
||||
|
||||
/** Sume's `{ error: { code, message, request_id, retry_after_seconds, details } }` envelope and failed jobs. */
|
||||
function record(value: unknown): Record<string, unknown> | undefined {
|
||||
return value !== null && typeof value === "object" && !Array.isArray(value)
|
||||
? value as Record<string, unknown> : undefined;
|
||||
}
|
||||
function text(value: unknown): string | undefined {
|
||||
return typeof value === "string" && value.trim().length > 0 ? value.trim() : undefined;
|
||||
}
|
||||
|
||||
// Responses may carry a signed upload or result URL. Keep the reason, not its access capability.
|
||||
export function safeSumeReason(value: string): string {
|
||||
return value.replace(/https?:\/\/\S+/giu, "[redacted-url]");
|
||||
}
|
||||
|
||||
export class SumeServiceError extends EndpointServiceError {}
|
||||
|
||||
export class SumeHttpError extends EndpointHttpError {
|
||||
constructor(status: number, response: { readonly headers: Headers }, bodyText: string,
|
||||
request: { readonly method: string; readonly path: string; readonly model?: string }) {
|
||||
let body: Record<string, unknown> | undefined;
|
||||
try { body = record(JSON.parse(bodyText)); } catch { /* Non-JSON gateway failures still have HTTP evidence. */ }
|
||||
const error = record(body?.error);
|
||||
const code = text(error?.code) ?? "SUME_HTTP_ERROR";
|
||||
const reason = text(error?.message) ?? (error === undefined ? text(bodyText.slice(0, 2000)) : undefined);
|
||||
const details = error?.details === undefined ? undefined : JSON.stringify(error.details).slice(0, 1000);
|
||||
const requestId = text(error?.request_id) ?? text(response.headers.get("x-sume-request-id"));
|
||||
const serviceRetryAfter = typeof error?.retry_after_seconds === "number" ? Math.round(error.retry_after_seconds * 1000) : undefined;
|
||||
const retryAfter = serviceRetryAfter ?? retryAfterMs(response.headers);
|
||||
const facts = [
|
||||
`Sume HTTP ${status}`, code, `${request.method} ${request.path}`,
|
||||
...(request.model === undefined ? [] : [`model=${request.model}`]),
|
||||
...(requestId === undefined ? [] : [`request=${requestId}`]),
|
||||
...(retryAfter === undefined ? [] : [`retry-after=${retryAfter}ms`]),
|
||||
];
|
||||
const message = [reason, details].filter((item) => item !== undefined).join(" ");
|
||||
super(code, `${facts.join("; ")}${message.length === 0 ? "" : `: ${safeSumeReason(message)}`}`,
|
||||
status, retryAfter);
|
||||
}
|
||||
}
|
||||
|
||||
/** A terminal `failed` or `canceled` job; `undefined` otherwise. */
|
||||
export function sumeJobFailure(status: string, job: Record<string, unknown> | undefined, id: string): SumeServiceError | undefined {
|
||||
if (status !== "failed" && status !== "canceled") return undefined;
|
||||
const error = record(job?.error);
|
||||
const code = text(error?.code) ?? text(error?.public_reason) ?? (status === "canceled" ? "SUME_JOB_CANCELED" : "SUME_JOB_FAILED");
|
||||
const upstream = text(record(error?.details)?.provider_error_message);
|
||||
const reason = [text(error?.message) ?? text(job?.error), upstream].filter((item) => item !== undefined).join(" ") || undefined;
|
||||
return new SumeServiceError(code,
|
||||
`Sume job ${id} ${status}; ${code}${reason === undefined ? "" : `: ${safeSumeReason(reason)}`}`);
|
||||
}
|
||||
2
packages/provider-sume/src/index.ts
Normal file
2
packages/provider-sume/src/index.ts
Normal file
@@ -0,0 +1,2 @@
|
||||
export { createSumeProvider, sumeProviderModuleRef } from "./provider.js";
|
||||
export type { CreateSumeProviderOptions } from "./provider.js";
|
||||
127
packages/provider-sume/src/mapping.ts
Normal file
127
packages/provider-sume/src/mapping.ts
Normal file
@@ -0,0 +1,127 @@
|
||||
import type { ModuleRef } from "@hypit/hypit/protocol";
|
||||
import type { GenerationWireMapping } from "@hypit/generation";
|
||||
|
||||
const SEEDANCE: ModuleRef = { name: "@hypit/seedance", version: "1" };
|
||||
const MINIMAX: ModuleRef = { name: "@hypit/minimax-h3", version: "1" };
|
||||
const GROK: ModuleRef = { name: "@hypit/grok-imagine", version: "1" };
|
||||
const PIXVERSE: ModuleRef = { name: "@hypit/pixverse", version: "1" };
|
||||
const GPT_IMAGE: ModuleRef = { name: "@hypit/gpt-image", version: "1" };
|
||||
const NANO_BANANA: ModuleRef = { name: "@hypit/nano-banana", version: "1" };
|
||||
const SEEDREAM: ModuleRef = { name: "@hypit/seedream", version: "1" };
|
||||
|
||||
/**
|
||||
* Sume catalog ids and the body fields each one documents.
|
||||
*
|
||||
* Video models post to `POST /v1/video-router/generate`, whose body carries every reference as a
|
||||
* flat URL field. Image models post to `POST /v1/images`; its `input_references` array of
|
||||
* `{ type, image_url: { url } }` objects is built in routes.ts from the `images` URLs written here.
|
||||
* Ports the service has no field for are written under their own names so routes.ts can drop an
|
||||
* authored "off" value and refuse an authored "on" value before anything is uploaded.
|
||||
*/
|
||||
const seedanceFields = {
|
||||
prompt: { as: "value", field: "prompt" },
|
||||
firstFrame: { as: "url", field: "image_url", resourceFields: ["personReference"] },
|
||||
lastFrame: { as: "url", field: "end_image_url", resourceFields: ["personReference"] },
|
||||
referenceImage: { as: "urlArray", field: "reference_image_urls", resourceFields: ["personReference"] },
|
||||
referenceVideo: { as: "urlArray", field: "reference_video_urls", resourceFields: ["personReference"] },
|
||||
referenceAudio: { as: "urlArray", field: "reference_audio_urls" },
|
||||
duration: { as: "value", field: "duration" },
|
||||
aspectRatio: { as: "value", field: "aspect_ratio" },
|
||||
resolution: { as: "value", field: "resolution" },
|
||||
generateAudio: { as: "value", field: "generate_audio" },
|
||||
webSearch: { as: "value", field: "web_search" },
|
||||
} as const satisfies GenerationWireMapping["fields"];
|
||||
|
||||
const seedance = (name: string): GenerationWireMapping => ({
|
||||
capability: { module: SEEDANCE, name }, result: "video", routes: [{ model: name }], fields: seedanceFields,
|
||||
});
|
||||
|
||||
const pixverse = (name: "pixverse-v6" | "pixverse-c1"): GenerationWireMapping => ({
|
||||
capability: { module: PIXVERSE, name }, result: "video", routes: [{ model: name }],
|
||||
fields: {
|
||||
prompt: { as: "value", field: "prompt" },
|
||||
firstFrame: { as: "url", field: "image_url" },
|
||||
lastFrame: { as: "url", field: "end_image_url" },
|
||||
referenceImage: { as: "urlArray", field: "reference_image_urls" },
|
||||
...(name === "pixverse-v6" ? {
|
||||
referenceVideo: { as: "urlArray" as const, field: "reference_video_urls" },
|
||||
multiClip: { as: "value" as const, field: "multi_clip" },
|
||||
seed: { as: "value" as const, field: "seed" },
|
||||
} : {}),
|
||||
duration: { as: "value", field: "duration" },
|
||||
// PixVerse's quality band is Sume's resolution.
|
||||
quality: { as: "value", field: "resolution" },
|
||||
aspectRatio: { as: "value", field: "aspect_ratio" },
|
||||
generateAudio: { as: "value", field: "generate_audio" },
|
||||
},
|
||||
});
|
||||
|
||||
export const sumeMappings: readonly GenerationWireMapping[] = [
|
||||
seedance("seedance-2"),
|
||||
seedance("seedance-2-fast"),
|
||||
seedance("seedance-2-mini"),
|
||||
seedance("seedance-2.5"),
|
||||
{
|
||||
capability: { module: MINIMAX, name: "minimax-h3" }, result: "video",
|
||||
routes: [{ model: "minimax-h3" }],
|
||||
fields: {
|
||||
prompt: { as: "value", field: "prompt" },
|
||||
firstFrame: { as: "url", field: "image_url" },
|
||||
lastFrame: { as: "url", field: "end_image_url" },
|
||||
referenceImage: { as: "urlArray", field: "reference_image_urls" },
|
||||
referenceVideo: { as: "urlArray", field: "reference_video_urls" },
|
||||
referenceAudio: { as: "urlArray", field: "reference_audio_urls" },
|
||||
duration: { as: "value", field: "duration" },
|
||||
aspectRatio: { as: "value", field: "aspect_ratio" },
|
||||
resolution: { as: "value", field: "resolution" },
|
||||
},
|
||||
},
|
||||
{
|
||||
capability: { module: GROK, name: "grok-imagine-video-1.5-preview" }, result: "video",
|
||||
routes: [{ model: "grok-imagine-video-1.5" }],
|
||||
fields: {
|
||||
prompt: { as: "value", field: "prompt" },
|
||||
images: { as: "urlArray", field: "images" },
|
||||
duration: { as: "value", field: "duration" },
|
||||
aspectRatio: { as: "value", field: "aspect_ratio" },
|
||||
resolution: { as: "value", field: "resolution" },
|
||||
},
|
||||
},
|
||||
pixverse("pixverse-v6"),
|
||||
pixverse("pixverse-c1"),
|
||||
{
|
||||
capability: { module: GPT_IMAGE, name: "gpt-image-2" }, result: "image",
|
||||
routes: [{ model: "openai/gpt-image-2" }],
|
||||
fields: {
|
||||
prompt: { as: "value", field: "prompt" },
|
||||
images: { as: "urlArray", field: "images" },
|
||||
aspectRatio: { as: "value", field: "aspect_ratio" },
|
||||
resolution: { as: "value", field: "resolution" },
|
||||
background: { as: "value", field: "background" },
|
||||
},
|
||||
},
|
||||
// Sume serves Nano Banana 2 as its successor, Nano Banana 2.1.
|
||||
...([["nano-banana-2", "google/nano-banana-2.1"], ["nano-banana-pro", "google/nano-banana-pro"]] as const).map(([name, model]) => ({
|
||||
capability: { module: NANO_BANANA, name }, result: "image" as const,
|
||||
routes: [{ model }],
|
||||
fields: {
|
||||
prompt: { as: "value" as const, field: "prompt" },
|
||||
images: { as: "urlArray" as const, field: "images" },
|
||||
aspectRatio: { as: "value" as const, field: "aspect_ratio" },
|
||||
resolution: { as: "value" as const, field: "resolution" },
|
||||
outputFormat: { as: "value" as const, field: "output_format" },
|
||||
},
|
||||
})),
|
||||
{
|
||||
capability: { module: SEEDREAM, name: "seedream-5-lite" }, result: "image",
|
||||
routes: [{ model: "bytedance-seed/seedream-5-lite" }],
|
||||
fields: {
|
||||
prompt: { as: "value", field: "prompt" },
|
||||
images: { as: "urlArray", field: "images" },
|
||||
aspectRatio: { as: "value", field: "aspect_ratio" },
|
||||
quality: { as: "value", field: "quality" },
|
||||
outputFormat: { as: "value", field: "output_format" },
|
||||
nsfwCheck: { as: "value", field: "nsfw_check" },
|
||||
},
|
||||
},
|
||||
];
|
||||
264
packages/provider-sume/src/provider.ts
Normal file
264
packages/provider-sume/src/provider.ts
Normal file
@@ -0,0 +1,264 @@
|
||||
import type { AsyncEndpoint, EndpointCredential, EndpointInvocationContext, EndpointOutcome } from "@hypit/hypit/endpoint";
|
||||
import { EndpointServiceError, credentialRef, defineEndpoint, wakeAfter } from "@hypit/hypit/endpoint";
|
||||
import type { CredentialRef, EndpointActionLimits, ResourceStore } from "@hypit/hypit/endpoint";
|
||||
import { EndpointHttpError, EndpointResponseError, EndpointTransportError, withRequestDeadline } from "@hypit/hypit/endpoint/http";
|
||||
import type { GenerationArtifactUrlResolver } from "@hypit/generation";
|
||||
import { canonicalize } from "@hypit/hypit/protocol";
|
||||
import type { BlobRef, CanonicalValue } from "@hypit/hypit/protocol";
|
||||
import { sumeRouteForCapability, sumeRoutes } from "./routes.js";
|
||||
import { SumeHttpError, SumeServiceError, sumeJobFailure } from "./errors.js";
|
||||
|
||||
export const sumeProviderModuleRef = { name: "@hypit/provider-sume", version: "1" } as const;
|
||||
|
||||
export type CreateSumeProviderOptions = {
|
||||
readonly instance?: string;
|
||||
readonly pool?: string;
|
||||
readonly baseUrl?: string;
|
||||
readonly apiKey?: CredentialRef;
|
||||
readonly defaultConcurrency?: number;
|
||||
readonly actionLimits?: EndpointActionLimits;
|
||||
readonly pollIntervalMs?: number;
|
||||
readonly requestTimeoutMs?: number;
|
||||
readonly operationTimeoutMs?: number;
|
||||
readonly fetch?: typeof globalThis.fetch;
|
||||
/** Publish a referenced Resource at a URL the service can fetch; replaces Sume's upload URL. */
|
||||
readonly publicAssetUrl?: (artifact: BlobRef, artifacts: ResourceStore, fields?: Readonly<Record<string, string | number | boolean>>) => Promise<string>;
|
||||
};
|
||||
|
||||
type Handle = {
|
||||
readonly contract: "hypit.sume-operation@1";
|
||||
readonly jobId: string;
|
||||
readonly route: string;
|
||||
readonly startedAt: number;
|
||||
readonly urls?: readonly string[];
|
||||
};
|
||||
|
||||
const extensions: Readonly<Record<string, string>> = {
|
||||
"image/png": "png", "image/jpeg": "jpg", "image/webp": "webp",
|
||||
"audio/mpeg": "mp3", "audio/wav": "wav", "audio/x-wav": "wav", "audio/aac": "aac", "audio/mp4": "m4a",
|
||||
"video/mp4": "mp4", "video/quicktime": "mov", "video/webm": "webm",
|
||||
};
|
||||
|
||||
function assert(condition: unknown, message: string): asserts condition { if (!condition) throw new Error(message); }
|
||||
function object(value: unknown, subject: string): Record<string, unknown> {
|
||||
assert(value !== null && typeof value === "object" && !Array.isArray(value), `${subject} must be an object`);
|
||||
return value as Record<string, unknown>;
|
||||
}
|
||||
function apiBaseUrl(value: string): string {
|
||||
let trimmed = value.trim();
|
||||
while (trimmed.endsWith("/")) trimmed = trimmed.slice(0, -1);
|
||||
assert(trimmed.length > 0, "Sume base URL is empty");
|
||||
return trimmed;
|
||||
}
|
||||
function apiKey(credentials: Readonly<Record<string, EndpointCredential>>): string {
|
||||
const value = credentials.apiKey?.secret;
|
||||
if (typeof value !== "string" || value.length === 0) {
|
||||
throw new SumeServiceError("SUME_CREDENTIAL_UNAVAILABLE",
|
||||
"Sume apiKey credential is unavailable; store a Sume API key for this Endpoint");
|
||||
}
|
||||
return value;
|
||||
}
|
||||
function httpsUrl(value: unknown, subject: string): string {
|
||||
assert(typeof value === "string" && /^https:\/\//u.test(value), `${subject} has no HTTPS URL`);
|
||||
return value;
|
||||
}
|
||||
function failure(error: unknown): EndpointOutcome {
|
||||
if (!(error instanceof EndpointServiceError || error instanceof EndpointHttpError
|
||||
|| error instanceof EndpointTransportError || error instanceof EndpointResponseError)) throw error;
|
||||
const code = error instanceof EndpointServiceError || error instanceof EndpointHttpError ? error.code : "SUME_ERROR";
|
||||
return { status: "failed", failure: { code, message: error.message } };
|
||||
}
|
||||
function retrying(error: unknown, handle: CanonicalValue, pollIntervalMs: number, deadlineAt?: number): EndpointOutcome | undefined {
|
||||
const delay = error instanceof EndpointTransportError ? pollIntervalMs
|
||||
: error instanceof EndpointHttpError && (error.status === 429 || error.status >= 500)
|
||||
? error.retryAfterMs ?? pollIntervalMs : undefined;
|
||||
if (delay === undefined) return undefined;
|
||||
return { status: "pending", handle, wakeAt: Math.min(Date.now() + delay, deadlineAt ?? Number.MAX_SAFE_INTEGER), progress: { phase: "retrying" } };
|
||||
}
|
||||
function unknownSubmission(error: unknown): EndpointOutcome | undefined {
|
||||
return error instanceof EndpointTransportError ? { status: "failed", failure: {
|
||||
code: "SUME_SUBMISSION_UNKNOWN",
|
||||
message: `Sume submission transport failed; remote outcome is unknown: ${error.message}`,
|
||||
} } : undefined;
|
||||
}
|
||||
|
||||
class SumeClient {
|
||||
constructor(readonly baseUrl: string, readonly timeout: number, readonly fetcher: typeof globalThis.fetch) {}
|
||||
|
||||
/** One authenticated call; returns the `data` member of Sume's envelope. */
|
||||
async json(path: string, key: string, init: RequestInit = {}): Promise<Record<string, unknown>> {
|
||||
return await withRequestDeadline(this.timeout, async ({ signal, wait }) => {
|
||||
const response = await wait(this.fetcher(`${this.baseUrl}${path}`, {
|
||||
...init, signal, headers: { authorization: `Bearer ${key}`, ...(init.headers ?? {}) },
|
||||
}));
|
||||
const text = await wait(response.text());
|
||||
if (!response.ok) {
|
||||
const input = typeof init.body === "string" ? JSON.parse(init.body) as Record<string, unknown> : undefined;
|
||||
throw new SumeHttpError(response.status, response, text, {
|
||||
method: init.method ?? "GET", path, ...(typeof input?.model === "string" ? { model: input.model } : {}),
|
||||
});
|
||||
}
|
||||
let body: unknown;
|
||||
try { body = text.length === 0 ? {} : JSON.parse(text); } catch { throw new EndpointResponseError(`Sume returned invalid JSON (${response.status})`); }
|
||||
return object(object(body, "Sume response").data, "Sume response data");
|
||||
}, () => new EndpointTransportError("Sume request timed out", { timeout: true }));
|
||||
}
|
||||
|
||||
/** Ask for an upload URL, PUT the Resource there and return the public HTTPS URL a job reads. */
|
||||
async upload(artifact: BlobRef, resources: ResourceStore, key: string): Promise<string> {
|
||||
const kind = artifact.mediaType.split("/", 1)[0] ?? "";
|
||||
assert(kind === "image" || kind === "video" || kind === "audio",
|
||||
`Sume accepts image, audio and video references, not ${artifact.mediaType}`);
|
||||
const bytes = await resources.get(artifact.resource);
|
||||
assert(bytes !== undefined && bytes.byteLength === artifact.size, `Reference Resource ${artifact.resource} is unavailable or has changed`);
|
||||
const slot = await this.json("/v1/files/generate-upload-url", key, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
content_type: artifact.mediaType,
|
||||
size_bytes: artifact.size,
|
||||
filename: `${artifact.resource}.${extensions[artifact.mediaType] ?? "bin"}`,
|
||||
}),
|
||||
});
|
||||
const uploadUrl = httpsUrl(slot.upload_url, "Sume upload slot");
|
||||
const headers = slot.upload_headers === undefined ? {} : object(slot.upload_headers, "Sume upload headers") as Record<string, string>;
|
||||
await withRequestDeadline(this.timeout, async ({ signal, wait }) => {
|
||||
const response = await wait(this.fetcher(uploadUrl, {
|
||||
method: "PUT", signal, headers: { "content-type": artifact.mediaType, ...headers }, body: new Uint8Array(bytes),
|
||||
}));
|
||||
// The upload URL is presigned; its path is a credential and stays out of the error.
|
||||
if (!response.ok) throw new SumeHttpError(response.status, response, "", { method: "PUT", path: "[upload-url]" });
|
||||
await wait(response.arrayBuffer());
|
||||
}, () => new EndpointTransportError("Sume upload timed out", { timeout: true }));
|
||||
return httpsUrl(slot.public_url, "Sume upload slot");
|
||||
}
|
||||
|
||||
async download(url: string): Promise<{ readonly bytes: Uint8Array; readonly mediaType: string }> {
|
||||
return await withRequestDeadline(this.timeout, async ({ signal, wait }) => {
|
||||
const response = await wait(this.fetcher(url, { signal }));
|
||||
if (!response.ok) throw new SumeHttpError(response.status, response, "", { method: "GET", path: "[result-url]" });
|
||||
return { bytes: new Uint8Array(await wait(response.arrayBuffer())), mediaType: response.headers.get("content-type")?.split(";", 1)[0] ?? "application/octet-stream" };
|
||||
}, () => new EndpointTransportError("Sume download timed out", { timeout: true }));
|
||||
}
|
||||
}
|
||||
|
||||
function resolverFor(client: SumeClient, context: EndpointInvocationContext, publicAssetUrl: CreateSumeProviderOptions["publicAssetUrl"]): GenerationArtifactUrlResolver {
|
||||
const resolved = new Map<string, Promise<string>>();
|
||||
return (artifact, fields) => {
|
||||
const existing = resolved.get(artifact.resource);
|
||||
if (existing !== undefined) return existing;
|
||||
const promise = publicAssetUrl === undefined
|
||||
? client.upload(artifact, context.resources, apiKey(context.credentials))
|
||||
: publicAssetUrl(artifact, context.resources, fields);
|
||||
resolved.set(artifact.resource, promise);
|
||||
return promise;
|
||||
};
|
||||
}
|
||||
|
||||
/** Job statuses before a terminal one. */
|
||||
const pendingStatuses: readonly string[] = ["queued", "processing"];
|
||||
|
||||
function endpoint(client: SumeClient, pollIntervalMs: number, maxOperationMs: number, publicAssetUrl: CreateSumeProviderOptions["publicAssetUrl"]): AsyncEndpoint {
|
||||
return {
|
||||
async start(context) {
|
||||
try {
|
||||
const route = sumeRouteForCapability(context.need.capability);
|
||||
assert(route !== undefined, "Sume does not implement this exact capability");
|
||||
const request = route.prepare(context.need.constraints);
|
||||
await context.reportProgress?.({ phase: `Preparing Sume request: ${request.model}` });
|
||||
const body = await request.compile(resolverFor(client, context, publicAssetUrl));
|
||||
await context.reportProgress?.({ phase: `Submitting Sume request: ${request.model}` });
|
||||
let response: Record<string, unknown>;
|
||||
try {
|
||||
response = await client.json(request.media === "image" ? "/v1/images" : "/v1/video-router/generate", apiKey(context.credentials), {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json", "idempotency-key": context.operation },
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
} catch (error) {
|
||||
const unknown = unknownSubmission(error);
|
||||
if (unknown !== undefined) return unknown;
|
||||
throw error;
|
||||
}
|
||||
const job = response.job === undefined ? undefined : object(response.job, "Sume job");
|
||||
const jobId = job?.id ?? response.request_id;
|
||||
if (typeof jobId !== "string" || jobId.length === 0) throw new EndpointResponseError("Sume response has no job id");
|
||||
const handle: Handle = { contract: "hypit.sume-operation@1", jobId, route: route.key, startedAt: Date.now() };
|
||||
const receipt = { id: jobId };
|
||||
await context.checkpoint?.({ handle: canonicalize(handle), receipt });
|
||||
return { ...wakeAfter(canonicalize(handle), pollIntervalMs, Date.now(), { phase: "submitted" }), receipt };
|
||||
} catch (error) {
|
||||
return failure(error);
|
||||
}
|
||||
},
|
||||
async poll(context) {
|
||||
let operationDeadlineAt: number | undefined;
|
||||
try {
|
||||
const handle = object(context.handle, "Sume handle") as unknown as Handle;
|
||||
const route = sumeRouteForCapability(context.need.capability);
|
||||
assert(route !== undefined && handle.contract === "hypit.sume-operation@1" && handle.route === route.key, "Sume handle is invalid");
|
||||
const receipt = { id: handle.jobId };
|
||||
operationDeadlineAt = handle.startedAt + maxOperationMs;
|
||||
if (Date.now() >= operationDeadlineAt) {
|
||||
return { status: "failed", receipt, failure: { code: "SUME_OPERATION_TIMEOUT", message: `Sume job ${handle.jobId} exceeded this Provider's operationTimeoutMs (${maxOperationMs}); remote outcome is unknown` } };
|
||||
}
|
||||
const key = apiKey(context.credentials);
|
||||
const id = encodeURIComponent(handle.jobId);
|
||||
const state = await client.json(`/v1/jobs/${id}/status`, key);
|
||||
const status = String(state.sume_status);
|
||||
if (pendingStatuses.includes(status)) {
|
||||
return { ...wakeAfter(canonicalize(handle), pollIntervalMs, Date.now(), { phase: status }), receipt };
|
||||
}
|
||||
const job = state.job === undefined || state.job === null ? undefined : object(state.job, "Sume job");
|
||||
const rejected = sumeJobFailure(status, job, handle.jobId);
|
||||
if (rejected !== undefined) return { ...failure(rejected), receipt };
|
||||
if (status !== "completed") throw new EndpointResponseError(`Sume returned unknown job status ${status}`);
|
||||
const result = object((await client.json(`/v1/jobs/${id}/result`, key)).result, "Sume job result");
|
||||
const artifacts = Array.isArray(result.artifacts) ? result.artifacts.map((item, index) => object(item, `Sume artifact ${index + 1}`)) : [];
|
||||
const media = artifacts.filter((item) => item.type === route.result);
|
||||
if (media.length === 0) throw new EndpointResponseError(`Sume job completed without ${route.result} artifacts`);
|
||||
const urls = media.map((item, index) => httpsUrl(item.url, `Sume ${route.result} artifact ${index + 1}`));
|
||||
return { status: "ready", handle: canonicalize({ ...handle, urls }), receipt };
|
||||
} catch (error) {
|
||||
return retrying(error, context.handle, pollIntervalMs, operationDeadlineAt) ?? failure(error);
|
||||
}
|
||||
},
|
||||
async collect(context) {
|
||||
try {
|
||||
const handle = object(context.handle, "Sume handle") as unknown as Handle;
|
||||
const route = sumeRouteForCapability(context.need.capability);
|
||||
assert(route !== undefined && handle.route === route.key && Array.isArray(handle.urls), "Sume collection route differs");
|
||||
await context.reportProgress?.({ phase: "Receiving generated files" });
|
||||
const blobs: BlobRef[] = [];
|
||||
for (const url of handle.urls) {
|
||||
const downloaded = await client.download(url);
|
||||
blobs.push(await context.resources.put(downloaded.bytes, downloaded.mediaType));
|
||||
}
|
||||
return { status: "completed", result: { value: route.packageResult(blobs) }, receipt: { id: handle.jobId } };
|
||||
} catch (error) {
|
||||
return retrying(error, context.handle, pollIntervalMs) ?? failure(error);
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export function createSumeProvider(options: CreateSumeProviderOptions = {}) {
|
||||
const requestTimeoutMs = options.requestTimeoutMs ?? 300_000;
|
||||
const operationTimeoutMs = options.operationTimeoutMs ?? 30 * 60_000;
|
||||
for (const [name, value] of Object.entries({ requestTimeoutMs, operationTimeoutMs })) {
|
||||
assert(Number.isSafeInteger(value) && value > 0, `Sume ${name} must be a positive integer`);
|
||||
}
|
||||
const client = new SumeClient(apiBaseUrl(options.baseUrl ?? "https://api.sume.com"), requestTimeoutMs, options.fetch ?? globalThis.fetch);
|
||||
const asyncEndpoint = endpoint(client, options.pollIntervalMs ?? 5_000, operationTimeoutMs, options.publicAssetUrl);
|
||||
return defineEndpoint({
|
||||
instance: options.instance ?? "sume.default", pool: options.pool ?? options.instance ?? "sume.default",
|
||||
pricing: { kind: "page", url: "https://www.sume.com/pricing/api" },
|
||||
credentials: { apiKey: options.apiKey ?? credentialRef("local", "sume.api-key") },
|
||||
credentialInputs: { apiKey: { label: "Sume API key" } },
|
||||
defaultConcurrency: options.defaultConcurrency ?? 2,
|
||||
...(options.actionLimits === undefined ? {} : { actionLimits: options.actionLimits }),
|
||||
capabilities: sumeRoutes.map((route) => ({
|
||||
capability: route.capability, returns: route.returns, lifecycle: "asynchronous" as const, endpoint: asyncEndpoint, capacity: route.capability.name, supports: route.supports,
|
||||
})),
|
||||
});
|
||||
}
|
||||
166
packages/provider-sume/src/routes.ts
Normal file
166
packages/provider-sume/src/routes.ts
Normal file
@@ -0,0 +1,166 @@
|
||||
import {
|
||||
compileWireRequest,
|
||||
generationTypes,
|
||||
sealGeneratedImageSet,
|
||||
sealGeneratedVideoSet,
|
||||
selectWireModelForRequest,
|
||||
} from "@hypit/generation";
|
||||
import type { GenerationArtifactUrlResolver, GenerationRequest, GenerationWireMapping } from "@hypit/generation";
|
||||
import { canonicalize } from "@hypit/hypit/protocol";
|
||||
import type { BlobRef, CanonicalValue, CapabilityRef, StoredValue, TypeRef } from "@hypit/hypit/protocol";
|
||||
import type { EndpointRequest, EndpointSupport } from "@hypit/hypit/endpoint";
|
||||
import { sumeMappings } from "./mapping.js";
|
||||
|
||||
export type SumePreparedRequest = {
|
||||
readonly model: string;
|
||||
readonly media: "image" | "video";
|
||||
readonly compile: (resolve: GenerationArtifactUrlResolver) => Promise<Record<string, unknown>>;
|
||||
};
|
||||
|
||||
export type SumeRoute = GenerationWireMapping & {
|
||||
readonly key: string;
|
||||
readonly returns: TypeRef;
|
||||
readonly supports: (request: EndpointRequest) => EndpointSupport;
|
||||
readonly prepare: (constraints: CanonicalValue) => SumePreparedRequest;
|
||||
readonly packageResult: (artifacts: readonly BlobRef[]) => StoredValue;
|
||||
};
|
||||
|
||||
function scalar(request: GenerationRequest, port: string): string | number | boolean | undefined {
|
||||
const value = request.ports[port]?.[0];
|
||||
return typeof value === "string" || typeof value === "number" || typeof value === "boolean" ? value : undefined;
|
||||
}
|
||||
function count(request: GenerationRequest, port: string): number {
|
||||
return request.ports[port]?.length ?? 0;
|
||||
}
|
||||
|
||||
/** Ratios the model packages offer that the Sume catalog row for the model does not list. */
|
||||
const UNLISTED_RATIOS: Readonly<Record<string, readonly string[]>> = {
|
||||
"grok-imagine-video-1.5-preview": ["2:3", "3:2"],
|
||||
"pixverse-v6": ["2:3", "3:2", "auto"],
|
||||
"pixverse-c1": ["2:3", "3:2"],
|
||||
"gpt-image-2": ["3:2", "2:3", "2:1", "1:2", "3:1", "1:3", "21:9", "9:21"],
|
||||
"seedream-5-lite": ["21:9"],
|
||||
};
|
||||
|
||||
/** Sume's per-model input ranges, checked before any reference is uploaded. */
|
||||
function rejection(mapping: GenerationWireMapping, request: GenerationRequest): string | undefined {
|
||||
const { name } = mapping.capability;
|
||||
const model = selectWireModelForRequest(mapping, request);
|
||||
const ratio = scalar(request, "aspectRatio");
|
||||
if (ratio !== undefined && UNLISTED_RATIOS[name]?.includes(String(ratio)) === true) {
|
||||
return `Sume ${model} does not render aspect ratio ${String(ratio)}`;
|
||||
}
|
||||
if (mapping.capability.module.name === "@hypit/seedance") {
|
||||
if (scalar(request, "webSearch") === true) return `Sume ${model} has no web_search field`;
|
||||
if (name === "seedance-2.5" && scalar(request, "duration") === -1) {
|
||||
return "Sume seedance-2.5 takes a duration of 4 to 30 seconds, not automatic";
|
||||
}
|
||||
}
|
||||
if (name === "minimax-h3" && scalar(request, "duration") === 4) {
|
||||
return "Sume minimax-h3 takes a duration of 5 to 15 seconds";
|
||||
}
|
||||
if (name === "grok-imagine-video-1.5-preview") {
|
||||
if (count(request, "images") > 1) return "Sume grok-imagine-video-1.5 takes a prompt alone or one first-frame image";
|
||||
if (ratio === "auto" && count(request, "images") === 0) {
|
||||
return "Sume grok-imagine-video-1.5 needs an explicit aspect ratio without a first frame";
|
||||
}
|
||||
}
|
||||
if (name === "pixverse-v6" || name === "pixverse-c1") {
|
||||
if (count(request, "firstFrame") > 0 || count(request, "lastFrame") > 0) {
|
||||
return `Sume ${model} takes a prompt or reference images, not frames`;
|
||||
}
|
||||
if (count(request, "referenceVideo") > 0) return `Sume ${model} takes no reference videos`;
|
||||
if (scalar(request, "multiClip") === true) return `Sume ${model} has no multi-clip field`;
|
||||
if (scalar(request, "seed") !== undefined) return `Sume ${model} takes no seed`;
|
||||
if (scalar(request, "duration") === 1) return `Sume ${model} takes a duration of 2 to 15 seconds`;
|
||||
}
|
||||
if (name === "gpt-image-2") {
|
||||
if (scalar(request, "background") !== undefined) return "Sume openai/gpt-image-2 has no background field";
|
||||
if (count(request, "images") > 10) return "Sume openai/gpt-image-2 accepts up to ten reference images";
|
||||
}
|
||||
if (name === "nano-banana-2" && count(request, "images") > 10) {
|
||||
return "Sume google/nano-banana-2.1 accepts up to ten reference images";
|
||||
}
|
||||
if (name === "seedream-5-lite") {
|
||||
if (scalar(request, "quality") !== "basic") {
|
||||
return "Sume bytedance-seed/seedream-5-lite renders at 2K only, which is quality basic";
|
||||
}
|
||||
if (scalar(request, "nsfwCheck") === false) {
|
||||
return "Sume bytedance-seed/seedream-5-lite has no switch to turn its content check off";
|
||||
}
|
||||
if (count(request, "images") > 10) return "Sume bytedance-seed/seedream-5-lite accepts up to ten reference images";
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/** Fields the model package states that Sume names differently, spells differently or omits. */
|
||||
function normalize(mapping: GenerationWireMapping, input: Record<string, unknown>): Record<string, unknown> {
|
||||
const { name } = mapping.capability;
|
||||
if (input.web_search === false) delete input.web_search;
|
||||
// MiniMax spells its native tier 768P; Sume's catalog lists 768p.
|
||||
if (name === "minimax-h3" && input.resolution === "768P") input.resolution = "768p";
|
||||
if (name === "grok-imagine-video-1.5-preview") {
|
||||
const images = input.images as readonly string[] | undefined;
|
||||
delete input.images;
|
||||
// A first frame keeps its own shape, and Sume applies aspect_ratio to text-to-video only.
|
||||
if (images !== undefined && images.length > 0) {
|
||||
input.image_url = images[0];
|
||||
delete input.aspect_ratio;
|
||||
}
|
||||
}
|
||||
if (name === "pixverse-v6" && input.multi_clip === false) delete input.multi_clip;
|
||||
if (mapping.result === "image") {
|
||||
const images = input.images as readonly string[] | undefined;
|
||||
delete input.images;
|
||||
if (images !== undefined && images.length > 0) {
|
||||
input.input_references = images.map((url) => ({ type: "image_url", image_url: { url } }));
|
||||
}
|
||||
}
|
||||
if (mapping.capability.module.name === "@hypit/nano-banana" && input.output_format === "jpg") input.output_format = "jpeg";
|
||||
if (name === "seedream-5-lite") {
|
||||
delete input.quality;
|
||||
delete input.nsfw_check;
|
||||
}
|
||||
return input;
|
||||
}
|
||||
|
||||
function capabilityKey(capability: CapabilityRef): string {
|
||||
return `${capability.module.name}@${capability.module.version}#${capability.name}`;
|
||||
}
|
||||
|
||||
export const sumeRoutes: readonly SumeRoute[] = sumeMappings.map((mapping) => ({
|
||||
...mapping,
|
||||
key: capabilityKey(mapping.capability),
|
||||
returns: mapping.result === "image" ? generationTypes.imageSet : generationTypes.videoSet,
|
||||
supports: (request) => {
|
||||
const reason = rejection(mapping, request.constraints as unknown as GenerationRequest);
|
||||
return reason === undefined ? { status: "supported" } : { status: "unsupported", reason };
|
||||
},
|
||||
prepare: (constraints) => {
|
||||
const request = constraints as unknown as GenerationRequest;
|
||||
const reason = rejection(mapping, request);
|
||||
if (reason !== undefined) throw new Error(reason);
|
||||
const model = selectWireModelForRequest(mapping, request);
|
||||
return {
|
||||
model,
|
||||
media: mapping.result === "image" ? "image" : "video",
|
||||
compile: async (resolve) => ({
|
||||
model,
|
||||
...normalize(mapping, (await compileWireRequest(mapping, request, resolve)).input as Record<string, unknown>),
|
||||
mode: "async",
|
||||
}),
|
||||
};
|
||||
},
|
||||
packageResult: (artifacts) => ({
|
||||
kind: "inline",
|
||||
value: canonicalize(mapping.result === "image"
|
||||
? sealGeneratedImageSet({ images: artifacts })
|
||||
: sealGeneratedVideoSet({ videos: artifacts })),
|
||||
}),
|
||||
}));
|
||||
|
||||
const byCapability = new Map(sumeRoutes.map((route) => [route.key, route]));
|
||||
|
||||
export function sumeRouteForCapability(capability: CapabilityRef): SumeRoute | undefined {
|
||||
return byCapability.get(capabilityKey(capability));
|
||||
}
|
||||
22
packages/provider-sume/test/routes.test.ts
Normal file
22
packages/provider-sume/test/routes.test.ts
Normal file
@@ -0,0 +1,22 @@
|
||||
import test from "node:test";
|
||||
import { assertMappingCoversPorts } from "@hypit/generation";
|
||||
import { gptImage2Ports } from "@hypit/gpt-image";
|
||||
import { grokImaginePorts } from "@hypit/grok-imagine";
|
||||
import { minimaxH3Ports } from "@hypit/minimax-h3";
|
||||
import { nanoBananaPorts } from "@hypit/nano-banana";
|
||||
import { pixversePorts } from "@hypit/pixverse";
|
||||
import { seedancePorts } from "@hypit/seedance";
|
||||
import { seedream5LitePorts } from "@hypit/seedream";
|
||||
|
||||
import { sumeMappings } from "../src/mapping.js";
|
||||
|
||||
// A port added to one of these model packages must be mapped here before Sume can serve it.
|
||||
test("every Sume mapping covers its model's ports", () => {
|
||||
const tables = {
|
||||
...seedancePorts, ...grokImaginePorts, ...nanoBananaPorts, ...pixversePorts,
|
||||
"minimax-h3": minimaxH3Ports, "gpt-image-2": gptImage2Ports, "seedream-5-lite": seedream5LitePorts,
|
||||
};
|
||||
for (const mapping of sumeMappings) {
|
||||
assertMappingCoversPorts(tables[mapping.capability.name as keyof typeof tables], mapping);
|
||||
}
|
||||
});
|
||||
46
pnpm-lock.yaml
generated
46
pnpm-lock.yaml
generated
@@ -201,6 +201,9 @@ importers:
|
||||
'@hypit/provider-pollo':
|
||||
specifier: workspace:*
|
||||
version: link:packages/provider-pollo
|
||||
'@hypit/provider-sume':
|
||||
specifier: workspace:*
|
||||
version: link:packages/provider-sume
|
||||
'@hypit/provider-tokendance':
|
||||
specifier: workspace:*
|
||||
version: link:packages/provider-tokendance
|
||||
@@ -2111,6 +2114,49 @@ importers:
|
||||
specifier: workspace:*
|
||||
version: link:../runtime-local
|
||||
|
||||
packages/provider-sume:
|
||||
dependencies:
|
||||
'@hypit/generation':
|
||||
specifier: workspace:^0.1.0
|
||||
version: link:../generation
|
||||
devDependencies:
|
||||
'@hypit/endpoint':
|
||||
specifier: workspace:*
|
||||
version: link:../endpoint
|
||||
'@hypit/gpt-image':
|
||||
specifier: workspace:*
|
||||
version: link:../gpt-image
|
||||
'@hypit/grok-imagine':
|
||||
specifier: workspace:*
|
||||
version: link:../grok-imagine
|
||||
'@hypit/hypit':
|
||||
specifier: workspace:*
|
||||
version: link:../..
|
||||
'@hypit/minimax-h3':
|
||||
specifier: workspace:*
|
||||
version: link:../minimax-h3
|
||||
'@hypit/nano-banana':
|
||||
specifier: workspace:*
|
||||
version: link:../nano-banana
|
||||
'@hypit/pixverse':
|
||||
specifier: workspace:*
|
||||
version: link:../pixverse
|
||||
'@hypit/protocol':
|
||||
specifier: workspace:*
|
||||
version: link:../protocol
|
||||
'@hypit/runtime':
|
||||
specifier: workspace:*
|
||||
version: link:../runtime
|
||||
'@hypit/runtime-local':
|
||||
specifier: workspace:*
|
||||
version: link:../runtime-local
|
||||
'@hypit/seedance':
|
||||
specifier: workspace:*
|
||||
version: link:../seedance
|
||||
'@hypit/seedream':
|
||||
specifier: workspace:*
|
||||
version: link:../seedream
|
||||
|
||||
packages/provider-tokendance:
|
||||
dependencies:
|
||||
'@hypit/generation':
|
||||
|
||||
@@ -29,6 +29,7 @@ const packageDirectories = [
|
||||
"provider-hiapi",
|
||||
"provider-monid",
|
||||
"provider-pollo",
|
||||
"provider-sume",
|
||||
"provider-tokendance",
|
||||
];
|
||||
const packageNames = packageDirectories.map((name) => `@hypit/${name}`);
|
||||
@@ -45,6 +46,7 @@ const optionalPackages = [
|
||||
"@hypit/provider-hiapi",
|
||||
"@hypit/provider-monid",
|
||||
"@hypit/provider-pollo",
|
||||
"@hypit/provider-sume",
|
||||
"@hypit/provider-tokendance",
|
||||
];
|
||||
|
||||
|
||||
@@ -56,6 +56,7 @@ npm install @hypit/provider-tokendance
|
||||
npm install @hypit/provider-hiapi
|
||||
npm install @hypit/provider-pollo
|
||||
npm install @hypit/provider-beatapi
|
||||
npm install @hypit/provider-sume
|
||||
npm install @hypit/provider-monid
|
||||
```
|
||||
|
||||
|
||||
Reference in New Issue
Block a user