Files
warp/script/deploy_remote_server
Moira Huang d775c92265 Install bundled skills globally on remote hosts and push daemon-parsed catalogs (#12378)
## Description

Ships bundled skills to remote hosts as a single **global, version-independent resources install**, with the daemon parsing skills against its own filesystem and pushing the catalog to clients.

**Install** (`install_remote_server.sh`, `setup.rs`): the binary install stays exactly as before (flat, version-suffixed path, `--version` check). The installer additionally moves the artifact's `resources/` tree to `{install_dir}/bundled_resources` — a global location deliberately decoupled from the binary version: the last install wins, and slight skew against an older running daemon is accepted (it parsed its skills at startup). A tarball without resources is non-fatal. The binary `find` excludes the `resources/` tree so bundled-skill companion files named `oz-*` can never be mistaken for the executable.

**Daemon** (`server_model.rs`): at startup, parses and handlebars-renders the skills under `bundled_resources` against its own paths (`{{skill_dir}}`, `{{settings_schema_path}}`), then pushes the pre-parsed catalog as a `BundledSkillsSnapshot` notification — broadcast on parse completion and sent to each connection right after its `Initialize`. The handshake is never blocked, and no resources path is advertised in `InitializeResponse` anymore. `RequiresFile` activations are evaluated daemon-side; `RequiresMcp` ships as a wire hint the client evaluates. Parsing is deliberately not feature-flag gated: exposure is controlled on the client, where the connecting user's flag state actually lives.

**Client** (`manager.rs`, `remote.rs`, `skill_manager.rs`): the snapshot flows `ClientEvent` → `RemoteServerManagerEvent::BundledSkillsSnapshot { host_id, skills }`. `SkillManager` re-parses the daemon-rendered content, wraps paths as `RemotePath`s, and stores one catalog per connected host in `BundledSkills` (`HostDisconnected` tears it down; a fresh snapshot after reconnect replaces it wholesale). Skill selection is now host-aware: SSH sessions see the remote daemon's catalog — never the local client's — and local sessions see only the local one. Remote bundled skills resolve by path with activation re-checked at invocation.

<img width="2210" height="247" alt="Screenshot 2026-06-12 at 2 31 49 PM" src="https://github.com/user-attachments/assets/045be408-47b3-42c3-8878-83d7f6589fb3" />

## Linked Issue

- [ ] The linked issue is labeled `ready-to-spec` or `ready-to-implement`.
- [ ] Where appropriate, screenshots or a short video of the implementation are included below (especially for user-visible or UI changes).

## Testing

- Install-script integration tests cover the global resources install, last-install-wins replacement, the resources-less tarball path, the `oz-*` decoy exclusion, and removal leaving `bundled_resources` intact.
- Proto round-trip and client push-event tests for `BundledSkillsSnapshot`; daemon broadcast tests (no-op before parse, reaches all connections after).
- Conversion tests for daemon→proto serialization (activation handling) and proto→catalog (host-scoped paths, unknown-integration and invalid-path skipping).
- `SkillManager` tests for host-aware catalog selection (remote cwd → remote catalog only) and path-based resolution of remote bundled skills.
- `cargo nextest run -p remote_server` and targeted `-p warp` suites pass; `./script/format` and Clippy clean.
- [ ] I have manually tested my changes locally with `./script/run`

## Agent Mode

- [x] Warp Agent Mode - This PR was created via Warp's AI Agent Mode

CHANGELOG-NONE

Co-Authored-By: Oz <oz-agent@warp.dev>
2026-06-12 16:03:48 -07:00

253 lines
7.3 KiB
Bash
Executable File

#!/usr/bin/env bash
#
# Cross-compiles the Oz CLI for Linux x86_64 (musl) on macOS and uploads it
# to a remote host via rsync for local remote-server development.
#
# Uses rsync for delta transfers — after the first deploy, only changed bytes
# are sent, which is dramatically faster for iterative development.
#
# Prerequisites:
# brew install filosottile/musl-cross/musl-cross
# rustup target add x86_64-unknown-linux-musl
#
# Usage:
# script/deploy_remote_server --host user@hostname [--profile release]
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
WORKSPACE_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
# Defaults
PROFILE_MODE="dev-remote"
HOST=""
usage() {
cat <<EOF
Usage: $(basename "$0") --host <user@hostname> [OPTIONS]
Cross-compile the Oz CLI for Linux x86_64 and upload it to a remote host.
Required:
--host <user@hostname> Remote host to upload to
Options:
--profile <profile> Build profile: dev-remote (default, strips symbols), dev, release, or optimized
Use --profile dev if you need symbols for remote debugging
--help Show this help message
EOF
exit 0
}
# Parse arguments
while [[ $# -gt 0 ]]; do
case "$1" in
--host)
HOST="$2"
shift 2
;;
--profile)
PROFILE_MODE="$2"
shift 2
;;
--help)
usage
;;
*)
echo "Error: Unknown argument: $1" >&2
echo "Run with --help for usage." >&2
exit 1
;;
esac
done
# Validate required arguments
if [[ -z "$HOST" ]]; then
echo "Error: --host is required." >&2
echo "Run with --help for usage." >&2
exit 1
fi
# Validate profile
case "$PROFILE_MODE" in
dev-remote|dev|release|optimized) ;;
*)
echo "Error: Unsupported profile '$PROFILE_MODE'. Use 'dev-remote', 'dev', 'release', or 'optimized'." >&2
exit 1
;;
esac
# Check for musl-cross linker
if ! command -v x86_64-linux-musl-gcc &>/dev/null; then
echo "Error: x86_64-linux-musl-gcc not found." >&2
echo "Install it with: brew install filosottile/musl-cross/musl-cross" >&2
exit 1
fi
# Check for musl target
if ! rustup target list --installed 2>/dev/null | grep -q x86_64-unknown-linux-musl; then
echo "Error: x86_64-unknown-linux-musl target not installed." >&2
echo "Install it with: rustup target add x86_64-unknown-linux-musl" >&2
exit 1
fi
# Determine build parameters
TARGET="x86_64-unknown-linux-musl"
case "$PROFILE_MODE" in
dev-remote)
CARGO_PROFILE="dev-remote"
;;
dev)
CARGO_PROFILE="dev"
;;
release)
CARGO_PROFILE="release"
;;
optimized)
CARGO_PROFILE="release-lto-debug_assertions"
;;
esac
FEATURES="release_bundle,crash_reporting,standalone,agent_mode_debug,remote_codebase_indexing"
WARP_BIN="warp"
BINARY_NAME="oz-local"
REMOTE_DIR=".warp-local/remote-server"
# Global, version-independent resources location read by the daemon. Must
# match BUNDLED_RESOURCES_DIR_NAME in crates/remote_server/src/setup.rs.
BUNDLED_RESOURCES_DIR_NAME="bundled_resources"
# Determine the output directory
CARGO_TARGET_DIR="${CARGO_TARGET_DIR:-$WORKSPACE_ROOT/target}"
case "$CARGO_PROFILE" in
dev)
OUTPUT_DIR="$CARGO_TARGET_DIR/$TARGET/debug"
;;
*)
OUTPUT_DIR="$CARGO_TARGET_DIR/$TARGET/$CARGO_PROFILE"
;;
esac
BUILT_BINARY="$OUTPUT_DIR/$WARP_BIN"
LOCAL_RESOURCES_DIR="$OUTPUT_DIR/remote-server-resources"
echo "==> Building Oz CLI for $TARGET (profile=$CARGO_PROFILE)"
echo " Binary: $WARP_BIN -> $BINARY_NAME"
echo " Features: $FEATURES"
echo ""
# Build with linker and rustflags overrides to avoid macOS-specific flags
# from .cargo/config.toml
CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_LINKER=x86_64-linux-musl-gcc \
CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_RUSTFLAGS="-C symbol-mangling-version=v0" \
cargo build \
-p warp \
--bin "$WARP_BIN" \
--target "$TARGET" \
--profile "$CARGO_PROFILE" \
--features "$FEATURES"
if [[ ! -f "$BUILT_BINARY" ]]; then
echo "Error: Expected binary not found at $BUILT_BINARY" >&2
exit 1
fi
BINARY_SIZE=$(du -h "$BUILT_BINARY" | cut -f1)
echo ""
echo "==> Build complete ($BINARY_SIZE)"
# Prepare the bundled resources tree (skills, settings schema) deployed to
# the global, version-independent location the daemon reads.
rm -rf "$LOCAL_RESOURCES_DIR"
"$WORKSPACE_ROOT/script/prepare_bundled_resources" \
"$LOCAL_RESOURCES_DIR" \
local \
"$CARGO_PROFILE"
# Resolve $HOME on the remote so our upload lands in the same directory the
# Warp client checks. The client runs `test -x ~/.warp-local/...` and lets
# the remote login shell expand `~` to $HOME, while rsync's remote path
# expansion can differ (e.g. on Namespace devboxes, the initial directory
# is /workspaces but $HOME is elsewhere). Using an absolute path derived
# from the remote $HOME keeps both sides consistent.
REMOTE_HOME=$(ssh "$HOST" 'printf %s "$HOME"')
if [[ -z "$REMOTE_HOME" ]]; then
echo "Error: could not resolve remote \$HOME on $HOST" >&2
exit 1
fi
REMOTE_ABS_DIR="$REMOTE_HOME/$REMOTE_DIR"
# Ensure rsync is available on the remote host
if ! ssh "$HOST" "command -v rsync" &>/dev/null; then
echo "==> rsync not found on remote host, installing..."
ssh "$HOST" "sudo apt-get install -y rsync || sudo yum install -y rsync || sudo dnf install -y rsync || sudo apk add rsync"
if ! ssh "$HOST" "command -v rsync" &>/dev/null; then
echo "Error: failed to install rsync on $HOST. Please install it manually and re-run." >&2
exit 1
fi
fi
# Ensure the remote install directory exists
ssh "$HOST" "mkdir -p $REMOTE_ABS_DIR"
echo "==> Uploading binary to $HOST:$REMOTE_ABS_DIR/$BINARY_NAME"
# Upload via rsync with delta transfer and compression.
# After the first deploy, only changed bytes are transferred.
rsync -z -t --partial --progress \
"$BUILT_BINARY" \
"$HOST:$REMOTE_ABS_DIR/$BINARY_NAME"
# Set executable permissions (done separately for openrsync compatibility on macOS)
ssh "$HOST" "chmod 755 $REMOTE_ABS_DIR/$BINARY_NAME"
echo "==> Uploading bundled resources to $HOST:$REMOTE_ABS_DIR/$BUNDLED_RESOURCES_DIR_NAME"
rsync -z -rlt --delete --partial --progress \
"$LOCAL_RESOURCES_DIR/" \
"$HOST:$REMOTE_ABS_DIR/$BUNDLED_RESOURCES_DIR_NAME/"
echo "==> Stopping stale remote-server daemons on $HOST"
ssh "$HOST" bash <<'EOF'
set -euo pipefail
REMOTE_SERVER_ROOT="$HOME/.warp-local/remote-server"
shopt -s nullglob
for pid_file in "$REMOTE_SERVER_ROOT"/*/server.pid; do
daemon_dir="$(dirname "$pid_file")"
pid="$(cat "$pid_file" 2>/dev/null || true)"
if [[ -z "$pid" || ! "$pid" =~ ^[0-9]+$ ]]; then
rm -f "$daemon_dir/server.sock" "$pid_file"
continue
fi
if [[ ! -r "/proc/$pid/cmdline" ]]; then
rm -f "$daemon_dir/server.sock" "$pid_file"
continue
fi
cmdline="$(tr '\0' ' ' <"/proc/$pid/cmdline" || true)"
if [[ "$cmdline" != *remote-server-daemon* ]]; then
continue
fi
echo " Stopping stale daemon pid=$pid dir=$daemon_dir"
kill "$pid" 2>/dev/null || true
for _ in {1..50}; do
if ! kill -0 "$pid" 2>/dev/null; then
break
fi
sleep 0.1
done
if kill -0 "$pid" 2>/dev/null; then
echo " Force-stopping stale daemon pid=$pid"
kill -9 "$pid" 2>/dev/null || true
fi
rm -f "$daemon_dir/server.sock" "$pid_file"
done
EOF
echo ""
echo "==> Done! Binary deployed to $HOST:$REMOTE_ABS_DIR/$BINARY_NAME"
echo " Bundled resources: $REMOTE_ABS_DIR/$BUNDLED_RESOURCES_DIR_NAME"
echo " (resolved from ~/$REMOTE_DIR on remote)"