mirror of
https://github.com/nearai/ironclaw.git
synced 2026-09-02 23:56:24 +08:00
198 lines
4.5 KiB
Plaintext
198 lines
4.5 KiB
Plaintext
---
|
|
title: Safety Layer
|
|
sidebarTitle: Safety Layer
|
|
description: Prompt injection defense and content validation
|
|
---
|
|
|
|
The Safety Layer provides multi-stage defense against prompt injection, data exfiltration, and malicious content.
|
|
|
|
## Overview
|
|
|
|
All external content passes through the Safety Layer before reaching the LLM:
|
|
|
|
<Frame>
|
|
<img src="/assets/safety-layer-overview.svg" alt="Safety Layer Overview Diagram" />
|
|
</Frame>
|
|
|
|
```
|
|
External Data → Validator → Sanitizer → Policy Engine → Leak Detector → LLM
|
|
```
|
|
|
|
## Components
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Validator" icon="check-circle">
|
|
Input validation: length, encoding, forbidden patterns.
|
|
</Card>
|
|
|
|
<Card title="Sanitizer" icon="shield">
|
|
Content escaping and dangerous pattern detection.
|
|
</Card>
|
|
|
|
<Card title="Policy Engine" icon="settings">
|
|
Severity-based rules with configurable actions.
|
|
</Card>
|
|
|
|
<Card title="Leak Detector" icon="search">
|
|
Scans for 15+ secret patterns in tool outputs.
|
|
</Card>
|
|
</CardGroup>
|
|
|
|
## Validator
|
|
|
|
Checks input before processing:
|
|
|
|
| Check | Action |
|
|
|-------|--------|
|
|
| **Length** | Reject if exceeds limit |
|
|
| **Encoding** | Reject invalid UTF-8 |
|
|
| **Null bytes** | Reject or strip |
|
|
| **Control chars** | Reject or escape |
|
|
|
|
## Sanitizer
|
|
|
|
Escapes dangerous content:
|
|
|
|
### Injection Patterns Detected
|
|
|
|
- Command chaining (`;`, `&&`, `||`)
|
|
- Subshells (`$()`, backticks)
|
|
- Path traversal (`../`)
|
|
- Null bytes
|
|
- Control characters
|
|
|
|
### Tool Output Wrapping
|
|
|
|
Tool outputs are wrapped before reaching the LLM:
|
|
|
|
```xml
|
|
<tool_output name="search" sanitized="true">
|
|
[escaped content here]
|
|
</tool_output>
|
|
```
|
|
|
|
The `sanitized="true"` attribute signals that content has been processed.
|
|
|
|
## Policy Engine
|
|
|
|
Rules-based enforcement with severity levels:
|
|
|
|
### Severity Levels
|
|
|
|
| Level | Action | Use Case |
|
|
|-------|--------|----------|
|
|
| **Critical** | Block + Alert | System compromise attempt |
|
|
| **High** | Block | Malicious content |
|
|
| **Medium** | Warn | Suspicious patterns |
|
|
| **Low** | Log | Minor issues |
|
|
|
|
### Policy Actions
|
|
|
|
- **Block** — Reject the content
|
|
- **Warn** — Allow with warning
|
|
- **Sanitize** — Clean and proceed
|
|
- **Review** — Flag for human review
|
|
|
|
## Leak Detector
|
|
|
|
Scans for 15+ secret patterns:
|
|
|
|
### Detected Patterns
|
|
|
|
| Pattern | Example |
|
|
|---------|---------|
|
|
| API keys | `sk-...`, `ak-...` |
|
|
| Tokens | `ghp_...`, `sess-...` |
|
|
| Private keys | `-----BEGIN RSA PRIVATE KEY-----` |
|
|
| Connection strings | `postgres://user:pass@...` |
|
|
| AWS credentials | `AKIA...` |
|
|
| GitHub tokens | `ghp_...` |
|
|
|
|
### Actions per Pattern
|
|
|
|
| Action | Behavior |
|
|
|--------|----------|
|
|
| **Block** | Reject the entire output |
|
|
| **Redact** | Mask the secret (e.g., `sk-****`) |
|
|
| **Warn** | Flag but allow |
|
|
|
|
## Shell Environment Scrubbing
|
|
|
|
The shell tool scrubs sensitive environment variables:
|
|
|
|
```rust
|
|
// Before: PATH, HOME, SECRET_KEY
|
|
// After: PATH, HOME
|
|
```
|
|
|
|
Prevents secrets from leaking via `env` or `$VAR` expansion.
|
|
|
|
## Command Injection Detection
|
|
|
|
Shell commands are checked for injection attempts:
|
|
|
|
```bash
|
|
# BLOCKED: Command chaining
|
|
cat file; rm -rf /
|
|
|
|
# BLOCKED: Subshell
|
|
echo $(cat /etc/passwd)
|
|
|
|
# BLOCKED: Path traversal
|
|
cat ../../../etc/passwd
|
|
```
|
|
|
|
## Configuration
|
|
|
|
Safety settings are configured via environment variables:
|
|
|
|
```bash
|
|
# Enable/disable safety layer
|
|
export SAFETY_ENABLED=true
|
|
|
|
# Configure severity thresholds
|
|
export SAFETY_SEVERITY_THRESHOLD=medium
|
|
```
|
|
|
|
## Integration
|
|
|
|
The Safety Layer runs automatically:
|
|
|
|
1. **Input validation** — Before processing user input
|
|
2. **Tool output scanning** — Before sending to LLM
|
|
3. **LLM response scanning** — Before displaying to user
|
|
|
|
## Troubleshooting
|
|
|
|
<AccordionGroup>
|
|
<Accordion title="Content blocked unexpectedly" icon="x-circle">
|
|
- Check policy severity threshold
|
|
- Review sanitizer rules
|
|
- Consider whitelisting specific patterns
|
|
</Accordion>
|
|
|
|
<Accordion title="Leaks not detected" icon="search">
|
|
- Pattern may not be in default list
|
|
- Add custom pattern via configuration
|
|
- Check leak detector is enabled
|
|
</Accordion>
|
|
|
|
<Accordion title="Performance impact" icon="gauge">
|
|
- Safety layer adds minimal overhead
|
|
- Most checks are O(n) on content size
|
|
- Disable specific checks if needed
|
|
</Accordion>
|
|
</AccordionGroup>
|
|
|
|
## Next Steps
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Secrets" icon="lock" href="/security/secrets">
|
|
Encryption and credential management
|
|
</Card>
|
|
|
|
<Card title="Sandbox" icon="container" href="/security/sandbox">
|
|
WASM and Docker isolation
|
|
</Card>
|
|
</CardGroup>
|