Commit Graph

237 Commits

Author SHA1 Message Date
安正超
7cc730d9c0 ci: isolate s3 test runner state (#3180) 2026-06-02 22:40:37 +08:00
CptOfEvilMinions
e91e513ab3 feat: Helm chart support extra volumes (#2982) 2026-06-01 16:52:35 +08:00
安正超
ac97ceb744 fix(config): restore default credential startup (#3114)
* fix(config): restore default credential startup

* fix: align e2e credentials with server env

* fix(config): restore default credential consistency

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-05-29 11:52:46 +00:00
houseme
2b82432f9e fix(ecstore): send valid ping body in remote locker (#3112)
* fix(ecstore): send valid ping body in remote locker

Build ping requests with a flatbuffer payload so health checks remain compatible with the ping response parser after restart.

* fix(bench): use multi-host warp target during failover

Normalize comma-separated warp host lists in run_object_batch_bench and let four-node failover bench pass BENCH_WARP_HOSTS so rolling restart does not pin load to a single restarting node.

* feat(health): add compat health probes with busy/KMS checks

  - Add /health/live liveness probe endpoint
  - Add busy protection (429) for readiness probes, gated by RUSTFS_HEALTH_COMPAT_BUSY_CHECK_ENABLE
  - Add KMS readiness check for /health/ready, gated by RUSTFS_HEALTH_COMPAT_KMS_READY_CHECK_ENABLE
  - Add lock quorum status caching with TTL to reduce RPC pressure
  - Consolidate health response building into build_health_response_parts
  - Register /health/live in console router and readiness gate
  - Remove MinIO references from newly added health code

* fix(health): decouple kms readiness from lock quorum
2026-05-29 08:02:50 +00:00
houseme
088c4bda43 fix(ecstore): harden issue3031 multipart validation path (#3106)
* fix(ecstore): harden issue3031 multipart validation path

- clear stale multipart part destinations before rename fan-out
- add repeated part overwrite regression coverage
- reduce remote disk startup false-fault escalation to suspect-first
- refine remote locker diagnostics and lower scanner leader-lock log noise
- add a dedicated 4-node issue3031 docker validation script

* refactor(admin): inline console version json macro

- drop the unused serde_json::json import in admin console
- call serde_json::json! inline in version_handler
- keep the console version response behavior unchanged

* fix(remote-disk): recover suspect health on probe success

- record probe success during remote disk health checks so suspect drives recover
- use async_with_vars for the remote disk health probe test
- make the missing-listener test assert the state transition more robustly
2026-05-28 14:26:31 +00:00
安正超
02a7d3c228 fix: derive run script CORS console port (#3068) 2026-05-23 04:52:46 +00:00
GatewayJ
c9f0f25f55 fix: bind run script to localhost (#3063) 2026-05-22 14:28:17 +00:00
Henry Guo
0985f0b37b feat(internode): label transport operation metrics (#3045)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-21 15:20:39 +00:00
Henry Guo
69dcf9e6cb fix(tooling): harden internode transport benchmark setup (#3037)
* refactor(config): centralize internode transport constants

* fix(bench): guard all ripgrep calls behind dry-run check

Move require_cmd rg and metrics collection inside the non-dry-run
path so that --dry-run works on hosts without rg installed.

* feat(tooling): cross-platform protoc setup for Linux and macOS

Make install-protoc.sh support Linux (x86_64, aarch64) alongside
macOS, and bump CI protoc from 29.3 to 33.1 to match the version
required by the gproto build script.

* fix(bench): record internode baseline error counts

* fix(skill): correct YAML frontmatter formatting for release-version-bump

* chore(ci): bump protoc version to 34.1

* fix(tooling): bump protoc 33.1 to 34.1 in install script, restore SKILL.md description

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-05-21 05:47:32 +00:00
houseme
dcbffb084f chore(deps): refresh workspace deps and linux fs_type gating (#3030)
* chore(deps): refresh workspace deps and linux fs_type gating

- refresh workspace dependency pins and lockfile updates

- remove now-unused crate dependency entries in multiple Cargo.toml files

- enable profiling export defaults in config and scripts/run.sh

- gate os::fs_type module/function/tests to Linux to avoid non-Linux dead_code warnings

* fix(utils): simplify fs_type linux gating

- keep fs_type module-level linux cfg in os::mod

- remove redundant linux cfg on get_fs_type and test module

* chore(deps): bump s3s git revision

- update workspace s3s dependency to rev 507e1312b211c3ddc214b03875d6fabd15d22ed5

- refresh Cargo.lock source entry for s3s

* chore(dev): allow mysql_async git source and env overrides

- allow mysql_async git source in deny.toml allow-git list

- make scripts/run.sh core env vars overrideable via existing shell env

* fix(utils): import get_fs_type in fs_type tests

- add explicit super::get_fs_type import in fs_type test module

- fix Linux E0425 unresolved function errors in unit tests

* chore(dev): tune run script observability defaults

- make profiling export env overrideable in scripts/run.sh

- set RUSTFS_OBS_SAMPLE_RATIO default from 2.0 to 1.0

- update allow-git review window comments in deny.toml

* test(obs): stabilize profiling env alias tests
2026-05-20 15:03:07 +00:00
Henry Guo
19b69abe5c feat(internode): harden p0 transport boundary and baseline tooling (#3017)
* feat(internode): p0 transport baseline and ci hardening

* fix(internode): avoid double wrapping transport errors

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-20 11:06:26 +00:00
Henry Guo
e929814edc feat(ecstore): add internode transport boundary and TCP baseline runner (#3010)
* docs: add internode data transport RFC

* feat: add internode operation metrics

* fix feedback

* fix(ci): fallback protoc token to github.token

* feat(ecstore): add internode transport boundary and baseline runner

* feat(internode): harden data transport baseline

* Revert "feat(internode): harden data transport baseline"

This reverts commit 5b8d6b8aa4.

* fix(internode): address baseline review comments

* fix(ci): pin setup-protoc to stable release

* fix(ci): install protoc via apt on linux

* fix(ci): restore protoc install for macos and windows

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-19 22:41:34 +00:00
houseme
25c6bdf490 perf(filemeta): phase-1~3 rename_data metadata optimization (#3011)
* chore(perf): harden amd64 profiling benchmark flow

* fix(profiling): isolate bench buckets and map protobuf conflict

* perf: avoid blocking owned local writes

* style: format profile admin handler

* docs: clarify observability trace validation

* perf: reduce mkdir overhead on local writes

* perf: add rename_data meta microbenchmark

* perf(filemeta): fast-path data_dir decode in version meta

* perf(filemeta): collapse data-dir lookup into one scan

* perf(filemeta): reduce scan allocs and refresh meta bench

* perf(ecstore): skip mkdir path on read-only open

* perf(filemeta): single-pass unshared data-dir scan

* perf(filemeta): add two-key inline remove fast path

* perf(filemeta): compare remove-two keys by bytes first

* bench(ecstore): add remove_two-only micro benchmark

* bench(ecstore): stabilize rename_data meta benchmark timing

* bench(ecstore): align rename_data path with remove_two

* perf(filemeta): avoid uuid string alloc in remove_two

* perf(filemeta): add fast-path for empty inline data

* perf(filemeta): streamline add_version match branch

* perf(filemeta): fast-return remove_key on miss

* perf(filemeta): speed up add_version insertion lookup

* style(ecstore): normalize formatting in perf-tuning files

* refactor(filemeta): unify inline data removal paths
2026-05-19 10:20:24 +00:00
houseme
6e12289339 fix(runtime): finalize issue 2941 profiling cleanup (#2983)
* perf(runtime): narrow profiling support and upgrade starshard

* style(notify): normalize starshard imports

* perf(ecstore): reduce list_path_raw coordination overhead

* docs(scripts): add issue 2941 perf capture workflow

* fix(runtime): finalize issue 2941 profiling cleanup

* build(deps): bump quick-xml to 0.40.0

* chore(scripts): untrack local perf capture guide

* fix(scripts): honor label in perf capture output
2026-05-16 11:09:04 +00:00
安正超
6898e720dd fix(security): harden proxy auth and default credentials (#2981)
* fix(security): harden proxy auth and default credentials

* fix(security): address proxy and credential feedback
2026-05-16 04:01:50 +00:00
yihong
824c4f7673 docs: fix some dead links (#2975)
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-16 10:06:53 +08:00
安正超
f84d8983ed test(s3): promote passing SSE multipart cases (#2900) 2026-05-10 13:16:23 +00:00
安正超
6896b38dc2 test(s3): promote lifecycle expiration header tests (#2858) 2026-05-07 13:08:42 +00:00
安正超
abc07a9dc4 fix(build): quote build script features argument (#2841) 2026-05-07 02:47:53 +00:00
安正超
96d41b6349 test(helm): cover standalone scale-to-zero rendering (#2831) 2026-05-06 21:13:40 +00:00
安正超
68fcbffcb6 test(build): cover build script feature flags (#2837) 2026-05-07 05:18:51 +08:00
安正超
4728abcff1 fix(security): document unsafe and TLS overrides (#2835) 2026-05-06 15:09:02 +00:00
安正超
b10db403b6 test(s3): promote passing copy metadata case (#2832) 2026-05-06 13:58:32 +00:00
Michael Graff
3898d524fe security: same-origin console CORS, fail-closed helm creds, deny.toml, sample-config hardening (#2769)
Signed-off-by: Michael Graff <explorer@flame.org>
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-05-06 07:34:44 +00:00
houseme
3dd0692917 refactor: unify credential env constants and deploy env usage (#2821)
Co-authored-by: Henry Guo <marshawcoco@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-05-06 04:30:29 +00:00
安正超
0d7e0a814f test(ci): cover Helm Recreate strategy rendering (#2752) 2026-04-30 12:14:38 +00:00
安正超
403997f2f8 test(ci): cover Helm chart version mapping (#2751) 2026-04-30 06:11:14 +00:00
houseme
c9622611ed fix(admin): harden site-replication identity handling and local verification for issue 2723 (#2730) 2026-04-29 03:27:55 +00:00
houseme
2953558f41 fix(lifecycle): prevent eager date-expiry deletion on config update (#2708) 2026-04-28 10:26:14 +00:00
houseme
50d03ef021 perf(memory): add reclaim signals and cache controls (#2689) 2026-04-26 16:42:35 +00:00
houseme
59f41eb86a feat(obs): improve metrics coverage and dashboard performance (#2682) 2026-04-25 18:51:29 +00:00
houseme
1e9c75a201 feat(ecstore): enforce local disk topology guardrails and expose device ids (#2679) 2026-04-25 06:02:02 +00:00
唐小鸭
d949d4e794 fix: avoid sending HEAD bodies over TLS HTTP/2 (#2648)
Signed-off-by: 唐小鸭 <tangtang1251@qq.com>
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-04-25 01:33:08 +00:00
houseme
bc37cc4001 refactor(logging): unify request-id propagation and fallback metrics (#2652) 2026-04-23 07:11:27 +00:00
houseme
3ac1d2ab0b fix(security): redact target debug logs and remove eval-based bench hook (#2637) 2026-04-21 21:21:01 +00:00
houseme
960c13a34b feat(storage): wire capacity/object perf tuning and add batch benchmark runners (#2628) 2026-04-21 07:20:57 +00:00
安正超
83bac39417 test(s3): promote passing compatibility cases (#2600) 2026-04-19 14:42:56 +00:00
安正超
ac443a90ce test(s3): reclassify passing compatibility cases (#2586)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-18 09:15:29 +00:00
houseme
1cbf156559 refactor(obs): migrate metrics runtime/schema and tighten migration guards (#2584) 2026-04-18 07:51:15 +00:00
houseme
ffcf18f5f3 fix(cache): wire trusted-proxy cache and remove stale cache traces (#2581) 2026-04-17 16:57:21 +00:00
安正超
68d3dba9fc fix: revert standalone #2351 artifacts (phase 5) (#2535) 2026-04-14 22:35:05 +08:00
John
11552eb722 fix(deploy): use standard rustfs log directory (#2491)
Signed-off-by: 80347547 <jianglong@oppo.com>
Co-authored-by: 80347547 <jianglong@oppo.com>
2026-04-11 21:48:05 +08:00
houseme
32bf8f5bf3 feat(storage): add direct chunk GET fast path (#2351)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-04-07 08:33:46 +08:00
houseme
d2901fd78c feat(admin): add audit target APIs and harden target source handling (#2350)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-04-04 09:07:22 +08:00
houseme
7172e151de fix: address correctness, safety, and concurrency issues (#2327)
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-30 00:30:57 +08:00
weisd
d637c4d342 fix(object-lock): recover remaining s3 tests (#2294) 2026-03-26 12:11:34 +08:00
weisd
59c437d901 feat(object-lock): complete legal hold enforcement (#2293) 2026-03-26 10:58:10 +08:00
houseme
fb2ced4d27 feat(obs): integrate dial9-tokio-telemetry for runtime tracing (#2285)
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-03-25 14:23:58 +08:00
houseme
19b8389dc4 fix(disk): Fix Usage Report Capacity Calculation (#2274)
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-03-24 23:47:30 +08:00
weisd
28f57b228c feat(s3): advance parity coverage (#2278) 2026-03-24 17:29:33 +08:00