fabi/Sylpheed
1
0
Fork 0
Code Issues 9 Pull Requests Actions Packages Projects Releases Wiki Activity

monorepo: one repository for the decoders, the port and the corpus

Browse Source 
Merges the Godot port into the reverse-engineering repository, preserving both
histories -- 1019 commits of corpus plus the port's 31, brought in by subtree
merge and then moved into place so git can follow each file across the rename.

The reason is not tidiness. The two-repo split forced the exporter to depend on
the decoders by pinned revision, and that created a whole class of failure that
now disappears: a sha reachable only from a topic branch, orphaned by a
squash-merge, breaking a fresh checkout silently at build time. It also forced a
live read-only mount of one agent's working tree into another's container, which
is why a contract file could move mid-iteration. With a path dependency, a
decoder change and the exporter change it requires land in the same commit or
not at all.

Canary stays separate: it is a fork tracking upstream.

New structure for the long term:

  docs/game/     how the game is NAVIGATED -- menus, modals, prompts, alerts,
                 and in-game flight. Written so nobody rediscovers it. Mostly
                 open questions on purpose; the in-game tutorials are the
                 resource for the flight half.
  docs/port/MODDING.md
                 modding as a constraint on the exporter TODAY, not a later
                 feature: one logical asset in one file (the disc splits nearly
                 everything, and resolving that is the exporter's job), names a
                 person recognises, PNG/OGG/OGV/JSON only, base-and-overrides so
                 re-exporting is always safe, provenance in every file.
  data/base + data/mods
                 generated tree and drop-in overrides, both gitignored
  exchange/      transient inter-agent files, deliberately outside history
  docs/agents/   the team protocol

Both the README and the navigation doc lead with the correction that cost the
most: the oracle is the real game under Xenia Canary. Reborn's renderer is a
hypothesis under test, it has been wrong, and treating it as ground truth
propagated into three documents and both agents before a human caught it.

Scripted modding stays possible without being built: no screen name is hardcoded
in GDScript and there is no native code in port/, which is what Godot Mod Loader
needs to be able to substitute behaviour later.
This commit is contained in:
MechaCat02
2026-08-29 11:34:46 +02:00
parent f44ebced59
commit 9fbb352ef0
45 changed files with 293 additions and 1523 deletions
Download Patch File Download Diff File Expand all files Collapse all files

259
README.md
View File

@@ -1,225 +1,66 @@
# Project Sylpheed: Arc of Deception — Reborn
# Sylpheed
A clean-room, open-source reimplementation of **Project Sylpheed: Arc of Deception** (Xbox 360, 2006).
A clean-room reverse engineering and port project for **Project Sylpheed: Arc of
Deception** (Xbox 360, 2007).
Built with **Rust** and the **Bevy** game engine. Runs natively on Windows, macOS, and Linux, and in the browser via WebAssembly.
Three things live here, in one repository so that a change spanning them lands as
one commit:
> **Legal note:** This project contains no original game code or assets. You must own a legitimate copy of Project Sylpheed to use this engine. Assets remain the intellectual property of SETA Corporation / Square Enix.
| | |
|---|---|
| **The decoders** | `crates/sylpheed-formats` — the disc's formats, read and verified disc-wide |
| **The port** | `port/` — a Godot 4 project, plus `crates/sylpheed-export` which converts a disc into the open asset tree it reads |
| **The corpus** | `docs/re/` — what has been reverse engineered, with its evidence, its retractions and its dead ends |
---
**You need your own copy of the game.** No game content is in this repository and
none ever will be. The exporter reads the disc you supply.
## Current Status: Milestone 1 — Asset Explorer
## The oracle is the real game
- [x] XISO disc image reading via `xdvdfs`
- [x] Xbox 360 texture de-tiling (Morton / Z-order)
- [x] XPR2 texture container parsing
- [x] Bevy custom `AssetLoader` for `.xpr` textures
- [x] Orbit camera viewer
- [x] CLI tools: extract, list, sniff, texture info, texture export
- [x] GitHub Actions CI (Windows + macOS + Linux + WASM)
- [x] WASM / web build target
- [ ] Mesh format (reverse engineering in progress)
- [ ] Audio format (XMA → PCM pipeline)
- [ ] Mission data format
> `sylpheed-cli` and the Explorer are **tools for verifying our decoding**. They
> are hypotheses under test and they have been wrong. When something must be
> checked against the truth, the truth is **the game running in Xenia Canary**,
> captured — not any renderer of ours.
---
This is stated first because getting it backwards is the most expensive mistake
this project has made.
## Repository Structure
## Layout
```
sylpheed-reborn/
└── sylpheed-viewer/ ← Milestone 1: asset explorer (Rust/Bevy workspace)
├── Cargo.toml ← Workspace root
├── crates/
│ ├── sylpheed-formats/ # Format parsers — no Bevy dependency
│ │ ├── xiso.rs # XISO / XDVDFS disc image reader
│ │ ├── texture.rs # XPR2 container + DXT de-tiling (Morton)
│ │ ├── vfs.rs # Virtual filesystem + magic-byte sniffer
│ │ ├── mesh.rs # Mesh parser (stub — RE in progress)
│ │ └── audio.rs # Audio parser (stub — XMA TODO)
│ │
│ ├── sylpheed-viewer/ # Bevy application
│ │ ├── lib.rs # App setup + WASM entry point
│ │ ├── main.rs # Native binary entry point
│ │ ├── asset_loader.rs # Custom Bevy AssetLoaders
│ │ ├── camera.rs # Orbit camera (LMB orbit, RMB pan, scroll zoom)
│ │ └── ui.rs # egui file browser + RE notes panel
│ │
│ └── sylpheed-cli/ # Command-line tools
│ └── main.rs # extract / list / sniff / texture info commands
│
├── assets/ ← Extracted game files (gitignored)
├── .github/workflows/
│ └── ci.yml # CI: Windows + macOS + Linux + WASM
├── Trunk.toml # WASM build configuration
└── justfile # All build recipes
crates/
sylpheed-formats/ the decoders. Disc-wide verified; the corpus is its spec
sylpheed-cli/ headless tools -- render a screen, dump a table, probe audio
sylpheed-viewer/ the Explorer: a human's window onto the disc. STATIC data only
sylpheed-export/ disc -> the open, moddable asset tree
port/ the Godot 4 project. Reads open formats ONLY
authored/ decisions that are NOT on the disc, each with its reason
data/
base/ generated by the exporter. Gitignored, never hand-edited
mods/ drop-in overrides. Yours
docs/
re/ the corpus: findings, refutations, method traps
game/ how the game is navigated -- menus, modals, flight
port/ the port's mission, its handoff contract, modding rules
agents/ how the agent team works together
tools/ capture harnesses, probes, the share tool
exchange/ transient inter-agent files. NOT in git
docker/ the agent containers
```
---
## Where to start
## Getting Started
* [`docs/re/INDEX.md`](docs/re/INDEX.md) — what is decoded
* [`docs/re/REFUTED.md`](docs/re/REFUTED.md) — what has been tested and died
* [`docs/re/METHOD.md`](docs/re/METHOD.md) — traps this project has already paid for
* [`docs/game/navigation.md`](docs/game/navigation.md) — how the game is navigated
* [`docs/port/MODDING.md`](docs/port/MODDING.md) — why the asset tree looks the way it does
### Prerequisites
Xenia Canary is a **separate** repository: it is a fork tracking upstream, and it
carries our instrumentation.
```bash
# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
## Conventions
# Install just (task runner)
cargo install just
# For web builds only
cargo install trunk
rustup target add wasm32-unknown-unknown
```
#### Linux — Bevy system dependencies
```bash
sudo apt-get install -y \
libasound2-dev libudev-dev libwayland-dev \
libxkbcommon-dev libx11-dev libxi-dev pkg-config
```
### Step 1 — Extract your disc
```bash
cd sylpheed-viewer
# Extract using the CLI tool
just extract /path/to/project_sylpheed.iso
# Or manually with xdvdfs
cargo install xdvdfs-cli
xdvdfs unpack project_sylpheed.iso ./assets/
```
### Step 2 — Run the viewer
```bash
cd sylpheed-viewer
just run # Native viewer
just web # WASM dev server → http://localhost:8080
```
The asset viewer opens an orbit camera scene. Use the left panel to browse extracted game files. Click a `.xpr` file to preview it as a texture.
---
## Build Commands
All commands run from the `sylpheed-viewer/` directory.
```bash
just run # Run the native viewer (debug)
just run-dev # Run with hot-reloading
just build-native # Build native release binary
just web # WASM dev server at localhost:8080
just build-web # WASM release build → ./dist/
just extract game.iso # Extract an ISO to ./assets/
just sniff # Identify file formats in ./assets/
just sniff-unknown # Show only unrecognised formats (RE focus)
just test # Run all tests
just ci # Full CI: fmt + lint + test + WASM check
```
---
## Technology Stack
| Layer | Choice | Reason |
|-------|--------|--------|
| Language | **Rust** | Memory safety, performance, cross-platform |
| Game engine | **Bevy 0.15** | ECS-first, WASM-native, data-driven |
| XISO reading | **xdvdfs** | Pure Rust, reads Xbox 360 disc images |
| Binary parsing | **binrw** | Derive-macro based, ideal for RE work |
| Web bundler | **Trunk** | Bevy's standard WASM build tool |
| CLI | **clap** | Asset extraction and inspection tools |
| Debug UI | **bevy_egui** | In-viewer asset browser and RE notes panel |
---
## Key Architectural Decisions
**`sylpheed-formats` has zero Bevy dependency.** All binary format parsers live here and are testable with plain `cargo test`. Bevy integration is a thin layer on top in `sylpheed-viewer`.
**WASM is a first-class target.** XISO reading is gated behind `#[cfg(not(target_arch = "wasm32"))]` since the browser can't read local files. On the web, assets must be pre-extracted and served over HTTP.
**Modding is designed in from the start.** The virtual filesystem (`vfs.rs`) is the single choke point for all asset reads — a mod loader only needs to intercept that one layer to override any file.
---
## Reverse Engineering Notes
### Known file formats
| Path pattern | Format | Status | Notes |
|---|---|---|---|
| `*.XPR`, `*.XPR2` | XPR2 texture | ✅ Parsing | DXT1/3/5, DXN, Morton de-tiling done |
| `DEFAULT.XEX` | Xbox EXE 2 | 🔬 Study | Main executable — load in Ghidra (PPC BE) |
| `*.XWB`, `*.XSB` | XACT audio | ⏳ TODO | Wave/sound banks; audio is XMA codec |
| `*.pak`, `*.p00` | SETA archive | ⏳ Unknown | Paired header/data format |
| mesh files | Unknown | ⏳ TODO | Run `just sniff-unknown` to find candidates |
### RE workflow
```bash
# 1. Extract and map the disc
just extract game.iso
just sniff-unknown # shows hex magic bytes of unknown files
# 2. Hex-inspect candidates
# Groups of 12 bytes at offsets: likely f32 XYZ vertices
# Groups of 6 bytes: likely u16 triangle indices
# 3. Cross-reference in Ghidra
# Load DEFAULT.XEX with PowerPC Big-Endian processor
# Search string refs to file extensions → find load functions
# 4. Implement parser
# Add binrw #[derive(BinRead)] struct in sylpheed-formats/src/mesh.rs
# cargo test -p sylpheed-formats
```
### Next steps (Milestone 2)
1. **Mesh format** — fingerprint with `sniff-unknown`, implement `mesh.rs`
2. **Audio** — parse XWB headers, batch-convert XMA to WAV via `ffmpeg`
3. **Mission data** — format unknown; starts after mesh/texture loading works
4. **Flight model** — document by playing the original; implement as Bevy system
---
## Constraints
1. **Never copy decompiled or disassembled game code** — reimplement behavior through observation only.
2. **Assets stay gitignored** — `assets/` and `*.iso` are excluded. Never commit game files.
3. **Keep `sylpheed-formats` Bevy-free** — parsers must be testable without a GPU.
4. **WASM must always compile** — CI enforces `cargo check --target wasm32-unknown-unknown`.
5. **`justfile` is the source of truth** for build commands — add new recipes there.
---
## Milestone Roadmap
| Milestone | Goal | Status |
|---|---|---|
| 1 | Asset Explorer | 🚧 In Progress |
| 2 | Flying Tech Demo | ⏳ Planned |
| 3 | Combat Prototype | ⏳ Planned |
| 4 | Mission 1 Playable | ⏳ Planned |
| 5 | Full Game (all 16 missions) | ⏳ Planned |
| 6 | Mod SDK | ⏳ Planned |
---
## Useful References
- [xdvdfs](https://github.com/antangelo/xdvdfs) — XISO disc image reader (used in this project)
- [Xenia emulator](https://github.com/xenia-canary/xenia-canary) — GPU/API reference (cloned at `../../xenia-canary/`)
- [xboxdevwiki](https://xboxdevwiki.net) — Xbox 360 hardware documentation
- [binrw docs](https://binrw.rs) — binary format parsing framework
- [Free60 Project](https://free60project.github.io/wiki/) — open Xbox 360 hardware docs
Confidence is per claim, never per document: ✅ `CONFIRMED` · 🟡 `PROBABLE` ·
❔ `HYPOTHESIS` · ❌ `REFUTED`. A withdrawn result is kept with its reasoning
rather than deleted — that is why the numbers here can be trusted.