Files
AudioSystem/AGENTS.md
T
2026-06-10 11:04:45 +08:00

376 lines
20 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# AGENTS.md — AudioSystem (OCES)
## Project
Unity 2022.3.62f3, URP, C#. Custom audio & haptic middleware under the namespace **OCES**.
---
## Agent Guidelines
- **称呼**:在沟通中始终称呼用户为「思谦」或「王思谦」。
- **存疑必问**:遇到任何不确定、拿不准、信息缺失或模糊的情况,不要自行推测或假设。先收集、整理好所有问题,然后停下来向思谦提问确认。宁可多问,不要猜错。
---
## Architecture
```
Assets/Scripts/OCES/
├── ResourceLoader.cs ← 资源缓存与同步/异步加载
├── Metronome.cs ← demo/test helper
├── PlaySoundBind.cs ← UI bind helpers
├── SetStateBind.cs ← 状态切换 bind
├── SetPropertyBind.cs ← 属性控制 bind
├── Audio/
│ ├── Generated/ ← AUTO-GENERATED, do not edit
│ │ ├── AudioObject.cs ← AudioObject + AudioObjectConfig(数据类 + 加载表)
│ │ ├── AudioGroup.cs ← AudioGroup + AudioGroupConfig
│ │ ├── AudioConsts.cs ← Cues.Play_* 常量, NameDictionaries, Parameters(GameState enum + EnumIds)
│ │ ├── MusicSegment.cs ← MusicSegment + MusicSegmentConfig
│ │ ├── MusicContainer.cs ← MusicContainer + MusicContainerConfig
│ │ ├── MusicPath.cs ← MusicPath + MusicPathConfig
│ │ ├── MusicTransition.cs ← MusicTransition + MusicTransitionConfig
│ │ ├── AmbiencePath.cs ← AmbiencePath + AmbiencePathConfig
│ │ ├── AmbienceTransition.cs ← AmbienceTransition + AmbienceTransitionConfig
│ │ └── FileManager.cs ← FileManager (OCES.Audio 命名空间, IBinarySerializable + 字节读取)
│ │
│ └── HandWritten/ ← hand-written runtime code
│ ├── AudioSystem.cs ← 主入口:持有 SfxSystem + MusicSystem,对外暴露 Play / SetState / SetLowpass
│ ├── SfxSystem.cs ← 音效系统:节流/并发/优先级/打断、连续容器播放、ActiveSound 管理
│ ├── AudioSourcePool.cs ← AudioSource 对象池
│ ├── AudioObject.cs ← partial: AudioObjectConfig 的 Switch 解析逻辑
│ ├── AudioExtendSettings.cs ← 集中配置 ScriptableObject(路径/Mixer组/导入参数/淡出淡入曲线等)
│ ├── HandWrittenDefinitions.cs ← 枚举定义(KillMode, MixingType, ContainerType, AlignMode, CallbackFlags, 等)+ IBinarySerializable 接口 + IPathEntry/ITransitionConfig 接口
│ ├── DebugInfoCollector.cs ← 运行时调试信息收集(仅在 UNITY_EDITOR || DEVELOPMENT_BUILD 编译)
│ ├── VolumeStepResolver.cs ← 音量阶梯变化(VolumeStep)计算
│ ├── PitchStepResolver.cs ← 音高阶梯变化(PitchStep)计算
│ ├── LongAudio/ ← interactive music & ambience systems
│ │ ├── MusicSystem.cs ← 音乐/环境音总控,持有 MusicChannelPlayer + AmbienceChannelPlayer,由 AudioSystem 驱动
│ │ ├── MusicStateRouter.cs ← 基于路径规则的全量状态匹配路由 + StateGroupRegistry 注册表
│ │ ├── MusicChannelPlayer.cs ← 音乐通道:Transition 切换、节拍对齐、淡入淡出
│ │ ├── AmbienceChannelPlayer.cs ← 环境音通道:Transition 切换、淡入淡出(无节拍对齐)
│ │ ├── LongAudioContainerPlayer.cs ← Container 递归播放引擎,被两个 Channel 共用
│ │ ├── ChannelFader.cs ← 音量淡入淡出 + SyncPoint 同步点管理
│ │ ├── BeatClock.cs ← 节拍/小节/Grid 定时回调
│ │ ├── AudioContainerSelector.cs ← SFX Container 的随机/顺序选择(含 LimitRepetition
│ │ ├── MusicSegment.cs ← partial: MusicSegment.IsOffBeat + MusicSegmentConfig.Validate()
│ │ └── MusicContainer.cs ← partial: MusicContainerConfig 的 GetBeatsPerBar/Validate
│ └── Editor/ ← Editor-only tools
│ ├── AudioImportTool.cs ← 批量音频导入设置
│ └── AudioExtendSettingsProvider.cs ← Project Settings 面板 "Audio Extend"
├── Haptic/
│ ├── Generated/ ← AUTO-GENERATED, do not edit
│ │ └── HapticObject.cs ← HapticObject + HapticObjectConfig(数据类 + 加载表)
│ │
│ └── Handwritten/ ← hand-written runtime code
│ ├── HapticSystem.cs ← 触感反馈主系统(Preset / Emphasis / Constant / Advance 四种模式)
│ ├── HapticSystemSettings.cs ← 触感配置 ScriptableObject(资源路径)
│ └── HandwrittenDefinitions.cs ← IBinarySerializable 接口 + FileManager + HapticType 枚举
```
---
## Generated code boundary — critical
Everything under `Assets/Scripts/OCES/*/Generated/` is auto-generated from Excel data tables. Every file has the header comment: `auto generated by tools(注意:千万不要手动修改本文件)`.
- **Generated files**: data classes (`AudioObject`, `AudioGroup`, `MusicSegment`, `MusicContainer`, `MusicPath`, `MusicTransition`, `AmbiencePath`, `AmbienceTransition`, `HapticObject`), config loader tables (`*Config` classes), `AudioConsts.cs` (Cues IDs, NameDictionaries, Parameters enums), `FileManager.cs` (in `OCES.Audio` namespace)
- **If you need to change data**: edit the source `.xlsx` in `DataTables/`, then re-run the ExcelTool to regenerate `.bytes` configs + C# classes
- **Partial class pattern**: `AudioObject.cs`, `AudioObjectConfig`, `MusicSegment`, `MusicSegmentConfig`, `MusicContainer`, `MusicContainerConfig` exist in both `Generated/` (serialization) and `HandWritten/` (runtime logic). Add runtime methods to the `HandWritten/` partial, never to `Generated/`.
- **Generated data protocols**: Generated `MusicPath` and `AmbiencePath` classes have their `ITransitionConfig` / `IPathEntry` interface implementations added via `HandWrittenDefinitions.cs` using `partial class` declarations.
## Data pipeline
```
DataTables/*.xlsx (gitignored, source of truth for game data)
↓ ExcelTool
Resources/AudioData/*.bytes ← binary configs loaded at runtime via AudioConfigLoader
Resources/HapticData/*.bytes
↓ also generates
Assets/Scripts/OCES/*/Generated/*.cs ← C# data classes + AudioConsts
```
`Tools/` and `DataTables/` are gitignored. The Excel tool and source spreadsheets live outside version control.
## Configuration: AudioExtendSettings
`AudioExtendSettings` is a **ScriptableObject singleton** that centralises all configurable parameters for the audio system. It replaces hardcoded path strings and magic numbers.
- **Asset location**: `Assets/Settings/AudioExtendSettings.asset`
- **Editor entry**: `Project Settings > Audio Extend` (via `AudioExtendSettingsProvider`)
- **Creation**: `Tools/Audio/Create AudioExtendSettings Asset` (menu item)
- **Runtime injection**: `AudioSystem.Awake()` reads the `[SerializeField]` inspector reference and assigns it to `AudioExtendSettings.Instance`
### Configured parameters
| Section | Key fields |
|---------|-----------|
| Resource Paths | `audioConfigPath`, `audioResourcePath`, `audioMixerPath` |
| AudioMixer Groups | `sfxGroupPath`, `voiceGroupPath`, `accentSfxGroupPath`, `musicGroupPath`, `ambienceGroupPath` |
| Lowpass Filter | `lowpassParamName`, `lowpassEnabledCutoff` (440Hz), `lowpassDisabledCutoff` (22000Hz), `lowpassTweenDuration` |
| Audio Import | `compressionFormat`, `sfxSampleRate` (22050), `musicSampleRate` (44100), `musicQuality`, `sfxQuality`, `decompressThreshold` (5s), `streamingThreshold` (15s) |
| Fade | `defaultFadeOutEase` (InSine), `defaultFadeInEase` (OutSine) |
**All code must read mixer group paths from `AudioExtendSettings.Instance.*GroupPath`** — do not hardcode path strings.
## Duplicate types — intentional
`FileManager` and `IBinarySerializable` exist independently in both `OCES.Audio` and `OCES.Haptic` namespaces. They are not shared.
- **Audio**: `OCES.Audio.IBinarySerializable` and `OCES.Audio.FileManager` are defined in `Audio/Generated/FileManager.cs`
- **Haptic**: `OCES.Haptic.IBinarySerializable` and `OCES.Haptic.FileManager` are defined in `Haptic/Handwritten/HandwrittenDefinitions.cs`
When adding haptic code, use `OCES.Haptic.*`; for audio, use `OCES.Audio.*`.
## ResourceLoader
`OCES.ResourceLoader` at the OCES root is a shared resource caching layer used by both `AudioSystem` and `HapticSystem`. It provides:
- `LoadSync<T>(string path)` — cached synchronous load via `Resources.Load<T>`
- `LoadAsync<T>(string path, Action<T> onComplete)` — async load with callback deduplication (concurrent requests for the same path share one load operation)
- Injected into `AudioSystem` and `HapticSystem` during `Awake()`
Both `AudioConfigLoader` and `HapticConfigLoader` use `ResourceLoader` to load `TextAsset` config bytes.
## API conventions
### AudioSystem — primary entry point
```csharp
// Core playback — prefer uint overloads
AudioSystem.Instance.Play(uint audioId); // via Cues.Play_* constants
AudioSystem.Instance.Play(int audioId); // convenience, casts to uint
AudioSystem.Instance.Play(AudioObject audioObject); // direct object (resolves Switch containers)
// State-driven music/ambience switching
AudioSystem.Instance.SetState<TEnum>(TEnum state); // drives MusicStateRouter → channel switching
// Lowpass filter
AudioSystem.Instance.SetLowpass(bool enable); // tweens mixer lowpass via DOTween
// Music sync callbacks (subscribed via events)
AudioSystem.Instance.OnBeat += containerId => { };
AudioSystem.Instance.OnBar += containerId => { };
AudioSystem.Instance.OnGrid += containerId => { };
```
- `Play(string)` is `[Obsolete]` — string-based lookup is ambiguous for shared/duplicate names. It survives for backward compatibility only.
- `SetState<TEnum>` updates the `MusicStateRouter` which matches `MusicPath` / `AmbiencePath` tables and switches the appropriate Container via `MusicChannelPlayer` / `AmbienceChannelPlayer`.
- **New state enums**: register via `StateGroupRegistry.Register<TEnum>(typeId)` in `Parameters.EnumIds.RegisterAllGameState()` (generated in `AudioConsts.cs`).
### SfxSystem — internal SFX runtime
`SfxSystem` is held by `AudioSystem` and not exposed publicly, but its methods are accessible via `AudioSystem` internally:
- `Stop(uint audioId)` — stop all active instances of a given audio ID
- `SetVolume(uint audioId, float targetVolume)` — live volume adjustment
- `SetPitch(uint audioId, float targetPitch)` — live pitch adjustment
SfxSystem manages: throttle (`ThrottleCount`), min interval (`MinInterval`), priority preemption (`KillMode`), concurrent instance tracking, continuous container playback, and haptic auto-trigger.
### HapticSystem
- `HapticSystem.Instance.Play(uint hapticId)` — called automatically by SfxSystem when `AudioObject.Haptic` is set; direct calls log a warning (debug-only).
- `HapticSystem.Instance.Stop(uint? hapticId = null)` — stops current haptic playback
- Supports four `HapticType`s: `Preset`, `Emphasis`, `Constant`, `Advance` (`.haptic` file)
#### Device capability & fallback
`HapticController.Play()` (`Lofelt/.../HapticController.cs:350`) uses a three-tier decision chain:
1. **Advanced device** (`DeviceCapabilities.meetsAdvancedRequirements`) → full `.haptic` playback via `LofeltHaptics`
2. **Gamepad** (`GamepadRumbler.CanPlay()`) → gamepad rumble
3. **Basic OS support** (`DeviceCapabilities.isVersionSupported`) → **always falls back to a Preset**
4. None of above → silent no-op
**Advance (`.haptic` file) fallback is Preset-only** — it never degrades to Emphasis or Constant. The fallback Preset is configured via `HapticObject.FallbackPreset` in the data table.
| HapticType | Advanced | Android basic | iOS basic |
|------------|----------|---------------|-----------|
| **Advance** | Full `.haptic` | → Preset | → Preset |
| **Emphasis** | JSON clip | 50ms max-amplitude pulse | → amplitude-mapped Preset |
| **Constant** | JSON clip with modulation | max-amplitude for `duration` | → hardcoded `HeavyImpact` |
Emphasis and Constant have their own internal fallback chains (template-based JSON clips on advanced, raw motor patterns or Presets on basic), but these are independent direct-invoke paths — they are never used as fallback targets for Advance.
---
## Interactive Music System
### State-driven routing
1. `AudioSystem.SetState<TEnum>(state)` → forwards to `MusicSystem.OnStateChanged(state)`
2. `MusicStateRouter.SetState()` stores the state, then performs full-match against `MusicPath` and `AmbiencePath` tables
3. Path format: `TypeID1,子状态|TypeID2,子状态…` — all conditions must be met (AND logic). Priority field selects the best match when multiple paths satisfy.
4. Best-matched `ContainerId` is dispatched to `MusicChannelPlayer.SwitchTo()` and `AmbienceChannelPlayer.SwitchTo()`
Key class: `MusicStateRouter` — maintains `ActiveStates` dictionary, exposes `LastMusicPathId` / `LastAmbiencePathId` for Transition lookup.
### StateGroupRegistry
Programmer-maintained enum-to-TypeId mapping. Usage:
```csharp
// In Parameters.EnumIds.RegisterAllGameState() (generated):
StateGroupRegistry.Register<Parameters.GameState>(1);
```
The integer `1` must match the TypeId values in the MusicPath/AmbiencePath Excel tables.
### Transition system
- `MusicChannelPlayer` handles music Transitions with **beat-aligned switching** (supports `AlignMode.Immediate` / `Beat` / `Bar`)
- `AmbienceChannelPlayer` handles ambience Transitions with **simple cross-fade** (no beat alignment)
- Both use `ChannelFader` for volume ramping via DG.Tweening `DOFade`
- Transition resolution: matches `SourceContainerId → DestinationContainerId` with `-1` wildcard support; highest-ID exact match wins
- `SyncPoint.SameAsCurrentSegment` allows new music to start at the same playback position as the outgoing track
### BeatClock
Drives music-sync callbacks (`OnBeat`, `OnBar`, `OnGrid`) using `AudioSettings.dspTime`:
- Parses BPM from `MusicContainer.Bpm` (falls back to inherited BPM from parent Container)
- Parses time signature from `MusicContainer.TimeSig` (e.g. `"4/4"`)
- Grid interval from `MusicContainer.Grid` (in bars)
- Automatically disables callbacks when Blend containers have multiple divergent BPMs
### Container playback
`LongAudioContainerPlayer` recursively plays a `MusicContainer` tree and supports:
- **Container types**: `Random`, `Sequence`, `Blend`
- **Random mode**: shuffle (不放回) with optional no-repeat history
- **Sequence mode**: global step cursor per container
- **Blend mode**: simultaneous playback of all child segments
- **Off-beat detection**: `MusicSegmentConfig.Validate()` checks `StartOffset`/`EndOffset` against actual clip lengths
- Returns `ContainerPlayHandle` for external stop/fade control
---
## SFX Runtime Logic
### Container playback
AudioObject supports container types (`ContainerType`):
| Type | Behavior |
|------|----------|
| `Random` | Random selection from child clips, with `LimitRepetition` and `RandomType` (Standard / Shuffle 不放回) |
| `Sequence` | Step-through in order |
| `Blend` | (reserved for long audio) |
| `Switch` | State-driven selection via `SwitchGroupId` — resolved by `AudioObjectConfig.PreParseSwitchMappings()` |
`ContainerPlayMode`:
- `false` (Step): play one clip at a time
- `true` (Continuous): play all clips in sequence with gap, supports Random/Sequence
### Selection helpers
- `AudioContainerSelector` — manages random histories, sequence cursors, `LimitRepetition` queues. Shared across all SFX instances.
- `PitchStepResolver` — incremental pitch shifting per `PitchStep` (+ semitones) within `PitchStepThreshold` (ms), clamped by `PitchStepLimit`
- `VolumeStepResolver` — incremental volume adjustment per `VolumeStep` (dB) within `VolumeStepThreshold` (ms)
### Priority & preemption (KillMode)
When `ThrottleCount` or `Group` limits are exceeded:
- `KillMode.Oldest` — evict the oldest playing instance
- `KillMode.Newest` — silence the newest (incoming) instance
### Mixing groups (MixingType)
| Enum | Mixer path (from AudioExtendSettings) |
|------|--------------------------------------|
| `Sfx` | `sfxGroupPath` |
| `Voice` | `voiceGroupPath` |
| `Accented` | `accentSfxGroupPath` |
### ActiveSound
Tracks each playing sound instance: `AudioSource`, `AudioObject`, `StartTime`, `CurrentLoopCount`, `Coroutine`, `Pitch`, `Volume`, `State` (`Pending` / `Playing` / `Finished`).
---
## Editor tools
### AudioImportTool
Batch-applies import settings to all `.wav`/`.ogg` files in the audio resources directory. Now fully configuration-driven via `AudioExtendSettings`.
- **Menu**: `Tools/Audio/Apply Audio Import Settings`
- **CLI entry**: `OCES.Audio.AudioImportTool.RunCli` (via `-executeMethod`)
- **Classification**: Auto-detects Music/SFX/Voice by checking `MusicSegment` table → `AudioObject` table `MixingType` → filename heuristics
- **Import rules** (all from `AudioExtendSettings`):
- SFX/Voice: `sfxSampleRate`, DecompressOnLoad (or Streaming if ≥ `streamingThreshold`)
- Music: `musicSampleRate`, DecompressOnLoad (≤ `decompressThreshold`), Streaming (≥ `streamingThreshold`), CompressedInMemory (between)
- Files referenced by `AudioObject.Haptic` force `preloadAudioData = true`
### AudioExtendSettingsProvider
Provides a custom Project Settings tab (`Project Settings > Audio Extend`) for editing `AudioExtendSettings.asset` with full serialized property inspection.
### ExcelTool
`Assets/Editor/ExcelTool.cs` — invokes external shell script to regenerate data from Excel. Currently disabled (`[UnityEditor.MenuItem]` is commented out).
---
## Debugging
`DebugInfoCollector` (compiled only under `UNITY_EDITOR || DEVELOPMENT_BUILD`):
- Displays current active states, clip concurrent counts, and active sound list to a `UI.Text` component
- Updated every frame by `SfxSystem.Update()` and `MusicStateRouter.SetState()`
- Singleton pattern, accessed via `DebugInfoCollector.Instance`
---
## Third-party dependencies
- **DOTween** (`Assets/Scripts/DG/DOTween/`) — used for low-pass filter tweening (`DOTween.To`), fade-in/fade-out curves (`AudioSource.DOFade`)
- **NiceVibrations / Lofelt** (`Assets/Scripts/Lofelt/NiceVibrations/`) — haptic feedback with native plugins per platform (`Assets/Plugins/{Android,iOS,macOS,Windows}/`)
---
## Audio file naming conventions
- `au_sfx_*` — sound effects
- `au_bgm_*` — background music
- `au_music_*` — music segments (used by the interactive music system)
- `au_stream.ogg` — streaming audio
---
## Resources paths
All paths are configured via `AudioExtendSettings` at runtime. The defaults are:
| Resource | Path | Access |
|----------|------|--------|
| Audio clips | `Resources/Audios/` | `ResourceLoader.LoadSync<AudioClip>` |
| Audio configs | `Resources/AudioData/` | `AudioConfigLoader.Load<T>` (file-based) or `HapticConfigLoader.Load<T>` (Resources-based) |
| Haptic configs | `Resources/HapticData/` | `HapticConfigLoader.Load<T>` |
| Haptic clips | `Resources/Haptics/` | `.haptic` files for advanced haptic playback |
| AudioMixer | `Resources/Audios/Master` | `Resources.Load<AudioMixer>` |
---
## Key enums (defined in `HandWrittenDefinitions.cs`)
| Enum | Values | Purpose |
|------|--------|---------|
| `KillMode` | `Oldest`, `Newest` | Priority preemption strategy |
| `MixingType` | `Sfx`, `Voice`, `Accented` | Output mixer group routing |
| `ContainerType` | `Random`, `Sequence`, `Blend`, `Switch` | Container playback strategy |
| `BlendCrossFadeType` | `Exponential`, `Linear`, `Logarithmic` | (reserved) |
| `AlignMode` | `Immediate`, `Beat`, `Bar` | Music transition timing |
| `CallbackFlags` | `MusicSyncBeat`, `MusicSyncBar`, `MusicSyncGrid` | (reserved) |
| `ActiveSoundState` | `Pending`, `Playing`, `Finished` | Playback lifecycle |
| `SyncPoint` | `Start`, `SameAsCurrentSegment` | Crossfade sync mode |
| `SyncSegment` | `Start`, `LastPlayedSegment` | (reserved) |
`Parameters.GameState` (generated in `AudioConsts.cs`) is the default game state enum: `Home`, `Game`, `Win`, `Lose`, `TestA`, `TestB`.