AudioSystem/AGENTS.md

AGENTS.md — AudioSystem (OCES)

Project

Unity 2022.3.62f3, URP, C#. Custom audio & haptic middleware under the namespace OCES.

---

Agent Guidelines

• 称呼：在沟通中始终称呼用户为「思谦」或「王思谦」。
• 存疑必问：遇到任何不确定、拿不准、信息缺失或模糊的情况，不要自行推测或假设。先收集、整理好所有问题，然后停下来向思谦提问确认。宁可多问，不要猜错。

---

Architecture

Assets/Scripts/OCES/
├── ResourceLoader.cs       ← 资源缓存与同步/异步加载
├── Metronome.cs            ← demo/test helper
├── PlaySoundBind.cs        ← UI bind helpers
├── SetStateBind.cs         ← 状态切换 bind
├── SetPropertyBind.cs      ← 属性控制 bind
│
├── Audio/
│   ├── Generated/          ← AUTO-GENERATED, do not edit
│   │   ├── AudioObject.cs     ← AudioObject + AudioObjectConfig（数据类 + 加载表）
│   │   ├── AudioGroup.cs      ← AudioGroup + AudioGroupConfig
│   │   ├── AudioConsts.cs     ← Cues.Play_* 常量, NameDictionaries, Parameters(GameState enum + EnumIds)
│   │   ├── MusicSegment.cs    ← MusicSegment + MusicSegmentConfig
│   │   ├── MusicContainer.cs  ← MusicContainer + MusicContainerConfig
│   │   ├── MusicPath.cs       ← MusicPath + MusicPathConfig
│   │   ├── MusicTransition.cs ← MusicTransition + MusicTransitionConfig
│   │   ├── AmbiencePath.cs    ← AmbiencePath + AmbiencePathConfig
│   │   ├── AmbienceTransition.cs ← AmbienceTransition + AmbienceTransitionConfig
│   │   └── FileManager.cs     ← FileManager (OCES.Audio 命名空间, IBinarySerializable + 字节读取)
│   │
│   └── HandWritten/        ← hand-written runtime code
│       ├── AudioSystem.cs           ← 主入口：持有 SfxSystem + MusicSystem，对外暴露 Play / SetState / SetLowpass
│       ├── SfxSystem.cs             ← 音效系统：节流/并发/优先级/打断、连续容器播放、ActiveSound 管理
│       ├── AudioSourcePool.cs       ← AudioSource 对象池
│       ├── AudioObject.cs           ← partial: AudioObjectConfig 的 Switch 解析逻辑
│       ├── AudioExtendSettings.cs   ← 集中配置 ScriptableObject（路径/Mixer组/导入参数/淡出淡入曲线等）
│       ├── HandWrittenDefinitions.cs ← 枚举定义（KillMode, MixingType, ContainerType, AlignMode, CallbackFlags, 等）+ IBinarySerializable 接口 + IPathEntry/ITransitionConfig 接口
│       ├── DebugInfoCollector.cs    ← 运行时调试信息收集（仅在 UNITY_EDITOR || DEVELOPMENT_BUILD 编译）
│       ├── VolumeStepResolver.cs    ← 音量阶梯变化（VolumeStep）计算
│       ├── PitchStepResolver.cs     ← 音高阶梯变化（PitchStep）计算
│       ├── LongAudio/               ← interactive music & ambience systems
│       │   ├── MusicSystem.cs               ← 音乐/环境音总控，持有 MusicChannelPlayer + AmbienceChannelPlayer，由 AudioSystem 驱动
│       │   ├── MusicStateRouter.cs           ← 基于路径规则的全量状态匹配路由 + StateGroupRegistry 注册表
│       │   ├── MusicChannelPlayer.cs         ← 音乐通道：Transition 切换、节拍对齐、淡入淡出
│       │   ├── AmbienceChannelPlayer.cs      ← 环境音通道：Transition 切换、淡入淡出（无节拍对齐）
│       │   ├── LongAudioContainerPlayer.cs    ← Container 递归播放引擎，被两个 Channel 共用
│       │   ├── ChannelFader.cs               ← 音量淡入淡出 + SyncPoint 同步点管理
│       │   ├── BeatClock.cs                  ← 节拍/小节/Grid 定时回调
│       │   ├── AudioContainerSelector.cs     ← SFX Container 的随机/顺序选择（含 LimitRepetition）
│       │   ├── MusicSegment.cs               ← partial: MusicSegment.IsOffBeat + MusicSegmentConfig.Validate()
│       │   └── MusicContainer.cs             ← partial: MusicContainerConfig 的 GetBeatsPerBar/Validate
│       └── Editor/                ← Editor-only tools
│           ├── AudioImportTool.cs            ← 批量音频导入设置
│           └── AudioExtendSettingsProvider.cs ← Project Settings 面板 "Audio Extend"
│
├── Haptic/
│   ├── Generated/          ← AUTO-GENERATED, do not edit
│   │   └── HapticObject.cs    ← HapticObject + HapticObjectConfig（数据类 + 加载表）
│   │
│   └── Handwritten/        ← hand-written runtime code
│       ├── HapticSystem.cs           ← 触感反馈主系统（Preset / Emphasis / Constant / Advance 四种模式）
│       ├── HapticSystemSettings.cs   ← 触感配置 ScriptableObject（资源路径）
│       └── HandwrittenDefinitions.cs ← IBinarySerializable 接口 + FileManager + HapticType 枚举

---

Generated code boundary — critical

Everything under Assets/Scripts/OCES/*/Generated/ is auto-generated from Excel data tables. Every file has the header comment: auto generated by tools(注意:千万不要手动修改本文件).

• Generated files: data classes (AudioObject, AudioGroup, MusicSegment, MusicContainer, MusicPath, MusicTransition, AmbiencePath, AmbienceTransition, HapticObject), config loader tables (*Config classes), AudioConsts.cs (Cues IDs, NameDictionaries, Parameters enums), FileManager.cs (in OCES.Audio namespace)
• If you need to change data: edit the source .xlsx in DataTables/, then re-run the ExcelTool to regenerate .bytes configs + C# classes
• Partial class pattern: AudioObject.cs, AudioObjectConfig, MusicSegment, MusicSegmentConfig, MusicContainer, MusicContainerConfig exist in both Generated/ (serialization) and HandWritten/ (runtime logic). Add runtime methods to the HandWritten/ partial, never to Generated/.
• Generated data protocols: Generated MusicPath and AmbiencePath classes have their ITransitionConfig / IPathEntry interface implementations added via HandWrittenDefinitions.cs using partial class declarations.

Data pipeline

DataTables/*.xlsx   (gitignored, source of truth for game data)
       ↓  ExcelTool
Resources/AudioData/*.bytes   ← binary configs loaded at runtime via AudioConfigLoader
Resources/HapticData/*.bytes
       ↓  also generates
Assets/Scripts/OCES/*/Generated/*.cs   ← C# data classes + AudioConsts

Tools/ and DataTables/ are gitignored. The Excel tool and source spreadsheets live outside version control.

Configuration: AudioExtendSettings

AudioExtendSettings is a ScriptableObject singleton that centralises all configurable parameters for the audio system. It replaces hardcoded path strings and magic numbers.

• Asset location: Assets/Settings/AudioExtendSettings.asset
• Editor entry: Project Settings > Audio Extend (via AudioExtendSettingsProvider)
• Creation: Tools/Audio/Create AudioExtendSettings Asset (menu item)
• Runtime injection: AudioSystem.Awake() reads the [SerializeField] inspector reference and assigns it to AudioExtendSettings.Instance

Configured parameters

Section	Key fields
Resource Paths	audioConfigPath, audioResourcePath, audioMixerPath
AudioMixer Groups	sfxGroupPath, voiceGroupPath, accentSfxGroupPath, musicGroupPath, ambienceGroupPath
Lowpass Filter	lowpassParamName, lowpassEnabledCutoff (440Hz), lowpassDisabledCutoff (22000Hz), lowpassTweenDuration
Audio Import	compressionFormat, sfxSampleRate (22050), musicSampleRate (44100), musicQuality, sfxQuality, decompressThreshold (5s), streamingThreshold (15s)
Fade	defaultFadeOutEase (InSine), defaultFadeInEase (OutSine)

All code must read mixer group paths from AudioExtendSettings.Instance.*GroupPath — do not hardcode path strings.

Duplicate types — intentional

FileManager and IBinarySerializable exist independently in both OCES.Audio and OCES.Haptic namespaces. They are not shared.

• Audio: OCES.Audio.IBinarySerializable and OCES.Audio.FileManager are defined in Audio/Generated/FileManager.cs
• Haptic: OCES.Haptic.IBinarySerializable and OCES.Haptic.FileManager are defined in Haptic/Handwritten/HandwrittenDefinitions.cs

When adding haptic code, use OCES.Haptic.*; for audio, use OCES.Audio.*.

ResourceLoader

OCES.ResourceLoader at the OCES root is a shared resource caching layer used by both AudioSystem and HapticSystem. It provides:

• LoadSync<T>(string path) — cached synchronous load via Resources.Load<T>
• LoadAsync<T>(string path, Action<T> onComplete) — async load with callback deduplication (concurrent requests for the same path share one load operation)
• Injected into AudioSystem and HapticSystem during Awake()

Both AudioConfigLoader and HapticConfigLoader use ResourceLoader to load TextAsset config bytes.

API conventions

AudioSystem — primary entry point

// Core playback — prefer uint overloads
AudioSystem.Instance.Play(uint audioId);                // via Cues.Play_* constants
AudioSystem.Instance.Play(int audioId);                 // convenience, casts to uint
AudioSystem.Instance.Play(AudioObject audioObject);     // direct object (resolves Switch containers)

// State-driven music/ambience switching
AudioSystem.Instance.SetState<TEnum>(TEnum state);      // drives MusicStateRouter → channel switching

// Lowpass filter
AudioSystem.Instance.SetLowpass(bool enable);           // tweens mixer lowpass via DOTween

// Music sync callbacks (subscribed via events)
AudioSystem.Instance.OnBeat += containerId => { };
AudioSystem.Instance.OnBar  += containerId => { };
AudioSystem.Instance.OnGrid += containerId => { };

• Play(string) is [Obsolete] — string-based lookup is ambiguous for shared/duplicate names. It survives for backward compatibility only.
• SetState<TEnum> updates the MusicStateRouter which matches MusicPath / AmbiencePath tables and switches the appropriate Container via MusicChannelPlayer / AmbienceChannelPlayer.
• New state enums: register via StateGroupRegistry.Register<TEnum>(typeId) in Parameters.EnumIds.RegisterAllGameState() (generated in AudioConsts.cs).

SfxSystem — internal SFX runtime

SfxSystem is held by AudioSystem and not exposed publicly, but its methods are accessible via AudioSystem internally:

• Stop(uint audioId) — stop all active instances of a given audio ID
• SetVolume(uint audioId, float targetVolume) — live volume adjustment
• SetPitch(uint audioId, float targetPitch) — live pitch adjustment

SfxSystem manages: throttle (ThrottleCount), min interval (MinInterval), priority preemption (KillMode), concurrent instance tracking, continuous container playback, and haptic auto-trigger.

HapticSystem

• HapticSystem.Instance.Play(uint hapticId) — called automatically by SfxSystem when AudioObject.Haptic is set; direct calls log a warning (debug-only).
• HapticSystem.Instance.Stop(uint? hapticId = null) — stops current haptic playback
• Supports four HapticTypes: Preset, Emphasis, Constant, Advance (.haptic file)

Device capability & fallback

HapticController.Play() (Lofelt/.../HapticController.cs:350) uses a three-tier decision chain:

1. Advanced device (DeviceCapabilities.meetsAdvancedRequirements) → full .haptic playback via LofeltHaptics
2. Gamepad (GamepadRumbler.CanPlay()) → gamepad rumble
3. Basic OS support (DeviceCapabilities.isVersionSupported) → always falls back to a Preset
4. None of above → silent no-op

Advance (.haptic file) fallback is Preset-only — it never degrades to Emphasis or Constant. The fallback Preset is configured via HapticObject.FallbackPreset in the data table.

HapticType	Advanced	Android basic	iOS basic
Advance	Full .haptic	→ Preset	→ Preset
Emphasis	JSON clip	50ms max-amplitude pulse	→ amplitude-mapped Preset
Constant	JSON clip with modulation	max-amplitude for duration	→ hardcoded HeavyImpact

Emphasis and Constant have their own internal fallback chains (template-based JSON clips on advanced, raw motor patterns or Presets on basic), but these are independent direct-invoke paths — they are never used as fallback targets for Advance.

---

Interactive Music System

State-driven routing

1. AudioSystem.SetState<TEnum>(state) → forwards to MusicSystem.OnStateChanged(state)
2. MusicStateRouter.SetState() stores the state, then performs full-match against MusicPath and AmbiencePath tables
3. Path format: TypeID1,子状态|TypeID2,子状态… — all conditions must be met (AND logic). Priority field selects the best match when multiple paths satisfy.
4. Best-matched ContainerId is dispatched to MusicChannelPlayer.SwitchTo() and AmbienceChannelPlayer.SwitchTo()

Key class: MusicStateRouter — maintains ActiveStates dictionary, exposes LastMusicPathId / LastAmbiencePathId for Transition lookup.

StateGroupRegistry

Programmer-maintained enum-to-TypeId mapping. Usage:

// In Parameters.EnumIds.RegisterAllGameState() (generated):
StateGroupRegistry.Register<Parameters.GameState>(1);

The integer 1 must match the TypeId values in the MusicPath/AmbiencePath Excel tables.

Transition system

• MusicChannelPlayer handles music Transitions with beat-aligned switching (supports AlignMode.Immediate / Beat / Bar)
• AmbienceChannelPlayer handles ambience Transitions with simple cross-fade (no beat alignment)
• Both use ChannelFader for volume ramping via DG.Tweening DOFade
• Transition resolution: matches SourceContainerId → DestinationContainerId with -1 wildcard support; highest-ID exact match wins
• SyncPoint.SameAsCurrentSegment allows new music to start at the same playback position as the outgoing track

BeatClock

Drives music-sync callbacks (OnBeat, OnBar, OnGrid) using AudioSettings.dspTime:

• Parses BPM from MusicContainer.Bpm (falls back to inherited BPM from parent Container)
• Parses time signature from MusicContainer.TimeSig (e.g. "4/4")
• Grid interval from MusicContainer.Grid (in bars)
• Automatically disables callbacks when Blend containers have multiple divergent BPMs

Container playback

LongAudioContainerPlayer recursively plays a MusicContainer tree and supports:

• Container types: Random, Sequence, Blend
• Random mode: shuffle (不放回) with optional no-repeat history
• Sequence mode: global step cursor per container
• Blend mode: simultaneous playback of all child segments
• Off-beat detection: MusicSegmentConfig.Validate() checks StartOffset/EndOffset against actual clip lengths
• Returns ContainerPlayHandle for external stop/fade control

---

SFX Runtime Logic

Container playback

AudioObject supports container types (ContainerType):

Type	Behavior
Random	Random selection from child clips, with LimitRepetition and RandomType (Standard / Shuffle 不放回)
Sequence	Step-through in order
Blend	(reserved for long audio)
Switch	State-driven selection via SwitchGroupId — resolved by AudioObjectConfig.PreParseSwitchMappings()

ContainerPlayMode:

• false (Step): play one clip at a time
• true (Continuous): play all clips in sequence with gap, supports Random/Sequence

Selection helpers

• AudioContainerSelector — manages random histories, sequence cursors, LimitRepetition queues. Shared across all SFX instances.
• PitchStepResolver — incremental pitch shifting per PitchStep (+ semitones) within PitchStepThreshold (ms), clamped by PitchStepLimit
• VolumeStepResolver — incremental volume adjustment per VolumeStep (dB) within VolumeStepThreshold (ms)

Priority & preemption (KillMode)

When ThrottleCount or Group limits are exceeded:

• KillMode.Oldest — evict the oldest playing instance
• KillMode.Newest — silence the newest (incoming) instance

Mixing groups (MixingType)

Enum	Mixer path (from AudioExtendSettings)
Sfx	sfxGroupPath
Voice	voiceGroupPath
Accented	accentSfxGroupPath

ActiveSound

Tracks each playing sound instance: AudioSource, AudioObject, StartTime, CurrentLoopCount, Coroutine, Pitch, Volume, State (Pending / Playing / Finished).

---

Editor tools

AudioImportTool

Batch-applies import settings to all .wav/.ogg files in the audio resources directory. Now fully configuration-driven via AudioExtendSettings.

• Menu: Tools/Audio/Apply Audio Import Settings
• CLI entry: OCES.Audio.AudioImportTool.RunCli (via -executeMethod)
• Classification: Auto-detects Music/SFX/Voice by checking MusicSegment table → AudioObject table MixingType → filename heuristics
• Import rules (all from AudioExtendSettings):
  • SFX/Voice: sfxSampleRate, DecompressOnLoad (or Streaming if ≥ streamingThreshold)
  • Music: musicSampleRate, DecompressOnLoad (≤ decompressThreshold), Streaming (≥ streamingThreshold), CompressedInMemory (between)
  • Files referenced by AudioObject.Haptic force preloadAudioData = true

AudioExtendSettingsProvider

Provides a custom Project Settings tab (Project Settings > Audio Extend) for editing AudioExtendSettings.asset with full serialized property inspection.

ExcelTool

Assets/Editor/ExcelTool.cs — invokes external shell script to regenerate data from Excel. Currently disabled ([UnityEditor.MenuItem] is commented out).

---

Debugging

DebugInfoCollector (compiled only under UNITY_EDITOR || DEVELOPMENT_BUILD):

• Displays current active states, clip concurrent counts, and active sound list to a UI.Text component
• Updated every frame by SfxSystem.Update() and MusicStateRouter.SetState()
• Singleton pattern, accessed via DebugInfoCollector.Instance

---

Third-party dependencies

• DOTween (Assets/Scripts/DG/DOTween/) — used for low-pass filter tweening (DOTween.To), fade-in/fade-out curves (AudioSource.DOFade)
• NiceVibrations / Lofelt (Assets/Scripts/Lofelt/NiceVibrations/) — haptic feedback with native plugins per platform (Assets/Plugins/{Android,iOS,macOS,Windows}/)

---

Audio file naming conventions

• au_sfx_* — sound effects
• au_bgm_* — background music
• au_music_* — music segments (used by the interactive music system)
• au_stream.ogg — streaming audio

---

Resources paths

All paths are configured via AudioExtendSettings at runtime. The defaults are:

Resource	Path	Access
Audio clips	Resources/Audios/	ResourceLoader.LoadSync<AudioClip>
Audio configs	Resources/AudioData/	AudioConfigLoader.Load<T> (file-based) or HapticConfigLoader.Load<T> (Resources-based)
Haptic configs	Resources/HapticData/	HapticConfigLoader.Load<T>
Haptic clips	Resources/Haptics/	.haptic files for advanced haptic playback
AudioMixer	Resources/Audios/Master	Resources.Load<AudioMixer>

---

Key enums (defined in HandWrittenDefinitions.cs)

Enum	Values	Purpose
KillMode	Oldest, Newest	Priority preemption strategy
MixingType	Sfx, Voice, Accented	Output mixer group routing
ContainerType	Random, Sequence, Blend, Switch	Container playback strategy
BlendCrossFadeType	Exponential, Linear, Logarithmic	(reserved)
AlignMode	Immediate, Beat, Bar	Music transition timing
CallbackFlags	MusicSyncBeat, MusicSyncBar, MusicSyncGrid	(reserved)
ActiveSoundState	Pending, Playing, Finished	Playback lifecycle
SyncPoint	Start, SameAsCurrentSegment	Crossfade sync mode
SyncSegment	Start, LastPlayedSegment	(reserved)

Parameters.GameState (generated in AudioConsts.cs) is the default game state enum: Home, Game, Win, Lose, TestA, TestB.