[G1-Sync] Manual knowledge update

2026-05-10 22:08:15 +09:00
parent 21ac3ed255
commit 504fd5fb42
3011 changed files with 380280 additions and 206977 deletions
@@ -0,0 +1,385 @@
+---
+id: ios-audio-video-avkit
+title: iOS Audio/Video — AVFoundation / AVKit
+category: Coding
+status: draft
+source_trust_level: B
+verification_status: conceptual
+created_at: 2026-05-09
+updated_at: 2026-05-09
+tags: [ios, audio, video, vibe-coding]
+tech_stack: { language: "Swift", applicable_to: ["iOS"] }
+applied_in: []
+aliases: [AVFoundation, AVPlayer, AVAudioEngine, AVKit, AirPlay, picture-in-picture]
+---
+
+# iOS Audio / Video
+
+> AVFoundation = audio/video 의 native. **AVPlayer (playback), AVAudioEngine (record/process), AVKit (UI)**. Background audio, AirPlay, PiP.
+
+## 📖 핵심 개념
+- AVPlayer: video / audio 재생.
+- AVAudioEngine: 정밀 audio (record, mix).
+- AVAudioSession: system 과 협의.
+- HLS / MP4 / DASH 지원.
+
+## 💻 코드 패턴
+
+### AVPlayer (간단 video)
+```swift
+import AVKit
+import SwiftUI
+
+struct PlayerView: View {
+    let player = AVPlayer(url: URL(string: "https://...mp4")!)
+    
+    var body: some View {
+        VideoPlayer(player: player)
+            .onAppear { player.play() }
+            .onDisappear { player.pause() }
+    }
+}
+```
+
+### AVPlayerViewController (UIKit)
+```swift
+import AVKit
+
+let player = AVPlayer(url: url)
+let vc = AVPlayerViewController()
+vc.player = player
+present(vc, animated: true) { player.play() }
+```
+
+### Background audio
+```swift
+import AVFoundation
+
+// AppDelegate / SwiftUI App
+do {
+    try AVAudioSession.sharedInstance().setCategory(.playback, mode: .default)
+    try AVAudioSession.sharedInstance().setActive(true)
+} catch {
+    print(error)
+}
+```
+
+```xml
+<!-- Info.plist -->
+<key>UIBackgroundModes</key>
+<array>
+    <string>audio</string>
+</array>
+```
+
+→ App backgrounded = audio 계속.
+
+### Now Playing Info
+```swift
+import MediaPlayer
+
+MPNowPlayingInfoCenter.default().nowPlayingInfo = [
+    MPMediaItemPropertyTitle: "Song Title",
+    MPMediaItemPropertyArtist: "Artist",
+    MPMediaItemPropertyArtwork: MPMediaItemArtwork(boundsSize: image.size) { _ in image },
+    MPMediaItemPropertyPlaybackDuration: duration,
+    MPNowPlayingInfoPropertyElapsedPlaybackTime: 0.0,
+]
+```
+
+→ Lock screen + Control Center + AirPods.
+
+### Remote command (lock screen control)
+```swift
+let center = MPRemoteCommandCenter.shared()
+center.playCommand.addTarget { _ in player.play(); return .success }
+center.pauseCommand.addTarget { _ in player.pause(); return .success }
+center.nextTrackCommand.addTarget { _ in nextTrack(); return .success }
+```
+
+### Picture-in-Picture
+```swift
+let pip = AVPictureInPictureController(playerLayer: playerLayer)
+pip?.delegate = self
+
+// 자동 또는 manual
+pip?.startPictureInPicture()
+```
+
+```xml
+<!-- Info.plist + Capability: Background Modes - Audio + Picture in Picture -->
+```
+
+### AirPlay
+```swift
+let routePicker = AVRoutePickerView()
+view.addSubview(routePicker)
+```
+
+→ AirPlay 자동 (Audio session 가 .playback).
+
+### HLS streaming
+```swift
+let asset = AVURLAsset(url: URL(string: "https://.../master.m3u8")!)
+let item = AVPlayerItem(asset: asset)
+let player = AVPlayer(playerItem: item)
+```
+
+→ HLS 가 native. Adaptive bitrate.
+
+### Download (offline)
+```swift
+import AVFoundation
+
+let session = AVAssetDownloadURLSession(...)
+let task = session.makeAssetDownloadTask(asset: asset, ...)
+task.resume()
+
+// Track progress
+task.delegate = self
+```
+
+→ HLS 가 offline 가능.
+
+### AVAudioEngine (정밀)
+```swift
+let engine = AVAudioEngine()
+let player = AVAudioPlayerNode()
+
+engine.attach(player)
+engine.connect(player, to: engine.mainMixerNode, format: nil)
+
+let file = try AVAudioFile(forReading: url)
+player.scheduleFile(file, at: nil)
+
+try engine.start()
+player.play()
+```
+
+→ Effect, mixing, recording.
+
+### Recording
+```swift
+let recorder = try AVAudioRecorder(url: url, settings: [
+    AVFormatIDKey: kAudioFormatMPEG4AAC,
+    AVSampleRateKey: 44100,
+    AVNumberOfChannelsKey: 1,
+])
+recorder.record()
+
+// Stop
+recorder.stop()
+```
+
+```swift
+// Mic permission
+AVAudioApplication.requestRecordPermission { granted in
+    if granted { ... }
+}
+```
+
+```xml
+<key>NSMicrophoneUsageDescription</key>
+<string>Record audio for ...</string>
+```
+
+### AudioKit (high-level)
+```swift
+import AudioKit
+
+let oscillator = Oscillator()
+let mixer = Mixer(oscillator)
+let engine = AudioEngine()
+engine.output = mixer
+try engine.start()
+
+oscillator.start()
+oscillator.frequency = 440
+```
+
+→ AudioKit 가 AVAudioEngine 의 wrapper.
+
+### Effect chain
+```swift
+let player = AVAudioPlayerNode()
+let reverb = AVAudioUnitReverb()
+reverb.loadFactoryPreset(.cathedral)
+reverb.wetDryMix = 50
+
+engine.attach(reverb)
+engine.connect(player, to: reverb, format: nil)
+engine.connect(reverb, to: engine.mainMixerNode, format: nil)
+```
+
+→ Multi-effect chain.
+
+### Real-time (low latency)
+```swift
+session.setPreferredIOBufferDuration(0.005)  // 5 ms
+session.setPreferredSampleRate(48000)
+```
+
+→ Latency ↓ — battery / CPU ↑.
+
+### AirPods 통합
+```swift
+// .playback 가 자동 AirPods.
+// Spatial audio (iOS 14+):
+session.setCategory(.playback, mode: .spokenAudio, options: [.allowBluetoothA2DP])
+```
+
+### Spatial audio (Dolby Atmos)
+```swift
+// AVPlayer 가 자동.
+// Track metadata 가 Dolby = automatic spatial.
+```
+
+### Video composition (edit)
+```swift
+let composition = AVMutableComposition()
+let videoTrack = composition.addMutableTrack(withMediaType: .video, preferredTrackID: kCMPersistentTrackID_Invalid)
+
+try videoTrack?.insertTimeRange(
+    CMTimeRange(start: .zero, duration: asset.duration),
+    of: asset.tracks(withMediaType: .video).first!,
+    at: .zero
+)
+
+// Export
+let exporter = AVAssetExportSession(asset: composition, presetName: AVAssetExportPresetHighestQuality)
+exporter?.outputURL = outputUrl
+exporter?.outputFileType = .mp4
+exporter?.exportAsynchronously { ... }
+```
+
+→ Trim / merge / overlay.
+
+### Capture (camera + mic)
+```swift
+let session = AVCaptureSession()
+session.sessionPreset = .high
+
+if let device = AVCaptureDevice.default(for: .video),
+   let input = try? AVCaptureDeviceInput(device: device) {
+    session.addInput(input)
+}
+
+let output = AVCaptureMovieFileOutput()
+session.addOutput(output)
+
+session.startRunning()
+
+// Record
+output.startRecording(to: url, recordingDelegate: self)
+```
+
+### CMSampleBuffer (frame-by-frame)
+```swift
+let videoOutput = AVCaptureVideoDataOutput()
+videoOutput.setSampleBufferDelegate(self, queue: queue)
+session.addOutput(videoOutput)
+
+func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
+    // 매 frame 처리 (ML, filter, ...)
+}
+```
+
+→ Vision / Core ML pipeline.
+
+### Now Playing — TimeControl
+```swift
+// Periodic time observer
+let interval = CMTime(seconds: 1, preferredTimescale: CMTimeScale(NSEC_PER_SEC))
+player.addPeriodicTimeObserver(forInterval: interval, queue: .main) { time in
+    let currentTime = time.seconds
+    nowPlayingInfo[MPNowPlayingInfoPropertyElapsedPlaybackTime] = currentTime
+    MPNowPlayingInfoCenter.default().nowPlayingInfo = nowPlayingInfo
+}
+```
+
+### Audio session interruption
+```swift
+NotificationCenter.default.addObserver(forName: AVAudioSession.interruptionNotification, object: nil, queue: .main) { note in
+    guard let info = note.userInfo,
+          let typeValue = info[AVAudioSessionInterruptionTypeKey] as? UInt,
+          let type = AVAudioSession.InterruptionType(rawValue: typeValue) else { return }
+    
+    if type == .began {
+        player.pause()
+    } else if type == .ended {
+        if let opts = info[AVAudioSessionInterruptionOptionKey] as? UInt,
+           AVAudioSession.InterruptionOptions(rawValue: opts).contains(.shouldResume) {
+            player.play()
+        }
+    }
+}
+```
+
+→ 전화 / Siri 가 interrupt.
+
+### Route change (헤드폰 unplug)
+```swift
+NotificationCenter.default.addObserver(forName: AVAudioSession.routeChangeNotification, ...) { note in
+    // 헤드폰 unplug = 자동 pause (iOS 가).
+}
+```
+
+### Performance
+```
+- Background queue 가 audio processing.
+- Main thread 가 UI 만.
+- Lazy load (큰 file).
+- Cache HLS segment.
+```
+
+### SwiftUI VideoPlayer (간단)
+```swift
+import AVKit
+import SwiftUI
+
+VideoPlayer(player: AVPlayer(url: url)) {
+    Text("Custom overlay")
+}
+```
+
+→ Built-in (iOS 14+).
+
+### 함정
+```
+- AVAudioSession 안 setCategory: silent mode 가 audio mute.
+- Background mode 안 enable: app close 시 audio 멈춤.
+- Now Playing 안 update: lock screen 가 stale.
+- Memory leak (player not released): file 도 hold.
+```
+
+## 🤔 의사결정 기준
+| 작업 | API |
+|---|---|
+| 단순 video | VideoPlayer |
+| Custom UI | AVPlayer + AVPlayerLayer |
+| Streaming | HLS + AVPlayer |
+| Recording | AVAudioRecorder |
+| Mix / effect | AVAudioEngine |
+| 정밀 frame | CMSampleBuffer |
+| Background music | .playback + Background Modes |
+| AirPlay | .playback (자동) |
+
+## ❌ 안티패턴
+- **AudioSession 무시**: silent mode 깨짐.
+- **Main thread audio process**: lag.
+- **Background mode 안 enable**: backgrounded 멈춤.
+- **Now Playing 안 update**: lock screen 잘못.
+- **Interruption handle 안**: phone call 후 재생 X.
+- **HLS 없이 mp4 streaming**: buffering 자주.
+- **Memory leak**: file 안 release.
+
+## 🤖 LLM 활용 힌트
+- AVPlayer / VideoPlayer 가 default.
+- AVAudioEngine 가 정밀.
+- Now Playing + Remote Command 가 lock screen.
+- HLS 가 streaming 표준.
+
+## 🔗 관련 문서
+- [[Mobile_Spatial_Audio_Video]]
+- [[iOS_Background_Tasks]]
+- [[iOS_Push_Notifications]]