[G1-Sync] Manual knowledge update

2026-05-10 22:08:15 +09:00
parent 21ac3ed255
commit 504fd5fb42
3011 changed files with 380280 additions and 206977 deletions
@@ -0,0 +1,380 @@
+---
+id: mobile-camera-ar-patterns
+title: Mobile Camera / AR — AVFoundation / CameraX / ARKit
+category: Coding
+status: draft
+source_trust_level: B
+verification_status: conceptual
+created_at: 2026-05-09
+updated_at: 2026-05-09
+tags: [mobile, camera, ar, vibe-coding]
+tech_stack: { language: "Swift / Kotlin", applicable_to: ["iOS", "Android"] }
+applied_in: []
+aliases: [AVCaptureSession, CameraX, ARKit, ARCore, RealityKit, Sceneform, AR foundation]
+---
+
+# Mobile Camera / AR
+
+> Camera = native API. **AVFoundation (iOS), CameraX (Android), ARKit / ARCore for AR**.
+
+## 📖 핵심 개념
+- Capture session.
+- Frame processing (real-time).
+- Permission.
+- AR (anchor, plane, light).
+
+## 💻 코드 패턴
+
+### iOS Camera (AVFoundation)
+```swift
+let session = AVCaptureSession()
+session.sessionPreset = .high
+
+guard let device = AVCaptureDevice.default(for: .video),
+      let input = try? AVCaptureDeviceInput(device: device) else { return }
+
+session.addInput(input)
+
+let output = AVCapturePhotoOutput()
+session.addOutput(output)
+
+let preview = AVCaptureVideoPreviewLayer(session: session)
+preview.frame = view.bounds
+view.layer.addSublayer(preview)
+
+session.startRunning()
+
+// Capture
+output.capturePhoto(with: AVCapturePhotoSettings(), delegate: self)
+```
+
+### iOS frame processing
+```swift
+let videoOutput = AVCaptureVideoDataOutput()
+videoOutput.setSampleBufferDelegate(self, queue: DispatchQueue.global(qos: .userInitiated))
+session.addOutput(videoOutput)
+
+func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
+    guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return }
+    
+    // 매 frame:
+    // - Vision / Core ML inference.
+    // - Filter / draw.
+}
+```
+
+### iOS permission
+```xml
+<!-- Info.plist -->
+<key>NSCameraUsageDescription</key>
+<string>Used to take photos.</string>
+```
+
+```swift
+AVCaptureDevice.requestAccess(for: .video) { granted in
+    if granted { setupCamera() }
+}
+```
+
+### Android CameraX
+```kotlin
+val cameraProvider = ProcessCameraProvider.getInstance(context).get()
+
+val preview = Preview.Builder().build().also {
+    it.setSurfaceProvider(viewFinder.surfaceProvider)
+}
+
+val imageCapture = ImageCapture.Builder().build()
+
+val analyzer = ImageAnalysis.Builder().build().also {
+    it.setAnalyzer(executor) { imageProxy ->
+        // 매 frame.
+        imageProxy.close()
+    }
+}
+
+cameraProvider.bindToLifecycle(
+    lifecycleOwner,
+    CameraSelector.DEFAULT_BACK_CAMERA,
+    preview, imageCapture, analyzer
+)
+```
+
+→ Modern Android camera API.
+
+### CameraX permission
+```kotlin
+val launcher = registerForActivityResult(ActivityResultContracts.RequestPermission()) { granted ->
+    if (granted) setupCamera()
+}
+
+launcher.launch(Manifest.permission.CAMERA)
+```
+
+```xml
+<uses-permission android:name='android.permission.CAMERA' />
+```
+
+### Photo capture
+```swift
+// iOS
+output.capturePhoto(with: AVCapturePhotoSettings(), delegate: self)
+
+func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {
+    let data = photo.fileDataRepresentation()
+    // Save / process.
+}
+```
+
+```kotlin
+// Android
+imageCapture.takePicture(
+    ContextCompat.getMainExecutor(context),
+    object : ImageCapture.OnImageCapturedCallback() {
+        override fun onCaptureSuccess(image: ImageProxy) {
+            // Process.
+            image.close()
+        }
+    }
+)
+```
+
+### Video record (iOS)
+```swift
+let movieOutput = AVCaptureMovieFileOutput()
+session.addOutput(movieOutput)
+
+let url = FileManager.default.temporaryDirectory.appendingPathComponent('video.mov')
+movieOutput.startRecording(to: url, recordingDelegate: self)
+
+// Stop
+movieOutput.stopRecording()
+```
+
+### Video record (Android)
+```kotlin
+val recorder = Recorder.Builder().setQualitySelector(QualitySelector.from(Quality.HD)).build()
+val videoCapture = VideoCapture.withOutput(recorder)
+
+cameraProvider.bindToLifecycle(lifecycleOwner, CameraSelector.DEFAULT_BACK_CAMERA, videoCapture)
+
+val recording = videoCapture.output
+    .prepareRecording(context, fileOutputOptions)
+    .start(executor) { event ->
+        // Status.
+    }
+
+// Stop
+recording.stop()
+```
+
+### iOS ARKit
+```swift
+import ARKit
+import RealityKit
+
+let arView = ARView(frame: view.bounds)
+view.addSubview(arView)
+
+let config = ARWorldTrackingConfiguration()
+config.planeDetection = .horizontal
+arView.session.run(config)
+
+// Place 3D object on plane
+arView.session.delegate = self
+
+func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
+    for anchor in anchors {
+        if let plane = anchor as? ARPlaneAnchor {
+            // Add 3D model.
+        }
+    }
+}
+```
+
+### Android ARCore
+```kotlin
+// Sceneform 또는 ARCore SDK
+val session = Session(context)
+val config = Config(session)
+config.planeFindingMode = Config.PlaneFindingMode.HORIZONTAL
+session.configure(config)
+
+// 매 frame.
+val frame = session.update()
+val planes = frame.getUpdatedTrackables(Plane::class.java)
+```
+
+→ Sceneform 가 deprecated. ARCore 직접.
+
+### Cross-platform AR
+```
+- Unity AR Foundation: Unity 친화.
+- React Native: react-native-arkit / react-native-arcore.
+- Flutter: ar_flutter_plugin.
+
+→ Native 가 가장 강력. Cross-platform 가 limit.
+```
+
+### ML on frame
+```swift
+// iOS Vision + Core ML
+let request = VNCoreMLRequest(model: model) { request, error in
+    // Result.
+}
+
+let handler = VNImageRequestHandler(cmSampleBuffer: sampleBuffer)
+try? handler.perform([request])
+```
+
+```kotlin
+// Android ML Kit
+val detector = TextRecognition.getClient(TextRecognizerOptions.DEFAULT_OPTIONS)
+val image = InputImage.fromMediaImage(imageProxy.image!!, imageProxy.imageInfo.rotationDegrees)
+
+detector.process(image).addOnSuccessListener { result ->
+    // text.
+}
+```
+
+### Performance
+```
+60 FPS = 16.7 ms / frame.
+ML 가 너무 느린 = drop frame.
+
+→ Background thread.
+Smaller model.
+Skip frame (every 2nd).
+```
+
+### Use case
+```
+- Photo / video capture.
+- QR / barcode (ML Kit / Vision).
+- Document scan.
+- AR (place, measure).
+- Filter (Snapchat 식).
+- OCR.
+- Face detect (Bumble 식).
+```
+
+### Camera permission UX
+```
+사용자 가 deny 후:
+- Settings 가 manual 변경.
+- Re-prompt 안 됨.
+
+→ "Why need" 명확. 첫 prompt 신중.
+```
+
+### Battery / heat
+```
+Camera 가 큰 battery + heat.
+- 사용 안 시 stop session.
+- 매 frame ML 가 필요 시만.
+- Resolution 가 use case 따라.
+```
+
+### iOS LiDAR (modern)
+```swift
+let config = ARWorldTrackingConfiguration()
+config.sceneReconstruction = .mesh
+
+if ARWorldTrackingConfiguration.supportsSceneReconstruction(.mesh) {
+    // iPhone Pro 이상.
+}
+```
+
+→ Depth + 3D mesh.
+
+### Photo metadata
+```swift
+photo.metadata     // GPS, exif, ...
+```
+
+```kotlin
+// CameraX 가 자동 set EXIF.
+```
+
+→ Privacy 신중.
+
+### Streaming (live broadcast)
+```
+- HLS / DASH 로 stream.
+- WebRTC (real-time).
+
+→ 큰 bandwidth.
+```
+
+### Filter / effect
+```swift
+// Core Image
+let context = CIContext()
+let filter = CIFilter.gaussianBlur()
+filter.setValue(ciImage, forKey: kCIInputImageKey)
+let output = filter.outputImage
+```
+
+→ Real-time gauss / sharpen / etc.
+
+### React Native
+```
+- react-native-vision-camera (modern).
+- expo-camera.
+- frame processor (worklet for ML).
+```
+
+### Flutter
+```
+- camera plugin.
+- image_picker.
+- ML 가 별 plugin.
+```
+
+### Production tips
+```
+1. Permission flow 신중.
+2. Resolution = use case 따라.
+3. Background thread (ML).
+4. Battery / heat 인지.
+5. Privacy (metadata strip).
+6. Test 가 매 device (low-end).
+```
+
+### 함정
+```
+- Permission deny: re-prompt 안 됨.
+- High resolution = slow + 큰 file.
+- ML 이 main thread: jank.
+- Background = stop session.
+- AR 가 모든 device 안 됨 (LiDAR / GPU).
+```
+
+## 🤔 의사결정 기준
+| 작업 | 추천 |
+|---|---|
+| iOS native | AVFoundation |
+| Android native | CameraX |
+| AR iOS | ARKit + RealityKit |
+| AR Android | ARCore |
+| ML detect | Vision (iOS) / ML Kit (Android) |
+| Cross-platform | RN Vision Camera / Flutter camera |
+| Document scan | VisionKit (iOS) / ML Kit (Android) |
+
+## ❌ 안티패턴
+- **Permission deny + no fallback**: bad UX.
+- **High res + ML: drop frame.
+- **Background + session running**: battery.
+- **Metadata 가 store**: privacy leak.
+- **AR 가 매 device 가정**: 안 됨.
+
+## 🤖 LLM 활용 힌트
+- AVFoundation (iOS) / CameraX (Android) 가 modern.
+- ARKit / ARCore 가 native AR.
+- ML 가 Vision / ML Kit.
+- Resolution + permission + privacy.
+
+## 🔗 관련 문서
+- [[Android_CameraX_Patterns]]
+- [[iOS_Audio_Video_AVKit]]
+- [[Android_ML_Kit_Health]]