Files

T

Antigravity Agent 504fd5fb42 [G1-Sync] Manual knowledge update

2026-05-10 22:08:15 +09:00

8.3 KiB

Raw Blame History

id, title, category, status, source_trust_level, verification_status, created_at, updated_at, tags, tech_stack, applied_in, aliases

title

Mobile Camera / AR

Camera = native API. AVFoundation (iOS), CameraX (Android), ARKit / ARCore for AR.

📖 핵심 개념

Capture session.
Frame processing (real-time).
Permission.
AR (anchor, plane, light).

💻 코드 패턴

iOS Camera (AVFoundation)

let session = AVCaptureSession()
session.sessionPreset = .high

guard let device = AVCaptureDevice.default(for: .video),
      let input = try? AVCaptureDeviceInput(device: device) else { return }

session.addInput(input)

let output = AVCapturePhotoOutput()
session.addOutput(output)

let preview = AVCaptureVideoPreviewLayer(session: session)
preview.frame = view.bounds
view.layer.addSublayer(preview)

session.startRunning()

// Capture
output.capturePhoto(with: AVCapturePhotoSettings(), delegate: self)

iOS frame processing

let videoOutput = AVCaptureVideoDataOutput()
videoOutput.setSampleBufferDelegate(self, queue: DispatchQueue.global(qos: .userInitiated))
session.addOutput(videoOutput)

func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
    guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return }
    
    // 매 frame:
    // - Vision / Core ML inference.
    // - Filter / draw.
}

iOS permission

<!-- Info.plist -->
<key>NSCameraUsageDescription</key>
<string>Used to take photos.</string>

AVCaptureDevice.requestAccess(for: .video) { granted in
    if granted { setupCamera() }
}

Android CameraX

val cameraProvider = ProcessCameraProvider.getInstance(context).get()

val preview = Preview.Builder().build().also {
    it.setSurfaceProvider(viewFinder.surfaceProvider)
}

val imageCapture = ImageCapture.Builder().build()

val analyzer = ImageAnalysis.Builder().build().also {
    it.setAnalyzer(executor) { imageProxy ->
        // 매 frame.
        imageProxy.close()
    }
}

cameraProvider.bindToLifecycle(
    lifecycleOwner,
    CameraSelector.DEFAULT_BACK_CAMERA,
    preview, imageCapture, analyzer
)

→ Modern Android camera API.

CameraX permission

val launcher = registerForActivityResult(ActivityResultContracts.RequestPermission()) { granted ->
    if (granted) setupCamera()
}

launcher.launch(Manifest.permission.CAMERA)

<uses-permission android:name='android.permission.CAMERA' />

Photo capture

// iOS
output.capturePhoto(with: AVCapturePhotoSettings(), delegate: self)

func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {
    let data = photo.fileDataRepresentation()
    // Save / process.
}

// Android
imageCapture.takePicture(
    ContextCompat.getMainExecutor(context),
    object : ImageCapture.OnImageCapturedCallback() {
        override fun onCaptureSuccess(image: ImageProxy) {
            // Process.
            image.close()
        }
    }
)

Video record (iOS)

let movieOutput = AVCaptureMovieFileOutput()
session.addOutput(movieOutput)

let url = FileManager.default.temporaryDirectory.appendingPathComponent('video.mov')
movieOutput.startRecording(to: url, recordingDelegate: self)

// Stop
movieOutput.stopRecording()

Video record (Android)

val recorder = Recorder.Builder().setQualitySelector(QualitySelector.from(Quality.HD)).build()
val videoCapture = VideoCapture.withOutput(recorder)

cameraProvider.bindToLifecycle(lifecycleOwner, CameraSelector.DEFAULT_BACK_CAMERA, videoCapture)

val recording = videoCapture.output
    .prepareRecording(context, fileOutputOptions)
    .start(executor) { event ->
        // Status.
    }

// Stop
recording.stop()

iOS ARKit

import ARKit
import RealityKit

let arView = ARView(frame: view.bounds)
view.addSubview(arView)

let config = ARWorldTrackingConfiguration()
config.planeDetection = .horizontal
arView.session.run(config)

// Place 3D object on plane
arView.session.delegate = self

func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
    for anchor in anchors {
        if let plane = anchor as? ARPlaneAnchor {
            // Add 3D model.
        }
    }
}

Android ARCore

// Sceneform 또는 ARCore SDK
val session = Session(context)
val config = Config(session)
config.planeFindingMode = Config.PlaneFindingMode.HORIZONTAL
session.configure(config)

// 매 frame.
val frame = session.update()
val planes = frame.getUpdatedTrackables(Plane::class.java)

→ Sceneform 가 deprecated. ARCore 직접.

Cross-platform AR

- Unity AR Foundation: Unity 친화.
- React Native: react-native-arkit / react-native-arcore.
- Flutter: ar_flutter_plugin.

→ Native 가 가장 강력. Cross-platform 가 limit.

ML on frame

// iOS Vision + Core ML
let request = VNCoreMLRequest(model: model) { request, error in
    // Result.
}

let handler = VNImageRequestHandler(cmSampleBuffer: sampleBuffer)
try? handler.perform([request])

// Android ML Kit
val detector = TextRecognition.getClient(TextRecognizerOptions.DEFAULT_OPTIONS)
val image = InputImage.fromMediaImage(imageProxy.image!!, imageProxy.imageInfo.rotationDegrees)

detector.process(image).addOnSuccessListener { result ->
    // text.
}

Performance

60 FPS = 16.7 ms / frame.
ML 가 너무 느린 = drop frame.

→ Background thread.
Smaller model.
Skip frame (every 2nd).

Use case

- Photo / video capture.
- QR / barcode (ML Kit / Vision).
- Document scan.
- AR (place, measure).
- Filter (Snapchat 식).
- OCR.
- Face detect (Bumble 식).

Camera permission UX

사용자 가 deny 후:
- Settings 가 manual 변경.
- Re-prompt 안 됨.

→ "Why need" 명확. 첫 prompt 신중.

Battery / heat

Camera 가 큰 battery + heat.
- 사용 안 시 stop session.
- 매 frame ML 가 필요 시만.
- Resolution 가 use case 따라.

iOS LiDAR (modern)

let config = ARWorldTrackingConfiguration()
config.sceneReconstruction = .mesh

if ARWorldTrackingConfiguration.supportsSceneReconstruction(.mesh) {
    // iPhone Pro 이상.
}

→ Depth + 3D mesh.

Photo metadata

photo.metadata     // GPS, exif, ...

// CameraX 가 자동 set EXIF.

→ Privacy 신중.

Streaming (live broadcast)

- HLS / DASH 로 stream.
- WebRTC (real-time).

→ 큰 bandwidth.

Filter / effect

// Core Image
let context = CIContext()
let filter = CIFilter.gaussianBlur()
filter.setValue(ciImage, forKey: kCIInputImageKey)
let output = filter.outputImage

→ Real-time gauss / sharpen / etc.

React Native

- react-native-vision-camera (modern).
- expo-camera.
- frame processor (worklet for ML).

Flutter

- camera plugin.
- image_picker.
- ML 가 별 plugin.

Production tips

1. Permission flow 신중.
2. Resolution = use case 따라.
3. Background thread (ML).
4. Battery / heat 인지.
5. Privacy (metadata strip).
6. Test 가 매 device (low-end).

함정

- Permission deny: re-prompt 안 됨.
- High resolution = slow + 큰 file.
- ML 이 main thread: jank.
- Background = stop session.
- AR 가 모든 device 안 됨 (LiDAR / GPU).

🤔 의사결정 기준

작업	추천
iOS native	AVFoundation
Android native	CameraX
AR iOS	ARKit + RealityKit
AR Android	ARCore
ML detect	Vision (iOS) / ML Kit (Android)
Cross-platform	RN Vision Camera / Flutter camera
Document scan	VisionKit (iOS) / ML Kit (Android)

❌ 안티패턴

Permission deny + no fallback: bad UX.
**High res + ML: drop frame.
Background + session running: battery.
Metadata 가 store: privacy leak.
AR 가 매 device 가정: 안 됨.

🤖 LLM 활용 힌트

AVFoundation (iOS) / CameraX (Android) 가 modern.
ARKit / ARCore 가 native AR.
ML 가 Vision / ML Kit.
Resolution + permission + privacy.

8.3 KiB Raw Blame History