---
id: android-ml-kit-health
title: Android ML Kit / Health Connect / On-device AI
category: Coding
status: draft
source_trust_level: B
verification_status: conceptual
created_at: 2026-05-09
updated_at: 2026-05-09
tags: [android, mlkit, health, on-device, vibe-coding]
tech_stack: { language: "Kotlin", applicable_to: ["Android"] }
applied_in: []
aliases: [ML Kit, Health Connect, MediaPipe, on-device ML, AICore, Gemini Nano]
---

# Android ML Kit / Health Connect / On-device AI

> Google ML Kit (built-in ML), Health Connect (cross-app health), MediaPipe (advanced ML), Gemini Nano (on-device LLM, Pixel 9+).

## 📖 핵심 개념
- ML Kit: 일반 ML task 빠른 사용.
- Health Connect: data 통합 + permissions.
- MediaPipe: vision / LLM 자체 모델.
- Gemini Nano: AICore — on-device LLM.

## 💻 코드 패턴

### ML Kit — Text recognition
```kotlin
implementation("com.google.mlkit:text-recognition:16.0.0")
```

```kotlin
import com.google.mlkit.vision.text.TextRecognition
import com.google.mlkit.vision.text.latin.TextRecognizerOptions

val recognizer = TextRecognition.getClient(TextRecognizerOptions.DEFAULT_OPTIONS)

val image = InputImage.fromBitmap(bitmap, rotation)
val result = recognizer.process(image).await()  // suspend extension

for (block in result.textBlocks) {
    for (line in block.lines) {
        println(line.text)
    }
}
```

→ OCR. 영수증 / 명함.

### ML Kit — Barcode
```kotlin
implementation("com.google.mlkit:barcode-scanning:17.2.0")

val scanner = BarcodeScanning.getClient()
val barcodes = scanner.process(image).await()

for (barcode in barcodes) {
    when (barcode.valueType) {
        Barcode.TYPE_URL -> println("URL: ${barcode.url?.url}")
        Barcode.TYPE_WIFI -> println("WiFi: ${barcode.wifi?.ssid}")
        Barcode.TYPE_TEXT -> println(barcode.rawValue)
    }
}
```

### ML Kit — Face detection
```kotlin
val options = FaceDetectorOptions.Builder()
    .setPerformanceMode(FaceDetectorOptions.PERFORMANCE_MODE_FAST)
    .setLandmarkMode(FaceDetectorOptions.LANDMARK_MODE_ALL)
    .setClassificationMode(FaceDetectorOptions.CLASSIFICATION_MODE_ALL)
    .build()

val detector = FaceDetection.getClient(options)
val faces = detector.process(image).await()

for (face in faces) {
    val bounds = face.boundingBox
    val smilingProb = face.smilingProbability ?: 0f
    val leftEye = face.getLandmark(FaceLandmark.LEFT_EYE)?.position
}
```

### ML Kit — Translation
```kotlin
implementation("com.google.mlkit:translate:17.0.2")

val options = TranslatorOptions.Builder()
    .setSourceLanguage(TranslateLanguage.KOREAN)
    .setTargetLanguage(TranslateLanguage.ENGLISH)
    .build()

val translator = Translation.getClient(options)
translator.downloadModelIfNeeded().await()

val translation = translator.translate("안녕").await()
// → "Hello"
```

### ML Kit — Pose / Body
```kotlin
implementation("com.google.mlkit:pose-detection:18.0.0-beta3")

val options = PoseDetectorOptions.Builder()
    .setDetectorMode(PoseDetectorOptions.STREAM_MODE)
    .build()

val detector = PoseDetection.getClient(options)
val pose = detector.process(image).await()

val nose = pose.getPoseLandmark(PoseLandmark.NOSE)?.position
val leftWrist = pose.getPoseLandmark(PoseLandmark.LEFT_WRIST)?.position
```

→ Fitness app.

### MediaPipe (advanced)
```kotlin
implementation("com.google.mediapipe:tasks-vision:0.10.20")

val options = ImageClassifier.ImageClassifierOptions.builder()
    .setBaseOptions(BaseOptions.builder().setModelAssetPath("model.tflite").build())
    .setMaxResults(5)
    .build()

val classifier = ImageClassifier.createFromOptions(context, options)
val result = classifier.classify(MPImage.fromBitmap(bitmap))

for (cat in result.classifications()[0].categories()) {
    println("${cat.categoryName()}: ${cat.score()}")
}
```

→ Custom TFLite 모델.

### Gemini Nano (on-device, Pixel 9+)
```kotlin
implementation("com.google.ai.edge.aicore:aicore:0.0.1-exp01")

val options = generationConfig {
    context = context  // Activity / Application
    temperature = 0.2f
    topK = 16
    maxOutputTokens = 256
}

val generativeModel = GenerativeModel(generationConfig = options)

val response = generativeModel.generateContent("Summarize this article: ...").text
```

→ Cloud 호출 없이. Privacy + offline + free.

→ ⚠️ Pixel 9 / 일부 device 만. Compatibility check.

### Health Connect setup
```kotlin
implementation("androidx.health.connect:connect-client:1.1.0-alpha07")
```

```xml
<!-- AndroidManifest.xml -->
<uses-permission android:name="android.permission.health.READ_STEPS" />
<uses-permission android:name="android.permission.health.WRITE_STEPS" />
<uses-permission android:name="android.permission.health.READ_HEART_RATE" />

<queries>
    <package android:name="com.google.android.apps.healthdata" />
</queries>
```

```kotlin
import androidx.health.connect.client.HealthConnectClient
import androidx.health.connect.client.records.StepsRecord
import androidx.health.connect.client.permission.HealthPermission

val healthConnectClient = HealthConnectClient.getOrCreate(context)

val permissions = setOf(
    HealthPermission.getReadPermission(StepsRecord::class),
    HealthPermission.getWritePermission(StepsRecord::class),
)

// Request
val launcher = registerForActivityResult(
    PermissionController.createRequestPermissionResultContract()
) { granted -> /* ... */ }

launcher.launch(permissions)
```

### Read steps
```kotlin
val response = healthConnectClient.readRecords(
    ReadRecordsRequest(
        recordType = StepsRecord::class,
        timeRangeFilter = TimeRangeFilter.between(
            Instant.now().minusSeconds(3600 * 24),
            Instant.now()
        )
    )
)

val totalSteps = response.records.sumOf { it.count }
```

### Write steps
```kotlin
healthConnectClient.insertRecords(listOf(
    StepsRecord(
        count = 5000,
        startTime = Instant.now().minusSeconds(3600),
        endTime = Instant.now(),
        startZoneOffset = ZoneOffset.UTC,
        endZoneOffset = ZoneOffset.UTC,
    )
))
```

### Aggregation
```kotlin
val agg = healthConnectClient.aggregate(
    AggregateRequest(
        metrics = setOf(StepsRecord.COUNT_TOTAL),
        timeRangeFilter = TimeRangeFilter.between(start, end)
    )
)

val total = agg[StepsRecord.COUNT_TOTAL] ?: 0L
```

### Background sync
```kotlin
class HealthSyncWorker(ctx: Context, params: WorkerParameters) : CoroutineWorker(ctx, params) {
    override suspend fun doWork(): Result {
        val steps = readSteps()
        syncToServer(steps)
        return Result.success()
    }
}

// Schedule
val request = PeriodicWorkRequestBuilder<HealthSyncWorker>(15, TimeUnit.MINUTES).build()
WorkManager.getInstance(ctx).enqueueUniquePeriodicWork("health-sync", KEEP, request)
```

### Privacy
```
- 명시적 사용자 consent
- Data minimization (read only what needed)
- 사용자 가 access / delete 가능
- GDPR / HIPAA compliance (US)
```

### CameraX + ML Kit
```kotlin
val analyzer = ImageAnalysis.Analyzer { imageProxy ->
    val mediaImage = imageProxy.image ?: return@Analyzer
    val image = InputImage.fromMediaImage(mediaImage, imageProxy.imageInfo.rotationDegrees)
    
    barcodeScanner.process(image)
        .addOnSuccessListener { barcodes -> /* ... */ }
        .addOnCompleteListener { imageProxy.close() }
}
```

→ Real-time camera + ML.

### TFLite (custom 모델)
```kotlin
implementation("org.tensorflow:tensorflow-lite:2.16.1")

val interpreter = Interpreter(loadModelFile())

val input = ByteBuffer.allocateDirect(...)
val output = ByteBuffer.allocateDirect(...)

interpreter.run(input, output)
```

→ 자체 모델 (TF / PyTorch → TFLite).

### Audio classification
```kotlin
implementation("com.google.mediapipe:tasks-audio:0.10.20")

val options = AudioClassifier.AudioClassifierOptions.builder()
    .setBaseOptions(BaseOptions.builder().setModelAssetPath("yamnet.tflite").build())
    .build()

val classifier = AudioClassifier.createFromOptions(context, options)
val result = classifier.classify(MPAudioData.create(buffer, sampleRate))

for (cat in result.classifications()[0].categories()) {
    println("${cat.categoryName()}: ${cat.score()}")
    // "Music", "Speech", "Bark", ...
}
```

### Subject segmentation (BG removal)
```kotlin
implementation("com.google.mlkit:subject-segmentation:16.0.0-beta1")

val segmenter = SubjectSegmentation.getClient()
val result = segmenter.process(image).await()

val foregroundBitmap = result.foregroundBitmap
// 배경 X — 사용자 / 사람 만
```

### Document scanner
```kotlin
implementation("com.google.android.gms:play-services-mlkit-document-scanner:16.0.0-beta1")

val options = GmsDocumentScannerOptions.Builder()
    .setGalleryImportAllowed(true)
    .setPageLimit(5)
    .setResultFormats(RESULT_FORMAT_PDF, RESULT_FORMAT_JPEG)
    .build()

GmsDocumentScanning.getClient(options)
    .getStartScanIntent(activity)
    .addOnSuccessListener { intent -> startActivityForResult(intent, ...) }
```

→ Document scan + auto crop + PDF.

### Smart reply (chat)
```kotlin
implementation("com.google.mlkit:smart-reply:17.0.4")

val smartReply = SmartReply.getClient()
val conversation = listOf(
    TextMessage.createForRemoteUser("Hi", System.currentTimeMillis(), "user_1"),
)

val result = smartReply.suggestReplies(conversation).await()

for (suggestion in result.suggestions) {
    println(suggestion.text)
    // "Hi!", "Hello", "Hey there"
}
```

### Battery / performance
```
On-device ML = 빠름 + free + private.
But:
- Battery 사용
- Memory
- Model size (10-100 MB)

→ 측정 + throttle.
```

### Cloud vs on-device
```
On-device:
+ Free (no API cost)
+ Private (no upload)
+ Offline
+ Low latency
- Limited model size
- Battery / memory

Cloud (Vertex AI / Gemini API):
+ Bigger / better model
+ Always updated
- Cost
- Privacy
- Latency
```

→ 일반 task = on-device. Complex / accuracy critical = cloud.

### Edge AI (modern stack)
```
1. Quick task: ML Kit (built-in)
2. Custom: MediaPipe + TFLite
3. LLM: Gemini Nano (Pixel 9+)
4. Cloud fallback: Gemini API
```

## 🤔 의사결정 기준
| 작업 | 추천 |
|---|---|
| OCR / barcode / face | ML Kit |
| 자체 모델 | MediaPipe / TFLite |
| On-device LLM | Gemini Nano (Pixel 9+) |
| Health data | Health Connect |
| 일반 LLM | Cloud (Gemini API) |
| Real-time | CameraX + ML Kit |

## ❌ 안티패턴
- **모든 거 cloud LLM**: cost / privacy.
- **Health Connect 권한 한 번 + 모든 거**: minimum access.
- **PII model 학습 외부 send**: privacy violation.
- **Gemini Nano + 모든 device**: compatibility check.
- **Battery 무시**: 사용자 끄기.
- **모델 download 큰 + first launch**: progressive.

## 🤖 LLM 활용 힌트
- ML Kit = 가장 단순 + 빠른 시작.
- Health Connect = cross-app data.
- MediaPipe = custom + advanced.
- Gemini Nano = privacy-friendly LLM.

## 🔗 관련 문서
- [[Android_CameraX_Patterns]]
- [[AI_Local_LLM_Inference]]
- [[Mobile_Push_Deep]]