2nd/10_Wiki/Topics/AI_and_ML/Edge-AI-and-Computing.md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	tech_stack
wiki-2026-0508-edge-ai-and-computing	Edge AI and Computing	10_Wiki/Topics	verified	self	edge AI on-device AI edge computing TinyML NPU mobile inference	none	A	0.95	applied	ai infrastructure edge-computing on-device-ai latency tinyml quantization		2026-05-10	pending	language framework C / C++ / Python TFLite / ONNX Runtime / Core ML / NCNN / TinyML

Edge AI and Computing

매 한 줄

"매 cloud 의 X — 매 device 의 inference". 매 latency ↓ + 매 privacy ↑ + 매 bandwidth ↓ + 매 offline. 매 model: 매 quantized + pruned + distilled. 매 hardware: 매 NPU (Apple Neural Engine, Snapdragon Hexagon), TinyML MCU. 매 modern: 매 on-device LLM (Phi-3, Llama 3.2 1B, Gemma 2B).

매 핵심

매 motivation

• Latency: 매 ms 의 round-trip cloud → 매 sub-ms.
• Privacy: 매 data 의 device 의 stay.
• Cost: 매 cloud GPU 의 X.
• Offline: 매 connectivity 의 independent.

매 model compression

• Quantization: FP32 → INT8 → INT4 → 4-bit.
• Pruning: 매 zero weights.
• Distillation: 매 teacher → student.
• Architecture: 매 MobileNet, EfficientNet, MobileViT.

매 hardware

• Apple Neural Engine (16-core).
• Snapdragon Hexagon NPU.
• Google Tensor TPU edge.
• NVIDIA Jetson (Orin, Xavier).
• TinyML MCU: ESP32, Cortex-M.
• Coral Edge TPU.

매 framework

• Mobile: TFLite, Core ML, MediaPipe, ONNX Runtime Mobile.
• Embedded: TensorFlow Lite Micro, Edge Impulse, NCNN, MNN.
• LLM-on-device: llama.cpp, MLC LLM, Apple Foundation Models, MLX.

매 응용

1. Mobile photo: 매 portrait, HDR.
2. Voice: 매 wake word, ASR.
3. AR: 매 hand tracking.
4. IoT: 매 anomaly.
5. Automotive: 매 ADAS.
6. Wearable: 매 ECG, sleep.

매 modern (2024+)

• On-device LLM: 매 Phi-3-mini (3.8B INT4), Llama 3.2 1B/3B, Gemma 2B.
• Apple Foundation Models framework.
• Qualcomm AI Hub.
• Hybrid edge-cloud: 매 simple → device, complex → cloud.

💻 패턴

TFLite quantize (Python)

import tensorflow as tf

converter = tf.lite.TFLiteConverter.from_saved_model('saved_model/')
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = lambda: ((tf.cast(x[:1], tf.float32),) for x in calibration_data)
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8
tflite_model = converter.convert()
open('model_int8.tflite', 'wb').write(tflite_model)

Mobile inference (Core ML, Swift)

import CoreML
let config = MLModelConfiguration()
config.computeUnits = .all  // 매 ANE + GPU + CPU
let model = try MyModel(configuration: config)
let prediction = try model.prediction(input: input)

Android (TFLite + delegate)

val options = Interpreter.Options().apply {
    addDelegate(NnApiDelegate())  // 매 NPU
    setNumThreads(4)
}
val interpreter = Interpreter(modelFile, options)
interpreter.run(input, output)

llama.cpp (LLM on-device)

#include "llama.h"
struct llama_model_params model_params = llama_model_default_params();
model_params.n_gpu_layers = 99;  // 매 Metal / CUDA
struct llama_model* model = llama_load_model_from_file("model_q4.gguf", model_params);
struct llama_context_params ctx_params = llama_context_default_params();
struct llama_context* ctx = llama_new_context_with_model(model, ctx_params);

MLX (Apple, Python)

import mlx.core as mx
import mlx_lm

model, tokenizer = mlx_lm.load('mlx-community/Llama-3.2-1B-Instruct-4bit')
prompt = tokenizer.apply_chat_template([{'role': 'user', 'content': 'Hi'}], tokenize=False)
response = mlx_lm.generate(model, tokenizer, prompt, max_tokens=128)

TinyML (TF Lite Micro, C)

#include "tensorflow/lite/micro/all_ops_resolver.h"
constexpr int kArenaSize = 8 * 1024;
uint8_t tensor_arena[kArenaSize];

void setup() {
  static tflite::MicroErrorReporter error_reporter;
  static tflite::AllOpsResolver resolver;
  static tflite::MicroInterpreter interpreter(model, resolver, tensor_arena, kArenaSize);
  interpreter.AllocateTensors();
}

void loop() {
  TfLiteTensor* input = interpreter.input(0);
  // 매 fill from sensor
  interpreter.Invoke();
  TfLiteTensor* output = interpreter.output(0);
}

Distillation (PyTorch)

def distill_loss(student_logits, teacher_logits, target, T=3, alpha=0.5):
    soft = F.kl_div(F.log_softmax(student_logits / T, -1),
                    F.softmax(teacher_logits / T, -1), reduction='batchmean') * T * T
    hard = F.cross_entropy(student_logits, target)
    return alpha * soft + (1 - alpha) * hard

Pruning (PyTorch)

import torch.nn.utils.prune as prune
for module in model.modules():
    if isinstance(module, nn.Linear):
        prune.l1_unstructured(module, 'weight', amount=0.5)
        prune.remove(module, 'weight')

Hybrid edge-cloud

def smart_dispatch(query, device_capability):
    complexity = estimate_complexity(query)
    if complexity < THRESHOLD and device_capability.has_local_llm:
        return local_llm.generate(query)
    return cloud_llm.generate(query)

Power-aware scheduling

void schedule_inference() {
    if (battery_level < 0.2) {
        use_quantized_model();  // 매 INT8
        skip_low_priority_inferences();
    } else {
        use_full_model();
    }
}

Latency benchmark

import time
def benchmark_tflite(interpreter, input_data, n=100):
    times = []
    for _ in range(n):
        t0 = time.perf_counter()
        interpreter.set_tensor(input_idx, input_data)
        interpreter.invoke()
        times.append(time.perf_counter() - t0)
    return {'p50': sorted(times)[n // 2], 'p99': sorted(times)[int(n * 0.99)]}

매 결정 기준

상황	Approach
Mobile vision	TFLite + NNAPI / Core ML
Mobile LLM	MLX / llama.cpp / MLC
MCU (mW power)	TinyML / TF Lite Micro
Privacy critical	On-device only
Latency critical	Edge + 매 cache
Variable complexity	Hybrid edge-cloud

기본값: 매 INT8 quantize + 매 NPU delegate + 매 device-tier model + 매 hybrid 의 fallback.

🔗 Graph

• 부모: Machine-Learning · Distributed-Systems
• 변형: TinyML · Mobile-AI
• 응용: LLM_Optimization_and_Deployment_Strategies · LLM_Optimization_and_Deployment_Strategies
• Adjacent: NPU · Federated-Learning

🤖 LLM 활용

언제: 매 mobile app. 매 IoT. 매 privacy. 매 low-latency. 언제 X: 매 huge model only. 매 frequent retrain.

❌ 안티패턴

• Cloud model 의 device 의 push: 매 ROM / RAM 의 fail.
• No quantization: 매 latency / battery.
• Single delegate hardcode: 매 device 의 fail.
• Edge-only stubborn: 매 hybrid 의 win 의 miss.
• No power awareness: 매 battery drain.

🧪 검증 / 중복

• Verified (TFLite docs, MLX, Apple WWDC, Qualcomm AI Hub).
• 신뢰도 A.

🕓 Changelog

날짜	변경
2026-04-26	EDGE-AI auto
2026-05-08	Phase 1
2026-05-10	Manual cleanup — quantize + 매 TFLite / MLX / llama.cpp / TinyML / hybrid code