2nd/10_Wiki/Topics/AI_and_ML/Autonomous-Polling-Wait-Automation.md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	tech_stack
wiki-2026-0508-autonomous-polling-wait	Autonomous Polling & Wait Automation	10_Wiki/Topics	verified	self	폴링 자동화 async wait agent loop state polling webhook fallback hybrid wait	none	B	0.85	applied	agent polling async automation notebooklm research-loop state-machine retry exponential-backoff		2026-05-10	pending	language framework TypeScript / Python Async/Await / Promise

Autonomous Polling & Wait Automation

📌 한 줄 통찰

"매 sleeping researcher". 매 long-running task (3-10 분) 의 완료 의 agent 가 자동 감지 + 매 next step 으로 transition. 매 manual button click 의 X. 매 10초 polling + 매 webhook fallback + 매 timeout 의 hybrid.

📖 핵심

매 polling pattern

• 매 short interval (1-30 sec) 의 state check.
• 매 max attempts / timeout.
• 매 simple, 매 stateless.

매 vs webhook

측면	Polling	Webhook
Setup	Simple	Complex (public URL)
Latency	Polling interval	Near-zero
Server load	High (N polls)	Low (1 call)
Reliability	Self-managed	Webhook 의 lost OK
Use case	Behind firewall	Public service

→ 매 hybrid 의 best.

매 polling strategies

Fixed interval

• 매 simple.
• 매 short job 의 OK.

Exponential backoff

• 매 wait = base × 2^n.
• 매 server-friendly.

Adaptive

• 매 ETA estimate.
• 매 progress-based.

Long polling

• 매 server 의 hold connection.
• 매 latency ↓.

매 long-running 의 pattern

1. Submit job → 매 job_id.
2. Poll status until complete.
3. Retrieve result when ready.
4. Webhook as fallback (optional).
5. Timeout + manual fallback.

매 NotebookLM Deep Research case

• 매 average 3-10 min.
• 매 10 sec polling × 60 = 매 max 10 min.
• 매 status: "queued" → "running" → "completed" / "error".
• 매 completed → 매 result fetch.

매 design 의 challenge

1. Quota: 매 too frequent → 매 API rate limit.
2. Stale state: 매 status 의 update 의 lag.
3. Network failure: 매 retry 의 idempotent.
4. Timeout: 매 server-side retry 의 inflight.
5. Resource leak: 매 polling 의 stop 보장.

매 best practice

• Initial delay: 매 즉시 poll X.
• Exponential + cap: 매 max interval.
• Jitter: 매 thundering herd 방지.
• Cancellation: 매 abort signal.
• Observability: 매 attempt count log.
• Idempotency: 매 result fetch 의 retry-safe.

💻 패턴

Basic polling (TS)

async function pollUntilDone<T>(
  fetchStatus: () => Promise<{ done: boolean; result?: T }>,
  options: { intervalMs?: number; maxAttempts?: number; timeoutMs?: number } = {},
): Promise<T> {
  const { intervalMs = 10_000, maxAttempts = 60, timeoutMs = 600_000 } = options;
  const start = Date.now();
  
  for (let i = 0; i < maxAttempts; i++) {
    if (Date.now() - start > timeoutMs) throw new Error('Timeout');
    
    const status = await fetchStatus();
    if (status.done) return status.result!;
    
    await new Promise(r => setTimeout(r, intervalMs));
  }
  throw new Error('Max attempts exceeded');
}

Exponential backoff with jitter

async function pollWithBackoff<T>(
  fetchStatus: () => Promise<{ done: boolean; result?: T }>,
  options: { baseMs?: number; maxMs?: number; maxAttempts?: number } = {},
): Promise<T> {
  const { baseMs = 1000, maxMs = 30_000, maxAttempts = 30 } = options;
  
  for (let i = 0; i < maxAttempts; i++) {
    const status = await fetchStatus();
    if (status.done) return status.result!;
    
    const delay = Math.min(maxMs, baseMs * 2 ** i);
    const jittered = delay * (0.5 + Math.random() * 0.5);
    await new Promise(r => setTimeout(r, jittered));
  }
  throw new Error('Max attempts');
}

Hybrid (poll + webhook)

async function awaitJobHybrid(jobId: string, webhookUrl?: string): Promise<Result> {
  // 매 webhook 의 우선 setup
  const webhookPromise = webhookUrl 
    ? listenForWebhook(jobId, webhookUrl, { timeoutMs: 600_000 })
    : null;
  
  // 매 polling 의 fallback
  const pollingPromise = pollUntilDone(
    () => api.getJobStatus(jobId),
    { intervalMs: 10_000, timeoutMs: 600_000 },
  );
  
  // 매 둘 다 race
  return Promise.race([webhookPromise, pollingPromise].filter(Boolean));
}

Cancellation (AbortController)

async function pollCancellable<T>(
  fetchStatus: (signal: AbortSignal) => Promise<{ done: boolean; result?: T }>,
  signal: AbortSignal,
): Promise<T> {
  while (!signal.aborted) {
    const status = await fetchStatus(signal);
    if (status.done) return status.result!;
    await sleep(10_000, signal);
  }
  throw new DOMException('Cancelled', 'AbortError');
}

function sleep(ms: number, signal: AbortSignal): Promise<void> {
  return new Promise((resolve, reject) => {
    const t = setTimeout(resolve, ms);
    signal.addEventListener('abort', () => {
      clearTimeout(t);
      reject(new DOMException('Cancelled', 'AbortError'));
    });
  });
}

Webhook handler (FastAPI)

from fastapi import FastAPI, BackgroundTasks
import asyncio

pending: dict[str, asyncio.Future] = {}

@app.post('/webhooks/job-done')
async def job_done(payload: dict):
    job_id = payload['id']
    if job_id in pending:
        pending[job_id].set_result(payload)
    return {'ok': True}

async def wait_for_webhook(job_id: str, timeout: float = 600):
    future = asyncio.Future()
    pending[job_id] = future
    try:
        return await asyncio.wait_for(future, timeout=timeout)
    finally:
        pending.pop(job_id, None)

Idempotent result fetch

def fetch_result_idempotent(job_id, max_retries=3):
    for attempt in range(max_retries):
        try:
            response = api.get_result(job_id)
            return response.data
        except TransientError as e:
            if attempt == max_retries - 1: raise
            sleep(2 ** attempt)
        except PermanentError:
            raise

Progress-aware polling

def poll_progress(job_id):
    last_progress = 0
    while True:
        status = api.get_status(job_id)
        if status.done: return status.result
        
        if status.progress > last_progress:
            log(f'Job {job_id}: {status.progress*100:.1f}%')
            last_progress = status.progress
        
        # 매 ETA 기반 의 dynamic
        remaining_eta = (1 - status.progress) * status.elapsed / max(status.progress, 0.01)
        next_poll = min(30, max(2, remaining_eta / 5))
        sleep(next_poll)

🤔 결정 기준

상황	Pattern
Fast (1-30 sec)	Fixed 1-2 sec polling
Medium (1-10 min)	5-10 sec polling
Long (10 min-hour)	Hybrid (webhook + polling)
Variable	Exponential backoff
Cancellable	AbortController
Resource-constrained	Webhook only
Behind firewall	Polling only

기본값: Hybrid (webhook + 10 sec polling) + jitter + cancellation.

🔗 Graph

• 부모: Async-Programming · API-Design
• 변형: Server-Sent-Events · Exponential-Backoff
• 응용: NotebookLM · Agent-Loop
• Adjacent: Circuit-Breaker · AbortController

🤖 LLM 활용

언제: 매 long-running job. 매 agent automation. 매 third-party API integration. 매 batch inference orchestration. 언제 X: 매 streaming (SSE 가 better). 매 sub-second job.

❌ 안티패턴

• No timeout: 매 무한 hang.
• No jitter: 매 thundering herd.
• Too short interval: 매 quota burn.
• No cancel: 매 resource leak.
• No idempotent fetch: 매 retry 의 corruption.
• Webhook only (firewall): 매 silent loss.
• Tight retry on permanent error: 매 useless burn.

🧪 검증 / 중복

• Verified (AWS / Stripe / Replicate / GitHub API patterns).
• 신뢰도 B.
• Related: Webhook-Pattern · Async-Job-Queue · Retry-with-Backoff · Agent-Loop.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — polling pattern + webhook + 매 TS / Python code (basic, backoff, hybrid, cancellation)