v2.2.256: 코어 채팅 큰 입력 청킹·통합 + 실제 컨텍스트 창 정렬 + 모델 핸들 race 수정

큰 입력 시 "Failed to acquire LM Studio model handle … Operation canceled"
로 턴 전체가 죽던 문제를 3계층으로 해결. 일반 채팅(코어 경로)은 그동안
단일 예산 호출이라 약한 모델·큰 입력에서 무너졌다 — 그 갭을 메움.

- 핸들 race 수정: getModelHandle 을 재시도 루프 안으로 이동. 취소/죽은-핸들
  류 에러는 SDK 재생성 후 1회 자동 재시도(실제 사용자 취소는 존중). 라이프
  사이클의 동시 로드가 abort 되며 SDK 가 coalesce 한 JIT 조회까지 죽던 것.
- Phase 1 실제 창 정렬: llm.getContextLength()(캐시)로 실측 창에 예산 클램프.
  설정값보다 작은 창으로 로드된 경우 서버 truncation/빈 답변 차단. 배지에 표시.
- Phase 2 코어 Map-Reduce: 단일 입력이 (유효 창 × ratio) 초과 시 청크→질의
  인지형 추출→통합. 부분/전체 폴백, 무관 시 정직 신호. 동시성 기본 2.
- Phase 3 메타 노출: 진행/결과 배지 표시, [조각 k] 출처 옵트인.

신규 설정 5종. /meet·/review 전용 경로는 불변. 테스트 +25건, 전체 684 통과.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-19 18:05:44 +09:00
parent 6adbc2a6fa
commit 76d5fedfb5
13 changed files with 883 additions and 19 deletions
+45
View File
@@ -39,6 +39,17 @@ export interface ILMStudioClient {
* "Model is disposed!" or "lock() request could not be registered" error.
*/
getModelHandle(modelKey: string, options?: { refresh?: boolean }): Promise<LLM>;
/**
* The model's *actually-loaded* context window in tokens (LM Studio's
* `llm.getContextLength()`), or `undefined` if it can't be determined.
*
* The user-facing `g1nation.contextLength` setting is only a budgeting
* intent — the real ceiling is whatever window the model was loaded with.
* Budgeting against the larger of the two silently overflows the server,
* which then truncates the prompt or emits EOS as the first token (empty
* answer). Cached per-key because it only changes on reload.
*/
getModelContextLength(modelKey: string): Promise<number | undefined>;
isReachable(): Promise<boolean>;
setBaseUrl(httpBaseUrl: string): void;
}
@@ -84,8 +95,10 @@ export class LMStudioClient implements ILMStudioClient {
private _wsUrl: string | undefined;
private _loadedCache: { value: string[]; expiresAt: number } | undefined;
private _downloadedCache: { value: string[]; expiresAt: number } | undefined;
private _contextLengthCache = new Map<string, { value: number; expiresAt: number }>();
private static readonly DEFAULT_LOADED_CACHE_TTL_MS = 5000;
private static readonly DEFAULT_DOWNLOADED_CACHE_TTL_MS = 60_000;
private static readonly DEFAULT_CONTEXT_LENGTH_CACHE_TTL_MS = 60_000;
constructor(httpBaseUrl: string) {
this.setBaseUrl(httpBaseUrl);
@@ -98,6 +111,7 @@ export class LMStudioClient implements ILMStudioClient {
this._sdk = undefined;
this._loadedCache = undefined;
this._downloadedCache = undefined;
this._contextLengthCache.clear();
}
}
@@ -170,6 +184,7 @@ export class LMStudioClient implements ILMStudioClient {
invalidateCaches(): void {
this._loadedCache = undefined;
this._downloadedCache = undefined;
this._contextLengthCache.clear();
}
async listLoaded(): Promise<string[]> {
@@ -243,6 +258,36 @@ export class LMStudioClient implements ILMStudioClient {
}
}
async getModelContextLength(modelKey: string): Promise<number | undefined> {
const key = (modelKey || '').trim();
if (!key) return undefined;
const now = Date.now();
const cached = this._contextLengthCache.get(key);
if (cached && cached.expiresAt > now) return cached.value;
try {
// Reuses the same handle the stream will use. If the model isn't
// loaded yet this forces a JIT load — acceptable since the very next
// step streams from it anyway. Best-effort: any failure (incl. the
// load-coalescing "Operation canceled" race) falls back to undefined
// so the caller keeps the configured window.
const handle: any = await this.getSdk().llm.model(key);
const len = typeof handle?.getContextLength === 'function'
? await handle.getContextLength()
: undefined;
if (typeof len === 'number' && Number.isFinite(len) && len > 0) {
this._contextLengthCache.set(key, {
value: len,
expiresAt: now + LMStudioClient.DEFAULT_CONTEXT_LENGTH_CACHE_TTL_MS,
});
return len;
}
return undefined;
} catch (e: any) {
logError('Failed to query LM Studio model context length.', { modelKey: key, error: e?.message ?? String(e) });
return undefined;
}
}
async isReachable(): Promise<boolean> {
try {
await this.getSdk().llm.listLoaded();