2nd/10_Wiki/Topics/Architecture/CST.md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	tech_stack
wiki-2026-0508-cst	CST	10_Wiki/Topics	verified	self	Concrete Syntax Tree Parse Tree Lossless Syntax Tree	none	A	0.95	applied	parser compiler ast cst ide refactoring		2026-05-10	pending	language framework Polyglot tree-sitter/Roslyn/rust-analyzer

CST

매 한 줄

"매 Concrete Syntax Tree (CST) = 매 source text 의 모든 문자 (whitespace, comment, trivia) 까지 보존하는 lossless tree.". 매 AST 가 의미만 남기고 trivia 를 버린 반면, CST 는 매 round-trip print 가 가능. 매 2026 모던 IDE / formatter / refactor tool 의 backbone — tree-sitter, Roslyn, rust-analyzer 가 모두 CST.

매 핵심

매 AST vs CST

• AST: 매 abstract — 매 keyword/punctuation 생략, semantics 만.
• CST: 매 concrete — 매 every token + whitespace + comment.
• Round-trip: 매 CST → exact source text 복원 가능.
• Incremental update: 매 edit 시 affected subtree 만 reparse.

매 use-case where CST > AST

• Code formatter (prettier, gofmt, rustfmt): 매 trivia 제어 필요.
• Refactor tool: 매 comment 보존 + 정확한 source range.
• Linter with auto-fix: 매 source position 정확.
• Syntax highlighting: 매 every token 위치 필요.
• IDE error tolerance: 매 partial parse 도 tree 반환.

매 응용

1. tree-sitter: 매 GitHub/Atom/Neovim grammar — 매 incremental.
2. Roslyn (C#): 매 SyntaxNode 가 CST.
3. rust-analyzer: 매 rowan library 의 lossless syntax tree.
4. Babel @babel/parser: 매 AST + tokens (semi-CST).
5. Prettier: 매 input 의 doc-IR 변환.

💻 패턴

tree-sitter — incremental CST

const Parser = require('tree-sitter');
const Java = require('tree-sitter-java');
const parser = new Parser();
parser.setLanguage(Java);

const source = `class A { int x = 1; }`;
const tree = parser.parse(source);

// edit: insert at offset 17
const newSource = `class A { int x = 1; int y = 2; }`;
tree.edit({
  startIndex: 19, oldEndIndex: 19, newEndIndex: 31,
  startPosition: {row: 0, column: 19},
  oldEndPosition: {row: 0, column: 19},
  newEndPosition: {row: 0, column: 31},
});
const newTree = parser.parse(newSource, tree); // reuses unchanged subtrees

Roslyn — preserve trivia in refactor

var tree = CSharpSyntaxTree.ParseText(source);
var root = tree.GetRoot();
var oldMethod = root.DescendantNodes().OfType<MethodDeclarationSyntax>().First();
var newMethod = oldMethod
    .WithIdentifier(SyntaxFactory.Identifier("RenamedMethod"))
    .WithLeadingTrivia(oldMethod.GetLeadingTrivia())
    .WithTrailingTrivia(oldMethod.GetTrailingTrivia());
var newRoot = root.ReplaceNode(oldMethod, newMethod);
File.WriteAllText(path, newRoot.ToFullString());  // round-trip, comments intact

rust-analyzer rowan

use rowan::{GreenNode, SyntaxNode};

// Green tree = immutable, shareable
let green: GreenNode = parse(source);
// Red tree = lazy wrapper with parent pointers + offset
let syntax: SyntaxNode<MyLang> = SyntaxNode::new_root(green);

for token in syntax.descendants_with_tokens() {
    println!("{:?} @ {:?}", token.kind(), token.text_range());
    // includes whitespace, comments, error tokens
}

Lossless tree → AST view

// AST is a typed view over CST
pub struct FnDef(SyntaxNode);
impl FnDef {
    pub fn name(&self) -> Option<Name> {
        self.0.children().find_map(Name::cast)
    }
    pub fn body(&self) -> Option<Block> {
        self.0.children().find_map(Block::cast)
    }
}
// Original text, comments, whitespace = always retrievable via self.0.text()

CST-based formatter

function format(node: CSTNode, ctx: PrintCtx): Doc {
  switch (node.type) {
    case 'function_declaration':
      return concat([
        node.leadingTrivia.filter(isComment),  // preserve doc comments
        'function ',
        format(node.name, ctx),
        format(node.params, ctx),
        ' ',
        format(node.body, ctx),
      ]);
    // ...
  }
}

Error-tolerant parse

// tree-sitter inserts ERROR / MISSING nodes; tree still walkable
const tree = parser.parse(`function f() { return ; }`);
tree.rootNode.descendantsOfType('ERROR').forEach(n => {
  console.log('parse error at', n.startPosition);
});
// IDE keeps working even with incomplete code

Source range — exact byte spans

foreach (var diag in compilation.GetDiagnostics()) {
    var span = diag.Location.SourceSpan;
    Console.WriteLine($"[{span.Start}..{span.End}] {diag.GetMessage()}");
}

매 결정 기준

상황	CST or AST
Code formatter	CST (trivia 필요)
Compiler optimization pass	AST (semantics 만)
IDE / refactor tool	CST
Pure code generation (template)	AST
Syntax highlighter	CST tokens
Bytecode emit	AST + lowered IR

기본값: developer-tool 빌드 = CST first (tree-sitter or rowan); compiler = CST → AST → IR.

🔗 Graph

• 부모: Parser
• 변형: AST · Lossless_Syntax_Tree
• 응용: Code_Formatter · IDE · Refactoring_Tools

🤖 LLM 활용

언제: parser tool 선택, refactor 구현, formatter 설계. 언제 X: runtime semantic analysis (type checker 영역 — AST 기반).

❌ 안티패턴

• Regex 로 source rewrite: 매 trivia 망가지고 nested 구조 무시.
• AST refactor + reprint: 매 comment 모두 사라짐.
• Full reparse on every keystroke: 매 IDE freeze — incremental 필수.
• Mutable shared CST: 매 race condition; immutable green tree 사용.

🧪 검증 / 중복

• Verified (tree-sitter docs, Roslyn API ref, "Lossless Syntax Tree" Aleksey Kladov 2020).
• 신뢰도 A.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — CST vs AST, tree-sitter, Roslyn, rowan patterns