모델 경량화의 이론적 통찰: Low-Rank Adaptation의 표현력과 수학적 기초 (수학적 분석)
LoRA(Low-Rank Adaptation) 논문에 대한 수학적 분석 연구. 사전 훈련된 모델을 타겟 모델과 일치시키기 위한 최소 LoRA adapter 순위(rank)를 수학적으로 분석하고, 모델 아키텍처(깊이, 너비)가 최소 순위에 미치는 영향을 이론적으로 규명. 선형 모델·FNN·트랜스포머까지 확장하여 LoRA 표현력과 근사 오차를 정량화하며, Eckart-Young-Mirsky 정리를 행렬 곱 경우로 일반화. 특이값 기반 적응적 rank 할당 방법론 제안 (Y. Zeng, K. Lee).
LoRALow-Rank AdaptationModel CompressionParameter-Efficient Fine-tuningMathematical AnalysisTheoretical AnalysisEckart-Young-Mirsky TheoremNeural Network TheoryTransformer ArchitectureSingular Value Decomposition
Interpretable Automated Machine Learning via Large Language Model Reasoning: Combining Transparency with Performance
한국인공지능학회 2025 추계학술대회 (단독저자 제출)
논문 리뷰
----------------------- REVIEW 1 ---------------------
SUBMISSION: 56
TITLE: Interpretable Automated Machine Learning via Large Language Model Reasoning: Combining Transparency with Performance
AUTHORS: Jonghyun Lee
----------- Overall evaluation -----------
SCORE: 2 (accept)
----- TEXT:
This paper proposes an LLM-driven AutoML system that enhances transparency while maintaining strong performance. The authors outline three key components of their LLM-based AutoML framework, which together enable human-level reasoning and competitive results. Overall, the work is well-motivated and offers valuable insights to the research community.
----------------------- REVIEW 2 ---------------------
SUBMISSION: 56
TITLE: Interpretable Automated Machine Learning via Large Language Model Reasoning: Combining Transparency with Performance
AUTHORS: Jonghyun Lee
----------- Overall evaluation -----------
SCORE: 2 (accept)
----- TEXT:
The idea of using LLM models for feature selection, model selection, and interpreting prediction results is impressive. The direction of research is promising, and it has the potential to shed light on LLMs as reasoning tools for machine learning's areas of optimal selection and interpretability. This paper is valuable for presentation at the JKAIA2025 conference.
However, the content is limited due to the primitive experiments and settings. DNN and various machine learning tasks can be candidates for this approach. If you can find a way to combine this approach and DPO, it can strengthen the paper. Anyway, this paper can serve as a good starting point for further study of LLMs' usage in machine learning.
----------------------- REVIEW 3 ---------------------
SUBMISSION: 56
TITLE: Interpretable Automated Machine Learning via Large Language Model Reasoning: Combining Transparency with Performance
AUTHORS: Jonghyun Lee
----------- Overall evaluation -----------
SCORE: 0 (borderline paper)
----- TEXT:
The work's core idea—using an LLM to synthesize standard statistical signals with lightweight, human-readable reasoning—has practical appeal, especially for education and for teams wary of black-box AutoML, and the multi-seed reporting is a welcome step toward more honest evaluation. However, technical depth and empirical rigor feel modest: the algorithmic choices are limited (RF, SVM/LogReg, K-means), ablations disentangling the LLM's value from simple score aggregation are missing, and unsupervised performance lags a tuned baseline by about 10.6% on Iris; moreover, cross-dataset gains are mixed and the scope is confined to small tabular benchmarks. The paper is clearly written and the interpretability angle is genuine, but stronger baselines (e.g., Auto-sklearn/TPOT with reporting parity), richer ablations, and broader domains would sharpen its contribution.
LLM 기반 AutoML: (1) 특성 분석·선택 (2) 모델 선택·실행 (3) 결과 분석·통찰 생성. 통계 지표 MI, RF 중요도, Pearson ρ를 LLM에 제공하고, 통계와 LLM 통찰을 결합한 특성 선택으로 투명성과 성능을 동시에 확보. 50–90% 차원 축소, 95.33%±3.58% 분류 정확도.
S∗=argmaxS⊆F[αXi∈S∑stats(Xi)+β⋅LLM-Insight(S,task)] AutoMLInterpretabilityLarge Language ModelsFeature SelectionExplainable AILLM ReasoningZero Human InterventionDimensionality ReductionModel SelectionAutomated Result Analysis