rStar Deep Dive: How MCTS & Mutual Reasoning Boost LLaMA2-7B Accuracy from 12% to 64% Without Fine-tuning

Hong-Wei Wu published on 2025-12-10 included in category Paper Introduction

Discover rStar, a breakthrough AI framework by Microsoft & Harvard that boosts Small Language Models (SLMs) like LLaMA2-7B. Learn how rStar uses Monte Carlo Tree Search (MCTS) and Mutual Reasoning to improve math problem-solving accuracy from 12% to 64% without fine-tuning or GPT-4. Explore the future of Inference Scaling Laws now.

Your Base Model Is Smarter Than You Think: How MCMC Sampling Can Outperform RL

Hong-Wei Wu published on 2025-12-02 included in category Paper Introduction

Discover how "Reasoning with Sampling" challenges RL in LLMs using Distribution Sharpening and MCMC. Learn why your base model is smarter than you think in this deep dive into inference-time compute.

Beyond Prompt Engineering: Inside Agentic Context Engineering, The Future of Self-Evolving AI

Hong-Wei Wu published on 2025-10-27 included in category Paper Introduction

Discover Agentic Context Engineering (ACE), a new method for creating self-improving LLMs. This breakdown of the latest research paper explains how ACE solves key issues like Context Collapse and Brevity Bias to build smarter, constantly evolving AI.

Dynamic Cheatsheet: The Secret to Self-Improving LLMs that Learn During Inference

Hong-Wei Wu published on 2025-10-15 included in category Paper Introduction

Unlock self-improving LLMs with the Dynamic Cheatsheet method. Learn how test-time learning and adaptive memory with retrieval and curation can boost agent performance.

Let AI Write Flawless SQL: A Deep Dive into the SQL-of-Thought Multi-Agent Framework

Hong-Wei Wu published on 2025-10-12 included in category Paper Introduction

Dive into SQL-of-Thought, a novel multi-agent framework designed to significantly boost LLM performance on Text-to-SQL tasks. This article breaks down its unique agentic workflow, guided error correction, and how providing an SQL Error Taxonomy leads to more accurate queries.

Beyond Top-k: How Adaptive-k Dynamically Selects the Best Context for RAG Without Latency

Hong-Wei Wu published on 2025-09-22 included in category Paper Introduction

Struggling with choosing the right 'k' in your RAG system? Discover Adaptive-k, a novel, no-tuning method that dynamically selects the best context for LLMs, improving performance without sacrificing speed.

SENSE Explained: How Strong and Weak LLMs Achieve State-of-the-Art Text-to-SQL (ACL 2024)

Hong-Wei Wu published on 2025-09-15 included in category Paper Introduction

Discover how the SENSE paper (ACL 2024) boosts Text-to-SQL performance for open-source models. This article breaks down its two-stage method of using synthetic data from strong LLMs for SFT and weak LLMs for DPO to achieve state-of-the-art results.