Artificial Intelligence
直接偏好优化
Direct Preference OptimizationA reinforcement learning method for aligning large language models with human preferences by optimizing a reward model directly from preference data without requiring a separate policy network. / 一种强化学习方法，用于将大型语言模型与人类偏好对齐，通过直接从偏好数据优化奖励模型，而无需单独的策略网络。
- Knowledge type
- 工艺技术
- First proposed or discovered
- Proposer
- Turkey Farm member
- Creator
- 农场主
- Last updated
- 2026-10-29 18:06:49
Sub-knowledge points
Reward modelingPolicy optimizationKL divergence penalty
Higher-order knowledge points
Reinforcement Learning from Human FeedbackLarge Language Model Alignment
Main parameters and performance indicators
preference dataset: pairs of (prompt, chosen response, rejected response)learning rate: hyperparameter for optimizerbeta: coefficient for KL divergence penalty
Open problems
Sample efficiencyStability during trainingScalability to complex tasks
Related literature
ranked by community starsFa-DPO: Faithfulness-Aware Direct Preference Optimization with risk-adaptive preference learning for medical question answeringOpen source
Related products and links
No item recorded yet
Creation and modification trail
1 recordsRecord title: Direct Preference Optimization.
Node update feedback
review required