Introduction

In the realm of large language models (LLMs), there has been a constant pursuit to enhance the capabilities of smaller models without compromising their efficiency. The traditional approach has been to use imitation learning, where smaller models learn from the outputs generated by large foundation models (LFMs). However, this approach has been marred by several challenges, including limited imitation signals from shallow LFM outputs, small-scale homogeneous training data, and a lack of rigorous evaluation. This often leads to smaller models imitating the style but not the reasoning process of LFMs.

The paper Orca: Progressive Learning from Complex Explanation Traces of GPT-4 introduces Orca, a 13-billion parameter model designed to imitate the reasoning process of large foundation models (LFMs) such as GPT-4. Unlike traditional large language models (LLMs), Orca employs a unique training approach that combines progressive learning and teacher assistance to overcome the capacity gap between smaller student models and their larger counterparts.

Training Methodology

 
Orca's training process consists of two stages.

In the first stage, Orca is trained on FLAN-5M, which includes ChatGPT augmentations. This intermediate teacher assistant helps bridge the capacity gap between Orca and GPT-4, which has a significantly larger parameter size. By leveraging ChatGPT's capabilities, Orca benefits from improved imitation learning performance.

In the second stage, Orca undergoes training on FLAN-1M, which incorporates GPT-4 augmentations. This progressive learning approach follows a curriculum learning paradigm, where the student model learns from easier examples before tackling more challenging ones. By gradually exposing Orca to increasingly complex reasoning and step-by-step explanations, the model enhances its reasoning abilities and mimicking skills.

Advantages and Contributions

 
Orca's training methodology offers several advantages over traditional LLMs.

Firstly, it addresses the capacity gap issue by utilizing an intermediate teacher model, allowing Orca to learn from a more capable source. This approach has been shown to improve imitation learning performance for smaller student models.

Secondly, the progressive learning aspect of Orca's training enables the model to build upon its knowledge incrementally. By starting with simpler examples and gradually introducing more complex ones, Orca develops a stronger foundation for reasoning and explanation generation.

Furthermore, Orca's ability to imitate the reasoning process of LFMs like GPT-4 opens up possibilities for enhanced performance in various tasks. By tapping into the rich signals provided by GPT-4's explanation traces and step-by-step thought processes, Orca gains valuable insights and improves its own capabilities.

Performance Benchmarks

Orca has shown remarkable performance in complex zero-shot reasoning benchmarks. It outperforms traditional state-of-the-art instruction-tuned models like Vicuna-13B by over 100% on benchmarks like Big-Bench Hard (BBH) and over 42% on AGIEval. Additionally, Orca achieves the same scores as ChatGPT on the BBH benchmarks and shows competitive performance on professional and academic exams such as the SAT, LSAT, GRE, and GMAT. This is particularly impressive considering that these are zero-shot settings without chain-of-thought, and Orca still performs competitively while trailing behind GPT-4.

Implications and Future Directions

The development of Orca represents a significant advancement in the field of LLMs. By learning from rich signals and imitating the reasoning process of LFMs, Orca is able to perform complex reasoning tasks with a high degree of accuracy. This has wide-ranging implications, especially in areas where complex reasoning and problem-solving are required.

Moreover, this research indicates that learning from step-by-step AI model explanations is a promising direction for improving model capabilities. This opens up new avenues for research and development in the field of LLMs.

Conclusion

 
Orca presents a novel approach to training large language models, combining progressive learning and teacher assistance to enhance imitation learning. By leveraging intermediate teacher models and gradually exposing the student model to more complex examples, Orca overcomes the capacity gap and improves its reasoning and explanation generation abilities. The paper's findings contribute to the advancement of imitation learning techniques and have implications for the development of future language models.

For more details on Orca and its research, refer to the introductory article from Microsoft and the accompanying research paper.