Open-Source Models & Evaluation

Days 3–5

The rest of Week 3 is about standing on the shoulders of existing high-performing models: pulling them, adapting them, and rigorously evaluating the results.

Using open-source models

• Pulling and using open-source models and architectures.
• Fine-tuning, plus pre- and post-training, to adapt a model to your domain.
• Leveraging high-performing deep learning models in your own scientific area.

Improving and adapting

• Improving open-source architectures rather than starting from scratch.
• Using models to inform mechanism, not just to predict.

Validation and evaluation

• Validating and evaluating model results.
• Evaluating the performance of deep learning models and of agentic systems.
• Avoiding the trap of impressive-looking results that don't hold up.

Connecting science to impact

• Using these models to connect Earth systems science with impact in other domains, and to communicate that impact to non-technical stakeholders.

Outcomes

By the end of the course you should be able to pull, use, and fine-tune open-source models, evaluate them and your agentic systems rigorously, and use them to inform mechanism and communicate impact.