GitHub Copilot: Evaluation Models Now Available in Auto Model Selection for Individual Plans
GitHub CopilotGitHub Copilot now allows evaluation models, experimental pre-release models identified internally rather than by official name, to be served automatically through Copilot auto model selection for users on individual (non-enterprise) plans. This lets GitHub route a portion of auto-selected traffic to models still under evaluation so it can gather real-world performance data. Individuals who prefer not to receive these models can opt out at any time via the Evaluation models in Copilot auto model selection setting under AI controls in their GitHub settings.
Sources & Mentions
3 external resources covering this update
Evaluation Models Enter Auto Model Selection
GitHub Copilot's auto model selection feature, which automatically picks the most appropriate model for a given request rather than requiring the user to choose, now includes evaluation models in the pool of options it can serve to users on individual plans.
What Evaluation Models Are
Evaluation models are experimental or pre-release models that GitHub is still assessing. They are typically identified by internal codenames rather than their official product names, because they have not yet been formally launched. By routing a share of auto-selected requests to these models, GitHub can collect real-world usage and quality signals before deciding whether to promote a model to general availability.
This change applies specifically to Copilot plans for individuals (non-enterprise). When auto model selection is active, it may now choose an evaluation model to handle a request.
A Caution on Quality
GitHub notes that, by their experimental nature, evaluation models may perform worse than established models on certain categories of prompts, including security-related tasks. As always, users should carefully review and validate any generated code, particularly for security, and rely on human review across a range of models before incorporating suggestions into production.
How to Opt Out
Individuals who do not want evaluation models served to them can disable the behavior at any time:
- Click your profile picture, then click Settings.
- At the top of the page, click AI controls.
- For the Evaluation models in Copilot auto model selection setting, select Disabled from the dropdown.
Once disabled, auto model selection will only draw from generally available models for that user.
Why It Matters
For everyday Copilot users on individual plans who rely on auto model selection, this change means they may occasionally be served a model that is still under evaluation, without that model being named in the UI. For most users this is transparent and may surface improved capabilities early, but the opt-out gives those who prefer only stable, named models a simple way to keep their experience consistent.