Topics

Backstage tour: Windows 10 development roadmap, schedule, release steps #backstagetour


 

Hi all,

For folks on Insiders subgroup: please read the participation survey results and the accompanying piece carefully.

As a companion to a post I sent to Insiders subgroup regarding Windows Insider Program participation survey (now closed), coupled with what has become a parade of questions about feature update releases, I’ll take you on a tour as to how a feature update is developed and released in stages (hence the #BackstageTour hashtag, reserved for posts like this; for recent family members, these posts are meant to explain certain details behind Windows 10 features; past topics have included Windows installation and emoji panel internals; please hold on as it can get really geeky):

Every six months or so, Microsoft releases big (and sometimes small) feature updates. Except for Version 1909 (November 2019 Update), every feature update was a new Windows 10 release (1909 was an enablement package that simply flipped the light switch). Every feature update is supported for at least 18 months (exceptions include fall feature updates for Windows 10 Enterprise and Education and Enterprise long-term servicing channel (LTSC), supported for 30 months and 10 years, respectively). Feature updates are not the end of the story as far as Windows 10 releases are concerned: a Windows 10 build that will eventually become the release everyone will see comes from a long line of development builds, made a bit more complicated these days due to Azure, source code management, and changes to Windows Insider fast ring.

A feature update is essentially a collection of features. Each Windows feature team at Microsoft works on a specific feature and related changes: Windows Terminal and console support, accessibility, Azure infrastructure, app development teams, Windows Subsystem for Linux, File Explorer, among many others. As far as Windows 10 development is concerned, these teams communicate with each other for one purpose: compressing (are you ready for this) 300 gigabytes of source code into a 3 gigabyte installer ready for public deployment every six months or so.

As feature development is a continuous process, not all changes are packaged into a feature update. Several factors play out in feature update development cycle:

  1. Release date: in order to meet semi-annual release commitments, Microsoft finishes development on a feature update months in advance. Prior to late 2019, a given Windows 10 build is finalized several weeks prior to public release; in 2019, this is now in the hands of Azure development schedule (Azure releases their “feature updates” in June and December; more on this factor below).
  2. Testing and maturity: a given feature goes through development, various testing stages (inside the feature team and by Windows Insiders), and finalization, which often takes months (some of them took years). An iteration of a feature ready for public deployment in the next feature update is marked as “stable” so changes can be packaged as part of a Windows 10 build.
  3. Feature flags: most changes are controlled by feature flags, a “gate” that tells Windows to use old or new parts of a given feature. If the new change is deemed stable, the corresponding change will go live; if not, it’ll be turned off, with Windows asked to use the old code.

 

A typical feature update development goes like this:

  1. Developers work on features based on feedback from users and internal roadmaps. Features are developed inside feature branches in order to isolate changes (these branches are housed inside Git branches private to Microsoft and are identified by “feature branch level”).
  2. The Azure team announces the deadline by which changes slated for the next feature update should be finalized. This announcement happens months in advance and typical deadlines are June (fall release) and December (spring release). This is so that all Windows developers can be on the same page as far as base source code is concerned.
  3. Whenever a feature team commits changes to a feature branch, potential changes (along with feature flags for these if any) are merged into rs_prerelease branch.
  4. At least once a day (except in weekends), whatever changes are present in rs_prerelease branch are compiled into a new Windows 10 build. This build is then sent to “canary ring”, a combination of automated tests and human engineers. No-one outside of Microsoft can join this ring.
  5. After a build passes canary ring, it is sent to “selfhost ring” where more Windows developers test the build and the build is subjected to more automated tests. This happens at least once a week (sometimes several times a week). Just like canary ring, no-one outside Microsoft can join this ring.
  6. After selfhost ring approves a build, it is then presented to Windows Insiders on fast ring. Typically a build compiled on Friday is presented to fast ring Insiders the following week unless a showstopper bug shows up. Fast ring builds show up on a weekly basis.
  7. From time to time, a fast ring build goes out to Microsoft employees for final internal testing. The purpose of this “Microsoft ring” is to let a Windows 10 build go through real-life testing inside Microsoft. As the name implies, only Microsoft employees have access to Microsoft ring.
  8. During all this time, a stable feature will be kept in the upcoming feature update, otherwise the feature is disabled by turning off the associated feature flag so feature teams can improve the disabled feature. Feature flags (controlled by feature teams) are employed on the rings described above (that is, all rings except slow and release preview Insider rings will see feature flag toggles to some extent). And since the prerelease branch can include all sorts of changes, fast ring Insiders are effectively running bits of changes that won’t even show up in the next feature update.
  9. Several weeks prior to Azure release deadline (late May and early November), a release (staging) branch is created to house final (or close to final) bits of the upcoming feature update. This is so the upcoming feature update can be finalized and to prepare future changes for Insiders.
  10. Sometime in June and December, the feature update in question will be finalized and signed by Microsoft. The final build is then ready for servicing changes (via cumulative updates). This also marks the appearance of final bits of the upcoming feature update on Microsoft ring, and will show up to Insider slow ring testers a few days later.
  11. Once a feature update build is signed, various product teams (PC, Xbox, HoloLens, etc.) are then tasked with finalizing the upcoming feature update with last minute changes. This continues until the day the feature update is released to release preview ring, and eventually, to everyone.
  12. In the meantime, Microsoft employees and fast ring Insiders are testing changes slated for future feature updates and the cycle repeats. Future feature iterations may end up on the next feature update, disabled for further development, or as in the case with Version 1909, parts of it were backported to 1903 and incorporated as cumulative updates.

 

At this point you might be wondering if fast ring Insiders can move to slow ring and vice versa. In the past, this was possible as fast ring Insiders were the first group of Insiders to receive the final feature update build; not anymore – fast ring Insiders are set to receive development builds permanently (unless this changes again). As I noted earlier, fast ring Insiders test all sorts of changes, some of which are not even meant for upcoming feature updates. In fact, as of June 11, 2020, fast ring Insiders are not only testing things slated for 20H2 (Manganese), but also parts of 21H1 (Iron), which won’t show up until next year. Or, to demonstrate the gravity of the situation, fast ring Insiders are actually testing (or preparing to receive) bits of 21H2 (Cobalt) and 22H1 (Nickel), which won’t show up until 2022!

That last bit is the true reason why I sent out Windows Insider Program participation survey to members of Insiders subgroup: to remind people that being an Insider means you are committing to providing early feedback. This is more so now that fast ring is set to receive development builds all the time. This is why you cannot just switch between Insider rings anymore, especially if you want to move from fast ring to slow ring – the only way to leave fast ring properly is a clean install.

Hope this helps.

 

References:

 

Cheers,

Joseph


Ernie Gobble
 

Thanks for making it more clear to meEmoji

 



Ernie Gobble
Fort Knox, Ky
U.S. Army Retired


 
   


On Thursday, June 11, 2020, 04:47:14 AM EDT, Joseph Lee <joseph.lee22590@...> wrote:


Hi all,

For folks on Insiders subgroup: please read the participation survey results and the accompanying piece carefully.

As a companion to a post I sent to Insiders subgroup regarding Windows Insider Program participation survey (now closed), coupled with what has become a parade of questions about feature update releases, I’ll take you on a tour as to how a feature update is developed and released in stages (hence the #BackstageTour hashtag, reserved for posts like this; for recent family members, these posts are meant to explain certain details behind Windows 10 features; past topics have included Windows installation and emoji panel internals; please hold on as it can get really geeky):

Every six months or so, Microsoft releases big (and sometimes small) feature updates. Except for Version 1909 (November 2019 Update), every feature update was a new Windows 10 release (1909 was an enablement package that simply flipped the light switch). Every feature update is supported for at least 18 months (exceptions include fall feature updates for Windows 10 Enterprise and Education and Enterprise long-term servicing channel (LTSC), supported for 30 months and 10 years, respectively). Feature updates are not the end of the story as far as Windows 10 releases are concerned: a Windows 10 build that will eventually become the release everyone will see comes from a long line of development builds, made a bit more complicated these days due to Azure, source code management, and changes to Windows Insider fast ring.

A feature update is essentially a collection of features. Each Windows feature team at Microsoft works on a specific feature and related changes: Windows Terminal and console support, accessibility, Azure infrastructure, app development teams, Windows Subsystem for Linux, File Explorer, among many others. As far as Windows 10 development is concerned, these teams communicate with each other for one purpose: compressing (are you ready for this) 300 gigabytes of source code into a 3 gigabyte installer ready for public deployment every six months or so.

As feature development is a continuous process, not all changes are packaged into a feature update. Several factors play out in feature update development cycle:

  1. Release date: in order to meet semi-annual release commitments, Microsoft finishes development on a feature update months in advance. Prior to late 2019, a given Windows 10 build is finalized several weeks prior to public release; in 2019, this is now in the hands of Azure development schedule (Azure releases their “feature updates” in June and December; more on this factor below).
  2. Testing and maturity: a given feature goes through development, various testing stages (inside the feature team and by Windows Insiders), and finalization, which often takes months (some of them took years). An iteration of a feature ready for public deployment in the next feature update is marked as “stable” so changes can be packaged as part of a Windows 10 build.
  3. Feature flags: most changes are controlled by feature flags, a “gate” that tells Windows to use old or new parts of a given feature. If the new change is deemed stable, the corresponding change will go live; if not, it’ll be turned off, with Windows asked to use the old code.

 

A typical feature update development goes like this:

  1. Developers work on features based on feedback from users and internal roadmaps. Features are developed inside feature branches in order to isolate changes (these branches are housed inside Git branches private to Microsoft and are identified by “feature branch level”).
  2. The Azure team announces the deadline by which changes slated for the next feature update should be finalized. This announcement happens months in advance and typical deadlines are June (fall release) and December (spring release). This is so that all Windows developers can be on the same page as far as base source code is concerned.
  3. Whenever a feature team commits changes to a feature branch, potential changes (along with feature flags for these if any) are merged into rs_prerelease branch.
  4. At least once a day (except in weekends), whatever changes are present in rs_prerelease branch are compiled into a new Windows 10 build. This build is then sent to “canary ring”, a combination of automated tests and human engineers. No-one outside of Microsoft can join this ring.
  5. After a build passes canary ring, it is sent to “selfhost ring” where more Windows developers test the build and the build is subjected to more automated tests. This happens at least once a week (sometimes several times a week). Just like canary ring, no-one outside Microsoft can join this ring.
  6. After selfhost ring approves a build, it is then presented to Windows Insiders on fast ring. Typically a build compiled on Friday is presented to fast ring Insiders the following week unless a showstopper bug shows up. Fast ring builds show up on a weekly basis.
  7. From time to time, a fast ring build goes out to Microsoft employees for final internal testing. The purpose of this “Microsoft ring” is to let a Windows 10 build go through real-life testing inside Microsoft. As the name implies, only Microsoft employees have access to Microsoft ring.
  8. During all this time, a stable feature will be kept in the upcoming feature update, otherwise the feature is disabled by turning off the associated feature flag so feature teams can improve the disabled feature. Feature flags (controlled by feature teams) are employed on the rings described above (that is, all rings except slow and release preview Insider rings will see feature flag toggles to some extent). And since the prerelease branch can include all sorts of changes, fast ring Insiders are effectively running bits of changes that won’t even show up in the next feature update.
  9. Several weeks prior to Azure release deadline (late May and early November), a release (staging) branch is created to house final (or close to final) bits of the upcoming feature update. This is so the upcoming feature update can be finalized and to prepare future changes for Insiders.
  10. Sometime in June and December, the feature update in question will be finalized and signed by Microsoft. The final build is then ready for servicing changes (via cumulative updates). This also marks the appearance of final bits of the upcoming feature update on Microsoft ring, and will show up to Insider slow ring testers a few days later.
  11. Once a feature update build is signed, various product teams (PC, Xbox, HoloLens, etc.) are then tasked with finalizing the upcoming feature update with last minute changes. This continues until the day the feature update is released to release preview ring, and eventually, to everyone.
  12. In the meantime, Microsoft employees and fast ring Insiders are testing changes slated for future feature updates and the cycle repeats. Future feature iterations may end up on the next feature update, disabled for further development, or as in the case with Version 1909, parts of it were backported to 1903 and incorporated as cumulative updates.

 

At this point you might be wondering if fast ring Insiders can move to slow ring and vice versa. In the past, this was possible as fast ring Insiders were the first group of Insiders to receive the final feature update build; not anymore – fast ring Insiders are set to receive development builds permanently (unless this changes again). As I noted earlier, fast ring Insiders test all sorts of changes, some of which are not even meant for upcoming feature updates. In fact, as of June 11, 2020, fast ring Insiders are not only testing things slated for 20H2 (Manganese), but also parts of 21H1 (Iron), which won’t show up until next year. Or, to demonstrate the gravity of the situation, fast ring Insiders are actually testing (or preparing to receive) bits of 21H2 (Cobalt) and 22H1 (Nickel), which won’t show up until 2022!

That last bit is the true reason why I sent out Windows Insider Program participation survey to members of Insiders subgroup: to remind people that being an Insider means you are committing to providing early feedback. This is more so now that fast ring is set to receive development builds all the time. This is why you cannot just switch between Insider rings anymore, especially if you want to move from fast ring to slow ring – the only way to leave fast ring properly is a clean install.

Hope this helps.

 

References:

 

Cheers,

Joseph