Ladies and gentlemen,
For Insiders: This is the boarding call for Air Redstone flight 14901 for fast ring pass holders. Please assemble at the Windows Update gate. As you board the “plane”, please read the instruction booklet from our captain:
For stable build users: as fast ring Insiders are installing the first Redstone 2 development build, let me take you on a short tour of some terminology used and how Insiders test things:
Insiders and Microsoft use special terms to denote life of an Insider build, testing and what not. Here is a concise dictionary of them:
· Flight: synonymous with build release. This term comes from Microsoft where engineers use the term “flighting” to denote build release process.
· Captain: In our group, “captain” refers to the head of Windows Insider Program. Currently, it is Dona Sarkar, a seasoned Microsoft engineer who took over this operation from Gabriel Aul, vice-president of engineering systems team a few weeks ago.
· Ring: A group of testers and/or developers.
· Redstone 2: the next feature upgrade (due sometime next year).
Rings: There are at least six rings (three internal, three external) that designate groups of testers. The rings are:
· Canary: Windows developers themselves who work on specific features. Each team of Windows developers can generate feature builds, test it and commit their changes into a repository inside Microsoft. These feature branches are tracked by sites such as msv-next, Build Feed and others.
· Windows and Devices: This includes Windows developers, testers, management and others.
· Microsoft internal: Microsoft employees.
· Insider fast: bleeding-edge, late alpha code, hits the air once approved by Microsoft.
· Insider slow: early beta quality, default for new entrants. A fast ring build will be promoted to this ring once telemetry and feedback says it is stable enough.
· Insider release preview: early access to cumulative updates, drivers, new app versions and next feature upgrades.
Apart from release preview, these rings test next feature upgrade builds.
How Insiders test for things:
1. We read release notes posted by Dona Sarkar on the day new build(s) come out.
2. Installs new build(s).
3. Some would test new features, documenting them and giving feedback as we go.
4. Some would test for regressions, and we also report them via Feedback Hub. For testing, we use either a spare computer or a virtual machine (VM).
5. Sometimes, Microsoft engineers specializing in one or more areas would respond to feedback, as well as other Insiders providing additional comments. Communication takes place via Feedback Hub (sometimes shortened to FB or FB Hub; not to be confused with Facebook), emails or Twitter.
6. Next build is released, and this cycle repeats.
In my case, the steps I take are:
1. I have several VM’s that are configured for each Insider ring except RP (for RP, I use a second computer).
2. I read release notes and post an email like this to Insiders subgroup. I tend to use the headline similar to what you are seeing. The #InsiderBuilds hashtag is reserved for this purpose.
3. If this is a build promotion (fast to slow), I add a message to the end of the release thread, signifying that the build released to fast ring is now available for slow ring pass holders.
4. In case of release preview, I also add #KBAlert to let RP users know that it’s a CU (you saw this a few days ago).
5. After installing the new build(s), I would first try out new features and provide feedback and write internal documentation.
6. If I encounter regressions (mostly accessibility), I would test it with both Narrator and NVDA to make sure it is screen reader agnostic as much as possible, and if so, I open Feedback Hub and send in reports. I always include build number, subsystem (for example, WSL for Windows Subsystem for Linux, UIA for UI Automation and so on; in case of an app, the app name) along with steps to reproduce and possible solutions if any.
7. If a MS engineer notices these reports and asks me for more information (happened on Twitter several times), I provide additional data such as keystrokes used, more technical information and so on. A particular case was when Insiders could not access quick actions in Action Center via keyboard, which led to an engineer specializing in user interface responding to my feedback, and this issue was subsequently fixed a few days later.
8. If it turns out an issue is caused by different screen readers giving different interpretations i.e. Narrator says one thing, NVDA says another and omits important information, JAWS says something completely different and so on, then I would report it to screen reader developers (same standard as feedback sent to Microsoft). In case of NVDA, if it’s an issue that I can solve (such as unlabeled controls, announcing changes to labels and so on), I write a fix for it, which gets integrated into NVDA snapshots or Windows 10 App Essentials depending on the severity of the issue (most end up included in the add-on first before some are taken up by NV Access; a good case was when UAC dialog redesign was first shown: a fix was included in Windows 10 App Essentials before it was sent upstream to NV Access as a pull request, and this fix will make its debut later in August). Thus part of my job as an Insider is to pave the way for at least NVDA users to have a great experience when they use the resulting stable build months later (and sometimes I do publish technical information, and if that happens, I try my best to explain in a way that benefits all screen reader users, not just NVDA folks).
· For Insiders: please test various scenarios, including unexpected ones and new apps so we can make the next feature upgrade a breeze for others. Don’t forget to send appropriate feedback and interact with other Insiders, including those who do not use screen readers.
· For stable build users: keep your feedback coming. Please let Insiders know things we should test.
The next installment will feature how to leave Insider program and other remarks, due out on the day the next Redstone 2 build is released. Thanks.