[three]Bean

Work continues on Factory 2.0...

Recall that we have 1000 different problems we're trying to solve, but we're attempting to focus on an isolated subset for now: problems we've picked so that their solutions can enable higher-level problem solving in the coming months. The work currently includes a focus on:

• Preparing elementary build infrastructure for the Fedora 26 Alpha release.
• Deserializing pipeline processes that could be done more quickly in parallel.
• Building a dependency chain database, so that we can build smarter rebuild automation and pipeline analytics.
• Monitoring pipeline performance metrics, so that as we later improve things we can be sure we had an effect.

Overall, I'm happy with the work output of this sprint, but there are complications.

• We made significant progress on the module build infrastructure. The stack is maturing - the number of new features will be slowing as we start to prepare it for production for F26. Our proposals were approved by FESCo last week. We still have a long way to go, but we're currently on target to build an F26 Edition out of the base modules. Status: Green.
• Our deserialization effort hit some roadbumps as we negotiate how to interface with existing test-execution environments. We're not blocked at the moment and are in dialogue with our partners. We will slip a few weeks on our MvP deliverable here. Status: Yellow.
• The dependency chain MvP is ready for deployment efforts in Sprint 5. We underestimated how this work would depend on other message bus enablement pre-requisites, which has pushed our delivery date back a few weeks. Status: Yellow.
• Our performance metrics work is progressing, but much more slowly than we had initially expected. We promised a delivery date of Dec. 1st for this, but we will miss it by a wide margin and need to adjust expectations. The primary factor is that the metrics are just more complex than we anticipated. The pipeline is complex, therefore exacting meaningful measurements is complex. A separate factor is some unpredictability with write-access to the integration lab's ELK instance we are depending on. The delay in work here does not block our other efforts. Status: Red.

mbs-build-profiles, by threebean

Here we show how we're re-using the "installation profiles" feature of modulemd to define the buildsystem "build groups" for a module. It's a natural extension of the install profiles metadata which furthmore helps unblock the base-runtime team in their quest to produce the first generational core!

mbs-config, by fivaldi

In this video I'm trying to explain changes in configuration internals of Module Build Service. This aims to split configuration data/logic so that it will be simplier to use and understand for contributors/users.

modulemd1-mock, by jkaluza

In this demo I inform about the support of modulemd-1.0 format in Module Build Service, describes the build-order feature and shows how the module using this feature builds using the mock builder.

resultsdb-updater, by mprahl

This video shows the new microservice ResultsDB-Updater which listens on the CI message bus for test results and adds them to ResultsDB.

umb-brew, by mikeb

A demo of message publication from Brew to the Unified Message Bus. Explains the topic hierarchy and the message format.

umb-dist-git-demo, by mikeb

A demo showing the publication of messages from a dist-git repo to the Unified Message Bus. Shows how the repos are configured for publication on the server side, and explains a little bit about message format.

This was our first full sprint with the new team! Welcome, Jan Kaluza, Courtney Pacheco, Vera Karas, and Stanislav Ochotnicky. We're glad to have Filip Valder join us in sprint 4 starting today.

Our top priority in sprint 3 was making sure that the base runtime team isn't blocked. They have a big job ahead of themselves to curate and build a collection of base modules at the core of the distro, and they need to use our prototype build tooling to do it. Anytime they're blocked, the Factory 2.0 team is trying to chase down the solution -- fixing tracebacks and developing new features. Cheers to Matt Prahl and Jan Kaluza for staying on top of this.

Meanwhile, we're continuing apace with the Dependency Chain and Deserialization epics that we originally scheduled for work this quarter. Mike Bonnet has been chasing down difficult technical pre-requisites for the later (message bus enablement), Matt Prahl demoed his dependency chain web UI, and Courtney Pacheco is giving shape to our metrics project (so we can have some confidence that future pipeline changes we make actually improve the state of affairs).

As always, we're double-tasked with laying the groundwork for work in future quarters. Thanks to Stanislav Ochotnicky for starting the conversation with Platform representatives about workflow changes and continuous integration, and thanks to all those who participated in this round of resultsdb/CI discussions.

f26-changes, by threebean

Here I talk about the Change proposals we've filed with FESCo for the Fedora 26 release.

• There's one on the module-build-service.
• And another on changes to the compose tooling.

I talk about this in the video, but the changes are designed specifically to limit the amount of risk we impose on the rest of the release process and the Modularity initiative. We can use the F26 release to assess the viability of Modularity and decide then how far we want to go in the Fedora 27 timeframe.

mbs-mock-backend, by jkaluza

In this demo I describe what is Mock Builder and why it is useful to Module Build Service. I also show it in action briefly.

pdc-tangle-web, by mprahl

In this demo I talk about an Angular2 web app I wrote called PDC Tangle Web. The app queries PDC based on the user's search criteria to show an artifact's dependencies. This is a beta version, so please keep in mind more features will come, but feel free to make suggestions on GitHub for what you'd like to see added.

resultsdb, by threebean

This one's just pointing you to a blog post that is a transcription of an internal document on resultsdb and our plans for it.

I know that showing off a blog post isn't all that exciting, but I'm glad we've gotten to this point. It is the product of a long process of discussion and eventual agreement.

Today marks the end of Sprint #2 for the so-called Factory 2.0 project. So far, it's been a small-ish team with Matthew Prahl and myself (and part-time support from the awesome Michael Bonnet). We know which way we want to go now and we're going to pick up a few more people in the coming weeks.

We write status reports internally, but we're doing work in the open, in Fedora. We should probably bridge that gap and I figured the easiest way to share would be to post our demo videos here as well:

Running fedmsg services on other message bus backends

Storing rpm dependencies in PDC

Storing container dependencies in PDC

Querying dependencies from PDC

Standing up resultsdb with ansible

---

I'll try my best to post all of our future sprint report videos here. No secrets! The best place to find us is in #fedora-modularity and #fedora-admin. Happy Hacking!