[three]Bean

A number of us were funded to come together last weekend for a release engineering FAD in Westford. I'll save summaries of the whole event for writeups by others, but I will say that we planned work on the first day with an 'agile' exercise proposed and facilitated by imcleod. It worked well and mitigated any centrifugal forces acting on the FAD.

Read Adam Miller's post for a solid summary of the whole event. Here's just a run down of the things I personally got involved in:

• The runroot plugin - we've been talking about it forever -- an Internet search reveals a mention of it from a meeting in 2007 <http://fedoraproject.org/wiki/Extras/SteeringCommittee/Meeting-20070315>. What finally made it a necessity is pungi4. We're hoping to move the compose to using that, and it in turn relies on the runroot plugin in koji to do most of its work.

  A portion of it was open sourced last year. I got it installed in staging and then found we were missing some more parts to it. We sorted those all out, submitted some patches upstream to koji and got a staging proof of concept working. As of last week, we have it in production and a new channel of builders dedicated to it.

• Our staging koji instance was originally setup last year at the Bodhi2 FAD using the external repos feature of koji. This worked well enough for testing scratch builds and little things like that, but was insufficient for testing some larger parts of the releng process (the compose). We compared some options and have settled on a solution. Some pieces of that puzzle have been implemented but I've yet to finish putting it all together. Hopefully done this week.

• As far as planning goes, I thought the koji 2.0 discussions were quite fruitful, and I'm really excited at how the architecture it shaping up.

• We also had some good interstitial sessions talking about role and requirements for what has been heretofore called the ComposeDB. There's still more discussion to have before we can land on a planning document for it, but fwiw, we think we have a much better name for it than ComposeDB.

I'll end with noting that on the day before the FAD started, a number of us got together to plan for and hack on Fedora Hubs. decause and I stayed up late almost every night and wrote some pretty neat proofs of concept for it. There's good stuff running now in the prototype if you want to try to get it running on your local system. Fun, fun. Happy Hacking!

Over the past few days I've been trying to replicate our release-engineering infrastructure in our staging environment so we can try out changes with less worry, move faster in the future, etc. There been some progress but there are a lot of pieces -- this won't be done for a while.

The latest subject of my efforts has been koji. I had some authentication problems off the bat but I got it to the point now so that it is pulling it's CRL correctly from FAS. You can edit /etc/koji.conf and actually point your builds at it. It only has one builder (all we need) but they'll run and complete!:

[koji]
;server = http://koji.fedoraproject.org/kojihub
;weburl = http://koji.fedoraproject.org/koji
;topurl = https://kojipkgs.fedoraproject.org/
server = http://koji.stg.fedoraproject.org/kojihub
weburl = http://koji.stg.fedoraproject.org/koji
topurl = https://kojipkgs.stg.fedoraproject.org/

I even have the kojira service running now (it is a background service responsible for rebuilding and maintaining the repos that hang off of https://koji.fedoraproject.org/repos/ -- notably the rawhide/ one that I'm interested in for performing fake staging 'composes'. It is notoriously absent from the staging repos.)

This led me into the dark nest that is koji tags and targets. I've been using them for years to build my packages, but I never really understood how they all fit together and what was even the difference between a tag and a target. I wrote the following graph-generating script the try and make sense:

#!/usr/bin/env python
""" koji2graphviz.py - Visualize Koji Tags and their relationships.

Author:     Ralph Bean <rbean@redhat.com>
License:    LGPLv2+
"""

import multiprocessing.pool
import sys

from operator import itemgetter as getter

import koji

# https://pypi.python.org/pypi/graphviz
from graphviz import Digraph

N = 40

graph_options = {'format': 'png'}
tags_parent_child = Digraph(
    name='tags_parent_child', comment='Koji Tags, Parent/Child relationships',
    **graph_options)
tags_groups = Digraph(
    name='tags_groups', comment='Koji Tags and what Groups they are in',
    **graph_options)
tags_and_targets = Digraph(
    name='tags_and_targets', comment='Koji Tags and Targets',
    **graph_options)

client = koji.ClientSession('https://koji.fedoraproject.org/kojihub')
tags = client.listTags()
for tag in tags:
    tags_parent_child.node(tag['name'], tag['name'])


def get_relations(tag):
    sys.stdout.write('.')
    sys.stdout.flush()
    idx = tag['id']
    return (tag, {
        'parents': client.getInheritanceData(tag['name']),
        'group_list': sorted(map(getter('name'), client.getTagGroups(idx))),
        'dest_targets': client.getBuildTargets(destTagID=idx),
        'build_targets': client.getBuildTargets(buildTagID=idx),
        'external_repos': client.getTagExternalRepos(tag_info=idx),
    })


print "getting parent/child relationships with %i threads" % N
pool = multiprocessing.pool.ThreadPool(N)
relationships = pool.map(get_relations, tags)
print
print "got relationships for all %i tags" % len(relationships)

print "collating known groups"
known_groups = list(set(sum([
    data['group_list'] for tag, data in relationships
], [])))
for group in known_groups:
    tags_groups.node('group-' + group, 'Group: ' + group)

print "collating known targets"
known_targets = list(set(sum([
    [target['name'] for target in data['build_targets']] +
    [target['name'] for target in data['dest_targets']]
    for tag, data in relationships
], [])))
for target in known_targets:
    tags_and_targets.node('target-' + target, 'Target: ' + target)

print "building graph"
for tag, data in relationships:
    for parent in data['parents']:
        tags_parent_child.edge(parent['name'], tag['name'])
    for group in data['group_list']:
        tags_groups.edge(tag['name'], 'group-' + group)
    for target in data['build_targets']:
        tags_and_targets.edge('target-' + target['name'], tag['name'], 'build')
    for target in data['dest_targets']:
        tags_and_targets.edge(tag['name'], 'target-' + target['name'], 'dest')

print "writing"
tags_parent_child.render()
tags_groups.render()
tags_and_targets.render()
print "done"

Check out this first beast of a graph that it generates. It is a mapping of the parent/child inheritance relationship between tags (you can get at similar information with the $ koji list-tag-inheritance SOME_TAG command.

This next one shows koji tags and what 'targets' they have relationships with. There are two kind of relationships here. A target can be a "destination target" for a tag or it can be a "build target" for a tag.

Dennis Gilmore tells me that:

Targets are glue for builds. Targets define the tag used for the buildroot
and the tag that the resulting build is tagged to.

Trying to unpack that -- take the rawhide target in this graph. It gets its buildroot definition from the f22-build tag, and builds that succeed there are sent to the f22 tag.

Anyways, the next step for my compose-in-staging project is to get that rawhide-repo-holder target setup. I think kojira will notice that and start building the appropriate rawhide/ repo for the next bits down the pipeline. Feel free to reuse and modify the graph-generating script above, it was fun to write, and I hope it's useful to you some day. Happy Hacking!

Messing around, I came up with a read-only dashboard for Fedora Release Engineering. You can check it out live on threebean.org.

Right now all it does is try to show the status of the compose process which runs once a day for rawhide and the "branched" pre-release (that's F20 right now). If any one of the components of that process (say, the pungify part) is happening right now, it'll tell you that and have it show up in yellow in stead of blue. If the compose hasn't happened in over 20 hours, that text will show up in gray to indicate that it is "stale" or "out of date" (those words might be too strong, but whatever).

The point of this post is to ask for feedback -- should this thing do anything more? If so, then what? Should we make it live under fedoraproject.org?

(details: It's just HTML, CSS, and js and it makes its queries against datagrepper)