I’m a software engineer and team lead with a focus on architecture and automation. I’m happiest when I’m learning new things, the open source way. I used to do brain science back in school.
Since 2018, I’ve been serving as the Product Pipeline Architect for the Red Hat productization toolchain, globally. I am responsible for creating and communicating our technical strategy and measuring our progress against corresponding goals. I involve myself in the details of our next-gen department initiatives, helping dozens of teams resolve complex blockers and helping management understand the most critical gaps.
Before taking this leadership role, I had been working more directly on backend infrastructure services. I wrote fedmsg, Fedora Infrastructure’s distributed message bus, and most of the related services. Written in Python on top of zeromq, it has interesting resilience and security properties. To support fedmsg, there’s datanommer, which stores the fedmsg history in a postgres database (and marks up the stored messages with associated metadata). Closely related is the datagrepper JSON API, a web service that allows people and programs to query the fedmsg history. I wrote a sister service for it called statscache, which serves pre-computed statistics on the tens of millions of rows of data for use in visualizations. I then turned my attention to simplifying some of our architecture by extracting certain core functions into microservices. Among those is FMN, a centralized notifications service. It allows users to get direct IRC messages or emails about a variety of topics based on self-service preferences. This is particularly useful in conjunction with the upstream release monitoring service that I wrote for Fedora packagers called the-new-hotness. One of many more message bus services – I wrote github2fedmsg which allows contributors to integrate their GitHub activity into the Fedora message bus via self-service webhooks. It’s success prompted the implementation of a series of other mediator-pattern bridge services like bugzilla2fedmsg and zanata2fedmsg.
Through 2016-2018, I’ve been defining the vision and serving as team lead for the group building the next generation of Red Hat’s build to release pipeline. It’s been a real change in role for me. New challenges included high level architecture, evangelism across departments, and reports to senior management. If you’re interested, there’s a video of one of my talks on youtube from Red Hat’s DevConf.
In previous roles, I wrote web applications, the best of which integrate the assets of other services. By far the coolest one (technologically and socially) is Fedora Badges which uses the fedmsg stream to award “badges” to contributors based on their contributions. Although I didn’t write the initial go-through, I at one point became the maintainer of the Fedora Packages web app, working on enhancements, repairs, operations, and some sophisticated rewrites. (I wrote a handy CLI tool called pkgwat to go with it.) I wrote the frontend and much of the API for bodhi2, the Fedora Updates System - a system crucial for the production of the OS. In 2015, I started working on a large integration app called fedora-hubs that featured an interesting actively-invalidated cache mechanism.
I’m no a stranger to operations. Beyond the day-to-day basics, I’ve spearheaded a number of projects in Fedora Infrastructure’s ansible repo. Notably, I ported our entire proxy layer’s configuration from puppet to ansible, revamped a number of subsystems to dynamically generate themselves from our host inventory, reducing admin time required to setup new services (i.e., messaging endpoints, firewall zones, nagios, etc.) and I wrote various playbooks that we use during routine maintenance (i.e. playbooks that take hosts in and out of proxy rotation, dns, and nagios before and after performing DB maintenance or software upgrades).
I’ve tried my hand at release engineering responsibilities. I took on ownership of an app called the product-definition-center and wrote pdc-updater to keep it in sync with our systems. With an increased release cadence associated with modern expectations of continuous delivery, I moved to automate the detection of AMIs and Project Atomic images from our data lake in the getfedora.org build process (removing the need for human intervention to publish a release).
As far as frontend development goes, in 2015, I made the splashy whatcanidoforfedora.org for new contributors. The software is nicely generic and has been re-used by a number of other sites. I wrote and maintain the Fedora releng-dashboard, a JS-only app that display the latest status of release processes. And this one’s really cool – I wrote the landing page for apps.fedoraproject.org. And I’ll mention last that I made a little JS shim for all of our webapps called fedmenu to help give their disparate appearance a more unified theme.
For fun I wrote and maintain an awesome extension to taskwarrior called bugwarrior. It syncs issues from github, bitbucket, trac, bugzilla, megaplan, teamlab, redmien, jira, activecollab, phabricator and versionone to a local task database. I use it to produce my weekly timesheets – invaluable to my workflow. I’ll mention last here that my first major open source endeavor was as the mis-fortuned maintainer of toscawidgets2. It’s a fun story that I’ll tell you over a beer sometime.
Technical lead for the Red Hat productization toolchain, globally. Responsible for creating and communicating our technical strategy and measuring department progress against corresponding goals. Fluent in the details of our next-gen department initiatives, helping our dozens of teams resolve complex blockers and helping management understand the most critical gaps.
Project lead for the most high profile initiative in the DevOps organization. Was responsible for vision, technical architecture, team health and function, deliverables and timelines, and project evangelism throughout the company. Built and led a team of a dozen to develop, integrate, and deploy a pragmatic suite of services to satisfy next generation requirements on Red Hat’s build to release pipeline.
Conceived of, designed, built, deployed, and maintained dozens of infrastructure services for the Fedora Project. Served as community Development Manager, a technical lead role.
Highlight: Was responsible for the rollout of a distributed message bus, fedmsg, that connected 40+ services while minimizing dependant coupling (fragility). Was directly responsible for implementation, test, community organization, deployment of the project, as well as identification and solicitation of buy-in from impacted stakeholders.
Taught two courses on Free and Open Source Software Development; innovated classroom dynamics and curriculum by a number of methods including accepting pull requests on the syllabus.
Three roles in one: developer, sysadmin, and user support.
I built web applications to facilitate user account administration and self-service, administered a compute cluster and was responsible for porting all resources from SGE to SLURM, coordinated integration with Xsede and the Open Science Grid national clusters, and provided research computing consultation and programming assistance to graduate students and professors across the institute.
Previous positions listed on my linkedin profile.
activemq, amqp, ansible, aws, backbone.js, bash, bind, bootstrap, C, C++, collectd, cornice, css, django, dns, dnssec, docker, ec2, flask, fortran, ganglia, gettext, glance, grunt, gulp, gunicorn, handlebars.js, haproxy, html, http, httpd, iptables, iptables, javascript, jenkins, jira, jms, jquery, LaTeX, less, libvirt, linux, lisp, logstash, markdown, memcached, mod_wsgi, mosh, nagios, netcat, nginx, nodejs, nova, olpc/sugar, opengl, openshift, openssh, openssl, openstack, openvpn, pascal, php, puppet, pyramid, python, rabbitmq, redis, reStructuredText, route53, ruby, s3, sass, selinux, SGE, SLURM, stomp, strace, systemd, tmux, trello, twisted, unbound, varnish, vim, websockets, zeromq, zsh.
Master of Science and Bachelor of Science in Computer Science from Rochester Institute of Technology. Focused on artificial intelligence and minored in Psychology. Graduated 2009.