Like in previous years, a few of us from Fedora were at PyCon US in Portland Oregon for the week. The conference is over now (I'm sticking around for a day to explore the Pacific Northwest). Here are some of the highlights from the talks attended and the community sprint days:
Talks worth checking out
- K Lars Lohn's final keynote was out of control. None of us were ready for it. It wasn't even about python but I know everyone loved it. Parisa Tabriz's keynote on hacker mindset was very good (she's the "security princess" at Google) and Guido van Rossum's keynote on the state of python wandered off into an interesting autobiography about what made Python possible. If you're interested in software architecture, the Wednesday morning keynote on Plone and Zope by @cewing was an interesting overview of the evolution of that stack.
- Alex Gaynor's talk on automation for dev groups, cleverly titled "The cobbler's children have no shoes" was close to my heart. There was a salient point in the Q&A section about how while we often focus on automating workflows that are somehow problematic, sometimes that problem is a deeper social one. Automation can surface and inadvertantly exacerbate a tension between groups that have friction.
- For web development stuff, three talks are worth highlighting: @callahad of Mozilla (who is an awesome person) gave a talk on new mobile web technologies, Service Workers, Push, and App Manifests. It's worth a listen for people in the Fedora and Red Hat infrastructure ecosystem. @dshafik of Akamai gave a super interesting talk on HTTP/2 and the consequences for web devs. The short of it is that we have all these hacks in place that have become "best practice" over the years (sprite sheets, compressed and concatenated assets, bloated collection REST responses), none of which are necessary or desirable when we have HTTP/2 ready to go server-side. Sixty percent of browsers are ready to consume HTTP/2 apps and its all backwards compatible. Definitely worth looking into. Last but not least, if you do wev development, check out Sumana's talk titled "HTTP can do that?!" which goes over how to get the most out of HTTP/1 (something we've not always been the best at doing) -- very engaging.
- If you watch any of the talks here, check out Larry Hasting's talk on removing python's global interpreter lock. It's important if you use the language, deal with performance issues, and especially if you write C extensions. If none of those are you -- the details of the interpreter implementation are still super interesting. #gilectomy
- Of course the hallway track was the most valuable. I had good talks with @goodwillbits, @lvh, @sils1297, and too many others to mention.
For the community code sprints, I hacked with a couple other people on the test suite for koji which is the build system used by Fedora and many other RPM-based Linux distributions. We have a lot of web services and systems that go into producing the distro. Koji was one of the first that was written back in the day and it is starting to show its age. Getting test coverage up to a reasonable state is a pre-requisite for further refactoring (porting to python3, making it more modular, faster, etc..).
I wrote last week about how a few of us from Fedora were at PyCon US in Montreal for the week. It's all over and done with and we're back home now (I got a flu-like bug for the second time this season on the way home...) So, these are just some quick notes on what I did at the sprints!
Early on, I ported python-fedora to python3 and afterwards bkabrda picked up the torch and ported fedmsg and the expansive fedmsg_meta module. The one thing standing in the way of full-on python3 fedmsg is the M2Crypto library which will probably not see python3 compatibility anytime soon. Slavek courageously ported half of fedmsg's crypto stack to the python3-compatible cryptography library only to find that it didn't support the other half of the equation. We're keeping those changes in a branch until that gets caught up.
The most exciting bit was helping Nolski with his tool that puts fedmsg notifications on the OSX desktop. It totally works. Crazy, right?
I started a prototype of fedora-hubs which doesn't do much but display little dummy widgets, but it is useful for reflecting on how the architecture ought to work.
I wrote some code to get the fedmsg-notify desktop tool to pull its preferences from the FMN service. The changes work, but they required some server-side patches to FMN that are done, but haven't yet been rolled out to production (and we're in freeze for the Beta release anyways..).
In order to use your FMN preferences, you currently have to set a gsettings value by hand which is unacceptable and gross, but I'm not sure how to present it in the config UI. We can't just go all-in with FMN because there are other distros out there (Debian) which use fedmsg-notify but which don't run their own FMN service. We'll have to think on it and let it sit for a while.
Lastly, Bodhi2 saw some good work. (We fixed some bugs that needed to be hammered out before release and we actually have an RPM and installed it on a cloud node! Staging will be coming next once some el7 compat deps get sorted out.)
I was really glad to meet sijis for fun-time late-night hackery in the hotel lobby.
That's all I can remember. It was a whirlwind, as always. Happy Hacking!
A few of us from Fedora are at PyCon US in Montreal for the week. The conference portion is almost over and the sprints start tomorrow, but in the meantime here are some highlights from the best sessions I sat in on:
- @nnja gave a great talk on technical debt and how it can contribute to a "culture of despair".
- @sigmavirus24's talk on writing tests against python-requests was supremely useful. Using his material, I wrote a patch for anitya that solved an onerous and recurring issue with the test suite.
- Raymond Hettlinger gave a very nice talk on "moving beyond pep8" which was pretty relevant for my team and our code review practices. We write a lot of code which entails doing a lot of code review. His thesis: working in a cosmetic pep8 mindset causes you to often miss the elephant in the room when doing code review. Instructive.
- There was a very good talk on one particular company's experiences with a microservices architecture. It is of special interest to me and our work on the Fedora Infrastructure team with lots of good take-aways. The video of it hasn't been posted yet, but definitely search for it in the coming days.
- I quite disagreed with some of the method presented in the effective python session. No need for wrapper-class boilerplate -- just use itertools.tee(...)!
- Some others: distributed systems theory, interpreting your genome, systems stuff for non-systems people, and ansible were all very nice.
Some hacking happened in the interstitial periods!
- I wrote a prototype of a system to calculate and store statistics about fedmsg activity and some plugins for it. This will hopefully turn out to be quite useful for building community dashboards in the future (like a revamped releng dashboard or the nascent fedora-hubs).
- We ported python-fedora to python3! Hooray!
- The GSOC deadline really snuck up on us, so Pierre-Yves Chibon and I carved out some time to sit down and go over all the pending applications.
I'm really looking forwards to the sprints and the chance to work and connect with all our upstreams. We'll be holding a "Live From Pycon" video cast at some point. Details forthcoming. Happy Hacking!
So I got to attend PyCon US again this year and while its customary to write a little report when you return from such a thing, I feel like, now a month past, that I've lollygagged so long that the writer's block is getting worse, not better. Now my report is untimely and it feels odd to even publish it. Better late than never? Yes. Let's go with that!
Jessica McKellar's keynote was amazing. It was good in its own right, but was also important in so far as it framed the process of change ongoing in both the python community and its major conference. People know the state of women in FLOSS is abysmal. The effort to turn things around was clear and front and center this year: "outreach works".
Guid Van Rossum's keynote featured a "no to 2.8" image meaning, there is not nor will there be a python-2.8 release. For Fedora and its downstreams, it serves as a reminder that we need to beef up on python3 preparedness. We're making progress. Do pay attention to the proposed change to make python 3 the default in Fedora 22 and help clear the way where you can.
Diversion into ML
I attended a number of talks on machine learning and big data and got all inspired. I wrote this little throwaway program in between sessions. It reads in a bunch of newgroups and then tries to predict which newsgroup each posting belongs to (and after lots of tweaking, it got a 100% success rate which is crazy).
Ultimately, what I want to do is write a plugin for mailman3. In the mailman3/hyperkitty UI you can tag/categorize threads. Each community I imagine would end up tagging threads with their own jargon: "this thread is a flamewar", "this is a feature request", "this one's a bug report", "this is a moonshot".. or whatever. We could build a plugin that reads in all those tags for each community and then builds a predictive model that can be applied to new messages as they come in.
We could proactively tag messages using the community's own jargon! My little script seems to work well enough and we know how to build mailman3 plugins. At this point we just have to wait for the community to accumulate a set of real world tags we could train and test against.
Just being there puts us in a position to build relationships with other upstreams and the rest of the community but the PyCon sprints were concretely productive for us this year. A couple people from my team were present.
- Aurélien got his changes merged into mailman proper(!) and they prepared to cut a beta release of the whole suite(!!).
- Toshio put work in on warehouse, the next generation of PyPI.
- Luke tore it up on Bodhi2 (crystal ball -- we'll have it deployed before Flock this year).
- I got a barrel's worth of python3 porting done (fedmsg is now python3 ready!).
So, I've been mentoring for some of the Fedora projects a la the GNOME Outreach Program for Women and I realized today that I'd actually met one of the applicants before in person at the Sprints at PyCon 2012. She could only stop in to say "Hi" and couldn't participate though; she was caring for an infant at the time. (Her husband did get to participate).
Since the sprints are where you can build real relationships with upstream(s), develop skills, become inspired, have an impact, find a role model, be a role model, etc.. I thought it would be good for the conference (or some sponsors) to provide quality, on-site childcare at the sprints. I wrote to pycon-organizers and stuff seems to be moving ahead. @jessenoller tweeted:
PyCon US 2013 Survey: 'Would You Use Subsidized Childcare at PyCon?' http://bit.ly/V0g5jf
A survey. Nice!