[three]Bean

Querying user activity

Mar 24, 2014 | categories: fedmsg, fedora, datagrepper View Comments

In July, I wrote about some tools you can use to query Fedora package history. This post is just to point out that you can use the same approach to query user history. (It is the same data source that we use in Fedora Badges queries -- it also comes with a nice HTML output). Here's some example output from the console:

~❯ ./userwat ralph
2014-03-24T00:15:30 ralph submitted datagrepper-0.4.0-2.el6 to stable https://admin.fedoraproject.org/updates/datagrepper-0.4.0-2.el6
2014-03-24T00:15:28 ralph submitted python-fedbadges-0.4.1-1.el6 to stable https://admin.fedoraproject.org/updates/python-fedbadges-0.4.1-1.el6
2014-03-24T00:15:28 ralph submitted python-taskw-0.8.1-1.el6 to stable https://admin.fedoraproject.org/updates/python-taskw-0.8.1-1.el6
2014-03-24T00:15:27 ralph submitted python-tahrir-api-0.6.0-2.el6 to stable https://admin.fedoraproject.org/updates/python-tahrir-api-0.6.0-2.el6
2014-03-24T00:15:27 ralph submitted python-fedbadges-0.4.0-1.el6 to stable https://admin.fedoraproject.org/updates/python-fedbadges-0.4.0-1.el6
2014-03-23T13:51:16 ralph updated a ticket on the fedora-badges trac instance https://fedorahosted.org/fedora-badges/ticket/122
2014-03-21T17:08:21 ralph's packages.yml playbook run completed
2014-03-21T17:03:38 ralph started an ansible run of packages.yml
2014-03-21T16:31:48 ralph updated a ticket on the fedora-badges trac instance https://fedorahosted.org/fedora-badges/ticket/213
2014-03-21T15:09:10 ralph's python-bugzilla2fedmsg-0.1.3-1.el6 tagged into dist-6E-epel-testing by bodhi http://koji.fedoraproject.org/koji/taginfo?tagID=137

The tool isn't packaged at all, but here's the script if you'd like to copy and use it:

#!/usr/bin/env python
""" userwat - A script to query a user's history from fedmsg.
Author: Ralph Bean
License: LGPLv2+
"""

import datetime
import requests
import sys

format_date = lambda stamp: datetime.datetime.fromtimestamp(stamp).isoformat()


def make_request(user, page):
    response = requests.get(
        "https://apps.fedoraproject.org/datagrepper/raw",
        params=dict(
            meta=["subtitle", "link", "title"],
            start=1,
            user=[user],
            rows_per_page=100,
            page=page,
        )
    )

    return response.json()


def main(user):
    results = make_request(user, page=1)

    for i in range(results['pages']):
        page = i + 1
        results = make_request(user, page=page)

        for msg in results['raw_messages']:
            print format_date(msg['timestamp']),
            #print msg['meta']['title'],
            print msg['meta']['subtitle'],
            print msg['meta']['link']

if __name__ == '__main__':
    if len(sys.argv) != 2:
        print sys.argv
        print "Usage:  userwat <FAS_USERNAME>"
        sys.exit(1)

    username = sys.argv[1]
    main(username)
View Comments

Kerberized fedmsg? Probably not.

Feb 25, 2014 | categories: fedmsg, kerberos, fedora View Comments

Last week, someone contacted me about deploying fedmsg in a kerberos-enabled environment. They wanted to know if they could re-used their krb infrastructure to determine the authenticity of messages. I'm writing this blog post to help me think it through.

First, how fedmsg works

fedmsg doesn't encrypt its communications at all. Messages are either signed using X509 cert/key pairs and verified against against a Certificate Authority cert or signed and verified with GPG keys. Those keys/certs (as of now) need to be pre-shared.

Anyone can read the bus at any point. Anyone can write to "the bus" at any point. Only some messages are trusted. Read more in the fedmsg.crypto docs.

How I think Kerberos works

(... but I'm probably wrong. Have mercy on me).

Kerberos has two central servers that are used to manage trust between clients and servers:

  • The Authentication Server
  • The Ticket Granting Server

When a client wants to connect to a server...

  • It first contacts the Authentication Server. The Authentication Server responds with two things:
    • A "Client <-> Ticket-Granting Server" session key that is encrypted with the client's secret key.
    • A "Ticket Granting" Ticket that is encrypted with the Ticket Granting server's secret key.
  • The client then does two things:
    • The client decrypts the first part (the session key) which it can then use to contact the ticket granting server (it sends an "Authenticator"; it has a session with it now).
    • It sends the second part -- the encrypted Ticket Granting Ticket -- to the Ticket Granting server plus the ID of the service it ultimately wants to connect to.
  • The Ticket Granting Server then does two things:
    • The Ticket Granting server decrypts the Ticket Granting Ticket (with its own secret key)
    • It also uses the session key to decrypt the Authenticator.
  • Now the Ticket Granting Server is assured that the client is who they say they are. The Ticket Granting server then does two more things:
    • The Ticket Granting Server sends a "Client <-> The Service It Wanted To Connect To" ticket back to the client which is encrypted with The Service It Wanted To Connect To's secret key.
    • It also sends a session key for the client <-> server session back to the client.
  • Dutifully, the client sends this new ticket to the server with which it ultimately wants to be in a authenticated relationship.
  • That server then decrypts all this. It increments a timestamp and re-encrypts it with their new session key. It sends this back to the client.

It looks like when you issue kinit on a machine, that completes step 1. Steps 2-6 are carried out independantly by programs.

Kerberized fedmsg?

The advantage of pulling off a kerberized fedmsg would be that you wouldn't have to set up a separate public key infrastructure just for fedmsg -- you could re-use your kerberos infrastructure. You wouldn't have to generate a CA cert. You wouldn't have to generate certs for each sending service, sign them with the CA cert, and distribute them, manage them...

Is it possible?

Any fedmsg publishing process would have to first assume that the Ticket Granting Ticket is already set up via kinit. What happens next? Usually that TGT is used to request another ticket specifically for communication between the requesting program and another specific service. fedmsg doesn't work that way... Messages are published to anyone/everyone -- an unspecifed set of recipients. It is confusing because the client/server relationship is all twisted.

In step 2, there is no "target service ID" to request.

In step 4, there is no "target service" with a secret key to encrypt anything for.

Even if we faked something out with a 'virtual service' with its on secret key, to whom would the fedmsg publisher send the encrypted ticket in step 5?

The prospects don't look good for a kerberized fedmsg, but I'm open to ideas if you have them. Hit me up in #fedora-apps on freenode or on the messaging-sig if you want to talk more about it.

UPDATE - 2014-02-25, Simo Sorce tells me that kx509 is a thing that could be used to acquire a short-lived certificate as long as you have a TGT. Apparently it needs some work first, though.

View Comments

bugz.fp.o EOL and switchover

Jan 13, 2014 | categories: fedora-packages, bugz, fedora View Comments

During the F20 release cycle and holidays, the infrastructure team tries not to make any disruptive changes to our web services. Now its January, we have a spate of built-up changes lined up for deployment. Lots of changes have been pushed out already.

One major change still to come is the introduction of pkgdb2 and the retirement of pkgdb. The original pkgdb provided an interface to Red Hat bugzilla made friendlier by an alias at bugz.fedoraproject.org. For instance, visiting bugz.fedoraproject.org/nethack would take you to admin.fedoraproject.org/pkgdb/acls/bugs/nethack.

Once pkgdb1 is retired, that will no longer be the case; bugz.fedoraproject.org/nethack will instead take you to the replacement interface on the fedora-packages webapp: apps.fedoraproject.org/packages/nethack/bugs/all.

View Comments

Badges Authz Feature

Jan 09, 2014 | categories: fedora, badges View Comments

A quick note: a new release of the Fedora Badges app went out before the holidays. The main new feature is some authorization code that allows us to grant particular users the rights to award particular badges.

This will make some new stuff possible. For instance, the Beat Writer badges are to be awarded to people who take on the writing of a documentation "beat". We don't really have a way to measure that programmatically. With the new authorization code, we can authorize randomuser/immanetize, the docs project lead, to award those badges as makes sense. Similarly, we have authorized patches to hand out the Web Warrior and Write Stuff badges.

More to come... If you haven't noticed, riecatnor is doing some prodigious work on new badge art.

View Comments

Planet Spew and a Badge Bonanza

Dec 13, 2013 | categories: planet, datagrepper, fedora, badges View Comments

Last night, we found a cronjob in Fedora Infrastructure's puppet repo that had incorrect syntax. It had been broken since 2009:

"Let's fix it!"
"Yeah!"
"Fixing is always good."

The cronjob's job was to run tmpwatch on the Fedora Planet cache. When it ran for the first time since 2009, it nuked a lot of files.

  • This caused the planet scraper to re-scrape a great many blog posts, also for the first time since 2009.
  • This then caused the planet scraper to broadcast fedmsg messages indicating that new blog posts had been found (even though they were old ones).
  • The badge awarder picked up these messages and awarded a wheelbarrow's worth of new blogging badges to almost everyone.

Take a look at the datagrepper history according to zodbot:

threebean │ .quote BDG
   zodbot │  BDG, fedbadges +1097.35% over yesterday
   zodbot │       ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▆▆█▂▂▁▁▁  ⤆ over 24 hours
   zodbot │       ↑ 18:19 UTC 12/12           ↑ 18:19 UTC 12/13
threebean │ .quote PLN
   zodbot │  PLN, planet +69411.11% over yesterday
   zodbot │       ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▃█▁▁▁▁▁▁▁  ⤆ over 24 hours
   zodbot │       ↑ 18:18 UTC 12/12           ↑ 18:18 UTC 12/13
ianweller │ lol

That's a lot of planet messages. :) Congratulations on your new blogging badges!

View Comments
« Previous Page -- Next Page »

[three]Bean.org