[three]Bean

Ømq and fedmsg DIY

Nov 16, 2012 | categories: python, fedmsg, zeromq, fedora View Comments

Bill Peck wanted to know if he could consume messages from the Fedora Infrastructure message bus without using the fedmsg libs and stack. The answer is yes.

I wrote up an example for him and turns out it was pretty simple, simple enough to warrant being added as its own new section in the docs.

The downshot here is that you don't get the configuration helpers of fedmsg.config, the message manipulation helpers of fedmsg.text and most importantly, the message validation of fedmsg.crypto. The upshot is that you get a quick connection without all those dependencies:

#!/usr/bin/env python
""" You'll need to install python-zmq (pyzmq) for this one.. """

import json
import pprint
import zmq


def listen_and_print():
    # You can listen to stg at "tcp://stg.fedoraproject.org:9940"
    endpoint = "tcp://hub.fedoraproject.org:9940"
    # Set this to something like org.fedoraproject.prod.compose
    topic = 'org.fedoraproject.prod.'

    ctx = zmq.Context()
    s = ctx.socket(zmq.SUB)
    s.connect(endpoint)

    s.setsockopt(zmq.SUBSCRIBE, topic)

    poller = zmq.Poller()
    poller.register(s, zmq.POLLIN)

    while True:
        evts = poller.poll()  # This blocks until a message arrives
        topic, msg = s.recv_multipart()
        print topic, pprint.pformat(json.loads(msg))

if __name__ == "__main__":
    listen_and_print()

p.s., in other news Amit Saha has a post up on collecting stats from the fedmsg bus. This is exciting to me.

View Comments

Search github from the gnome-shell

Nov 07, 2012 | categories: python, github, gnome, fedora View Comments

After the first one, I wrote another search provider for gnome-shell >= 3.6 in python, this time for flipping through your github repositories.

Screencast on vimeo. Source on github. Fedora package review in bugzilla.

A standing problem is how to prompt the user for their authentication tokens. Right now, I require that the user maintain a ~/.search-github file by hand including their github username and password. Ugly, but it works.

Best, I think, would be to integrate with GNOME Online Accounts. However, it appears that I cannot write a plugin for that.

What other approaches are available? I could use a modal prompt kind of like when NetworkManager prompts you for a wifi or VPN password and could then save credentials to the keyring. I haven't yet figured out how to create that prompt yet though, so if anyone knows.. kindly point me in the right direction.

Other ideas?

View Comments

apps.fp.o

Nov 05, 2012 | categories: fedora View Comments

I made a landing page for the previously barren https://apps.fedoraproject.org/ (however, it's up only in our staging environment until after the freeze).

http://threebean.org/apps.fp.o.png

It's built using the Javascript Infovis Toolkit. The data model is kept in a nice little YAML file so we can easily update it from puppet.

If you want to toy with it, the source can be found on github.

View Comments

Querying the Fedora Packages webapp

Nov 02, 2012 | categories: python, gnome, fedora View Comments

This post is about the python client stack I've been building around the Fedora Packages webapp.

That webapp is a replacement for the old fedora-community. It indexes a ton of information from Fedora Infrastructure and presents it in a package-centric way. It was developed by J5, Luke Macken, and Máirín Duffy.

From the Gnome Shell

The latest addition is a gnome-shell-3.6-compatible search provider that includes hits from the Packages webapp in your gnome-shell search results. No Amazon sponsored results -- I promise.

It's called gnome-shell-search-fedora-packages and I made this screencast showing it in action.

Writing it has got me excited to make search providers for all kinds of things: github repositories, pinterest results, etc.. This SSH search provider is a great idea (althought it doesn't seem to work right now). "Search your ~/.ssh/config for named remote locations and pop open terminals to them on activation."

From the Command Line

You can also search the Fedora Packages app from the console with my pkgwat tool:

--- ~ » pkgwat search nethack
+------------------+-----------------------------------------------------+
| name             | summary                                             |
+------------------+-----------------------------------------------------+
| nethack          | A rogue-like single player dungeon exploration game |
| nethack-vultures | NetHack - Vulture's Eye and Vulture's Claw          |
| slashem          | Super Lotsa Added Stuff Hack - Extended Magic       |
| crossfire        | Server for hosting crossfire games                  |
| crossfire-client | Client for connecting to crossfire servers          |
+------------------+-----------------------------------------------------+

But you can also extract more specific information, like a summary:

--- ~ » pkgwat info nethack
+--------------+------------------------------------------------------------------------+
| Field        | Value                                                                  |
+--------------+------------------------------------------------------------------------+
| upstream_url | http://nethack.org                                                     |
| description  | NetHack is a single player dungeon exploration game that runs on a     |
|              | wide variety of computer systems, with a variety of graphical and text |
|              | interfaces all using the same game engine.                             |
|              |                                                                        |
|              | Unlike many other Dungeons & Dragons-inspired games, the emphasis in   |
|              | NetHack is on discovering the detail of the dungeon and not simply     |
|              | killing everything in sight - in fact, killing everything in sight is  |
|              | a good way to die quickly.                                             |
|              |                                                                        |
|              | Each game presents a different landscape - the random number generator |
|              | provides an essentially unlimited number of variations of the dungeon  |
|              | and its denizens to be discovered by the player in one of a number of  |
|              | characters: you can pick your race, your role, and your gender.        |
| name         | nethack                                                                |
| summary      | A rogue-like single player dungeon exploration game                    |
| link         | https://apps.fedoraproject.org/packages/nethack                        |
| devel_owner  | lmacken                                                                |
+--------------+------------------------------------------------------------------------+

Bodhi metadata:

--- ~ » pkgwat releases nethack
+---------------+----------------+-----------------+
| release       | stable_version | testing_version |
+---------------+----------------+-----------------+
| Rawhide       | 3.4.3-27.fc18  | Not Applicable  |
| Fedora 18     | 3.4.3-27.fc18  | None            |
| Fedora 17     | 3.4.3-26.fc17  | None            |
| Fedora 16     | 3.4.3-25.fc15  | None            |
| Fedora EPEL 6 | None           | None            |
| Fedora EPEL 5 | 3.4.3-12.el5.1 | None            |
+---------------+----------------+-----------------+

And everything else the Packages app provides:

--- ~ » pkgwat --help
usage: pkgwat [--version] [-v] [-q] [-h] [--debug]

CLI tool for querying the fedora packages webapp

optional arguments:
  --version      show program's version number and exit
  -v, --verbose  Increase verbosity of output. Can be repeated.
  -q, --quiet    suppress output except warnings and errors
  -h, --help     show this help message and exit
  --debug        show tracebacks on errors

Commands:
  bugs           List bugs for a package
  builds         List koji builds for a package
  changelog      Show the changelog for a package
  contents       Show contents of a package
  help           print detailed help for another command
  icon           Show the icon for a package
  info           Show details about a package
  releases       List active releases for a package
  search         Show a list of packages that match a pattern.
  updates        List bodhi updates for a package

From Python

Both pkgwat and the gnome shell search provider use a common Python API to get their information. It's available on PyPI as pkgwat.api and from Fedora as python-pkgwat-api.

You can find the full documentation on readthedocs.org:

import pprint
import pkgwat.api

pprint.pprint(pkgwat.api.releases("awesome"))
{u'rows': [{u'release': u'Rawhide',
            u'stable_version': u'3.4.13-1.fc18',
            u'testing_version': u'Not Applicable'},
           {u'release': u'Fedora 18',
            u'stable_version': u'3.4.13-1.fc18',
            u'testing_version': u'None'},
           {u'release': u'Fedora 17',
            u'stable_version': u'None',
            u'testing_version': u'None'},
           {u'release': u'Fedora 16',
            u'stable_version': u'None',
            u'testing_version': u'None'},
           {u'release': u'Fedora EPEL 6',
            u'stable_version': u'None',
            u'testing_version': u'None'},
           {u'release': u'Fedora EPEL 5',
            u'stable_version': u'None',
            u'testing_version': u'None'}],
 u'rows_per_page': 10,
 u'start_row': 0,
 u'total_rows': 6,
 u'visible_rows': 6}

And Ruby? - Awesomely, David Davis just started work on a Ruby implementation.

View Comments

Querying Gnome-Shell Search Providers Over DBUS with Python

Oct 31, 2012 | categories: python, dbus, gnome, fedora View Comments

I'm working on a gnome-shell search provider to query the Fedora Packages webapp with the pkgwat python api.

In older versions of the gnome-shell, this used to be super simple -- you would just drop an XML file in /usr/share/gnome-shell/open-search-providers that defined a URL for where to search for stuff.

GNOME changed the way this all worked in gnome-shell 3.6 and you have to have an actual dbus service that returns results now. I haven't figured that part all out yet but I'm on the way.

Part of the first step was figuring out how the existing search providers worked, so I wrote this little python snippet to query the Documents search provider. Maybe you'll find it useful:

import dbus
import pprint

bus = dbus.SessionBus()
proxy = bus.get_object(
    # Query the Documents search provider...
    'org.gnome.Documents.SearchProvider',
    '/org/gnome/Documents/SearchProvider',

    # Or query my own search provider instead...
    #'org.fedoraproject.fedorapackages.search',
    #'/org/fedoraproject/fedorapackages/search',
)

# which interface do we make our calls against?
kw = dict(dbus_interface="org.gnome.Shell.SearchProvider")

# This is what we want to search our docs for
term = "aw"  # = "foobartest"

ids = proxy.GetInitialResultSet(term, **kw)

result = proxy.GetResultMetas(ids, **kw)
result = [dict(item) for item in result]

pprint.pprint(result)

J5's tool d-feet was indispensable in figuring this out. This dbus-python tutorial was super useful too.

For my own search provider, I forked lmacken's fedmsg-notify as a starting point. More on that later.

View Comments
« Previous Page -- Next Page »

[three]Bean.org