[three]Bean
python3-pyramid packaged for Fedora
Apr 15, 2013 | categories: pyramid, fedora View CommentsIt took some lifting, but I got python3-pyramid packaged and in Fedora.
I know none of us are using python3 in the wild yet, but its another brick layed on the long road to the future.
Moksha Tutorial - Live Graph of User Activity
Sep 17, 2012 | categories: python, pyramid, fedora, moksha View CommentsI've been working recently on sprucing up the fedmsg docs and the underlying Moksha docs. It seemed a natural extension of that work to put together some Moksha tutorials like this one.
Here I'll be showing you how to add a websocket-powered d3 graph of user activity to a Pyramid app using, as I said, Moksha.
Note
You can find the source for this tutorial on github.
Bootstrapping
Note
Bootstrapping here is almost exactly the same as in the Hello World tutorial. So if you've gone through that, this should be simple.
The exception is the new addition of a tw2.d3 dependency.
Set up a virtualenv and install Moksha and Pyramid (install virtualenvwrapper if you haven't already):
$ mkvirtualenv tutorial $ pip install pyramid $ pip install moksha.wsgi moksha.hub $ # tw2.d3 for our frontend component. $ pip install tw2.d3 $ # Also, install weberror for kicks. $ pip install weberror
Use pcreate to setup a Pyramid scaffold, install dependencies, and verify that its working. I like the alchemy scaffold, so we'll use that one. Its kind of silly, though: we won't be using a database or sqlalchemy at all for this tutorial:
$ pcreate -t alchemy tutorial $ cd tutorial/ $ rm production.ini # moksha-hub gets confused when this is present. $ python setup.py develop $ initialize_tutorial_db development.ini $ pserve --reload development.ini
Visit http://localhost:6543 to check it out. Success.
Enable ToscaWidgets2 and Moksha Middlewares
Note
Enabling the middleware here is identical to the Hello World tutorial.
Moksha is framework-agnostic, meaning that you can use it with TurboGears2, Pyramid, or Flask. It integrates with apps written against those frameworks by way of a layer of WSGI middleware you need to install. Moksha is pretty highly-dependent on ToscaWidgets2 which has its own middleware layer. You'll need to enable both, and in a particular order!
Go and edit development.ini. There should be a section at the top named [app:main]. Change that to [app:tutorial]. Then, just above the [server:main] section add the following three blocks:
[pipeline:main] pipeline = egg:WebError#evalerror tw2 moksha tutorial [filter:tw2] use = egg:tw2.core#middleware [filter:moksha] use = egg:moksha.wsgi#middleware
You now have three new pieces of WSGI middleware floating under your pyramid app. Neat! Restart pserve and check http://localhost:6543 to make sure its not crashing.
Provide some configuration for Moksha
Warning
This is where things begin to diverge from the Hello World tutorial.
We're going to configure moksha to communicate with zeromq and WebSocket. As an aside, one of Moksha's goals is to provide an abstraction over different messaging transports. It can speak zeromq, AMQP, and STOMP on the backend, and WebSocket or COMET emulated-AMQP and/or STOMP on the frontend.
We need to configure a number of things:
- Your app needs to know how to talk to the moksha-hub with zeromq.
- Your clients need to know where to find their websocket server (its housed inside the moksha-hub).
Edit development.ini and add the following lines in the [app:tutorial] section. Do it just under the sqlalchemy.url line:
##moksha.domain = live.example.com moksha.domain = localhost moksha.notifications = True moksha.socket.notify = True moksha.livesocket = True moksha.livesocket.backend = websocket #moksha.livesocket.reconnect_interval = 5000 moksha.livesocket.websocket.port = 9998 zmq_enabled = True #zmq_strict = True zmq_publish_endpoints = tcp://*:3001 zmq_subscribe_endpoints = tcp://127.0.0.1:3000,tcp://127.0.0.1:3001
Also, add a new hub-config.ini file with the following (nearly identical) content. Notice that the only real different is the value of zmq_publish_endpoints:
[app:tutorial] ##moksha.domain = live.example.com moksha.domain = localhost moksha.livesocket = True moksha.livesocket.backend = websocket moksha.livesocket.websocket.port = 9998 zmq_enabled = True #zmq_strict = True zmq_publish_endpoints = tcp://*:3000 zmq_subscribe_endpoints = tcp://127.0.0.1:3000,tcp://127.0.0.1:3001
Emitting events when users make requests
This is the one tiny little nugget of "business logic" we're going to add. When a user anywhere makes a Request on our app, we want to emit a message that can then be viewed in graphs by other users. Pretty simple: we'll just emit a message on a topic we hardcode that has an empty dict for its body.
Add a new file, tutorial/events.py with the following content:
from pyramid.events import NewRequest from pyramid.events import subscriber from moksha.hub.hub import MokshaHub hub = None def hub_factory(config): global hub if not hub: hub = MokshaHub(config) return hub @subscriber(NewRequest) def emit_message(event): """ For every request made of our app, emit a message to the moksha-hub. Given the config from the tutorial, this will go out on port 3001. """ hub = hub_factory(event.request.registry.settings) hub.send_message(topic="tutorial.newrequest", message={})
Combining components to make a live widget
With those messages now being emitted to the "tutorial.newrequest" topic, we can construct a frontend widget with ToscaWidgets2 that listens to that topic (using a Moksha LiveWidget mixin). When a message is received on the client the javascript contained in onmessage will be executed (and passed a json object of the message body). We'll ignore that since its empty, and just increment a counter provided by tw2.d3.
Add a new file tutorial/widgets.py with the following content:
from tw2.d3 import TimeSeriesChart from moksha.wsgi.widgets.api.live import LiveWidget class UsersChart(TimeSeriesChart, LiveWidget): id = 'users-chart' topic = "tutorial.newrequest" onmessage = """ tw2.store['${id}'].value++; """ width = 800 height = 150 # Keep this many data points n = 200 # Initialize to n zeros data = [0] * n def get_time_series_widget(config): return UsersChart( backend=config.get('moksha.livesocket.backend', 'websocket') )
Rendering Moksha Frontend Components
With our widget defined, we'll need to expose it to our chameleon template and render it. Instead of doing this per-view like you might normally, we're going to flex Pyramid's events system some more and inject it (and the requisite moksha_socket widget) on every page.
Go back to tutorial/events.py and add the following new handler:
from pyramid.events import BeforeRender from pyramid.threadlocal import get_current_request from moksha.wsgi.widgets.api import get_moksha_socket from tutorial.widgets import get_time_series_widget @subscriber(BeforeRender) def inject_globals(event): """ Before templates are rendered, make moksha front-end resources available in the template context. """ request = get_current_request() # Expose these as global attrs for our templates event['users_widget'] = get_time_series_widget(request.registry.settings) event['moksha_socket'] = get_moksha_socket(request.registry.settings)
And lastly, go edit tutorial/templates/mytemplate.pt so that it displays users_widget and moksha_socket on the page:
<div id="bottom"> <div class="bottom"> <div tal:content="structure users_widget.display()"></div> <div tal:content="structure moksha_socket.display()"></div> </div> </div>
Running the Hub alongside pserve
When the moksha-hub process starts up, it will begin handling your messages. It also houses a small websocket server that the moksha_socket will try to connect back to.
Open up two terminals and activate your virtualenv in both with workon tutorial. In one of them, run:
$ moksha-hub -v hub-config.ini
And in the other run:
$ pserve --reload development.ini
Now open up two browsers, (say.. one chrome, the other firefox) and visit http://localhost:6543/ in both. In one of them, reload the page over and over again.. you should see the graph in the other one "spike" showing a count of all the requests issued.
ToscaWidgets2 Bugsprint Report
Mar 04, 2012 | categories: python, toscawidgets, pyramid, turbogears View CommentsHere's the rundown of the pre-PyCon tw2 bugsprint. The unstoppable Greg Jurman and I coordinated in google+ and IRC (with lots of ad-hoc visitors in both) and did all the damage we could do to the bug list. Our goal was to close enough that we could justify a solid 2.0 non-beta release but we didn't quite get there. We settled on 2.0 release candidate 1.
tl;dr - Big progress. We'll seal the deal on 2.0 at PyCon US next week.
Infrastructure:
- tw2 sourcecode is spread out in repos all over the place. We centralized everything at http://github.com/toscawidgets.
- We're using git-flow to manage team development and releases.
- We setup a http://cia.vc bot in #toscawidgets/IRC, which is awesome.
- We're in the continuous integration queue with Shining Panda.
Tickets closed:
- Duplicated TW Encoders
- Graceful degradation when JS disabled for tw2.dynforms
- HTML5 Prompt Text
- Configurable 'location' for resources
- Disable resource injection on a per-resource basis
- Document middleware configuration values
- Handling SCRIPT_NAME
- Internationalization
- Inline templates
- Refactor resources for better compatibility with tw0.9
- Automatic 'modname' for resources
- Shortcuts for controller registration / Tidy up tutorial
Tickets worked on, but not complete:
Tickets created:
TW2 Bugsprint; the Final Countdown
Mar 02, 2012 | categories: python, toscawidgets, pyramid, turbogears View CommentsTomorrow marks the start of the toscawidgets2 24 hour bugsprint and I couldn't be more stoked. We haven't even started yet but tw2 development pushed ahead anyways.
We have:
- redesigned the tw2 homepage.
- improved tw2.jqplugins.jqgrid's sqlalchemy integration.
- pumped out one or two miscellaneous fixups to tw2.jqplugins.ui.
- released tw2.jqplugins.elrte.
- released tw2.jqplugins.elfinder (which is amazing).
- released tw2.captcha based on tgcaptcha2.
- released an alpha version of tw2.d3. It's built with WebSocket support in mind. More on that soon!
- been excited to learn that the TurboGears2 team is aiming to standardize on tw2.
- been fielding a lot more support for tw2 in Pyramid 1.3 both on the mailing list and in #toscawidgets. There's a bug still lurking out there for windows+tw2+pyramid users. At this point, if you can reproduce it and tell us about it, that would be helpful.
- got our first reports of people using tw2 with bottle. It worked!
Goal: (solidly) close as many bugs as possible and push as many libraries from beta to release as we can.
Here's to winning that game.
Hacking tw2 resource injection (in pyramid)
Feb 11, 2012 | categories: python, toscawidgets, pyramid View CommentsIn #toscawidgets
this morning, zykes- was asking about how to do the
jquery removal hack I wrote about a month back but in Pyramid instead of
TurboGears2. First I'll summarize the problem, then show you our solution.
toscawidgets2 tw2.*
provides lots of handy web ui widgets
including a wrapper around jquery-ui, jqgrid, and lots of fancy data widgets.
One of the problems it solves for you is de-duplicating resources. Say you're
including two fancy jquery-related widgets. Both of them require that
jquery.js
be included on the page. tw2
includes a
piece of middleware that tracks all the resources required by the widgets being
rendered on a given request, reduces that list to a set with no duplicates,
orders it by dependency, and injects it into the
tag.
Usually this is fine, but in some cases you want an exception to the rule.
Say, like in my previous post, you want to include jquery.js yourself manually
and you'd rather not have tw2
stomping all over your javascript
namespace. You could disable tw2
injection of all
resources, but you want all the others included -- just not jquery.
There is no automatic detection and filtration flag implemented in
tw2
and it would be tough to do in the general case.
tw2
can't guarantee that it's own jquery.js and your jquery.js are
the same, or different, or included, or not included, or anything -- it doesn't
even try.
To get tw2
to not do what you don't want, you need to
un-register tw2
's jquery resource from the middleware
yourself (on each request). Previously we came up with a working
hack that does this in the context of a TurboGears2 app. Here's the same
concept applied to a Pyramid app using a Pyramid "tween".
import tw2.core.core import tw2.jquery def remove_jq_factory(handler, registry): """ Remove tw2 jquery_js from tw2's middleware registry. In order to use this, you need to add the following to your myapp/__init__.py file: config.add_tween('myapp.tween.remove_jq_factory') """ def remove_jq_tween(request): # Send the request on through to other tweens and to our app response = handler(request) # Before the response is modified by the tw2 middleware, let's remove # jquery_js from its registry. offending_links = [r.req().link for r in [ tw2.jquery.jquery_js, ]] local = tw2.core.core.request_local() local['resources'] = [ r for r in local.get('resources', list()) if r.link not in offending_links ] return response return remove_jq_tween
Next Page ยป