jump to navigation

Pylons and PrefixMiddleware July 16, 2014

Posted by PythonGuy in Pylons, Python.
add a comment

So Python Guy was trying to get his Pylons website working with Amazon’s Elastic Beanstalk behind its Elastic Load Balancers running SSL. Everything seemed to work fine until Python Guy noticed that redirect() was sending URLs that pointed to the wrong address.

Never fear, Python Guy is here! All that needed to be done was to put PrefixMiddleware in front of the app. See here for instructions!

Pyjamas + Pylons June 4, 2011

Posted by PythonGuy in Pyjamas, Pylons.
add a comment

I’m looking into Pyjamas, specifically combining it with Pylons.

First note: easy_install pyjamas does not work with virtualenv. This is not a good sign. Maybe because it is a build tool, this isn’t going to be as important as I thought. But something tells me something is fishy about the install process.

On Fedora:

$ sudo yum -y install pyjamas*

The base pyjamas package is missing a lot of things you’ll want to monkey around with, so you might as well install them all.

The pyjamas-examples package has all the examples. You may want to copy the entire thing into your home directory before modifying anything.

Pyjamas: I note that the build scripts don’t work out of the box, on account of /usr/share/pyjamas/bin/pyjsbuild is missing. This is very, very curious. Perhaps the pyjamas RPM doesn’t install in a complete state, or the build scripts are just plain wrong. Indeed, /usr/bin/pyjsbuild does exist, so the build scripts are wrong.

I’ve figured out a way to integrate Pylons and Pyjamas somewhat elegantly. Inside your Pylons project:

  1. Create a library directory for your_project/ui by touching your_project/ui/__init__.py
  2. Put your Pyjamas script in there in ClassName.py.
  3. Build your stuff with a script named build.sh which contains: /usr/bin/pyjsbuild -o your_project/public/ui your_project.ui.YourClass
  4. Copy the main HTML file (your_project.ui.YourClass.html) to your public/index.html.
  5. Modify the meta tag at the top of your new index.html to put “/ui/=” before the class name.

What we’re doing here is organizing your code so that all the Pyjamas stuff lives under your_project.ui. This unique namespace shouldn’t be relied upon but by things in that namespace, since whatever you rely on is going to be converted to Javascript.

That last step tells Pyjamas that all your stuff lives under the /ui/ path. I found this by using Firebug on the startup script.

Using Formencode for Real-World Code January 17, 2011

Posted by PythonGuy in Advanced Python, Formencode, Pylons.
add a comment

Formencode is nice and pretty, and fun to use. However, even the author admits there are problems. While it does really well with trivial things, more complicated things get complicated fast.

In my own real-world usage, I’ve tried to tame Formencode. I’ve found that taming it is quite difficult. Instead, it’s better to simply fall back to more primitive methods of form validation.

That’s not to say the code in Formencode isn’t useful, even when falling back! No, I still use Formencode’s beautiful Email validator, even in the most complex forms. Here’s an example how:


2011 Predictions for Python January 13, 2011

Posted by PythonGuy in Pylons, Python, Ruby, SQLAlchemy.
add a comment

I guess you’re supposed to make predictions in January. Here’s mine.

At the end of 2011, Python 2 will still be more popular than Python 3, but some important projects will migrate to Python 3 anyway.

Sometime in 2011, a lot of Ruby coders are going to wake up and start playing with something else. Not that Ruby’s bad, it’s just not the best. And it has some serious issues.

Sometime in 2011. Guido van Rossum will say at least one really wise thing that will completely redefine how we look at some aspect of day-to-day programming. Maybe two.

Sometime in 2011, PyPy will get a version of Python that runs faster than CPython.

Sometime in 2011, serious discussion will be had about whether we should all program in the PyPy dialect—just in case it does take off.

Sometime in 2011, Pyramid will be released and probably ignored for a long time. Who knows? Maybe it will be better than Pylons. Maybe not. It’s not really important because there won’t be a huge reason to shift to it. Regardless, the team developing Pyramid is going to learn a lot about the problems they were trying to solve, and propose something even shinier and better to solve all the problems they tried to solve in the past and a whole lot of new problems they invented because they are actually from the distant future.

Sometime in 2011, SQLAlchemy will become so intelligent it may actually be considered as a running mate in the 2012 presidential elections.

Sometime in 2011, someone’s going to give example code for their research paper in Python. And someone else is going to read that paper and understand it even though they don’t know Python. And then they are going to go through the rest of the month thinking that Python is really this cribbage people use for writing research papers. When someone finally points out it’s a real language and the example code was actually production in a working system, he’s going to do a double-take and spit his coffee all over. It’ll be a mess.

Taming SQLAlchemy January 11, 2011

Posted by PythonGuy in Pylons, SQLAlchemy.
add a comment

There is one Python module out there that is more magical than all the others combined. Those of you who know it know it as SQLAlchemy.

After working intimately with SQLAlchemy for over two years now, here are the recommendations I have for you, the SQLAlchemy newbie, before implementing your first major project.

One: Always stick all your SQLAlchemy stuff into three separate modules. One has all the specific types (these may be spread around a bit), another has the tables, and the last has the objects.

Two: Your SQLAlchemy objects are the ideal model objects in the MVC architecture. That is, you don’t need a wrapper around anything, like you do in other languages. Simply adapt the classes into your own specific needs. Add additional functions and methods as needed. Be sure that your do as much as you can in the mapper, because the SQLAlchemy mapper function is smarter than Einstein.

Three: Sometimes your queries get really hard, and it’s about that time that you fall back to your tables. Thankfully, you can combine your objects and your tables in queries. One trick I used is to have your query on the outside be for the object you want, and then have a subquery that is simply for the IDs you are interested in.

Four: Ask for help when needed! SQLAlchemy is extremely complicated, and even the experts need advice or help from time to time. The mailing list is awesome, and people there are friendly.

Five: Just like it takes a bit of time before you’re comfortable with iterators, it will take a while before you start passing SQLAlchemy Query objects around as if they were lists. The vast majority of your data work will be massaging the Query object, so get used to it as the currency in your data algorithms.

Six: Leave performance for last. You might identify things that may slow down your query that you’ll want to remember right away, however. For instance, if you are iterating through a loop and querying a table for each iteration, then ask yourself whether it makes sense to simply stick that all in a query. After all, SQL’s looping mechanism kicks butt on Python’s looping mechanism because your SQL database should be smarter than you’ll ever hope to be. (I am a huge user of PostgreSQL so this is true. MySQL? Oracle? SQL Server? Not so much.) Even if your database isn’t very smart, SQLAlchemy is. The important point is that you trust SQLAlchemy and your database to be more optimized than any code you could ever write, and then come back later to revisit performance issues.

Seven: Ignore the No-SQL databases. Your SQLAlchemy setup should be much more elegant and transparent than the best No-SQL databases out there. When you’ve identified bottlenecks that cannot be addressed by better data organization or indexing, or smarter indexing and caching, then you can think of even better ways to do all the above. A good No-SQL database can be a good cache, but that’s about as far as I’ll trust them.

In my work on Deal-Digger.com, using SQLAlchemy has meant I can implement 3 to 4 table relationships from the top of the Pylons stack to the bottom in about 10 minutes. There isn’t a whole lot to test because my job isn’t to test SQLAlchemy. The queries are legible, easy to work on, and honestly, fun to play with. I heartily endorse SQLAlchemy and encourage you to use it whenever you need to speak SQL!

Pylons Rocks! January 6, 2011

Posted by PythonGuy in Pylons, Python, SQLAlchemy.
add a comment

Python Rocks! Pylons Rocks! (Pyramid—I don’t know yet.)

Here’s what Python Guy does for work now. Go subscribe and get yourself some deals.

Content-type for Pylons Static Content February 6, 2008

Posted by PythonGuy in Pylons, Web Technologies.

So I was playing with my shiny new Firefox3 beta 2. I was admiring the wonderful features of the canvas object. And I started to want to really get into SVG.

Well, I ran into a problem. My pylons server wasn’t serving SVG files as SVG files.

I put “test.svg” into my project’s “public” directory. I browsed to the URL http://localhost:5000/test.svg . I noticed that Firefox didn’t recognize the file and asked me to open another app to render it. (eog is nice, and it does a terrific job rendering it, but I wanted to have Firefox do the hard work this time around.)

Just to be sure that Firefox could render the file, I browsed to the absolute path on my local hard drive. That is, “file://home/pythonguy/MyProject/myproject/public/test.svg” Sure enough, it renders fine.

What else to do? Well, I realize by reading the FAQ on SVG for Firefox that if the server doesn’t send the correct Content-type response, then Firefox won’t guess. Sure enough, this telnet session showed me what my server was saying:

$ telnet localhost 5000
GET /test.svg HTTP/1.0

HTTP/1.0 200 OK
Server: PasteWSGIServer/0.5 Python/2.5.1
Date: Wed, 06 Feb 2008 03:24:32 GMT
content-type: application/octet-stream

So there’s the problem. My Pylons server doesn’t recognize the .svg ending on the file.

Digging deeper, I found this part in “config/middleware.py”:

    # Static files
    javascripts_app = StaticJavascripts()
    static_app = StaticURLParser(config['pylons.paths']['static_files'])
    app = Cascade([static_app, javascripts_app, app])
    return app

Well, this certainly is interesting. The “Cascade” object lives in “paste/cascade.py”. All it does is try out the different apps until it gets one that doesn’t give an error response. The “StaticURLParser” object is the one that serves the static files.

Looking in “paste/urlparser.py” (thank goodness for explicit imports in Python!), I found a reference to “FileApp” in “paste/fileapp.py”. There, I see that the mime type is determined by the mimetypes module, part of the standard packages for Python.

Poking around mimetypes, I am not surprised to see that files ending in “.svg” are not recognized. (I am surprised, very happily surprised, that Pylons doesn’t try to redesign the wheel when it comes to MIME types.) Reading the documentation, it points out where it learns which file names represent which MIME types. On my system, that is “/etc/mime.types”.

Sure enough, SVG is missing from the list of MIME types in /etc/mime.types. Maybe it is out of date? (Unlikely, but possible.) Nope, it turns out that SVG isn’t registered yet as a valid MIME type.

I hand-edited the file to recognize SVG. Telnet now gives me:

$ telnet localhost 5000
GET /test.svg HTTP/1.0

HTTP/1.0 200 OK
Server: PasteWSGIServer/0.5 Python/2.5.1
Date: Wed, 06 Feb 2008 03:24:32 GMT
content-type: image/svg+xml

But Firefox still doesn’t recognize it. Nuts.

SQLAlchemy Rocks! Part II August 16, 2007

Posted by PythonGuy in Advanced Python, Myghty, Pylons, SQLAlchemy, Web Technologies.

As a follow up to the previous article, SQLAlchemy Rocks!, here’s how I used the page object.

Remember, I’m using Pylons. Here’s the controller.

    def index(self):
        page = int(request.params.get('p', 1))
        results_per_page = int(request.params.get('rpp', 10))

        c.portfolio_page = model.portfolios_page(page, results_per_page)

        return render_response('/home/index.myt')

And portfolios_page is defined as:

def portfolios_page(page, results_per_page):
    return Page(Portfolio.query().order_by(asc('name')),
            page, results_per_page)

Here, Portfolio is an ORM object, not a table.

The corresponding Myghty template (just the relevant bits):

Items <% c.portfolio_page.first+1 %>
to <% c.portfolio_page.last+1 %>
of <% c.portfolio_page.total %>
    <th>Net Worth</th>
%   for p in c.portfolio_page:
    <td><a href="<% h.url_for(controller="portfolio", portfolio=p.id) %>"><%
        p.name %></a></td>
    <td><% p.net_worth |money %></td>
    <td><% p.permissions(c.user) %></td>
%   # end for
Results per page: <% c.portfolio_page.results_per_page %>
Page: <% c.portfolio_page.page %>
Total pages: <% c.portfolio_page.pages %>

Hope this isn’t too much code. Take your time to grok it all. Feel free to ask specific questions in the comments.