Subscribe: Jeffrey P Shell: Python
Added By: Feedage Forager Feedage Grade B rated
Language: English
code  dtml  idx  list  much  new  object  objects  page  part  parts  python  sitename  system  web  xml rpc  xml  zope 
Rate this Feed
Rate this feedRate this feedRate this feedRate this feedRate this feed
Rate this feed 1 starRate this feed 2 starRate this feed 3 starRate this feed 4 starRate this feed 5 star

Comments (0)

Feed Details and Statistics Feed Statistics
Preview: Jeffrey P Shell: Python

Jeffrey P Shell: Python

Python related programming topics

Last Build Date: Sat, 11 Jan 2003 19:25:18 GMT

Copyright: Copyright 2003 Jeffrey P Shell

Moving Toulouse

Sat, 11 Jan 2003 19:24:27 GMT

Industrie Toulouse has moved to A new home ( and a new publishing system. Most of the archives will be there soon. The Radio site will remain up indefintely, but no new content will appear here.

Big Alpha Releases for Zope and Python

Fri, 03 Jan 2003 22:22:33 GMT

Two big alpha releases were made for the new year: Python 2.3a1 and Zope 3X a1. Python 2.3 doesn't make any major changes to the language (unlike previous Python 2 series releases, which brought things like list comprehensions, generators, nested scopes, iterators, etc). Andrew Kuchling, as always, covers the changes extensively. Of interest are - ability to import from .zip files, universal newline support!, a new logging package, a new Set datatype and default Boolean datatype. Zope 3X (x for "experimental" - a final Zope 3 release without the 'X' is expected to have some Zope 2 migration features) is a completely different beast from its ancestors. Its primary focus is still the Object Publishing mission that's been in place since the days of Bobo, but beneath the covers it is shockingly different from Zope's 1 and 2 before it. Zope 3 is also dubbed the component architecture project. Its design focuses on many interacting components with well-defined interfaces. It's an aggregation heavy design, as opposed to the inheritance heavy design of Zope 2. Adapters exist to take the burden off of objects for out-of-scope elements such as text indexing or sorting-by-size. As the author of a particular object, you can implement those interfaces (ISearchableText, ISized) on the class itself, or as separate classes. Code that requires searchable text or sizing information asks for an ISearchableText implementation for your class, and since that's the only interface that it's expecting to use, it doesn't matter if it gets an adapter or your class - it only matters that the ISearchableText interface is implemented correctly. Design by Contract finally makes a strong appearance in the Zope world. Zope 3 should also score big on the packaging and deployment front. Zope 3 uses a new configuration language called ZCML. ZCML is used to register new components at load time, and to load in still further components. It wrests control away from Python's import statements and need for special functions in such as the common Zope 2 initialize(context). As such, it can be not only more expressive, but allows control over the order in which items get initialized. Under the mantra of explicit is better than implicit, all new components have to be explicitly added (usually in the products.zcml file). A side effect of this is that heavily customized Zope distributions can be made. Some such distributions exist now with Plone installers and BizarShop. Zope 3's customization and configuration capabilities should allow similar distributions to be made that don't look like Zope at all, while still tying in to all of the available features of Zope. The combination of heavy Interface use combined with rich configuration options should mean that any part of the system could be kicked out in place of a new one, as long as the expected interfaces are implemented. This is a fundamental feature of component systems, and Zope 3 looks like it will live up to this feature. Another big feature of Zope 3 is the proliferation of View Components. Views can be individual ZPT pages, attached to a particular content interface via ZCML. But they can also be full-on components in their own right, usually as Python classes. And they're not limited to HTML. Other views may be written to correspond with the other publishing channels (ftp, xml-rpc, and others in the future). I wrote a post back in September about my early experiments augmenting other Zope 3 components, in which I include some example code making an XML-RPC view onto a Job Board component written by someone else. My final paragraph in that post reads: And finally, Zope 3 should yield a usable scalable means of adapting the works of other developers into new solutions by providing better control of product/service/view configuration and overrides, such as adding a new XML-RPC API to someone elses bug tracking system without having to alter that bug tracking system, or use secret Python hacks to alter behaviour. A few other positive note[...]

Problems with StructuredText

Wed, 13 Nov 2002 16:50:10 GMT

I was going to write up my own list of issues I've had with Zope's StructuredText (Classic and NG) implementation, but this document sums it up nicely. My expectation of software is, to quote Radiohead, "No alarms and no surprises." "Zope" Page Templates live up to that mantra, as does reStructuredText, so far.

reStructuredText First Impressions

Thu, 31 Oct 2002 05:10:23 GMT

Today I started writing up user documentation for the project I'm wrapping up. It's the first time I've used reStructuredText, and I must say that I'm liking it. As far as structured text markup languages go, it's pretty comprehensive and not *too* obtrusive. It's definitely good enough for most technical documentation needs, and is in fact a PEP Formatting Standard (see also PEP 12 Source). Other items of interest, particularly to Python programmers, are: I include the source links to show the relative cleanliness of the markup, which is the point of all Structured Text formats. sigh I thought I'd have more to say, but I'm really too tired. So - quick wrap up: I was pretty productive with this stuff right off the bat. Nice. For "Zope", there is a proposal to integrate reStructuredText into Zope, hopefully for 2.7.

Looking ahead to Python 2.3

Tue, 08 Oct 2002 19:26:29 GMT

I was trying to ascertain the state of "Python" 2.3 today, and PEP 283 offered up an interesting list of what is already in for 2.3, and a fairly long list of "planned features". What's in? Two new types - booleans and sets!. A fundamental datetime type or interface may make the release as well. A planned possible feature is an Iterator Tools module featuring "goodies from SML and Haskell". I've been using Python for over six years now (it's hard to believe it's been that long!), and it's been interesting to watch the language grow since 1.3. A nice thing about Python's growth is that it's been fairly smart - as it picks up new features, it also simplifies itself. A lot of the old confusions / problems are gone or going away, and helpful features have popped up -- the type/class dichotomy is on its way out (it's interesting to see the prototype 'bool' type written up in Python by subclassing from 'int' - see PEP 285); booleans are coming in (I've always liked the way Python has treated true/false values, but it's nice to have actual 'True' and 'False' objects, and the built in 'bool' constructor); much of the old 'string' module has become methods on the 'string' type, removing the oddities of having a bunch of procedures to operate on a very common object in an object-oriented language; there are more unifications in expressions (the 'in' operator can now be used to search for substrings in a string: if 'this' in 'there was this thing..': in place of if 'there was this thing..'.find('this') > -1:; this also applies for dictionary keys: 'somekey' in mydict in place of mydict.has_key('somekey')); iterators are offered throughout the language now, sometimes offering shortcuts for common operations (such as reading individual lines from a file), and keeping resource use down for others; and we finally have nested scopes. Not to mention the little delights like Generators and List Comprehensions.

A Zope 3 story - augmenting other components

Mon, 02 Sep 2002 08:47:43 GMT

It's very late, so I'll try to make this quick and write up more later. Today, I finally had enough free time in my coffers to download "Python" from CVS (aka - 2.3 pre-alpha) onto my old iMac, and augment that with "Zope 3" - the ComponentArchitecture project. Then I installed the simple but usable Job Board Example. I soon noticed that said JobBoardExample was missing out on a couple of things - an editing interface, and an XML-RPC interface. One of the notions of Zope 3 is that it's supposed to be easier for developers to embrace and extend other components by being able to write new "views" for them. So, I decided to write a JobEditView. I made a new Product (esentially, a plain Python package) called JobBoardEditor, and filled it with a couple of files -, (the page template to edit a single job with), and configure.zcml, the configuration file that ties it into the Zope system. It wasn't long until I could go to a Job object and traverse to my new edit form, and then have the submit go to my 'edit' method. But, at this point, I'm running into security issues that are beyond my measure to figure out, especially at 1:00 am with a beer headache. ";->" My other thing to try out was putting a simple XML-RPC interface on to the JobList. This was particularly interesting to me as I've found that while traditional "Zope" method calls should - in theory - work fine with something like XML-RPC, they seldom do. Sometimes it's because too much HTML is returned from a call, but most of the time lately (for me at least), it's that I want the server to do some processing of the results before sending them out to the client, using the relatively simple data types afforded to XML-RPC. As a result, I've been adding in extra Python Scripts particularly for XML-RPC, and letting them work in the proper places either via acquisition or some other clever tricks. It's not a bad system, but it's not exactly formalized or repeatable, since it's not in a formal Product. In Zope 3, this is different. I wrote a simple XMLRPCView class. 'View' objects in Zope 3 have two state members - 'request' (the incoming request object), and 'context' - the object the View is applied against. So, for me to add in the new functionality whereby I could just get a list of approved Job titles, I wrote the following Python file in my JobBoardEditor product: from Zope.Publisher.XMLRPC.XMLRPCView import XMLRPCView from ZopeProducts.JobBoardEx.IJob import JobState class JobListXMLRPCView(XMLRPCView): __implements__ = XMLRPCView.__implements__ def listJobTitles(self): joblist = self.context job_ids = joblist.query(state=JobState.Approved) out = [] for jid in job_ids: out.append(joblist[jid].summary) return tuple(out) Then, I added the following to JobBoardEditor/configure.zcml (after adding xmlns:xmlrpc='' to the head zopeConfigure element): With this configuration statement, I've added a new View to be published on the XMLRPC channel (which in Zope 3 runs on a different port than normal HTTP requests, which is probably wise) for objects that implement the IJobList interface. I was able to do all of this outside of the JobBoardEx product. I just had to ensure that the Product was added in after JobBoardEx by using the site's products.zcml file to affect the ordering. After restarting Zope 3 (which is a very slow restart on this machine), I was able to do this: Py$ s = Server('http://localhost:8281/zoo/joblistmethods') Py$ s.listJobTitles() ['test test test'] I actually had created two jobs through the web interface, but had only approved one. So I ducked in to the web interface to approve the other job, and then made the call again: Py$ s.listJobTitles() [...]

Aha! (Python Patch)

Thu, 18 Jul 2002 02:19:05 GMT

So, the mysterious core dumps seem to be over, thanks to this patch on regarding pthreads and stack sizes on BSD. I reverted back from the Tracker changes I made last night, and nothing crashed. I did have to make one small patch to Tracker though, introducing a fun filled positive lookahead assertion to a regular expression used to generate HREF's.

Issue Trackers Revisited

Wed, 17 Jul 2002 03:26:02 GMT

So finally, I have "Tracker" working with "Zope" 2.5.1 and "Python" 2.1.3 on FreeBSD 4.5. It seems that the restricted Python engine (which I presume is new/revamped in 2.5?) has a problem with expressions that are heavy with parenthesis. So, after crawling through a lot of DTML code (some of it quite old - some of it predating the 'let' tag!) by slowly moving a 'dtml-raise' statement around, I was able to find all (or at least most) of my problems. I just couldn't take "Roundup" any more. Actually, as a single developer or with another geek, it was alright. But it's still young, and doesn't have a lot of the great user/security/collaboration issues that Tracker has (and which CMF Collector partly has). And it doesn't have the workflow. And it's still too much tweak-work to configure ones own instance of "Roundup" beyond the two templates they have. I am going to try to stay active with it though, but it really feels like the goals of "Roundup" and "Tracker" are very different. Tracker is an old unwieldy beast, however, and it seems to take a fair bit of grunt work to get it to work every time Zope has a new revision. Ugh.

An optimization dilemma

Mon, 08 Jul 2002 15:28:15 GMT

I'm facing a nice little Python/Zope programmers dilemma. I have a critical "Zope" "Python" Script - Python code objects in Zope that are editable through the web, and are subject to being run in a "restricted mode" interpreter that enforces Zope security policies. It's quite long for a single script, weighing in at approximately 170 lines. But it does its job well, and is written with maintainability in mind (there's only one variable in there that I find non-obvious, and whose usage I might change). Unfortunately, it seems to do its job slowly. I think there are a few contributing factors to this:
  1. Restricted Python. I think the security overhead is slowing the execution down. In evaluating the script's code, it doesn't really need to go through this.
  2. SQL Calls. There are a lot of calls into SQL Methods, both read and write. I'm not sure if this can be helped, or indeed is having any real noticeable effect on the speed.
  3. Generating/Sending Emails. I think there might be an issue here. The MailDropHost Product by Jens Vagelpohl could help as it sends mail asynchronously.
With those points in mind, I've started tackling the first bullet by moving the code into an External Method, where it is not victim to the Restricted Python interpreter. As a bonus, the code can be checked into CVS. But now I face a new dilemma - inline or breakout? Since the code was originally written as a Zope Python Script, it was one big inline piece of execution. Now that it's an External Method, I have the opportunity to refactor, and turn the code into a class/singleton wherein the code is broken out into methods that are executed in a certain order. My newer instincts are pulling me in this direction. But I know that it brings in new overhead to the code whose speed I'm already concerned about. Now, instead of dealing with restricted Python, I'd have to deal with a number of new function calls and name lookups. And, possibly, Thread Safety issues. The code in question is very central business logic. In order of importance, it needs to be solid, maintainable, and relatively fast. It succeeds on the first two points (very solid, fairly maintainable). The third one is a matter of some debate, most of which is going on inside my head. Future writings in this space will look at the avenues explored and if they yielded any results.

Customizing Roundup, day one

Tue, 02 Jul 2002 16:58:21 GMT

I spent some more time working with "Roundup" this morning. When you create a new instance, one of the files it generates is contains code to initialize the database schema, and also contains an empty IssueClass class, subclassed off of the default roundup.roundupdb.IssueClass. I was able to take advantage of this to customize the emails sent out by "Roundup" to make them look a bit more like the ones sent out by Tracker (as mentioned in the "Python based Issue Trackers" post, Tracker generates the nicest emails of any issue tracking system I've worked with thus far).

Python based Issue Trackers

Tue, 02 Jul 2002 05:40:11 GMT

I've been on one end or another of three Python based issue trackers lately - Tracker, its descendant, CMFCollector, and it's competition in the Software Carpentry contest, Roundup. And there's always something there to make me unhappy. Tracker is one of the best issue trackers I have used. It's got a fairly solid user interface, a fairly solid built in Workflow, and hierarchical correspondence/state changing. It's one of the easiest web based issue tracking systems to use to follow the actual history of an item, and it's easy to see at a glance how actively an issue was discussed/reported on in the Tracker from the issue listing page. And Tracker sends out best emails of any bug tracking system I've seen. Sadly, it's pretty much dead (and in fact - it's been core dumping on my when I've attempted to use it with Zope 2.5.1 and Python 2.1.2/2.1.3). You can see it still in full action as the Tracker for the whole CMF project. CMFCollector is its replacement. I don't like it as much. It suffers from a flat transcript (by default in a non-monospaced font) that is harder to follow than the system used in Tracker. And the UI - by default - just isn't quite as much there, although this may be due to the pretty basic (or just plain weird) default CMF Skins. The pretty-much-default CMF Collector setup can be seen in action at The Zope Issue Tracker. The Plone (a massive skin/layer for the CMF) varation can be seen in action at The Plone Issue Collector. Roundup is not Zope based at all. It's a pure-python solution with a nice abstraction of layers (frontends/backends). It seems to be the most actively developed, but still feels young. Regardless of age, I've started installing it at work. The user interface is very basic (and kindof hard to read), and it suffers from the same flat-transcript issue that CMFCollector does. But it's fast, simple, and fairly solid. It has a few options for submitting/modifying issues - by email, by command line, and through the web. It can run as a standalone web server, as a CGI script, or through "Zope" via a frontend called ZRoundup. ZRoundup essentially passes and wraps Zope REQUEST data and passes it into Roundup, and returns the resulting HTML -- essentially a CGI-esque gateway for Zope. But it allows Roundup objects to be placed alongside other Zope content, like ZWiki's or just plain documents as part of a simple project management solution. I miss Tracker. And CMFCollector isn't yet a worthy replacement for it. Roundup is small, fast, and looks to be broken out enough to be easily programmable. I may even look into getting involved with its development.

Universal Newline Support, coming soon to Python

Sun, 23 Jun 2002 19:41:52 GMT

PEP 278 is something I (and I'm sure many others) have been wanting for a very long time. Now, in Python 2.3 (most likely), you'll be able to have code like f = open(filename, 'rU'), and have ALL newlines in the file be the single, normal, 'n' character. Of interest is the new file object attribute, newlines:
A file object that has been opened in universal newline mode gets a new attribute "newlines" which reflects the newline convention used in the file. The value for this attribute is one of None (no newline read yet), "\r", "\n", "\r\n" or a tuple containing all the newline types seen.
This is especially welcome on "Mac OS X", as noted in the PEP, due to OS X's double standards - Classic and Carbon apps typically use the CR symbol to end a line, while Cocoa and Unix apps typically use LF.

Reordering Items in a Zope ObjectManager

Sat, 15 Jun 2002 17:40:39 GMT

We have movable Parts!. That's right. With a little bit of code, a couple of unit tests, and some surprisingly simple UI work, Parts in an SCS Document can be re-arranged. This is yielding something I've been wanting to do (and have) for years - decent through-the-web Document editing, with drop in objects than can be reorganized easily. While I'm sure it's been done many times before, it was great seeing the results yesterday as part of my own framework and application. The work went fairly quickly, once I moved the methods to the right class and added Unit Tests. The secret is in the _objects attribute of Zope ObjectManager based instances. _objects is a tuple of dictionaries with the keys 'id' and 'meta_type'. ObjectManager classes use this information to know what objects in the ObjectManager are being managed as subobjects, and to define their ordering. Most people never see or know of this attributes existence as its use is well-handled by the common ObjectManager API's. But one can use this list to their advantage, such as reordering. So, in todays code we have what I used to reorder Parts in my PartContainer class. There are two important elements - finding the index of the object in question, and then swapping its place in the list. We can't use parts.index(part_id) because of the complex data structure of the list - although, if we had the object in question, we could try parts.index({'id': obj.getId(), 'meta_type': obj.meta_type}). But the following works, and could be applied to any ObjectManager based class. Note - there are already Products available to bring order to Folders in Zope, such as Ordered Folder. It's a feature that was Proposed for Zope 2.6, but was deferred.. class PartContainer(ObjectManager): def _findPartIndex(self, part_id, objectList=[]): """ Returns the index of a part in the object list """ idx = 0 for obj in objectList: if obj['id'] == part_id: break idx += 1 else: ## part not found raise IndexError(part_id) return idx security.declareProtected(change_scs_documents, 'movePartUp') def movePartUp(self, part_id): """ Move a part up. Raises IndexError if part is not found """ ## make a copy of our _objects list, which is a part of ObjectManager. ## _objects is a tuple of dictionaries with keys 'meta_type' and 'id'. parts = list(self._objects) idx = self._findPartIndex(part_id, parts) if idx == 0: ## Can't move past top, just exit return 0 ## swap fields, moving idx up parts[idx], parts[idx-1] = parts[idx-1], parts[idx] self._objects = tuple(parts) return 1 security.declareProtected(change_scs_documents, 'movePartDown') def movePartDown(self, part_id): """ Move a part down. Raises IndexError if part is not found """ ## make a copy of our _objects list, which is a part of ObjectManager. ## _objects is a tuple of dictionaries with keys 'meta_type' and 'id'. parts = list(self._objects) idx = self._findPartIndex(part_id, parts) if idx == len(parts)-1: ## Can't move past bottom, so just exit return 0 ## swap fields, moving idx up parts[idx], parts[idx+1] = parts[idx+1], parts[idx] self._objects = tuple(parts) return 1 [...]

Zopeishy vs Pythonishy

Thu, 13 Jun 2002 14:57:23 GMT

According to this program by Jarno Virtanen, this site is a tad more zopishy than pythonishy.

Andrew Kuchlings "What's New in Python 2.3" in development

Tue, 11 Jun 2002 16:31:28 GMT

Andrew Kuchling has published an early development revision of "What's New in Python 2.3", detailing some of the work that's already been checked in, and what's no longer bound to the __future__.

ZWiki and logical structures.

Mon, 10 Jun 2002 19:59:30 GMT

ZWiki is a sehr-cool beast, as far as Wikis go. A very attractive feature, seen all over the web site is Ken Manheimer's work on page/parenting relationships. This makes it so that Wiki pages, although stored in a flat namespace (all in one folder) can appear to be hierarchical, depending on which page generated which page (parenting can be altered of course). This in turn leads to cool things, like a structured table of contents (see the Zope 3 wiki's TOC Here). It's especially cool in that it's fairly easy to change the hierarchical structure of the Wiki - by looking at a page's backlinks, one can reparent a page to one or more of its linkers. Essentially, new logical structures can grow independent of the physical structure. I've used a similar feature in a recent simple content system, wherein all documents were stored in a single folder, but the users entering content could classify documents in one or more categories. The main pages on the site were basically database queries (actually, catalog queries) based on the classifications. Thus, the site was built out of a simple logical structure rather than a physical one, which allowed documents to appear in more than one place if they needed to be. It's no unique concept, "Radio" and some other blogging tools allow this, much to my delight. In the simple compound system I did for that customer, the flat namespace also allowed simple inter-document link management. The author could say "document A links to document B", and when document B was deleted, the link to it would also evaporate. For some reason, doing deep inter-object linking in "Zope" is still tricky business, but there's hope on the "Zope 3" horizon (some of which looks like it may be backported to Zope 2).

Templates, code, and HTML

Tue, 28 May 2002 18:15:14 GMT

Half of our power is out, but the airport base station and the gateways are still on. So, todays teaser medicine consists of a "Python" translation of some UserTalk code I wrote (the first major script I wrote in Radio), runnable as an External Method in "Zope". Also included is the UserTalk code, as well as some DTML and ZPT output. I show this because, personally, I loathe coming across HTML in code. Sorry Jon. Writing for templates increases possibilities for reuse, since the data producing code is independent from the code that displays it. While you can do this in pure python scripts (by having one script produce data, and have another script as a consumer that outputs HTML from within Python), I take a "why bother?" attitude. I've had to maintain that HTML generating code in the past - the long ago past in web time (1996). And I hated the maintenance chores that seemed to go along with even the most trivial HTML. Why do we keep reinventing old problems? The script-inside-HTML way is often as broken, in my mind, as the HTML-inside-scripts way. It always surprises me to see ASP/PHP code that looks like:: <% for row in data { out.writeln(''); out.writeln(''); out.writeln(''); } %>
' + + '
Why bother having the HTML abilities of ?SP/PHP languages at all? I can understand why it happens. I just don't think it should. ";->". But, in the words of Dennis Miller, "Of course, that's just my opinion, I could be wrong". In any case, I present the Goldberg Variations: import os radioWwwFolder = '/Applications/Radio Userland/www' def listOutlines(): out =[] path = os.path.join(radioWwwFolder, 'outlines') def visit(arg, dirname, fnames): for fname in fnames: if fname.endswith('.opml'): fn = os.path.splitext(fname)[0] # Split extension off if dirname != path: # Deal with subdirectories fn = os.path.join(dirname[len(path):], fn) arg.append(fn) os.path.walk(path, visit, out) return out ZPT = """
  1. Outline
""" DTML = """
""" USERTALK = """ on listOutlines() { local(out = "
    \n"); local( + "outlines"); on visit(f) { f = file.fileFromPath(f); if string.hasSuffix(".opml", f) { f = string.popSuffix(f, '.'); out = out + "
  1. " + f + "
  2. n"}; return true}; file.visitFolder(path, 1, @visit); out = out + "
"; return out } """ [...]

More Page Templates (a buttal of sorts?)

Fri, 24 May 2002 04:49:20 GMT

Jon Udell and Zope Newbies both write stuff along the lines of I decided again not to use them because they would add more complexity to my sites and wouldn't solve any problems -- I'm not about to start using Dreamweaver or GoLive any time soon. It's just me and Pepper or Mozilla. I don't have to work with any designers, or that the learning curve is too steep. Personally, I think that the learning curve for Page Templates is much smaller than DTML. The only problem is in trying to think like DTML when you really shouldn't be. While there are helpful conversion guides, I do think it best to approach ZPT on a fresh project rather than trying to retrofit them into something existing (by either adding on to a site/project, or trying to replace existing DTML). Page Templates have a wonderfully clean syntax in the TAL/TALES combination. There are very few TAL attributes, and their operations and operational order are clearly defined. TALES eliminates much of DTML's guesswork by eliminating the whole "is it an expression? a literal? a name lookup?" game. The METAL macro expression system is a little more complicated, but ultimately provides a clean way of componentizing HTML/XML, and can be effectively used to eliminate the standard_html_header/footer idiom (I strongly dislike the whole header/footer inclusion idea proffered by so many templating languages, which require you to maintain balance between two separate files/objects to try to give some notion of "standard look" to a site. METAL is much more along the lines of Dreamweavers and GoLive's Templates idea, where a page has editable "slots" between commonly shared data, and can thus be edited as a whole page). There are areas that Page Templates break down, especially in comparison with DTML. First, Page Templates are purely about HTML/XML. They're not good for dynamically generating style sheets, Javascript, or SQL, although they can be used to do so (but as the old maxim goes, just because you can doesn't mean you should). Second, DTML has a strong history as a reporting oriented templating language. TAL and TALES don't really take on this responsibility, and rather hand it off to other modules like ZTUtils (Zope Template Utilities), a woefully underdocumented piece of the pie. The Page Templates approach is ultimately better, I believe, because it's more open to new components being used to handle batching and useful widgets, while with DTML you got what was there. The 'dtml-tree' tag has been woefully stretched in order to allow many variations in how an expanding tree might find its data. ZTUtils offers handy utility modules, classes, and functions to replace all of this and allow a developer to better handle how it all might work, but it's currently much harder to do things like dtml-tree in Page Templates. But DTML has suffered from many problems over its lifetime. Until Python Scripts came to Zope, it was the primary way most people learned how to program Zope. It grew a lot of programming language style muscle, while retaining all of its reporting language heritage. Its namespace stack is a powerful but often untamed beast, and its full of grotesque shortcuts and workarounds. Anyone who's seen _[_['sequence-item']] knows what I mean. And anyone's who's used _[_['sequence-item']] probably had a momentary twinge of "ugh!" before moving on. DTML has gotten the job done. There are places where it continues to get the job done - the SQL specific tags are one of my favorite things about doing SQL in Zope. But DTML is full of accidental surprises - a common mistake is due to the fact th[...]

Sortof Agile Modeling

Thu, 16 May 2002 02:17:56 GMT

I started work on a new application today. We sat around the white board coming up with objects/data, and I did some rapid modeling of the white board data into a rough UML class diagram using OmniGraffle 2.0. OmniGraffle is a fine choice for anyone (on Mac OS X) doing some form of Agile Modeling. I've had a love-hate relationship with UML over the years, and especially with modeling tools. There are many problems (that have been acknowledged, and some tools even do their best to solve them), such as model and source code getting out of sync, or how do you effectively model a language like "Python"?. On top of these issues lie many of the issues that have plagued programming, especially in the age of internet development, for years - the need for speed (just hurry and get it done! document it later!), scope (a project is [often misconceived as] too small to spend time modeling}, or inability to map web based programming techniques into communicative diagrams (is a JSP page a class? A UI Component? Or is its rendered state a UI component? Huh?). And often, good development processes seem to be killed by the attempted formalization of a process. This is why it's been good to see the rise of practices like "Extreme Programming", "Agile Modeling", Refactoring, and so on. While there is definitely value to a heavy process like the Rational Unified Process or Catalysis, small shops and teams just can't take on that kind of overhead most of the time. So, like my story about "Testing Joys" where I was able to get over that often difficult first hump from idea to realization through unit tests, today I re-experienced the same thing with a quick model. And I emphasize re-experienced, because these feelings get lost quickly during the course of a project, where deadlines and other pressures often kill off even the best practices a developer tries to have. So, "Agile Modeling". Today is actually the first time I've read about it at all, besides seeing the book title. And I don't claim to follow any of their practices fully. I just like the general idea. Modeling is about communication. Sometimes that communication is in the form of ideas of how a system could be shaped. Sometimes that communication is in the form of details about how a system is shaped. And often, a tool can be too rigid which gets in the way of the "shaping of ideas", or it can be too typecast to a particular language or target environment to accurately express the details of what has been/is being produced. The latter bullet has gotten in my way plenty of times. Few tools support Python concepts. Fewer still support Zope-ish ones. At Zope Corp, we were never able to come up with a really good system that would let an engineer jump right in easily, even using a Python friendly tool like Object Domain. Furthermore, many UML tools available for the Mac OS are either pure Java, or some other cross-platform compilation, making them awkward to use on a Mac. So where does my take on "Agile Modeling" come in? Usually, I need diagrams most for sketching out ideas about a schema design or workflow model. A tool like OmniGraffle makes this easy, since it's really a very basic diagramming tool. And that's its strength. Visio and Concept Draw are both expansive diagramming tools, with more flexibility than their Pure-UML/CASE tool counterparts, but they're still too heavyweight. The learning curve is probably too steep for a developer to pick up quickly enough to do something like capture a schema concepts off of a white board. And still other tools, like Illus[...]

Marshalling - the good point of XML-RPC and SOAP.

Tue, 07 May 2002 23:07:28 GMT

Looking back at Sam Ruby's article, Google's Genius, he mentions my favorite point in favor of SOAP/RPC: Perhaps the most illumining part of Paul's essay is when he describes his optimized doSpellingSuggestion API.  In this case, he declares that XML is overkill for the job.  Unquestionably, omitting XML in some cases creates a tighter data stream.  It can also require custom marshallers and parsers to be written.  More tradeoffs to consider. The second to last sentence is important. In my view, XML "scraping" is not much better than HTML "scraping". The Google API's were immediately usable in everything from Ruby to Python to AppleScript to Frontier/Radio, and more, with only a SOAP library needed - nothing specific to Google. The calls were Google specific, yes, but the client immediately understood the results of the call. Even though it's a silly application, I would never have even attempted to write something like HyprSwank, as mentioned in "AppleScript Studio, XML-RPC, and Zope", if I had to parse the results out of my data structure myself. I'm not sure what XML parsing libraries even exist for AppleScript, but I'm sure if they do (or were to) exist, it couldn't be any easier to use than: tell application "" to set link_list to call xmlrpc {method name:"hyprSWNK", parameters:{}} Or in Python: import xmlprclib, pprint link_list = xmlrpclib.ServerProxy('').hyprSWNK() pprint.pprint(link_list) [{'sitename': 'mobileffe', 'link': ''}, {'sitename': 'Prada', 'link': ''}, {'sitename': 'ACFny', 'link': ''}, {'link': 'http://www.peopleusedtodream', 'sitename': 'People Used To Dream About The Future'}, {'sitename': 'David Lynch', 'link': ''}, {'sitename': '.tiln', 'link': ''}, {'link': '', 'sitename': 'educated people are stupid cowards'}, {'link': '', 'sitename': 'absolut antigirl !(absolut cybele)'}, {'sitename': 'no ~ type', 'link': ''}, {'sitename': ' +=', 'link': ''}, {'sitename': '', 'link': ''}, {'sitename': 'infotron', 'link': ''}, {'link': '', 'sitename': 'aaaaaaa n t i o r p;uss'}, {'sitename': 'nato0+55+3d', 'link': ''}, {'link': '', 'sitename': 'themoonstealingproject'}, {'link': '', 'sitename': 'musique:motpol @'}, {'link': '', 'sitename': 'finest cotton armbags since 1997 (HL)'}, {'sitename': 'One Bit Louder', 'link': ''}, {'sitename': 'purple', 'link': ''}, {'sitename': 'OFRPublications', 'link': ''}] While I could have written my own parser, written my own format, or spent time researching common XML formats and then scouring the web for an already written parser for my language or sitting down with a copy of "XML and Python", I chose not to. For as much as I rag sometimes on some of XML-RPC's shortfalls, it's much better than doing all of that! It gives one common format for a common internet use, and has spread around to many languages. If I had to write my parser, or wait for someone to write one, or deal with DOM navigation, in order to deal with GoogleML, I'd be much less curious about "hmm, what can I write t[...]

Fun With Generators

Mon, 06 May 2002 20:50:55 GMT

"The great thing about this arrangement is that I can substitute a different block generating function with zero effort, as sizeAndDigest doesn't need to know how its data is being generated. I could have done that with saved state and a callback to get the next chunk of data, but generators are a lot simpler." [Deadly Bloody Serious]

Python Generators

Mon, 06 May 2002 15:51:39 GMT

There's a post on Lambda the Ultimate today about Generators and Abstraction, with reference to "Python". Ruby has the concept of generators built in from the ground up. Last summer, I came across Ruby and started playing around with it, and annoyed everyone around me (I was working for Zope Corporation who own PythonLabs) by going on and on about Ruby's object model and generators. While I'm sure my little rants had little or nothing to do with the course of Python 2.2, it is nice to see Python picking up on simple generators, iterators, and the whole new classes thing. It has me excited about Python again. An interesting show of the power of generators and iterators comes from PEP 279, The enumerate() built-in function. It's a new built in function that gives to all collections (anything iterable) functionality similar to dict.items() (more specifically, dict.iteritems()) which loops over the keys (index) and values (item) of a dictionary. This example, from the PEP itself, shows a good use of generators and iterators in action, keeping evaluation of a loop lazy while being able to index it:
def enumerate(collection):
    'Generates an indexed series:  (0,coll[0]), (1,coll[1]) ...'     
    i = 0
    it = iter(collection)
    while 1:
        yield (i,
        i += 1

Boolean type for Python 2.3

Mon, 06 May 2002 09:19:10 GMT

I should pay attention to the comp.lang.python mailing list more often, but Usenet newsgroups seem to be a much bigger pain in the ass in these days of Broadband - it's hard to know whether your provider, or your providers provider, will really offer any usenet service. Anyways, PEP 285 brings about True and False to the Python language as natural members for the first time. One of the areas this will help in is the reduction of code like this: return not not foo #the twin 'not not' keywords cast foo into a boolean statement and replace it with: return bool(foo) Which, while brief, avoids the headslam that boolean logic games like 'not not' can be. One place this will be especially beneficial is with XML-RPC and SOAP, and other means of communicating with languages that have (and expect!) booleans. It's been sufficient so far in Python to use any true value or any false value for boolean use, and 1 and 0 for basic usage, but that's hard to communicate over the wire. A Python function may return '1' as an integer value. But it can also be returning '1' to indicate truth (as in predicates like 'isinstance()'). So... Interesting. Never even knew it was up for debate, let alone accepted.

Another Python Web Service package.

Sun, 05 May 2002 11:07:58 GMT

A seemingly forgotten Web Service package for Python. Even its developer seems to have left it alone for a while. It was to be part of Web Service support for "Zope", which has yet to materialize. Anyways, this is an interesting package to look at. It's the most complete of the few SOAP + Friends packages I've looked at for Python, with deep support for WSDL Reading and Writing among other things. It's powerful, flexible, yet still easy to use. And, it mostly works. Whether or not it continues to be developed, it's a good Python package design that's worth looking at.

Verbosity and Web Services, again. Does Dave Winer really get it?

Sat, 04 May 2002 09:07:25 GMT

Forward: a few hours ago, after getting home from dinner with friends, I was reading more on Web Services, and found this article on Wrapping Web Service APIs on ORN by Stephan Figgins. It somehow led me back to an old Scripting News post that somewhat rankled me when I first read it. I'm finally posting my response. At 2AM, saturday morning. Does Dave Winer get Web Services? This posting talks about the overhead involved in publishing a web service via .NET. I admit, my experience with Radio and Frontier is still very small. But, at least with "Radio", you have to write your Web Services script in a special folder, Web Services. How is that not overhead? Now, you're writing a single script adapter to another part of the system in order to publish it on the web services channel. This may make sense in an environment based on pure scripting, but what about real persistent objects that are already on the web - objects that are both Data and Behavior? "Zope", and Bobo before it, have been built on this concept from day one. Many a year ago, Amos Lattier wrote an article called The Bobo Way. I'm going to quote his article here, because (as I've said before), this sounds like familiar ground: For example if your app keeps track of company employees, you will probably want a separate Employee object for each employee. That way you can access these objects with URLs like: /Employees/amos/set_menu?lunch=spam This URL assumes that Employees is a mapping object that contains Employee objects. Employees["amos"] is an Employee object that represents the amos employee. The above URL calls amos' set_menu method with lunch=spam for arguments: Employees["amos"].set_menu(lunch=spam) Ed: this could also be Employees.amos.set_menu(lunch=spam) For contrast lets look at a non-bobo way of doing the same thing: / It seems to me like an awful lot of XML-RPC interfaces are not much better than this last example. They're usually a single function that is handed some sort of key to operate on another object with, rather than going to that object directly. It defeats polymorphism, it defeats generalization, and ultimately it can lead to a large and unwieldy adapter layer, tucked away in a Web Services directory somewhere. There are merits to this design. You are basically writing an Adapter pattern of sorts, mapping a Web Service friendly interface into whatever the internal system really has. And, if that's the only gateway into your system from the outside, you have some level of security by limiting the amount of access to a collection of XML-RPC dedicated functions instead of opening up access to every potential object. But there are merits to the other design too. In theory, you can program Zope the same way on the inside as on the outside, so long as you have the credential to do so. You have instances of classes out there, on the web, already. .NET seems to be following this route somewhat. Doing Web Services by this route is scarier. Why? Security for one reason. And, you just don't want to turn every single method on the object to be callable over the web, or - at the very least - you want to attach conditions. This is what Microsoft is doing. Let's look at how a simple 'return hello' class for Zope looks, disregarding any Zope OFS framework stuff (making the class addable through the web interface). Putting an inst[...]