[Michael T. Babcock] Remote image loader

The story

I was playing around with my links page which is populated dynamically by del.icio.us and came across their example to show favicon pictures alongside each list entry which looked quite nice I thought.

What I noticed as I was playing with and reloading the page was the number of slow outbound image requests my browser was making, fetching the (sometimes non-existent) images from each of the remote servers listed. I figured it should be easy enough to reference a dynamic image in Zope that downloads the remote images dynamically and keeps a cached copy for next time.

After a bit of research, I wrote up the following fairly simplistic Zope Extension (save in your Zope Extensions folder under your active Zope instance, not the original source directory) and the Python script that calls it as well:


# Copyright (C) 2006, Michael T. Babcock <zopista@mikebabcock.ca>
# Feel free to reuse at will; BSD or Zope licensing applies at your discretion.
# Consider giving me credit somewhere if you appreciated this.

import urllib
from StringIO import StringIO

def imagefetch(self, domain):
        """Return the favicon.ico file stored at http://domain/favicon.ico
        converted to PNG format and resized to 16x16 just in case"""

        image_file = StringIO()
        image_data = ""

                remote_image = urllib.urlopen("http://%s/favicon.ico" % domain)
                image_data = remote_image.read()
                return None

        if len(image_data) == 0:
                return False

                # write image_file
                return None

        # Convert to PNG
        import PIL
        png_image = PIL.Image.open(image_file)
        png_image = png_image.convert('RGB')
        png_image = png_image.resize((16,16),PIL.Image.ANTIALIAS)
        png_image_file = StringIO()
        png_image.save(png_image_file, "PNG")

        # Create image in cache
        image_id = "%s.png" % domain

                favicons = self['favicons']

                if favicons.hasObject(image_id):

                        png_image_file, 'favicon for %s' % domain)
                return None

        #image_info = getattr(self, image_id)

        return image_id

Once the above script is in your Extensions folder, add an extension to your images directory on your Zope server (or wherever you like) from within the browser interface specifying the function name (imagefetch) and the extension name (Zdynimage). You'll now be able to use the following Zope Script to dynamically load and cache remote images:


request = container.REQUEST

if not request.traverse_subpath[-1] == "favicon.png":
    return "<P>Usage: <code>/thisscript/<em>sitename</em>/favicon.png</code></P>"

# Take the final path element, use as domain name
    domain = str(request.traverse_subpath[-2])
    return "<P>Error: specify path after filename</P>"

url = "http://%s/favicon.ico" % domain
data = ""

image_id = "%s.png" % domain

# if not exists cached image, load it into cache from site
if not context.favicons.hasObject(image_id):
    ret = context.imagefetch(domain)

# return data of cached image
    if 0:
        return "<P>image %s in favicons</P>" % image_id
    data = context.favicons[image_id].data # line 25
except KeyError, e:
    data = context['delicious_medium_png'].data


return data


I'll bother typing this in shortly. For now I hope the code above helps you out.


I scale all the images to the size I want to display them in so as not to save more image data than necessary and also convert them to PNG format in the process. This may not be appropriate for other uses. Removing this functionality would be simple enough; its only a few lines of PIL code (and the Python Imaging Library is wonderful to work with).

Please note there's no domain matching. I'm writing a quick check that will make sure the script is called only from your own pages, to avoid remote abuse. Ironically this could lead to massive abuse of your server if you don't implement such measures (ironic since the purpose is to partly to avoid abusing remotely referenced servers).

Also realize there is no cache expiry supported or negative caching. The images downloaded stay on your server in the configured directory until you manually delete them and sites without said images will be probed for them on every reference (although this is already normal for normal page loads of those sites).

To do

As stated above, I need to write a little one or two-liner that makes sure the site referrer is within my domain(s). I'd also like to make its functionality a little more generic for the sake of other users. I'm not sure if its worth it for my favicons at a few kilobytes each, but dynamically expiring the images and/or actually doing HTTP ETag/Last-Modified checks like a real caching provider would be interesting (of course, this would already happen if I checked them every time since my webserver is behind a caching proxy for outbound requests). Deleting old, unreferenced images would obviously make sense and setting a configurable cache membership property within Zope would be good to avoid refresh delays. Any volunteers?

Stumble it! XFN Friendly Powered by DJBDNS Powered by Zope Valid CSS! Website Security Test

Served by:  Zope 2.7.6

Page Copyright © 2014, Michael T. Babcock. All Rights Reserved.

To contact me, send an E-mail to sawyoursite at this domain.

If you'd really like your mail server reported for spam, send me some junk mail to junk-yum@mikebabcock.me or devnull@mikebabcock.me. This site powered by djbdns