Red Artisan

Agile Ruby on Rails Specialists

Ruby Cocoa & Core Image at Cocoaheads

posted by crafterm, 11 Apr 2008

Last night saw our first Melbourne Cocoaheads meet up for the year. An awesome night was had by all, there were several talks given by various people including myself about RubyCocoa & Core Image, XCode Organiser, Python, and the iPhone.

I gave a presentation about Ruby Cocoa and some of the Core Image related research I had been working on to integrate it into Attachment Fu for processing file uploads in Rails applications. I’ve uploaded the slides from my talk if anyone is interested in them.

Screen Capture via Ruby Cocoa!

posted by crafterm, 12 Jan 2008

During the week a good friend of mine laid down the challenge to work out how to programatically create a screenshot of your Mac OSX desktop. The following article steps through the process of performing this, adding some charm to the operation to create a full desktop snapshot tool using Ruby Cocoa.

Before stepping into the code, I’ll first present a small extension module that all code snippets rely upon. It’s essentially a small extension to the OSX::CIImage class to simplify loading and saving of image files, and conversions between Core Image and Quartz Image objects (which we’ll need later).

extensions.rb

module OSX
  class CIImage      
    def save(target, format = OSX::NSJPEGFileType, properties = nil)
      bitmapRep = OSX::NSBitmapImageRep.alloc.initWithCIImage(self)
      blob = bitmapRep.representationUsingType_properties(format, properties)
      blob.writeToFile_atomically(target, false)
    end

    def cgimage
      OSX::NSBitmapImageRep.alloc.initWithCIImage(self).CGImage()
    end

    def self.from(filepath)
      raise Errno::ENOENT, "No such file or directory - #{filepath}" unless File.exists?(filepath)
      OSX::CIImage.imageWithContentsOfURL(OSX::NSURL.fileURLWithPath(filepath))
    end
  end
end

Capturing the Desktop

With our extensions code in place, we can use the new CGWindow API, recently added to Mac OSX 10.5, to snapshot our desktop:

capture-plain.rb

require 'osx/cocoa'
require 'extensions'

class Screen
  def self.capture
    screenshot = OSX::CGWindowListCreateImage(OSX::CGRectInfinite, OSX::KCGWindowListOptionOnScreenOnly, OSX::KCGNullWindowID, OSX::KCGWindowImageDefault)
    OSX::CIImage.imageWithCGImage(screenshot)
  end
end

Screen.capture.save('desktop.jpg')

An example screen capture is at the top of this article.

This works really well and is only a few lines of code. It’s quite fast, in fact the Apple documentation mentions ”For capturing pixels, the CGWindow API should demonstrate performance that is equal or better than the techniques used by the OpenGLScreenSnapshot and OpenGLScreenCapture samples”.

The only thing is that the user has no feedback that an actual screenshot was taken, other than the creation of the target image file.

So, lets build upon this and add some feedback.

Capturing the Desktop with a Fade operation

What we’ll do to make it obvious that a screenshot is being taken, is fade the desktop out to a black colour, take the screenshot of the original desktop content, and fade the desktop back to it’s original state.

The effect is similar to what OSX does while changing screen resolutions when you attach an external display or a projector to your Mac.

Implementation wise, we’ll add a fade operation to our Screen class, that will accept a block of code to perform in between fading the display out and back in again. The relevant Cocoa operations are documented in the Quartz Display Services guide.

Essentially we need to invoke CGAcquireDisplayFadeReservation() to obtain a fade reservation token, after which we can invoke CGDisplayFade() to fade the display to a solid colour and back. Once we’re done fading, we can release the fade reservation token with CGReleaseDisplayFadeReservation() (or allow it to time out).

capture-with-fade.rb

require 'osx/cocoa'
require 'extensions'

class Screen

  def self.capture
    fade do
      screenshot = OSX::CGWindowListCreateImage(OSX::CGRectInfinite, OSX::KCGWindowListOptionOnScreenOnly, OSX::KCGNullWindowID, OSX::KCGWindowImageDefault)
      OSX::CIImage.imageWithCGImage(screenshot)
    end
  end

  private

    def self.fade
      err, token = OSX::CGAcquireDisplayFadeReservation(OSX::KCGMaxDisplayReservationInterval)

      if err == OSX::KCGErrorSuccess
        begin
          OSX::CGDisplayFade(token, 0.3, OSX::KCGDisplayBlendNormal, OSX::KCGDisplayBlendSolidColor, 0, 0, 0, true)
          return yield if block_given?
        ensure
          OSX::CGDisplayFade(token, 0.3, OSX::KCGDisplayBlendSolidColor, OSX::KCGDisplayBlendNormal, 0, 0, 0, false)
          OSX::CGReleaseDisplayFadeReservation(token)
        end
      end
    end
end

Screen.capture.save('desktop.jpg')

Capturing the Desktop with a Fade and Snap Graphic

The fade looks really nice, but since other applications can also perform the same effect, lets embed a small graphic in between the fade of a camera to really show that we’re taking a picture of the desktop.

To do this, we need to capture the desktop in between the fade operation, and draw directly onto the display, without creating a window or any other graphical decorations. Capturing and drawing directly onto a display are also discussed in the Quartz documentation online.

Essentially, we need to use CGDisplayCapture(display) to capture a specified display. While it’s captured we have exclusive access to the display, and no other application will interfere with it. Then, we can use CGDisplayGetDrawingContext(display) to obtain a drawing context, and CGContextDrawImage() to draw an image directly to the display. Once we’re done showing our picture, we can then release our capture of the display using CGDisplayRelease(display).

The image we’ll use is above (or grab another one you’d like to use), save is as nikon.jpg alongside the following script.

capture-with-fade-and-graphic.rb

require 'osx/cocoa'
require 'extensions'

class Screen

  def self.capture
    fade do
      screenshot = OSX::CGWindowListCreateImage(OSX::CGRectInfinite, OSX::KCGWindowListOptionOnScreenOnly, OSX::KCGNullWindowID, OSX::KCGWindowImageDefault)
      OSX::CIImage.imageWithCGImage(screenshot)
    end
  end

  private

    def self.fade
      err, token = OSX::CGAcquireDisplayFadeReservation(OSX::KCGMaxDisplayReservationInterval)

      if err == OSX::KCGErrorSuccess
        begin
          OSX::CGDisplayFade(token, 0.3, OSX::KCGDisplayBlendNormal, OSX::KCGDisplayBlendSolidColor, 0, 0, 0, true)

          snap(token)

          return yield if block_given?
        ensure
          OSX::CGDisplayFade(token, 0.3, OSX::KCGDisplayBlendSolidColor, OSX::KCGDisplayBlendNormal, 0, 0, 0, false)
          OSX::CGReleaseDisplayFadeReservation(token)
        end
      end
    end

    def self.snap(token)
      display = OSX::CGMainDisplayID()

      if OSX::CGDisplayCapture(display) == OSX::KCGErrorSuccess
        begin
          ctx = OSX::CGDisplayGetDrawingContext(display)
          if ctx
            pic = OSX::CIImage.from('nikon.jpg')

            OSX::CGDisplayFade(token, 0.0, OSX::KCGDisplayBlendSolidColor, OSX::KCGDisplayBlendNormal, 0, 0, 0, true)

            # calculate middle of the screen for the images location
            display_width, display_height = OSX::CGDisplayPixelsWide(display), OSX::CGDisplayPixelsHigh(display)
            pic_width, pic_height = pic.extent.size.width, pic.extent.size.height
            position_x, position_y = (display_width - pic_width) / 2.0, (display_height - pic_height) / 2.0

            OSX::CGContextDrawImage(ctx, OSX::NSRectFromString("#{position_x} #{position_y} #{pic_width} #{pic_height}"), pic.cgimage)

            sleep(0.7)

            OSX::CGDisplayFade(token, 0.0, OSX::KCGDisplayBlendNormal, OSX::KCGDisplayBlendSolidColor, 0, 0, 0, true)
          end
        ensure
          OSX::CGDisplayRelease(display)
        end
      end
    end
end

Screen.capture.save('desktop.jpg')

Summary

First we created a simple capture class that used the new CGWindow API in Mac OSX 10.5, then we built upon that adding a fade effect around the actual screen capture. Next we drew an image directly on the display in between the fade, to make it even more obvious that we were taking a screenshot.

There we have it, a really useful tool for capturing screenshots of your desktop programatically.

Special Thanks!

  1. Thanks to Lachlan for the challenge mate! :)
  2. Thanks to Pete for your help with the Cocoa desktop fade semantics.
  3. Thanks to DSevilla for the camera image used above.

Cool Effects with Core Image!

posted by crafterm, 31 Dec 2007

In a previous article I described how to use Core Image as the backend image processor for Attachment Fu in your Rails applications. In that particular article we looked at supporting image scaling and thumbnails to be compatible with the other Attachment Fu backends such as RMagick and ImageScience.

With Core Image available however, we have an entire range of post processing filters available to use at our fingertips. In this article we’ll step through a few of these additional filter options that you can use to post process your images with.

Here’s a few examples of what we can do with Core Image post file upload. All of the following examples use the following input image taken in Berlin while at RailsConf EU (also used in the performance measurements article):

Greyscale or Sepia

A scaled version of the source image is cool, but how about an automatic greyscale or sepia version of the image:

Greyscale

Sepia

Code Fragment

module RedArtisan
  module CoreImage
    module Filters
      module Color

        def greyscale(color = nil, intensity = 1.00)
          create_core_image_context(@original.extent.size.width, @original.extent.size.height)

          color = OSX::CIColor.colorWithString("1.0 1.0 1.0 1.0") unless color

          @original.color_monochrome :inputColor => color, :inputIntensity => intensity do |greyscale|
            @target = greyscale
          end
        end

        def sepia(intensity = 1.00)
          create_core_image_context(@original.extent.size.width, @original.extent.size.height)

          @original.sepia_tone :inputIntensity => intensity do |sepia|
            @target = sepia
          end
        end

      end
    end
  end
end

Exposure and Noise Control

Another option for us is to automatically adjust exposure and noise parameters upon upload to brighten images up, or remove unwanted noise from lower quality images:

1 F-Stop

2 F-Stops

Noise Removal

Code Fragment

module RedArtisan
  module CoreImage
    module Filters
      module Quality

        def reduce_noise(level = 0.02, sharpness = 0.4)
          create_core_image_context(@original.extent.size.width, @original.extent.size.height)

          @original.noise_reduction :inputNoiseLevel => level, :inputSharpness => sharpness do |noise_reduced|
            @target = noise_reduced
          end
        end

        def adjust_exposure(input_ev = 0.5)
          create_core_image_context(@original.extent.size.width, @original.extent.size.height)

          @original.exposure_adjust :inputEV => input_ev do |adjusted|
            @target = adjusted
          end          
        end

      end
    end
  end
end

Watermarking

Sometimes we’d like to automatically add a watermark to our images, either with a single watermark image, or as a tiled watermark image:

Single Watermark

Tiled Watermark

Code Fragment

module RedArtisan
  module CoreImage
    module Filters
      module Watermark

        def watermark(watermark_image, tile = false, strength = 0.1)
          create_core_image_context(@original.extent.size.width, @original.extent.size.height)

          if watermark_image.respond_to? :to_str
            watermark_image = OSX::CIImage.from(watermark_image.to_str)
          end

          if tile
            tile_transform = OSX::NSAffineTransform.transform
            tile_transform.scaleXBy_yBy 1.0, 1.0

            watermark_image.affine_tile :inputTransform => tile_transform do |tiled|
              tiled.crop :inputRectangle => vector(0, 0, @original.extent.size.width, @original.extent.size.height) do |tiled_watermark|
                watermark_image = tiled_watermark
              end
            end
          end

          @original.dissolve_transition :inputTargetImage => watermark_image, :inputTime => strength do |watermarked|
            @target = watermarked
          end
        end

      end
    end
  end
end

Funky Effects

We can also use cool and funky effects used in applications like Photobooth, here’s an example using the edge colouring algorithm:

Edges

Core Fragment

module RedArtisan
  module CoreImage
    module Filters
      module Effects

        def edges(intensity = 1.00)
          create_core_image_context(@original.extent.size.width, @original.extent.size.height)

          @original.edges :inputIntensity => intensity do |edged|
            @target = edged
          end
        end

      end
    end
  end
end

The sign artwork works particularly well with this algorithm.

Core Image Processor

All of the code above is also available as a usable image processor via git.

Some examples of using the processor:

require 'red_artisan/core_image/processor'

# generate some test output images for various effects

processor = RedArtisan::CoreImage::Processor.new('berlin.jpg')

grey = processor.greyscale
grey.save 'results/berlin-grey.jpg'

sepia = processor.sepia
sepia.save 'results/berlin-sepia.jpg'

watermarked = processor.watermark('watermark_image.png')
watermarked.save 'results/berlin-watermarked.jpg'

watermarked = processor.watermark('watermark_image.png', true)
watermarked.save 'results/berlin-watermarked-tiled.jpg'

noise_reduced = processor.reduce_noise
noise_reduced.save 'results/berlin-noise-reduced.jpg'

exposure_adjusted = processor.adjust_exposure
exposure_adjusted.save 'results/berlin-exposure-adjusted-half-stop.jpg'

exposure_adjusted = processor.adjust_exposure(2.0)
exposure_adjusted.save 'results/berlin-exposure-adjusted-two-stops.jpg'

edge = processor.edges
edge.save 'results/berlin-edge.jpg'

Summary

The above shows us only a fraction of what can be done with the 100+ filters Core Image provides by default. There’s many other filters that let you create all sorts of effects with single and multiple images combined. Enjoy!

Core Image Performance

posted by crafterm, 16 Dec 2007

Following my previous post, a few people have asked me how well Core Image performs in comparison to RMagick and ImageScience when used as part of Attachment Fu. To answer these questions, I’ve spent a bit of time collecting some performance results.

Please note that as with all performance testing, results will vary according to many parameters such as hardware, processing power, and even the input image content itself, as you’ll see below.

Naturally Core Image should be faster since it will run hardware accelerated on my MacBook Pro, so please take these results more as an indication of library characteristics rather than final timings. The tests I performed also mirrored how each library was being invoked by Attachment Fu, and beyond performance there’s also several other tangible benefits to Core Image such as deployment and access via RubyCocoa to keep in mind as well.

The first test performs 5 successive resizes of a 1mb image under each library (from 1250x833 to 640x480). The input image is a complex photo (smaller version above) taken in Berlin of parts of the Berlin Wall, including a high colour range and lots of shapes and sizes:

Both RMagick and ImageScience yield consistent results as expected. Core Image’s resize values are interesting as successive resizes are twice as fast than the first, which I speculate would include an initial Core Image to OpenGL compilation phase of the resize operation being performed.

Similar characteristics can be observed with a slightly larger image size of around 2.5mb (2200x1467 to 640x480):

In both cases, Core Image performs really well, particularly on consecutive runs.

The actual image content being resized also affects results as well - the following test uses a blank image of identical dimensions to the 1mb Berlin image (1250x833 to 640x480):

Finally, the following graph shows the results of resizing 6 versions of the Berlin image with each library, at increasing size & dimensions to half their original size. The results plotted for each measurement are calculated over the average of 5 resizes of each image.

At small image sizes, we can see the difference in resize time to be essentially negligible, however as image size increase up to 500kb, the hardware acceleration of Core Image starts to shine, particularly around the 2.5mb - 5mb range and beyond where it screams along.

There does seem to be a limit to performance gains, on Friday Ben and I also made a few experiments with really large image sizes (TIFF files around 90mb, resizing from 5436x4080 to 640x480), and noticed that Core Image can be slower than Image Science under these conditions:

At this stage I can only speculate as to why this occurs, and it deserves further inspection.

Another interesting aspect to look at is the actual on disk size of the resized images. The following is a directory listing of the results after the 2.5mb image (2200x1467 resize to 640x480):

crafterm  staff  153492 16 Dec 22:30 core_image-complex_resized.jpg
crafterm  staff  288455 16 Dec 22:30 image_science-complex_resized.jpg
crafterm  staff  356827 16 Dec 22:30 rmagick-complex_resized.jpg

Core Image’s default compression range is quite impressive, and while I’m sure the other libraries compression factors can be fine tuned I suspect it would also affect their relative performance measurements.

All measurements shown in the graphs are in seconds, and the test machine was my MacBook Pro, 15” Intel Core 2 Duo, 2.16 Ghz with an ATI Radeon X1600 video card with 128mb vram. I’d certainly be keen to see how other types of hardware perform, so please feel free to run the test code on your system and send me the resultant graphs, and I’ll add them to the project.

All test code and input images (except for the 90mb example due to copyright) are available in a git repository online.

Attachment_fu magic with Core Image and Ruby Cocoa!

posted by crafterm, 13 Dec 2007

Like many of us in the Rails world, I use attachment_fu to handle file uploads in my Rails applications. Attachment_fu does a great job, in particular with it’s ability to scale and generate multiple sized copies of images with various processing back ends including ImageScience, RMagick and MiniMagic, not to mention it’s flexible storage options, all from the comfort of a DSL:

has_attachment :content_type => :image, 
               :resize_to    => '640x400',
               :storage      => :file_system, 
               :size         => 1.kilobyte .. 3.megabytes,
               :thumbnails   => { :small => '50x50', :medium => '320x200' }

Working on under Mac OS X though, it’s often been a challenge to get these underlying libraries installed. General community consensus is that RMagick leaks memory, and ImageScience is built upon FreeImage and RubyInline which requires a development environment to compile, install and run.

As developers MacPorts lets us all build and install these libraries comfortably, however the biggest beef I’ve had is that under the hood of every recent Mac OS X installation, there’s actually a great image processing library already available to us - Core Image.

Core Image

Core Image has been a part of Mac OS X since Tiger and is part of the QuartzCore framework, offering a flexible filter/pipeline based approach to manipulating images via transforms. Once of the most exciting things about Core Image though, is that it processes images using a subset of the OpenGL Shading Language, and when available will use the GPU to render, all in accelerated hardware near or in real time! In environments where the available GPU is not supported, Core Image automatically falls back to the CPU for processing seamlessly giving us the best of both worlds.

Until recently, access to Core Image has only been available via languages such as Objective-C. However with the release of Leopard, Ruby has become an officially supported language within XCode, and in particular RubyCocoa is available by default under every Leopard Mac OS X installation (in Tiger it can be installed separately).

So now it’s even easier for us to take full advantage of the underlying power of these Cocoa API’s, directly from Ruby itself.

Let’s get started!

In this article, I’ll describe how to add support for using Core Image as the image processing library within attachment_fu.

Once we’re finished, attachment_fu will handle all your uploads using Core Image for resizing and thumbnail generation (most likely hardware accelerated inside your Mac’s GPU), if you have Leopard it won’t require any 3rd party library to be installed to work, and it will handle any Mac OS X supported image format, which as of 10.5, includes RAW.

It’s even more particularly enticing if you deploy your Rails application to an XServe which includes a Core Image supported GPU.

To do this, we’ll need to perform the following steps:

  • Create an image manipulation class that uses Core Image
  • Integrate this new class into attachment_fu, by writing a new attachment_fu processor module
  • Optionally, update attachment_fu’s automatic image processing list, or rely on using the :processor directive in our has_attachment model definitions.

Create an image manipulation class that uses Core Image

Our first step is to create a new class that uses Core Image to resize and/or thumbnail a given image. To do this, we’ll need to define an API to accept the source image, some new dimensions and specify where the resized version should be rendered to.

Inside of the class, we’ll use Core Image Filters to perform the actual work.

Core Image Filters allow you to create a ‘pipeline’ of transforms that’s performed on a given image. Mac OS X includes many filters by default, allowing you to perform all sorts of effects on your image, but the one we’re interested in at the moment is the Lanczos Scale Transform filter.

The Lanczos Scale Transform produces a high quality, scaled version of the source image using a well defined algorithm.

In addition to this, we’ll pre-process the image with an Affine Clamp filter which will make the image infinitely big by clamping the image’s edges outwards. We do this so that there’s no edge imperfections introduced due to rounding/ceiling of dimensions when scaling. After the Lanczos scaling has done it’s trick, we then crop the image to our target dimensions for rendering.

Here’s an example of how we’ll use the classes API:

p = Processor.new OSX::CIImage.from(path_to_image)
p.resize(640, 480)
p.render do |result|
  result.save('resized.jpg', OSX::NSJPEGFileType)
end

Processor.resize(width, height) will perform a hard resize to the given dimensions, where as Processor.fit(size) will resize the image to a scale that fits its aspect ratio. These two methods are provided as attachment_fu includes some extra geometry processing code that lets us use RMagick style geometry stings to specify dimensions as fixed values, percentages, scales, or relative aspect ratio sizes that we’ll leverage off.

Here’s the actual classes implementation

vendor/core_image/processor.rb

require 'rubygems'
require 'osx/cocoa'
require 'active_support'

# Copyright (c) Marcus Crafter <crafterm@redartisan.com>
#
class Processor

  def initialize(original)
    @original = original
  end

  def resize(width, height)
    create_core_image_context(width, height)

    scale_x, scale_y = scale(width, height)

    @original.affine_clamp :inputTransform => OSX::NSAffineTransform.transform do |clamped|
      clamped.lanczos_scale_transform :inputScale => scale_x > scale_y ? scale_x : scale_y, :inputAspectRatio => scale_x / scale_y do |scaled|
        scaled.crop :inputRectangle => vector(0, 0, width, height) do |cropped|
          @target = cropped
        end
      end
    end
  end

  def fit(size)
    original_size = @original.extent.size
    scale = size.to_f / (original_size.width > original_size.height ? original_size.width : original_size.height)
    resize (original_size.width * scale).to_i, (original_size.height * scale).to_i
  end

  def render(&block)
    raise "unprocessed image: #{@original}" unless @target
    block.call @target
  end

  private

    def create_core_image_context(width, height)
        output = OSX::NSBitmapImageRep.alloc.initWithBitmapDataPlanes_pixelsWide_pixelsHigh_bitsPerSample_samplesPerPixel_hasAlpha_isPlanar_colorSpaceName_bytesPerRow_bitsPerPixel(nil, width, height, 8, 4, true, false, OSX::NSDeviceRGBColorSpace, 0, 0)
        context = OSX::NSGraphicsContext.graphicsContextWithBitmapImageRep(output)
        OSX::NSGraphicsContext.setCurrentContext(context)
        @ci_context = context.CIContext
    end

    def vector(x, y, w, h)
      OSX::CIVector.vectorWithX_Y_Z_W(x, y, w, h)
    end

    def scale(width, height)
      original_size = @original.extent.size
      return width.to_f / original_size.width.to_f, height.to_f / original_size.height.to_f
    end

end

module OSX
  class CIImage
    include OCObjWrapper

    def method_missing_with_filter_processing(sym, *args, &block)
      f = OSX::CIFilter.filterWithName("CI#{sym.to_s.camelize}")
      return method_missing_without_filter_processing(sym, *args, &block) unless f

      f.setDefaults if f.respond_to? :setDefaults
      f.setValue_forKey(self, 'inputImage')
      options = args.last.is_a?(Hash) ? args.last : {}
      options.each { |k, v| f.setValue_forKey(v, k.to_s) }

      block.call f.valueForKey('outputImage')
    end

    alias_method_chain :method_missing, :filter_processing

    def save(target, format, properties = nil)
      bitmapRep = OSX::NSBitmapImageRep.alloc.initWithCIImage(self)
      blob = bitmapRep.representationUsingType_properties(format, properties)
      blob.writeToFile_atomically(target, false)
    end

    def self.from(filepath)
      OSX::CIImage.imageWithContentsOfURL(OSX::NSURL.fileURLWithPath(filepath))
    end
  end
end

The processor class uses a few Ruby idioms, in particular with the filter processing and rendering code. The filtering code leverages blocks and method_missing to provide a declarative approach to defining filters and their parameters:

@original.affine_clamp :inputTransform => OSX::NSAffineTransform.transform do |clamped|
  clamped.lanczos_scale_transform :inputScale => inputScale, :inputAspectRatio => ratio do |scaled|
    scaled.crop :inputRectangle => vector do |cropped|
      @target = cropped
    end
  end
end

(note that the methods affine_clamp, lanczos_scale_transform and crop don’t actually exist on OSX::CIImage, they’re applied dynamically)

A few helper methods have also been added to OSX::CoreImage to ease construction and serialization. Rendered output file types can be NSBMPFileType, NSGIFFileType, NSJPEGFileType, NSPNGFileType, or NSTIFFFileType. See the XCode API documentation for an NSBitmapImageRep class for more information about the various properties for each type.

Integrating our core image processor into attachment_fu

Now that we have a class that can resize images using Core Image, we just need to integrate it with attachment_fu.

Attachment_fu’s design is quite modular, allowing us to add a new image processor via writing a module that’s mixed in at runtime. Essentially we need to define a module within the Technoweenie::AttachmentFu::Processors namespace, that will extend the functionality of the ‘process_attachment’ method.

The common approach is to do this via alias_method_chain, which decorates the existing process_attachment method with new functionality.

Here’s the new module:

technoweenie/attachment_fu/processors/core_image_processor.rb

require 'core_image/processor'

module Technoweenie # :nodoc:
  module AttachmentFu # :nodoc:
    module Processors
      module CoreImageProcessor
        def self.included(base)
          base.send :extend, ClassMethods
          base.alias_method_chain :process_attachment, :processing
        end

        module ClassMethods
          def with_image(file, &block)
            block.call OSX::CIImage.from(file)
          end
        end

        protected
          def process_attachment_with_processing
            return unless process_attachment_without_processing
            with_image do |img|
              self.width  = img.extent.size.width  if respond_to?(:width)
              self.height = img.extent.size.height if respond_to?(:height)
              resize_image_or_thumbnail! img
              callback_with_args :after_resize, img
            end if image?
          end

          # Performs the actual resizing operation for a thumbnail
          def resize_image(img, size)
            processor = Processor.new(img)
            size = size.first if size.is_a?(Array) && size.length == 1
            if size.is_a?(Fixnum) || (size.is_a?(Array) && size.first.is_a?(Fixnum))
              if size.is_a?(Fixnum)
                processor.fit(size)
              else
                processor.resize(size[0], size[1])
              end
            else
              new_size = [img.extent.size.width, img.extent.size.height] / size.to_s
              processor.resize(new_size[0], new_size[1])
            end

            processor.render do |result|
              self.width  = result.extent.size.width  if respond_to?(:width)
              self.height = result.extent.size.height if respond_to?(:height)
              result.save self.temp_path, OSX::NSJPEGFileType
              self.size = File.size(self.temp_path)
            end
          end          
      end
    end
  end
end

The module brings the core image based processing class we’ve written above into scope, and then proceeds to define a CoreImageProcessor module within the Technoweenie::AttachmentFu::Processors namespace. When this module is included, it then aliases process_attachment to add some new functionality, essentially the process_attachment_with_processing and resize_image methods.

Most of the code in this module surrounds calling the processor class’ API with the correct resize values based on the range of possible geometry values attachment_fu allows. The ImageScience and RMagick processors look quite similar.

Update attachment_fu’s automatic image processing list

Attachment_fu includes several processors which are tried in order at startup to work out which image processing engine to use, based on what underlying libraries, etc, are available on your machine.

This step is optional because we can either update this list so that our Core Image processor is part of this selection, or we can specify the processor directly when defining a has_attachment on a model:

has_attachment :content_type => :image, 
               :processor    => :core_image,
               :resize_to    => '640x400',
               :storage      => :file_system, 
               :size         => 1.kilobyte .. 3.megabytes,
               :thumbnails   => { :small => '50x50', :medium => '320x200' }

To update the default list, open up the attachment_fu.rb source file, and update the @@default_processors class variable from:

module Technoweenie
  module AttachmentFu
    @@default_processors = %w(ImageScience Rmagick MiniMagick)

to:

module Technoweenie
  module AttachmentFu
    @@default_processors = %w(CoreImage ImageScience Rmagick MiniMagick)

which will register the Core Image processor module for inclusion when attachment_fu searches for image processors.

Summary

We’ve created a class that uses Core Image to resize an image to given dimensions, using a high quality Lanczos Scale transform, via RubyCocoa. We’ve integrated it into attachment_fu by creating a new processor module that can be specified in the default image processing search list, or directly in a has_attachment definition.

I want it!

To make things easy, I’ve created a git repository that includes all of the above files so you can keep up to date, and the project format is in a structure you can export directly into your attachment_fu installation. To access the source, use the following command:

$> git clone git://git.redartisan.com/af_ci.git

This will check out the attachment_fu core image project that you can copy across into your attachment_fu installation (or perform a nice git export of the source directly).

Once you’ve installed the source files in your attachment_fu plugin, apply the supplied patch to update attachment_fu’s list of supported image processors. You’ll need to restart your Rails application to reload the plugin, after which you’ll be using Core Image to process attachments!

Future

I have several further enhancements and ideas surrounding the core image processor which I’ll talk about in further articles - if you have any improvements to the code, patches are also more than welcome. Enjoy!

[Update]

To facilitate updates to the code, I’ve imported the source into a git repository rather than distribute via the tar/gz the original post referenced.

[Update II]

Now that Technoweenie has imported his sources into github, I’ve created a child attachment_fu repository that includes all of the above updates to use Core Image with Attachment Fu.