Getting Started with Rubinius II: Coding!

posted by crafterm, 11 October 2007

In a previous article we examined the process of checking out Rubinius, building it from source and discussed its directory structure. In this article, we’ll take it one step further and examine the process of implementing an example method that can be contributed back to the project as a patch for inclusion in the official Rubinius source base.

If you haven’t checked out or built Rubinius please see my previous post which details the preliminary steps required before we can start implementing.

The feature we’ll implement is the File.link method, the implementation is quite simple and only a few lines of code but it will take us through the process of adding a method to an existing class with an existing spec, and will also take us into the system call layer where we’ll interact with the underlying operating system to perform a symlink.

In this case it’s not required however generally it’s a good idea to run the rake dev:setup rake task before implementation to ensure that we have pristine copies of our runtime archives available. We do this because the compiler itself requires that the runtime archives work, and if we introduce a defect it’s possible to enter the situation where we cannot compile a fix.

dev:setup essentially makes a backup of the runtime archives that will always be used for compilation. In our particular case the compiler doesn’t create any symlinks so this step is optional but it’s a good idea if you’re working on existing code or low level methods such as File.stat, Hash, Array, etc to do so.

Normally when using git we would create a feature branch, implement our specs and changes on that branch, commit it locally and then rebase the source code off the master branch before pushing it to the main repository (this is how Rubinius committers integrate their work into the main line development). In this article we’ll omit these stages as they’re well documented on the Rubinius project pages, and here we want to focus on the changes to be made to Rubinius itself.

Specification

Back to our new feature – a spec already exists for File.link and it’s in the spec/core/file/link_spec.rb file:

require File.dirname(__FILE__) + '/../../spec_helper'

describe "File.link" do
  before do 
    @file = "test.txt"
    @link = "test.lnk"     
    File.delete(@link) if File.exists?(@link)
    File.delete(@file) if File.exists?(@file)
    File.open(@file, "w+")
  end

  platform :not, :mswin do
    it "link a file with another" do
      File.link(@file, @link).should == 0
      File.exists?(@link).should == true
      File.identical?(@file, @link).should == true
    end

    it "raise an exception if the target already exists" do
      File.link(@file, @link)
      should_raise(Errno::EEXIST) { File.link(@file, @link) }
    end

    it "raise an exception if the arguments are of the wrong type or are of the incorrect number" do
      should_raise(ArgumentError) { File.link }
      should_raise(ArgumentError) { File.link(@file) }
    end
  end

  after do
    File.delete(@link)
    File.delete(@file)
  end
end

The core specification suite is laid out in the spec/core directory using the convention of a having a spec file per method on each class containing all behaviour for that corresponding method. Platform and bootstrap specs are in the spec/platform and spec/bootstrap directories respectively.

Examining the specification above, there’s three tests that are run on all non-mswin platforms (ie. those supporting the creation of symlinks). The tests ensure that when called, File.link creates a symlink between the source and target, or raises an exception either if the target already exists or if it’s given incorrect arguments.

This identifies what we need to implement.

Let’s run the spec to see what’s failing:

$> bin/mspec -f s spec/core/file/link_spec.rb

File.link
- link a file with another  (ERROR - 1)
- raise an exception if the target already exists (ERROR - 2)
- raise an exception if the arguments are of the wrong type or are of the incorrect number (ERROR - 3)


1)
File.link link a file with another  FAILED
No method 'link' on an instance of Class.: 
    Object(Class)#link (method_missing) at kernel/core/object.rb:98
                        main.__script__ at spec/core/file/link_spec.rb:14
                              Proc#call at kernel/core/context.rb:262
                          SpecRunner#it at spec/mini_rspec.rb:337
                                main.it at spec/mini_rspec.rb:369
                        main.__script__ at spec/core/file/link_spec.rb:24
                          main.platform at ./spec/core/file/../../spec_helper.rb:96
                        main.__script__ at spec/core/file/link_spec.rb:30
                              Proc#call at kernel/core/context.rb:262
                    SpecRunner#describe at spec/mini_rspec.rb:347
                          main.describe at spec/mini_rspec.rb:365
                        main.__script__ at spec/core/file/link_spec.rb:3
                              main.load at kernel/core/compile.rb:78
                   main.__eval_script__ at (eval):8
                             Array#each at kernel/core/array.rb:526
                  Integer(Fixnum)#times at kernel/core/integer.rb:19
                             Array#each at kernel/core/array.rb:526
                   main.__eval_script__ at (eval):5
                CompiledMethod#activate at kernel/core/compiled_method.rb:110
                        Compile.execute at kernel/core/compile.rb:34
                        main.__script__ at kernel/loader.rb:170
..snip..
$> 

From the stacktraces we can see:

No method 'link' on an instance of Class.

indicates that File.link doesn’t even exist inside the current implementation of File.

Design

The corresponding source file to implement File.link is in kernel/core/file.rb:

# depends on: io.rb

class File < IO
  ..snip..

  def self.new(path, mode)
    return open_with_mode(path, mode)
  end

  def self.open(path, mode="r")
    raise Errno::ENOENT if mode == "r" and not exists?(path)

    f = open_with_mode(path, mode)
    return f unless block_given?

    begin
      yield f
    ensure
      f.close unless f.closed?
    end
  end

  def self.exist?(path)
    out = Stat.stat(path, true)
    if out.kind_of? Stat
      return true
    else
      return false
    end
  end

  def self.file?(path)
    st = Stat.stat(path, true)
    return false unless st.kind_of? Stat
    st.kind == :file
  end

  ..snip..
end

Here we see methods implementing various parts of the File API. The above methods show the implementation of File.new, File.open, File.exist? and File.file? (to compare MRI’s implementation of the above methods check the file.c source file in the Ruby tar.gz source archive).

Lets look a first implementation of File.link. The primary behaviour of File.link is to create a hard link between two filenames. To do this we need to invoke the link(2) system call on the underlying operating system to create the link.

A quick examination of the link(2) man page yields:

$> man 2 link

LINK(2)             BSD System Calls Manual            LINK(2)

NAME
     link -- make a hard file link

SYNOPSIS
     #include <unistd.h>

     int
     link(const char *name1, const char *name2);

DESCRIPTION
     The link() function call atomically creates the specified directory entry
     (hard link) name2 with the attributes of the underlying object pointed at
     by name1 If the link is successful: the link count of the underlying
     object is incremented; name1 and name2 share equal access and rights to
     the underlying object.

     ..snip..

RETURN VALUES
     Upon successful completion, a value of 0 is returned.  Otherwise, a value
     of -1 is returned and errno is set to indicate the error.

     ..snip..

STANDARDS
     The link() function is expected to conform to IEEE Std 1003.1-1988
     (``POSIX.1'').

According to the man page, link(2) accepts the source and target of the symlink as paramaters, and returns an integer indicating success or failure.

FFI

To invoke link(2) we need to add a new method to the ffi layer inside of Rubinius. ffi stands for ‘foreign function interface’, and it’s a really neat way of being able to interact with system calls on the underlying operating system without needing to write a lot of stub or native integration code.

ffi bindings are compiled into the platform.rba archive, and since link(2) conforms to a POSIX standard the file we need to modify is kernel/platform/posix.rb.

Opening kernel/platform/posix.rb we’ll see blocks of code such as the following inside the Platform::POSIX module:

# file system
attach_function nil, 'access', [:string, :int], :int
attach_function nil, 'chmod',  [:string, :int], :int
attach_function nil, 'fchmod', [:int, :int], :int
attach_function nil, 'unlink', [:string], :int
attach_function nil, 'getcwd', [:string, :int], :string
attach_function nil, 'umask', [:int], :int

This code dynamically attaches methods to the module, and specifies the parameter types and return values of each method.

The general format of the ‘attach_function’ method is as follows:

__attach_function___ library, __method name__, [ parameters ], return value
  • library, library name to load dynamically, nil otherwise
  • name, name of the method to attach, this is also the name the method will be available as inside the module
  • parameters, array of symbols identifying the types this method accepts as parameters
  • return value, type of the return value

(attach_function can also accept several other formats of parameters, please take a closer look at kernel/platform/ffi.rb for more details)

Symbols are defined for most primitive types, ie: :short, :int, :long, :string, :char, etc, which can be used in the parameter list and return value specifier.

Following the examples above, link(2) can be attached to the Platform::POSIX module with one line of code:

attach_function nil, 'link', [:string, :string], :int

After adding this line of code to the Platform::POSIX module, we need to update the platform.rba archive to ensure it now includes knowledge of link(2) system call.

$> rake build:platform

Implementation

Now that we have access to the link(2) system call, we can invoke it via ffi from the file module.

Open up kernel/core/file.rb, and in between two existing methods, enter the following code:

def self.link(from, to)
  Platform::POSIX.link(from, to)
end

As with the platform archive, we’ll need to update the core archive:

$> rake build:core

Let’s re-run our specifications to see if it passes:

$> bin/mspec -f s spec/core/file/link_spec.rb

File.link
- link a file with another 
- raise an exception if the target already exists (ERROR - 1)
- raise an exception if the arguments are of the wrong type or are of the incorrect number


1)
File.link raise an exception if the target already exists FAILED
Expected EEXIST, nothing raised: 
          main.should_raise at ./spec/core/file/../../mspec_helper.rb:27
            main.__script__ at spec/core/file/link_spec.rb:21
                  Proc#call at kernel/core/context.rb:262
              SpecRunner#it at spec/mini_rspec.rb:337
                    main.it at spec/mini_rspec.rb:369
            main.__script__ at spec/core/file/link_spec.rb:24
              main.platform at ./spec/core/file/../../spec_helper.rb:96
            main.__script__ at spec/core/file/link_spec.rb:30
                  Proc#call at kernel/core/context.rb:262
        SpecRunner#describe at spec/mini_rspec.rb:347
              main.describe at spec/mini_rspec.rb:365
            main.__script__ at spec/core/file/link_spec.rb:3
                  main.load at kernel/core/compile.rb:78
       main.__eval_script__ at (eval):8
                 Array#each at kernel/core/array.rb:526
      Integer(Fixnum)#times at kernel/core/integer.rb:19
                 Array#each at kernel/core/array.rb:526
       main.__eval_script__ at (eval):5
    CompiledMethod#activate at kernel/core/compiled_method.rb:110
            Compile.execute at kernel/core/compile.rb:34
            main.__script__ at kernel/loader.rb:170

3 examples, 1 failures
$>

We’re in better shape, two spec’s are now passing, including the link test – we’re successfully creating a hard link between 2 filenames, but one spec is still failing in the area of handling error conditions, in particular when the target filename already exists.

Lets update our File.link implementation appropriately:

def self.link(from, to)
  raise Errno::EEXIST if exists?(to)
  Platform::POSIX.link(from, to)
end

and naturally, rebuild the core:

$> rake build:core

and re-run our specification:

$> bin/mspec -f s spec/core/file/link_spec.rb

File.link
- link a file with another 
- raise an exception if the target already exists
- raise an exception if the arguments are of the wrong type or are of the incorrect number


3 examples, 0 failures

hooray, all link specifications passed.

If the specs for File.link are complete (ie. document all areas of File.link’s behaviour), we are ready to submit a patch back to the Rubinius community. Alternatively, if some behaviour is lacking from the specs, we could now iterate through the above process adding a spec to document additional behaviour, and implement it following TDD/BDD practices until all expected behaviour has been added.

Patch

To create a patch we can use git and issue the command:

$> git diff > file_link.diff

This will create a patch for us containing the changes we made across the entire Rubinius project. We can then send this back to the community for inclusion into the official Rubinius repository, by submitting it in a Rubinius Lighthouse ticket.

Summary

We’ve stepped through the process of implementing a feature in Rubinius by examining the behaviour of a particular method via it’s corresponding specification tests. As part of the implementation we’ve added a binding to an underlying operating system call via the ffi layer in Rubinius, and then called upon that binding in the class where the functionality is expected.

We then ensured that all required behaviour including error conditions have been met by making sure the spec test suite passes. Finally we’ve created a patch using git that we can submit back to the Rubinius project via lighthouse.

Implementing a feature in Rubinius can certainly be as straightforward and as easy as what we’ve seen above. There’s many specifications that have been written that don’t have corresponding implementations, so pick a class, check it’s specs, write an implementation and join in on building a fantastic, extendable and awesome Ruby virtual machine! :)