Complex modifies the Math library in a way that breaks compatibility in many ways.
1 Math.log("10") # => 2.30258509299405
2 require "complex"
3 Math.log("10") # => raises NoMethodError: undefined method `polar' for "10":String
Running the Math specs after the Complex specs, exposes a total of 50 errors/failures.
bin/mspec -t r spec/ruby/1.8/library/complex/ spec/ruby/1.8/core/math/ ... 77 files, 433 examples, 977 expectations, 33 failures, 17 errors
See the attached complex_math.patch file for fixes.
1 require "complex"
2 Complex(3, 4).numerator # => Complex(3, 4)
3 Complex(3.6, 3).numerator # => raises NoMethodError: undefined method `denominator' for 3.6:Float
I think it would be more appropriate to raise a "real" exception.
1 require "complex"
2 Complex(3, 4).hash # => 14
3 Complex(4, 3).hash # => 14
1 require "rational"
2 Rational(3, 4).hash # => 14
3 Rational(4, 3).hash # => 14
The offending line is in net/ftp.rb at line 671:
1 if $![0, 3] != "500"
Exceptions do not respond to the method #[], thus this line results in a NoMethodError.
A fix for this problem is to change the line to:
1 if $!.message[0, 3] != "500"
The simplest way to get started writing specs is to clone the main Github repo, install the mspec gem, and read the wiki documentation here. Once you have written a spec and committed it to your local clone, use git format-patch to create a patch file and upload it in a new ticket. Please do not send pull requests.
If you have questions, the quickest response is available in the #rubyspec channel on freenode. Or you can send a message to the mailing list
The current "standard" version of the MatzRuby for the specs is 1.8.6 patchlevel 114, which is not currently the official stable version, but it appears to be the version used by the vast majority of folks at the moment.
Refer to the documentation for guards, and in particular the ruby_bug guard for how to work with potential bugs discovered in the current standard version.
The RubySpec project aims to write a complete executable specification for the Ruby programming language that is syntax-compatible with RSpec. RSpec is essentially a DSL (domain-specific language) for describing the behavior of code. This project contains specs that describe language syntax, core library classes, and standard library classes.
The specs generally serve two purposes: 1) to drive development, and 2) as a verification mechanism. These goal can sometimes be at odds. During development, the code and specs evolve together. For verification, it is desirable to have a highly stable set of specs that ideally have been audited for correctness. The specs have been extensively used and developed BDD style in the Rubinius project for almost 1.5 years with numerous contributions by other projects like JRuby. Consequently, the specs have attained a high degree of stability. However, there are a number of core library classes and numerous standard library classes for which specs still need to be written.
The following documents describe how the Rubinius specs are organized, the style used to write them, and special measures (guards) used to write a single body of specs that accommodates numerous different Ruby implementations.
The RubySpec project intends to provide a complete and exhaustive specification of the Ruby language and its libraries. Because Ruby is not completely isolated from its platform or execution environment, the spec files may contain guards: conditions placed around a spec or a set of specs to enable or disable them. The guard mechanism is currently implemented only in MSpec, although there should be no issues adding them to RSpec or other implementations. MSpec offers another means of inclusion and exclusion as well: tags and how they work together with guards is discussed in some detail further down.
Our reference implementation is generally the most recent stable release of the official Ruby implementation available from http://ruby-lang.org (often referred to as "MatzRuby" or "MRI.") The set of specs we have should describe the complete behaviour of that release all the way from how the if statement works to the behaviour of core classes like Hash and beyond. The specs are always updated to exactly specify that latest reference implementation as new releases come along. The reference implementation may also be referred to as "Standard," "Reference" or "Target." The current reference implementation version is always listed at Current Standard.
So, generally speaking the purpose of any given spec in our hierarchy is to specify how the Standard behaves for that particular aspect. If you see a plain spec, that spec is exactly the behaviour of the Standard. There are three cases in which guards would be used:
In some cases, MatzRuby exhibits different behaviour based on its execution environment. For example, the Kernel.fork method is not available on Windows and the maximum value of a Fixnum before it is promoted to a Bignum is different between 32-bit and 64-bit platforms. These and other similar cases, such as requiring superuser privileges for a Dir.chroot spec, are surrounded by guards:
platform_is :wordsize => 32 do
it "converts numbers to Bignums if they get too big" do
# 32-bit behaviour goes here
end
end
platform_is :wordsize => 64 do
it "converts numbers to Bignums if they get too big" do
# 64-bit behaviour goes here
end
end
#it blocks.) The guards are actually not connected to or dependent on eachother in any way; you could leave the 64-bit version out altogether. Typically all guards come in pairs so make sure you always implement both behaviours. In some cases a platform simply exhibits additional behaviour, which allows the spec writer to have an unguarded set of specs for the normal behaviour and then a set that augments that, guarded to only be run on the platforms that support it.
Sometimes a bug appears or is discovered in MatzRuby, either everywhere or specific to a certain platform or environment (the specs have unearthed dozens of bugs already.) The criteria for this particular guard is fairly stringent: at the very least, a bug must be filed against Ruby on their bug tracker. The bug ID is actually an argument (optional for now) to the #ruby_bug guard. It is advisable to first discuss the potential bug either among your fellow RubySpec developers or on the ruby-core mailing list, however, in order to verify that we are in fact dealing with an actual bug and not simply some unexpected behaviour. One of these guards might look like this:
ruby_bug "#someidnumber" do
it "produces 2 when adding 1 to 1" do
# CORRECT implementation goes here
(1 + 1).should == 2
end
end
If you only work with the specs as they relate to the reference implementation, you will never need to create these types of guards, but you may run into some perusing the existing specs. Broadly speaking all of the different alternative Ruby implementations, Rubinius, JRuby, IronRuby and so on, should exhibit the exact same behaviour as the reference implementation but in practice this is not always the case. There is also a bit of a conflict of interest: these being RubySpecs, we would prefer to not have a large amount of implementation guards littering them.
Tags are the alternative mechanism offered by MSpec for this purpose. They allow the users of the specs to locally exclude some specs based on their own criteria, without touching the specs themselves (presumably they will also have a separate, parallel subset of specs that is considered "correct" behaviour for that implementation in the case at hand.) In the future it is likely that the number of implementation guards will lessen in favour of using tags, but for now the guideline has been that tags are used for transient problems, for example to exclude a spec that the implementation is not capable of running yet. Guards, on the other hand, have been used to indicate a permanent difference. There are also two different types of implementation differences: most often a feature is just not supported in an implementation:
describe "Kernel.fork" do
not_supported_on :jruby do
it "does forky things" do
# Whatever the spec is
end
end
end
:rubinius and so on; the only exception is that the reference implementation is always called just :ruby.
The second case is that an implementation may implement a feature differently from the reference implementation. Rubinius, for example, has a Fixnum whose maximum value before conversion to Bignum is platform-independent, unlike MatzRuby. Here, two guards must be used:
not_compliant_on :rubinius do
it "automatically converts from a Fixnum to a Bignum when blah blah" do
# Spec
end
end
compliant_on :ruby, :jruby do
platform_is :wordsize => 32 do
# 32-bit spec
end
platform_is :wordsize => 64 do
# 64-bit spec
end
end
#compliant_on must currently have the :ruby specified although it is kind of implied already. This is likely to change in the near future.)
Sometimes the behavior of certain methods will change between different Ruby versions (or revisions!). If you're writing specs against several different patchlevels you can use the ruby_version_is guard:
ruby_version_is "" ... "1.9" do
it "adds two numbers" do
(1+1).should == 2
end
end
ruby_version_is "1.9" do
it "adds two numbers" do
(1+1).should == 3
end
end
In this case we're assuming that 1 + 1 will equal 2 in every version before 1.9 and that in 1.9 and following versions it will return 3.
The range syntax works just like normal Ranges:
"a".."b" # => "a" to "b", including "b".
"a"..."b" # => "a" to "b", excluding "b".
"a" # => covers everything before "a"
The exact semantics always depend on the particular guard in question. Usually they are self-evident but the MSpec Wiki contains a thorough description of all guard types and their uses. Two things are always the same, though: if a guard has multiple parameters, those are combined using OR logic: compliant_on :ruby, :jruby means either one of the two, not both (obviously enough.) If AND logic is desired ("Rubinius on 32-bit platforms"), the guards should be nested instead. The outer guards are naturally evoked before getting to the inner ones.
The first commit of what would become RubySpec is recorded in the Rubinius repository on Sat Dec 16 07:55:41 2006 +0000. At that point, Rubinius had only been publicly announced about two months before at RubyConf06 (see a nice summary of Evan's talk). We did not run much code, but Evan had already been writing tests for the development he was doing. The initial spec code was an ape of the unit test harness, which basically created a subprocess for each test method and made assertions on the text output from processing some Ruby code.
<more to follow>
There are many conceivable ways to organize the spec files. The structure is based on the Ruby language as well as the major components of a Ruby implementation.
The goal is to maintain locality by grouping related specs. Generally, there is a single element (method, syntax element) per file, and the files are organized into three main directories: language, core, and library. Below is a partial graphic of the directory tree.
spec
|-- core
| + -- array
| + -- bignum
| + -- binding
| + -- class
| + -- ...
| + -- time
| + -- true
| + -- unboundmethod
|-- fixtures
|-- language
+-- library
+ -- enumerator
+ -- ...
+ -- time
+ -- yaml
There are three primary challenges in combining the 1.8 and 1.9 specs:
The three issues above are addressed as follows:
language_version helper to conditionally run those specs. See MSpec Helpers and the language/method_spec.rb specs for examples.ruby_version_is guards as usual for any methods or method behaviors specific to a particular version. See MSpec Guards.:files config setting in either ruby.1.8.mspec or ruby.1.9.mspec for libraries or classes that should not be run in the respective version. See MSpec Configuration.The language directory contains specs for the Ruby language proper. There are numerous possible ways of categorizing the entities and concepts that make up a programming language. Ruby has a fairly large number of reserved words. These words significantly describe major elements of the language, including flow control constructs like "for" and "while", conditional execution like "if" and "unless", exceptional execution control like "rescue", etc. There are also literals for the basic "types" like String, Regexp, Array and Fixnum.
Behavioral specifications describe the behavior of concrete entities. Rather than using concepts of computation to organize these spec files, we use entities of the Ruby language. Consider looking at any syntactic element of a Ruby program. With (almost) no ambiguity, one can identify it as a literal, reserved word, variable, etc.
There is a spec file that corresponds to each literal construct and most reserved words, with the exceptions noted below. There are also several files that are more difficult to classify: all predefined variables, constants, and objects (predefined_spec.rb), the precedence of all operators (precedence_spec.rb), the behavior of assignment to variables (variables_spec.rb), the behavior of subprocess execution (execution_spec.rb), the behavior of the raise method as it impacts the execution of a Ruby program (raise_spec.rb), and the block entities like "begin", "do", "{ ... }" (block_spec.rb).
Several reserved words and other entities are combined with the primary reserved word or entity to which they are related:
predefined_spec.rb: false, true, nil, selffor_spec.rb: inif_spec.rb: then, elsifcase_spec.rb: whenthrow_spec.rb: catchThe core directory contains specs for the Ruby core library. These include classes such as Array, String, Regexp, Range, Fixnum, Float, etc. The core directory contains a subdirectory named after the classes in the core library. For example, specs for the Array class are in the array directory.
Within each subdirectory of core, the specs for each method of that class are placed in a separate file. For example, specs for Array#compact are places in spec/core/array/compact_spec.rb. Method names with characters like "?", "=", and "!" are in files named by stripping those characters. For example, specs for Array#compact! are in the same file as specs for Array#compact. All the spec files that are needed have already likely been created. (See the documentation for mkspec for details.)
The library directory contains specs for the classes of the Ruby standard library. The same naming convention used in the core directory applies here as well.
The 1.8 and 1.9 specs have been merged. See the Organization page for details about writing specs for the different versions.
Generally, RSpec specs describe the expected behavior of code. While RSpec is fairly young, there are some conventions for writing specs. The RubySpecs cover a wide variety of components, so we have developed some pragmatic conventions to handle the various situations. As noted below, some conventions are more rigid than others.
These conventions apply to all specs. Existing specs that deviate from these conventions need to be fixed. Consistency is the principle that will almost always trump other conventions. Consistency aids understanding and readability. There are many thousands of lines of code in the spec files, so the value of consistency cannot be overstated.
The specs uniformly use describe not context. The use of it is preferred over specify except in situations when the first word of the string is not a verb. The word "should" is unnecessary noise in the spec description strings and is not used. (The rationale is this: the spec string describes the expected behavior unconditionally. The code examples, on the other hand, set up an expectation that is tested with the call to the should method. The code examples can violate the expectation, but the spec string does not. The value of the spec string is as clearly as possible describing the behavior. Including "should" in that description adds no value.)
Whenever possible, the spec strings should be written to conform to very basic English sentence structure: subject + predicate. The spec strings also uniformly use double-quotes, not single-quotes. The minimum number of words should be used to describe the behavior. Only make distinctions when they add significant value to understanding the behavior. This is explained further below. The general rule across all the specs is to use the least amount of detail to unambiguously describe behavior. Add to the detail conservatively. This is conceptually consistent with doing the simplest thing that could work.
Ruby is a beautifully expressive language with optional parentheses. There is a distinct preference for omitting parentheses in the specs whenever they are not needed. In other words, parentheses should not be used unless necessary to make an expression syntactically or semantically correct.
The specs for the Ruby core and standard libraries use one describe block per method. For particularly complex methods, such as Array#[], more than one describe block may be used according to the nature of arguments the method takes.
The describe string should be "Constant.method" for class methods and "Constant#method" for instance methods. "Constant" is either a class or module name. For subclasses or submodules, the "Constant" name should be "Super::Sub". The describe string should not include arguments to the methods unless absolutely necessary to describe the behavior of the method. Keep in mind that in Ruby duck-typing is a deeply embedded concept. Many methods will take any object that responds to a particular method or acts like an instance of a particular class.
Nested describe blocks should not be used. Various automated process scripts depend on the describe string having the format explained above. Also, nested describe blocks complicate the structure of the specs. If a particular situation appears to greatly benefit from nested blocks, open a discussion about it on the mailing list.
# This is correct
describe "String#eql?" do
it "returns true if other has the same length and content" do
...
end
end
describe "Array#[]= with [index, count]" do
it "returns non-array value if non-array value assigned" do
...
end
end
Contrast the good example above with the one below. The following example deviates from the conventions for describe strings and uses "should" and single-quotes for the descriptions.
# This is NOT correct
describe "String#eql?(string)" do
it 'should return true if other has the same length and content' do
...
end
end
describe 'Array#[]=(index, count)' do
it 'returns non-array value if non-array value assigned' do
...
end
end
The vast majority of the spec files for the core library have already been created. To create template files for the standard library classes, refer to the mkspec documentation.
Many spec code examples refer to a particular class. To prevent name clashes with these different class definitions across all the specs, the classes should be scoped to a module. The convention is as follows:
module ObjectSpecs
class SomeClass
end
end
The module is named after the class for which the specs are being written. So, for the specs for Object, the module name is ObjectSpecs.
These utility classes are also referred to as fixtures. In the directory for each class, there is also a fixtures directory. Refer to the existing files for examples.
Ruby has a significant number of aliased methods. True aliases are identical methods, so the specs should be exactly the same for each aliased method. The following illustrates the convention for specs for aliased methods (or just otherwise identical interfaces.)
In rubyspec/1.8/core/array/shared/collect.rb
shared :array_collect do |cmd|
describe "Array##{cmd}" do
it "returns a copy of array with each element replaced by the value returned by block" do
a = ['a', 'b', 'c', 'd']
b = a.send(cmd) { |i| i + '!' }
b.should == ["a!", "b!", "c!", "d!"]
end
it "does not return subclass instances" do
MyArray[1, 2, 3].send(cmd) { |x| x + 1 }.class.should == Array
end
end
end
In rubyspec/1.8/core/array/collect_spec.rb
require File.dirname(__FILE__) + '/../../spec_helper'
require File.dirname(__FILE__) + '/shared/collect.rb'
describe "Array#collect" do
it_behaves_like :array_collect, :collect
end
In rubyspec/1.8/core/array/map_spec.rb
require File.dirname(__FILE__) + '/../../spec_helper'
require File.dirname(__FILE__) + '/shared/collect.rb'
describe "Array#map" do
it_behaves_like :array_collect, :map
end
Writing specs that use floating point values poses a problem because two values that look the same when rendered to a string may not actually be bitwise equal. Also, floating point operations can result in a value that differs based on the way the FPU carried out the operations.
Specs that compare floating point values should use #should_be_close with the TOLERANCE constant. For floating point values that are exact, but larger than the precision formatted with #to_s (e.g. 1093840198347109283720.00), use the expanded float literal not the truncated precision format that #to_s provides (e.g. don't use 1.09384019834711e+21).
For the language specs, there is nothing as convenient or as concrete as a particular method to spec. Review the discussion of the organization of the language specs. The general conventions apply here: use simple English to describe the behavior of the language entities and only add detail as needed. Use a single describe block initially and add distinguishing describe blocks as necessary. Use it rather than specify whenever possible.