Ramble On

My proposal for porting Parrot to RTEMS was accepted for the Google Summer of Code! Though it’s a little late, this post will introduce and explain what I’ll be doing.

Parrot

Parrot is a virtual machine designed with dynamic languages in mind. Historically, Parrot started out as the virtual machine for Perl 6 Rakudo, but now it has many languages (or “High Level Languages” or “HLLs”) being actively developed to run on top of the virtual machine. Parrot comes with a number of tools to make implementing new languages easier – the Parrot Compiler Tools. Like other VMs, Parrot can compile down to platform independent bytecode (called PBC). Parrot was designed with “HLL Interop” in mind – to quote the docs,

Parrot is also designed to provide interoperability between languages that compile to it. In theory, you will be able to write a class in Perl, subclass it in Python and then instantiate and use that subclass in a Tcl program.

How cool is that? That means progress for one HLL is progress for all. Any optimization to the underlying architecture is an optimization for every HLL. Any low-level bindings will automatically work for all HLLs.

RTEMS

RTEMS stands for Real-Time Executive for Multiprocessor Systems. It is a real time operating system designed for embedded systems. RTEMS is free and open source and POSIX (1003.1b API) compliant. RTEMS comes with a full ANSI C, ANSI C++ (with STL), and ADA support. RTEMS has a full TCP/IP stack as well as other networking features like HTTP servers and support for RPC. Finally, RTEMS has full debugging support through GDB.

RTEMS is cross-compiled – that means the programmer might be on an x86 architecture but compile programs to run on Sparc. RTEMS currently supports 116 hardware configurations (“Broad Support Packages” or “BSPs”) on a number of chip architectures (x86, ARM, Sparc) which can all co-exist on the same development machine and run with the same development code.

Potential

I think the potential here is staggering – not only does RTEMS get new languages to target the hardware, Parrot gets new platforms to test and run on. Someone could write something in Rakudo Perl using classes from other languages like Python or Ruby or Tcl, compile to bytecode and then be run on a hundred or so different hardware configurations.

Current Work

Of course it’s not all puppy dogs and ice cream – there’s work to do! Things don’t quite work yet, but by the end of the summer I hope to have Parrot’s bleeding edge building on RTEMS and running the test suite on the BSPs. I’ll update this blog once a week with my progress after my Tuesday meeting with my mentors at the Parrot Foundation and RTEMS. The first work that needs to get knocked out:

1. Hints Makefile – This Makefile will extract all the BSP information necessary for compiling Parrot and then generate a hints file that will be used by Parrot at compile time.
2. Cross compilation support for Configure.PL – We need to be able to specify which platform we want to target at compile time for Parrot. Currently Configure.PL figures this out automatically by checking the system that we are running, but with cross-compilation the system we are running is not the system we are targeting.
3. Write blog post detailing plan of attack

I was given a free copy of Codeulate Screencasts’ “Vim for Rails Developers“.  The screencast normally costs $9 US and lasts about 36 minutes.  I use Vim (MacVIM) everyday for a number of hours at work and am coding in Rails more and more (through work and other projects).  If you are even thinking about considering contemplating the possibility of developing Rails with Vim, you should buy this screencast.  There is enough distilled wisdom here to make it worth your time, and there is really nothing better than having someone not only tell you but show you how to do what needs to be done.  Screencasts are hands down the best way to get up to speed with a technology (short of having someone holding your hand or looking over your shoulder) and if you haven’t watched any yet, you need to check out learnivore.com. This screencast is among the best thought out and professionally executed.

Contents

Typing Faster

During this section the screencast covers the benefits of being able to touch type and at a somewhat rapid pace, not only in general but in Vim.  Though I’m fairly comfortable in this regard already, I can see how it would be of great benefit for someone who hasn’t used Vim a lot before.  The Vim cheatsheet recommended is really great – everyone should have a copy. More important is developing an understanding of the different modes and how to move around effortlessly.

A few years back I tried switching to Dvorak and found no speed increase in my typing.  One benefit from switching was that the one very common error when typing in QWERTY of transposing two letters is much less likely to happen in Dvorak. Of course your milage may vary.

Vim Rails plugin

This is the real meat of the screencast, and where the most value is.  Tim Pope’s Vim Rails plugin is extremely powerful and the screencast will showcase some of the most common features.

It would have been nice to see some instructions for the Vim newbies on how to install plugins.  Though I used “GetLatestVimScripts” before but have now switched to Vimana, a gem like plugin manager.  As long as you don’t mind a few perl modules on your system, you should really manage all your Vim scripts with Vimana.

Vim snipMate plugin

Again, another power plugin that everyone should be using.  The screencast demonstrates how to both use and customize some of the functionality.

Ctags

I hadn’t heard about ctags until this screencast.  Mac OS X ships with a version of ctags, but it looks like it is severely nerfed.  When I have some free time I’ll definitely build my own copy from source.  The screencast shows how to run and integrate ctags with Vim.

Ack

I recently switched over to start using Ack instead of grep, and have never looked back.  Ack is faster than grep is almost every situation, and intelligently handles source control directories (.svn, .git, etc.).  Ack also doesn’t rely upon a lot of perl modules and so can really install anywhere.  The screencast introduces Ack and shows how to use it as your Vim grep program.

Quick tips

The last section contains some quick tips that just make Vim life easier.  I don’t want to give away any secrets, but the ci” and :jumps commands are probably some of the most useful that will save time.

Verdict: Buy.

About You

Have you participated in Google Summer of Code before?: Yes

Detail your previous GSoC experience

Last year I worked with The Perl Foundation under Jonathan Leto to develop a CPAN module Math::Primality (github.com/bubaflub/math–primality).  The project was challenging but Jonathan was extremely helpful and patient and could always point me in the right direction.  He jumped in from time to time to help me out when I was stuck, and we finished ahead of schedule.  I’m very proud of the end product and is probably the best tested and best documented code I’ve produced.  I’ve been involved with the Perl community since, hacking on the Parrot VM (which is the VM that will run an implementation of Perl 6) and have been a committer for a number of months now.

Bio

I am currently 24 years old living in Champaign-Urbana, IL attending school full time and working part time as a web developer.  My undergraduate degree was in Math & Computer Science (with coursework focusing on Number and Coding Theory, Numerical Analysis, and Cryptography).  I’ve been a web developer since before beginning college and currently work supporting a legacy application in PHP + MySQL and develop new software in Ruby on Rails.  I had to pick up Ruby/Rails for this current job and am loving it.

About Your Project

Why do you use Ruby and/or Rails? How would you like to see them improve?

I started with Ruby on Rails for work but soon began using it to prototype my own projects.  I’ve been using jQuery for a number of years before that and love it as well.  I’d like to see Ruby on Rails become JavaScript framework agnostic, as this provides more choices for the developer.  Especially with the big push for modularity in Rails 3 we shouldn’t be tied to one framework.

Outline the specific project you’re proposing

This project will make ActionView javascript framework agnostic by providing an intermediate class that exposes a consistent API that multiple javascript frameworks can implement.  This project will be accompanied by tests to ensure correctness and documentation both for the new code and a tutorial for how to add other javascript frameworks. The project aims to not break any existing functionality and enable the developer to pick which framework they would like to use.

Why is this important to the Ruby and/or Rails communities at large? Why is this important to you?

I love Ruby on Rails.  I love jQuery.  It makes me sad to see the two not integrated well.  And although most of my work requires me to use completely custom jQuery, we should lower the barrier of entry for people to use JavaScript in their Rails applications.  Many throw out RJS completely simply because it uses Prototype and they don’t.  The intermediate class would not only allow the option to choose which JavaScript framework, but would make it easier to extend JavaScript functionality.

List a clear set of goals/milestones you’ll hit during the summer, along with a rough timeline. Be specific about your deliverables.

Assuming a timeline similar to GSoC

• When notified of acceptance until the beginning of the program: Documentation and testing of existing functionality. I am unsure of the existing test coverage.  The first step will be to fill out any missing documentation and tests to ensure I don’t break anything in the process.
• By midpoint of the internship: Replicate the existing functionality through the intermediate class. I will first work on developing an intermediate class that will be called instead of Prototype/Scriptaculous methods directly. By half way through the summer I would like to have this class completely functional, deprecating the old methods.
• By the end of the internship: Have at least jQuery functioning through the intermediate class. By the end of the internship I would like to have jQuery functioning through that same intermediate class, so one could simply specify “prototype” or “jquery” and get the same functionality without breaking any code.  If I am ahead of schedule, I will also begin integrating other popular javascript frameworks like mootools or YUI.

Abstract

Currently Crypt::Primes and Crypt::Random use Math::Pari for their mathematical processing. Bindings to Pari (through Math::Pari) are fragile and can slow Perl 5 development when new versions are released as bugs must be hunted. This project will move these two core modules to use Math::GMPz and Math::Primality (accessing the GMP library) which will provide identical functionality with less headaches. Since I do not own either of these modules I will provide a set of patches that the authors may choose to incorporate for this functionality.

Deliverables

• Crypt::Primes patches, documentation, and tests
• Crypt::Random patches, documentation, and tests
• The same for any other libraries that may depend upon them

Project Details

This project will attempt to remove Perl 5′s dependence upon Pari and the fragile Math::Pari bindings.  Last year through Google’s Summer of Code I developed Math::Primality to provide this type of functionality and now this project will integrate it into core modules.  The main idea is to remove the dependencies without having any noticeable affect to the end-user.  If there is insufficient tests or documentation, I will add them to ensure correct functioning and clarity. If time permits, I will profile the modules and attempt to optimize the performance of these core cryptographic libraries.

Project Schedule

April 26th – May 24th: Install and test Crypt::Primes and Crypt::Random as is.  Contact module author(s) to introduce myself and explain what is happening.  Determine if any other core modules need to be updated. I will also finish polishing Math::Primality (Issues #1 – #4 at http://github.com/bubaflub/math–primality/issues – Convert README to POD, use Dist::Zila, test with 5.12, and clarify documentation).

May 24th – July 12th: Work on patching Crypt::Primes.  Fill out any missing documentation and enhance test suite.

By July 12th: Have Crypt::Primes converted to use Math::GMPz and Math::Primality.

July 12th – August 9th: Work on patching Crypt::Random.  Fill out any missing documentation and enhance test suite.

By August 16th: Merge patches if possible.  Polish code and fill out test suite and documentation.

If I finish early or am ahead of schedule I will move on to other Crypt::* modules that are commonly used and depend upon Math::Pari, providing the same documentation and tests as above.

References and Likely Mentors

I’ve worked with Jonathan Leto on last year’s Google Summer of Code and work with him on Parrot currently.

License

Same terms as Perl itself.

Bio

My name is Bob Kuo and I am a 24 year old graduate student living in Champaign-Urbana, IL.  My undergraduate degree was in Math and Computer Science with classwork focusing on Number and Coding Theory, Numerical Analysis, and Algorithms.  I participated in last year’s Google Summer of Code with The Perl Foundation writing Math::Primality and finished ahead of schedule.  I have been involved with Parrot since the end of last year’s Google Summer of Code and am an active committer.  I currently work part-time as a web developer where I support a legacy PHP system and develop in Ruby/Rails.  I have been using Perl in web development and system administration for 4+ years.  I’ve contributed bug fixes to the CPAN modules CGI and Math::GMPz. You can see my open source code at github.com/bubaflub.

Eligibility

I am currently a full time student in graduate school at an accredited institution living in the United States.

Abstract

Parrot currently has two PMCs that wrap functionality in GMP: BigInt and BigNum. This project will move these PMCs out of the main parrot repository and replace them with bindings to the GMP library. The project will focus on getting main integer and floating point functions over from GMP, replicating in part Math::GMPz and Math::GMPf.

Benefits to the Perl/Open Source Community

Some members of the parrot community have expressed their disgust with the current BigNum implementation.  Not only would this project provide a more elegant and efficient implementation, but GMP bindings would be available to any language implemented on top of Parrot – hence Rakudo would gain “free” bindings to GMP.  GMP bindings are also foundational for any future cryptographic libraries that may need to be written.

Deliverables

• BigInt and BigNum PMCs based upon new GMP bindings that do not break current
• Access to crucial GMP integer and floating point functions through NCI
• Tests for every accessed function
• Documentation for every function
• Example code

Project Details

This project will provide access to the GMP library through the NCI (native call interface).  The focus will be to replace the existing BigInt and BigNum functionality by refactoring the PMCs to use the GMP NCI. The design will allow future work to bind to more functions or be easily modified if the GMP API changes in the future (though this is unlikely – GMP is fairly stable).  Though the bare-minimum amount of functionality required for BigInt and BigNum is proposed here, depending on time and speed of development I will essentially bind as many functions as I can.

Project Schedule

April 26th – May 24th: Begin and (hopefully) finalize design of new PMCs and GMP interface.  Start branch to move BigInt and BigNum out of main parrot repository.

May 24th – July 12th: Begin binding to necessary integer and float functions in GMP.  Write tests and documentation.

By July 12th: Have at least one of the PMCs functionally equivalent with it’s previous incarnation.  Have tests to prove this and documentation to explain it.

July 12th – August 9th: Finish second PMC and necessary NCI bindings.

By August 16th: Merge branch if it hasn’t already been merged into trunk.  Polish tests and documentation.

If I finish early or am ahead of schedule I will provide bindings, tests, and documentation to some of the other integer and float functions.  If I’m speeding through this, I will also provide the same for the GMP rational functions.

References and Likely Mentors

I’ve worked with Jonathan Leto on last year’s Google Summer of Code and work with him on Parrot currently.  Andrew Whitworth would also be a good mentor for this project as he works with parrot-linear-algebra and the HLL matrixy.

License

Artistic license 2.0 (same as Parrot)

Bio

My name is Bob Kuo and I am a 24 year old graduate student living in Champaign-Urbana, IL.  My undergraduate degree was in Math and Computer Science with classwork focusing on Number and Coding Theory, Numerical Analysis, and Algorithms.  I participated in last year’s Google Summer of Code with The Perl Foundation writing Math::Primality and finished ahead of schedule.  I have been involved with Parrot since the end of last year’s Google Summer of Code and am an active committer.  I currently work part-time as a web developer where I support a legacy PHP system and develop in Ruby/Rails.  I have been using Perl in web development and system administration for 4+ years.  I’ve contributed bug fixes to the CPAN modules CGI and Math::GMPz. You can see my open source code at github.com/bubaflub.

Eligibility

I am currently a full time student in graduate school at an accredited institution living in the United States.

Parrot on RTEMS

Google Summer of Code Program 2010 Project Proposal

Name: Bob Kuo
Email: (removed here but in my official proposal through Google)
University: Urbana Theological Seminary, Urbana, IL, USA

Project Abstract

This project seeks to enable code written in parrot to run on the RTEMS platform.

Project Description

Parrot is a multi-language, multi-platform virtual machine. The goal of this project is to enable parrot to configure and build for a variety of RTEMS BSPs. Currently, parrot cannot build out of directory nor can it cross-compile. This project would provide the necessary improvements to the parrot build system to enable parrot code to target RTEMS.

Project Deliverables

1. May 23 (coding begins) – I already have a development environment setup with RTEMS successfully building and compiling on sparc-sis. I will be focusing on an x86 target and I will also attempt to get Chris’ previous work on Parrot + RTEMS up and running.
2. June 15th – Work will begin on a testing framework to ensure that parrot is behaving correctly within RTEMS.
3. July 12th (Midterm Evaluation) – Chris’ work will be brought up to date so that Parrot + RTEMS can build of trunk with a small set of patches against the Makefile.
4. August 9th (Final Evaluation) – Testing framework is in full swing and bugs are being fixed in both parrot and RTEMS. Cross-compilation support is merged into the parrot trunk.
5. August 23rd (Final Results Announced) – Code is polished, documented, and tested.

Proposed Schedule

April 26 – May 23 2010 (study and prep)

I am less familiar with RTEMS and CVS (I have only used svn or git) so I will be studying the RTEMS system. I will work primarily with the x86 architecture and the i386/pc386 BSP as Parrot already runs very well on this architecture. I also at this point would like to get Chris’ previous work running on my test environment.

May 24 – June 15 2010 (testing)

Assuming I can get Chris’ previous work to build, I will be devising a way to get the parrot test suite to run on RTEMS. Most likely this will be limited to the “core” tests at first.

June 16 – July 12 2010 (trunk)

Once Chris’ work and a testing framework is in place, I will be attempting to get the latest parrot to build on RTEMS. Parrot 2.3.0 will be a longer-term supported release and work will be based off that release.

July 13 – August 9 2010 (patches)

Documentation and tests for both Parrot and RTEMS will be provided throughout the entire period. I hope to have a cross-compilation branch merged into Parrot trunk so that a set of patches against a Makefile are no longer necessary.

August 10 – August 23 2010 (cleanup)

Clean up code, improve documentation, and package any patches or extra files for delivery.

In the even that I’m ahead of schedule:

If I am ahead of schedule during and Parrot is running cleanly on x86 I will begin to extend support to other BSPs. This will involve someone at RTEMS who already has a large number of BSPs built on their system and using the development build farm. While Parrot 2.3.0 will be a longer supported release, it would be ideal if RTEMS+Parrot could track Parrot’s ongoing development.  Programs will be developed that will allow regular automated testing of RTEMS+Parrot.

Continued Involvement

Assuming that the project is a success, I can continue to maintain and support and improve Parrot on RTEMS. Since my work will most likely take me into the guts of RTEMS, I will be contributing to RTEMS for some time to come.

Future Improvements

1. Remove compiler and linker warnings
2. Fix broken tests
3. Support more BSPs

Major Challenges foreseen

My understanding of Makefiles is a bit shallow and cross-compilation is very new to me. I’ll need some expert help in determining how to best modify both Parrot and RTEMS build chains.

Possible Mentors

As Chris Johns has previous work on RTEMS+Parrot, he would be most helpful in mentoring me.  Dr. Joel Sherrill has been extremely helpful on IRC. Jonathan Leto (with Parrot) has worked on the Parrot side of this equation and would be helpful as well.

References

Chris’ previous work: http://www.rtems.org/ftp/pub/rtems/people/chrisj/parrot/
Parrot ticket for cross-compilation: http://trac.parrot.org/parrot/ticket/1516
Branch on parrot for cross-compilatoin: http://trac.parrot.org/parrot/browser/branches/tt1516_builddir_option

Relevant Background Experience

I have been a Parrot developer since the end of last Google’s Summer of Code and have commit access. Last year I worked with the Perl Foundation to develop Math::Primality and my work can be seen at http://github.com/bubaflub/math–primality.

Personal

I am a 24 year old grad student living in Champaign-Urbana, IL. My undergraduate degree was in Math and Computer Science with course work focusing on Number and Coding Theory, Numerical Analysis, and Algorithms. I work part-time as a web developer.

How did you learn about RTEMS?

From parrot people.

Anything else that makes you special:

I have a strange light-blue birthmark towards the top of my forehead.

Experience

Free Software Experience/Contributions:
I am a committer to parrot, have a module on CPAN (Math::Primality from last year’s Google Summer of Code), and have committed bug fixes to various open source projects (e.g. CGI.pm and Math::GMPz).

Language Skill Set
My part-time job focuses on PHP and Ruby. My open source work takes me mostly towards Perl, and have been working with Perl in either web development or system administration for 4+ years. Previously I worked in C/C++ and Java.

Related Research and Work Experience (if any):
Besides some parrot hacking and playing around with ANTLR for parser generation, nothing significant.

Reference Links and Web URLs (optional):

blog.bobkuo.com
IRC handle: bubaflub
github.com/bubaflub

Title:

IPv4, IPv6, MACADDR Native Column Types

Student:

Robert Kuo

Abstract:

Removing the UNSIGNED column type from Drizzle forced IP addresses to be stored as BIGINTs which is inefficient. This project will implement the IPv4, IPv6, and MACADDR native column types for Drizzle and the functions INET_NTOA() and INET_ATON(). This project will include inline documentation, patches to the official documentation, and test cases that verify correct operation and error handling.

Content:

Project Title
IPv4, IPv6, MACADDR Native Column Types

Name:

Bob Kuo

Email:

(removed from here but in my official proposal sent to Google)

IRC:
bubaflub

Synopsis

This project will implement the IPv4, IPv6, and MACADDR native column types as well as the functions INET_NTOA() and INET_ATON() as well as the documentation and test cases to support this new functionality.

Benefits to the Drizzle Community

According to a message on the mailing list this feature has already been requested by the community. As well as a felt need, this would bring Drizzle closer to MySQL functionality and compatibility.

I’ve worked with MySQL for more than 4 years and would love to get involved with Drizzle but haven’t had the time or resources to do so. I love the idea of a MySQL that is lean, community-driven, and cruft-free and Drizzle seems to fit the bill.

Deliverables

• The INET_NTOA() and INET_ATON() functions that convert between dotted-quad and an integer network address.
• The IPv4, IPv6, and MACADDR native column types so that one could type: CREATE TABLE sessions ( id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY , ip_address IPv4 NOT NULL );
• Test cases that prove correct operation and error handling of the functions
• Documentation in whatever format is suitable

Project Details

This project will focus on the functions and then use them internally to convert IP addresses to a space-efficient binary format – e.g., binary(4) for IPv4 and binary(16) for IPv6.

Project Schedule

Before the start date (April 20th – May 23rd)

• Meet Drizzle community
• Familiarize myself with Drizzle internals and build process
• Familiarize myself with bzr (I usually use git)

First half (May 23rd – July 13th)

• Spec out functionality with documentation and write failing tests
• Write INET_NTOA() and INET_ATON()
• Implement the IPv4 native column type

Second half (July 13th – August 10th)

• Finish documentation and test cases
• Implement the IPv6 native column type
• Implement the MACADDR native column type

In the event that I finish early:

• More documentation and tests
• Any optimizations that profiling or benchmarking reveals
• Any other utility functions related to these column types
• Work on the testing framework

Bio/Experience

My name is Bob Kuo and I am a 24 year old currently pursuing my masters and living in Champaign-Urbana, IL. My undergrad was in Math and Computer Science at the University of Illinois at Chicago with my classwork focusing on algorithms, coding and number theory, and numerical analysis. I have used MySQL for more than 4 years in both a developer and DBA role. I participated in last year’s Google Summer of Code working for The Perl Foundation and successfully finished the project ahead of schedule. My project code is available on the CPAN and at http://github.com/bubaflub/math–primality. I currently work part-time as a web developer for a local company and work in PHP and Ruby and Rails and can work in Perl, PHP, Ruby, C/C++ and with both MySQL and PostgreSQL.

All comments, critiques, or feedback are most welcome.

Thanks,

Bob Kuo

Title:

Implementation of algorithm to infer gene duplications in BioRuby

Student:

Robert Kuo

Abstract:

This project will implement an algorithm to detect gene duplications in BioRuby described in Zmasek and Eddy, 2001, “A simple algorithm to infer gene duplication and speciation events on a gene tree”, Bioinformatics, 17, 821-828. The project will include full documentation, tests, and examples.

Content:

Name: Bob Kuo
(the following have been removed from here but they are in my official proposal to Google)
Address:
Email Address:
Mobile Phone:
IRC Handle: bubaflub

I am 24 and currently pursuing my masters and living in Champaign-Urbana, IL. My undergraduate degree was in Math and Computer Science from the University of Chicago at Illinois, with my coursework focusing on number and coding theory, numerical analysis, and algorithms. I am currently employed part-time as a web developer working in PHP and Ruby on Rails. I participated in last years’ Google Summer of Code by working with the Perl Foundation (http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/dukeleto/t124022226790) which was successfully completed ahead of schedule.

I am interested in BioRuby because I have always had an interest in combining the sciences with programming and am interested in learning more about evolutionary biology. I believe I am well-suited for this project not because I am the best Ruby programmer in the world, but because I am willing to learn and work through algorithms step-by-step. There is a reference specification written in Java from which I can test my Ruby project. My work as a web developer has required me to be multi-lingual and am comfortable reading and writing Java, Perl, PHP, and Ruby.

You can see my open source work at http://github.com/bubaflub, and would recommend looking through http://github.com/bubaflub/math–primality, my work from last years’ Google Summer of Code with The Perl Foundation which has significant in-line documentation and many tests.

All code developed for this project would be released on GitHub and available under the same terms as BioRuby itself.

Plan:
Before the start date (April 20th):

1. Meet the BioRuby and Open Bioinformatics community
2. Familiarize myself with the BioRuby package and the phyloXML format
3. Read necessary papers (such as http://bioinformatics.oxfordjournals.org/cgi/content/abstract/17/9/821)
4. Familiarize myself with the Java implementation of the algorithm (http://www.phylosoft.org/forester/applications/sdi/)
5. Discuss and set expectations of code – dependencies, code style, tests, documentation, etc.

First half (May 23rd to July 13th):

1. Decide on which libraries
2. Spec out all necessary components with documentation and failing unit tests
3. Write integration tests that cover the entire algorithm (i.e. if I input “A” I should get “B”)
4. Begin implementing the algorithm

Second half (July 13th to August 10th):

1. Finish implementing the algorithm
2. Finish documentation and tests

In the even that I finish early:

1. Profiling and speeding up existing code
2. Extra documentation and tests
3. More examples
4. Extend the algorithm to use non-binary species and gene trees.

Obligations
I will continue to work part-time during the summer and may take summer classes. Last year these obligations were not problematic and did not affect my performance. During the day though I will be at work I will be available via email, IM, and IRC.
I welcome any feedback, comments, and critiques to this proposal.

Thanks,
Bob Kuo

We now have the RTEMS toolchain and a local checkout up to date on the system.  Now we must build a BSP that targets some specific architecture.  The getting starting docs recommend starting with sparc-sis, so that’s what we are going to do.  A much more concise wiki page focusing on Fedora Core 12 on Virtual Box has been put together, so I recommend for those following along at home they start there.

Change into the top of your build directory:

cd $B

This directory isn’t under version control, but all the subdirectories are checkouts from the official CVS repository.  Now that we have RTEMS configured for our host machine (Fedora Core 12 on x86) we can build different BSPs by configuring them out of directory. Most unix/linux users will be familiar with the regular incantation of “./configure && make && make install” that result in the configuration and building and installation of a program from source.  RTEMS (and, from what I understand from the docs, most cross-compilation) can be configured and built from outside the source tree.  This allows us to configure RTEMS once and then have a separate build for each different BSPs.

Let’s build the sparc-sis BSP.  First, create a folder to hold all the cross-compiled goodness:

mkdir b-sis
cd b-sis

Now configure RTEMS to build a sparc-sis-esque setup:

../rtems/configure --target=sparc-rtems4.10 --enable-rtemsbsp=sis --enable-tests=samples --disable-posix --disable-itron

and then build it

make

and then install it

su
make install

The “su” may be unnecessary but you may run into permission errors so it’s a safe bet.

You have now installed the sparc-sis on your system. We need to set one environment variable to tell the make processes which BSP to build for.  This environment variable allows you to have multiple BSPs installed at once and easily switch building targets on the fly.  So make sure you are in the “b-sis” directory:

cd $B/b-sis

And then search for the Makefile.inc file:

find . -name Makefile.inc

This should return the location of the Makefile.inc relative to your path.  I got the following:

./sparc-rtems4.10/c/sis/make/Makefile.inc

To find out your current directory, use

pwd

My system is setup (the virtual box development image provided by RTEMS) so I am currently at

/home/rtems/rtems-4.10-work/b-sis

Putting those two together the absolute path of the Makefile.inc (for sparc-sis) is:

/home/rtems/rtems-4.10-work/b-sis/sparc-rtems4.10/c/sis/make/Makefile.inc

We now need to set the environment variable $RTEMS_MAKEFILE_PATH.  Assuming your shell is bash or some slightly compatible variant, environment variables start with a dollar sign ($) and one word (including underscores) in all caps.  Go ahead and edit your .bash_profile to set this variable:

RTEMS_MAKEFILE_PATH=/home/rtems/rtems-4.10-work/b-sis/sparc-rtems4.10/c/sis/make
export RTEMS_MAKEFILE_PATH

There is perhaps a better place to put this, but .bash_profile will work.  Your .bash_profile is located in your home directory (which you can get to by typing “cd ~”) and is run every time a terminal is open, ensuring that this variable is present.  Note that we pointed to the directory where the Makefile.inc is located, not to the Makefile.inc itself.

Now, let’s build some examples.  Go to the examples provided:

cd $B/examples-v2

And marvel at all the examples that have been provided.  You can run “make” from this directory but since we are on sparc-sis and explicitly disabled POSIX some of the examples will not build.  So let’s go and build the hello world example:

cd hello/hello_world_c
make

If you see an error about not being able to find some .cfg or Makefile.inc, you’ve set your path incorrectly.  If you see the error that it can’t find /Makefile.inc, then your environment variable isn’t set at all.  Try “echo $RTEMS_MAKEFILE_PATH” to see what the variable is currently set as.  If it’s in your .bash_profile but not showing, you need to reload your .bash_profile by running “. ~/.bash_profile”.  Try the echo again and hopefully you will see the environment variable set.

Assuming it all went well at this point, you can load up the executable in GDB.  If you don’t know much about GDB,  you should go and read up about how to use it. Fire up the RTEMS sparc GDB:

sparc-rtems4.10-gdb o-optimize/hello.exe

You should be greeted with the RTEMS GDB built for sparc. You’ll need three commands to execute the program:

tar sim
load
r

The first command “tar sim” targets the simulator for sparc.  (See “help targets” for more information and options.) The second loads the program.  The third “r” is short for “run”.  You should see the basic “Hello World” program.

Let’s edit the program to do something slightly more interesting and prove that our build environment is mostly sane.  Go ahead and edit test.c:

gvim test.c

And add a few lines to change what is being printed out.  Run “make” again to rebuild the executable.  Now fire up GDB as before and make sure the output has changed.  You can see my diff and my results below:

For a potential Google Summer of Code project, I’ll be working on porting the Parrot VM to RTEMS. Though the steps are well documented on the RTEMS wiki, I’ll follow up with my own step-by-step instructions.

I’m running Mac OS X 10.6.3 on a Mac Book Pro with a 2.4 GHz Intel Core 2 Duo processor and 2 GB of RAM.  While doing development work with Ruby on Rails, I’ll have an instance of postgres, nginx, debug consoles, MacVIM, Open Office, Google Chrome, and Safari all open at once. I really need to upgrade to 4GB soon, which, according to crucial.com will only set me back about $100.  The whole point of this rant is to ensure that you have a beefy machine because virtualization and compiling can really tax a system.

Most of this information will come directly from the page “Virtual Machines for RTEMS Development” with only a few modifications or clarifications.

1. Grab a copy of virtual box.

Virtual Box is Sun’s GPL’d virtualization software which allows you to “boot” another operating system (like Windows or Ubuntu) as an application within your OS.  Running another OS simultaneously on your computer is useful for testing and developing software that needs to target multiple platforms. Parallels and VMWare are also excellent choices for virutalization.

The download is a standard DMG file which you must double-click to mount.  Follow the instructions on the installer; total time for me was less than 5 minutes.

2. Grab a copy of the RTEMS development image.

What we could do is use Virtual Box to setup a new guest OS from scratch and then download the components that we need, setup the paths, and then build all the tools.  Instead, the RTEMS people have provided a development image – a snapshot of a system already configured.  At the time of this writing, the latest image was a Fedora Core 12 system updated March 12th, 2010.

Note that the image is distributed over BitTorrent, and if you don’t have a BitTorrent client I recommend you pick up Transmission.  The image is a hefty 2.5 GB and when I downloaded it there was only one person seeding the file, capped at around 70-80 kbps and the download took several hours.  If you require to download this file anytime soon, please comment on this post and I’ll begin seeding the file as well.

At some point in the future when I have free time, I’ll install from scratch on CentOS and document it (either with screenshots or perhaps with a screencast).

Total time: 5+ hours of downloading

3. Create the OS in Virtual Box

Once you have downloaded file, you’ll need to unzip it and move it to some decent location on your computer.  Next, open up Virtual Box and select File -> Import Appliance.  On the first page of the wizard click “Choose” and then navigate to your unzipped directory and select the .ovf file.  Accept the default settings of the wizard and wait while the file is imported.  Note that the progress bar is a bit skittish and for me, started with an estimate of finishing in 18 hours and then suddenly jumping to 30 minutes.

Total time: 20 minutes or so

4. Boot the OS

Once the OS has been imported to your system, boot the OS up.  This part was a bit unnerving for me because I was presented with a black screen for 5 – 10 minutes.  Eventually a three-part progress bar at the bottom of the screen showed up and the GUI began to load.

Total time: ~10 minutes

5. Log in

Sounds simple enough.  Logging in as root is disabled through the window manager so log in as the “rtems” user.  Password is “rtems”.

Total time: <1 minute

6. Bring the system up to date

Open up a new terminal and test if you have an internet connection:

ping www.google.com

Clearly there are better / more scientific ways to test if you have a working internet connection, but this is quick and easy.  You should be receiving pings back from google; if not, there is something wrong with your Virtual Box setup.  If this is working, go ahead and drop down to root:

su

And then update all the packages.  The “yum” command is the package manager that comes with Red Hat flavors.  The “-y” option assumes yes to every prompt.

yum -y update

This command brings all the installed RPMs up to date.  It will ask you for the root password which is “Virtual0″ (zero, not a letter).  When I ran it it had to pull down 10+ MB of index files, sat there for a long time without a response (again, making me think my system had died), and then pulled another 100+ MB of updated packages.

Total time: ~20 minutes

7. Update RTEMS repository

Many convient variables are already setup, listed on the wiki page here. Typing

cd $r

will bring you to the root of the RTEMS source tree.

Next, run the command to update the source files

cvs up -Pd 2>&1 | grep -v cvs

I myself have never used CVS – I started with Subversion and moved onto git with this current job – so I couldn’t say off the top of my head what that command is doing.  Looking the man page for cvs (by typing “man cvs”) I find that the -P option prune empty directories – that is it will not download empty folders – and the -d option searches all subdirectories for updates.  While I would have called that “recursive” (and hence would have expected a “-r”), I guess I’m just spoiled with SVN and git.  The 2>&1 redirects standard error to standard output.  The pipe passes all this output onto grep, and the -v option removes any line that contains “cvs”.  All you will see on the terminal is a list of files that are updated.

Total time: 5-15 minutes (I wasn’t really paying attention)

8. Bootstrap

Finally, run the bootstrap command:

./bootstrap

This is a script that will compile all of the source code.  The wiki page warns us that “[t]he bootstrap will probably take 10-15 minutes to execute depending on the speed of your computer.” Having wised up through the process, I ran:

time ./bootstrap

The time command will return the amount of time another command takes to run.  I got this output:

real    19m50.354s
usr     4m27.944s
sys     6m9.864s

I use to know at some point the distinction of user time and system time but now I have forgotten and don’t feel like googling for it at 2:20 a.m. All you need to know is that it took me 19 minutes and about 50 seconds of real time to bootstrap the entire system.

In the next post, I’ll walk through getting a “Hello World” example started.