Those first three steps are now possible with a set of patches to the Parrot configuration code. Now all that remains is to copy the source code to the build directory so we can just run `make`. I’ve tested configuring out of directory and building in directory and you get exactly the same Makefile and build and it passes all coding tests. The patch right now is a bit ugly and should be refactored into a separate module.

After this is done, I’ll work on getting a mechanism to copy over all the source so we can do a full build out of directory. Hopefully this will be committed to the source, and I can begin working on getting Parrot to cross-compile. Since I’ve already got Parrot compiling with the RTEMS tool-chain, after all this is done I’ll create a bootstrapping Perl script that’ll help install the source, get you a system copy of Parrot, a RTEMS cross-compiled copy and build libparrot for linking.

Parrot does not come with a Makefile but comes with a Perl script called Configure.pl that runs a number configuration of steps. One steps determines what CPU and architecture we are running, one checks for the sizes of native types like ‘int’ and ‘float’ and ‘double’. Some steps wrangle templates to generate Makefiles and source code while others compile programs to check how a system compiler actually handles certain pieces of code. Essentially these steps capture all the information necessary to configure Parrot to compile correctly on whatever system we are running.

The problem is that with RTEMS our development system will not necessary be the same architecture as the target platform. For example, I’m running Fedora Core 12 on x86 but my code and compilers target sparc. So these automated tests won’t run. We need some secret sauce to get Parrot to compile on different hardware architectures for RTEMS.

The first part of the secret sauce was overriding the default compiler, linker, and other parts of the tool chain. This was simple because Configure.pl can accept the overrides not only through the command line, but we can also specify an external file to hold our entire configuration recipe. This helps keep things organized, makes it easier to generate, and avoids the problem of a max command line argument length.

The second step was to disable as many of the auto configuration steps as possible. Again, this was simple because the external file can also specify which steps we want to skip. Last week I disabled every step which generated a lot of errors from the configuration script. Reading these errors showed me exactly what information was necessary and from there I could either provide the information in a hints file (in lieu of running the step) or I could re-enable the step (if it didn’t run hardware dependent code).

This week I finished the hints file and removed the last remaining configuration bugs. That means Parrot can now run the Configure.pl script with RTEMS variables and Configure.pl will generate a Makefile and not throw any errors. It’s not time to celebrate quite yet – many of the values are hard-coded into the hints file rather than probed from the RTEMS. And the build still hangs on a few wrong values in the hints file.

To alleviate the first problem would require a re-write of the secret sauce with either autoconf or with perl. Autoconf would be nice because RTEMS already uses it and there are probably many examples of how to get exactly the information I want. Perl would be nice because I already know the language unlike autoconf / m4.

A rewrite is definitely in the future, but for now I will work on getting the build clean and a ‘Hello World’ example in PIR running before that. Once I’ve worked out all of the problems there I will begin to work on getting the Parrot test suite running, rewriting the secret sauce, testing multiple hardware configurations, and then even playing with High Level Languages (HLLs) that run on top of Parrot like Rakudo (Perl 6 on Parrot) or Cardinal (Ruby on Parrot). Finally, I’d like to round things out with a series of tutorials (or even screencasts) showing how to install RTEMS, Parrot, and some HLLs on a blank target environment.

The good news is that this step is mostly complete. The configure script now recognizes which compilers we must use, and, best of all, this information is generated based on the RTEMS installation that you have (rather than a static recipe).

Parrot’s configuration, however, is more than just to determine which compiler to run but also includes automatic probes for architecture specific information, such as byte order or the size of an int. Since our target architecture is not our host architecture, we can compile these automatic probes but we cannot run them. So the hints file will contain all of the information that we need to compile Parrot but can’t figure out on our own.

Work on the hints file has begun by first overloading the standard libraries that are linked to. Again, this is because the host architecture (with which the libraries are built against) is not the same as the target architecture and we need a few RTEMS specific libraries as well.

Right now the configuration script runs but errors on a few of the steps. I hope that by next week the entire configuration script can run and we can begin trying to build Parrot on RTEMS.

As always, you can track progress at my github repository for the project or pinging me on IRC… look for bubaflub hanging out on #RTEMS on freenode or #Parrot irc.perl.org.

Parrot

Parrot is a virtual machine designed with dynamic languages in mind. Historically, Parrot started out as the virtual machine for Perl 6 Rakudo, but now it has many languages (or “High Level Languages” or “HLLs”) being actively developed to run on top of the virtual machine. Parrot comes with a number of tools to make implementing new languages easier – the Parrot Compiler Tools. Like other VMs, Parrot can compile down to platform independent bytecode (called PBC). Parrot was designed with “HLL Interop” in mind – to quote the docs,

Parrot is also designed to provide interoperability between languages that compile to it. In theory, you will be able to write a class in Perl, subclass it in Python and then instantiate and use that subclass in a Tcl program.

How cool is that? That means progress for one HLL is progress for all. Any optimization to the underlying architecture is an optimization for every HLL. Any low-level bindings will automatically work for all HLLs.

RTEMS

RTEMS stands for Real-Time Executive for Multiprocessor Systems. It is a real time operating system designed for embedded systems. RTEMS is free and open source and POSIX (1003.1b API) compliant. RTEMS comes with a full ANSI C, ANSI C++ (with STL), and ADA support. RTEMS has a full TCP/IP stack as well as other networking features like HTTP servers and support for RPC. Finally, RTEMS has full debugging support through GDB.

RTEMS is cross-compiled – that means the programmer might be on an x86 architecture but compile programs to run on Sparc. RTEMS currently supports 116 hardware configurations (“Broad Support Packages” or “BSPs”) on a number of chip architectures (x86, ARM, Sparc) which can all co-exist on the same development machine and run with the same development code.

Potential

I think the potential here is staggering – not only does RTEMS get new languages to target the hardware, Parrot gets new platforms to test and run on. Someone could write something in Rakudo Perl using classes from other languages like Python or Ruby or Tcl, compile to bytecode and then be run on a hundred or so different hardware configurations.

Current Work

Of course it’s not all puppy dogs and ice cream – there’s work to do! Things don’t quite work yet, but by the end of the summer I hope to have Parrot’s bleeding edge building on RTEMS and running the test suite on the BSPs. I’ll update this blog once a week with my progress after my Tuesday meeting with my mentors at the Parrot Foundation and RTEMS. The first work that needs to get knocked out:

1. Hints Makefile – This Makefile will extract all the BSP information necessary for compiling Parrot and then generate a hints file that will be used by Parrot at compile time.
2. Cross compilation support for Configure.PL – We need to be able to specify which platform we want to target at compile time for Parrot. Currently Configure.PL figures this out automatically by checking the system that we are running, but with cross-compilation the system we are running is not the system we are targeting.
3. Write blog post detailing plan of attack

Currently Crypt::Primes and Crypt::Random use Math::Pari for their mathematical processing. Bindings to Pari (through Math::Pari) are fragile and can slow Perl 5 development when new versions are released as bugs must be hunted. This project will move these two core modules to use Math::GMPz and Math::Primality (accessing the GMP library) which will provide identical functionality with less headaches. Since I do not own either of these modules I will provide a set of patches that the authors may choose to incorporate for this functionality.

Deliverables

• Crypt::Primes patches, documentation, and tests
• Crypt::Random patches, documentation, and tests
• The same for any other libraries that may depend upon them

Project Details

This project will attempt to remove Perl 5′s dependence upon Pari and the fragile Math::Pari bindings.  Last year through Google’s Summer of Code I developed Math::Primality to provide this type of functionality and now this project will integrate it into core modules.  The main idea is to remove the dependencies without having any noticeable affect to the end-user.  If there is insufficient tests or documentation, I will add them to ensure correct functioning and clarity. If time permits, I will profile the modules and attempt to optimize the performance of these core cryptographic libraries.

Project Schedule

April 26th – May 24th: Install and test Crypt::Primes and Crypt::Random as is.  Contact module author(s) to introduce myself and explain what is happening.  Determine if any other core modules need to be updated. I will also finish polishing Math::Primality (Issues #1 – #4 at http://github.com/bubaflub/math–primality/issues – Convert README to POD, use Dist::Zila, test with 5.12, and clarify documentation).

May 24th – July 12th: Work on patching Crypt::Primes.  Fill out any missing documentation and enhance test suite.

By July 12th: Have Crypt::Primes converted to use Math::GMPz and Math::Primality.

July 12th – August 9th: Work on patching Crypt::Random.  Fill out any missing documentation and enhance test suite.

By August 16th: Merge patches if possible.  Polish code and fill out test suite and documentation.

If I finish early or am ahead of schedule I will move on to other Crypt::* modules that are commonly used and depend upon Math::Pari, providing the same documentation and tests as above.

References and Likely Mentors

I’ve worked with Jonathan Leto on last year’s Google Summer of Code and work with him on Parrot currently.

License

Same terms as Perl itself.

Bio

My name is Bob Kuo and I am a 24 year old graduate student living in Champaign-Urbana, IL.  My undergraduate degree was in Math and Computer Science with classwork focusing on Number and Coding Theory, Numerical Analysis, and Algorithms.  I participated in last year’s Google Summer of Code with The Perl Foundation writing Math::Primality and finished ahead of schedule.  I have been involved with Parrot since the end of last year’s Google Summer of Code and am an active committer.  I currently work part-time as a web developer where I support a legacy PHP system and develop in Ruby/Rails.  I have been using Perl in web development and system administration for 4+ years.  I’ve contributed bug fixes to the CPAN modules CGI and Math::GMPz. You can see my open source code at github.com/bubaflub.

Eligibility

I am currently a full time student in graduate school at an accredited institution living in the United States.

Parrot currently has two PMCs that wrap functionality in GMP: BigInt and BigNum. This project will move these PMCs out of the main parrot repository and replace them with bindings to the GMP library. The project will focus on getting main integer and floating point functions over from GMP, replicating in part Math::GMPz and Math::GMPf.

Benefits to the Perl/Open Source Community

Some members of the parrot community have expressed their disgust with the current BigNum implementation.  Not only would this project provide a more elegant and efficient implementation, but GMP bindings would be available to any language implemented on top of Parrot – hence Rakudo would gain “free” bindings to GMP.  GMP bindings are also foundational for any future cryptographic libraries that may need to be written.

Deliverables

• BigInt and BigNum PMCs based upon new GMP bindings that do not break current
• Access to crucial GMP integer and floating point functions through NCI
• Tests for every accessed function
• Documentation for every function
• Example code

Project Details

This project will provide access to the GMP library through the NCI (native call interface).  The focus will be to replace the existing BigInt and BigNum functionality by refactoring the PMCs to use the GMP NCI. The design will allow future work to bind to more functions or be easily modified if the GMP API changes in the future (though this is unlikely – GMP is fairly stable).  Though the bare-minimum amount of functionality required for BigInt and BigNum is proposed here, depending on time and speed of development I will essentially bind as many functions as I can.

Project Schedule

April 26th – May 24th: Begin and (hopefully) finalize design of new PMCs and GMP interface.  Start branch to move BigInt and BigNum out of main parrot repository.

May 24th – July 12th: Begin binding to necessary integer and float functions in GMP.  Write tests and documentation.

By July 12th: Have at least one of the PMCs functionally equivalent with it’s previous incarnation.  Have tests to prove this and documentation to explain it.

July 12th – August 9th: Finish second PMC and necessary NCI bindings.

By August 16th: Merge branch if it hasn’t already been merged into trunk.  Polish tests and documentation.

If I finish early or am ahead of schedule I will provide bindings, tests, and documentation to some of the other integer and float functions.  If I’m speeding through this, I will also provide the same for the GMP rational functions.

References and Likely Mentors

I’ve worked with Jonathan Leto on last year’s Google Summer of Code and work with him on Parrot currently.  Andrew Whitworth would also be a good mentor for this project as he works with parrot-linear-algebra and the HLL matrixy.

License

Artistic license 2.0 (same as Parrot)

Bio

My name is Bob Kuo and I am a 24 year old graduate student living in Champaign-Urbana, IL.  My undergraduate degree was in Math and Computer Science with classwork focusing on Number and Coding Theory, Numerical Analysis, and Algorithms.  I participated in last year’s Google Summer of Code with The Perl Foundation writing Math::Primality and finished ahead of schedule.  I have been involved with Parrot since the end of last year’s Google Summer of Code and am an active committer.  I currently work part-time as a web developer where I support a legacy PHP system and develop in Ruby/Rails.  I have been using Perl in web development and system administration for 4+ years.  I’ve contributed bug fixes to the CPAN modules CGI and Math::GMPz. You can see my open source code at github.com/bubaflub.

Eligibility

I am currently a full time student in graduate school at an accredited institution living in the United States.

Google Summer of Code Program 2010 Project Proposal

Name: Bob Kuo
Email: (removed here but in my official proposal through Google)
University: Urbana Theological Seminary, Urbana, IL, USA

Project Abstract

This project seeks to enable code written in parrot to run on the RTEMS platform.

Project Description

Parrot is a multi-language, multi-platform virtual machine. The goal of this project is to enable parrot to configure and build for a variety of RTEMS BSPs. Currently, parrot cannot build out of directory nor can it cross-compile. This project would provide the necessary improvements to the parrot build system to enable parrot code to target RTEMS.

Project Deliverables

1. May 23 (coding begins) – I already have a development environment setup with RTEMS successfully building and compiling on sparc-sis. I will be focusing on an x86 target and I will also attempt to get Chris’ previous work on Parrot + RTEMS up and running.
2. June 15th – Work will begin on a testing framework to ensure that parrot is behaving correctly within RTEMS.
3. July 12th (Midterm Evaluation) – Chris’ work will be brought up to date so that Parrot + RTEMS can build of trunk with a small set of patches against the Makefile.
4. August 9th (Final Evaluation) – Testing framework is in full swing and bugs are being fixed in both parrot and RTEMS. Cross-compilation support is merged into the parrot trunk.
5. August 23rd (Final Results Announced) – Code is polished, documented, and tested.

Proposed Schedule

April 26 – May 23 2010 (study and prep)

I am less familiar with RTEMS and CVS (I have only used svn or git) so I will be studying the RTEMS system. I will work primarily with the x86 architecture and the i386/pc386 BSP as Parrot already runs very well on this architecture. I also at this point would like to get Chris’ previous work running on my test environment.

May 24 – June 15 2010 (testing)

Assuming I can get Chris’ previous work to build, I will be devising a way to get the parrot test suite to run on RTEMS. Most likely this will be limited to the “core” tests at first.

June 16 – July 12 2010 (trunk)

Once Chris’ work and a testing framework is in place, I will be attempting to get the latest parrot to build on RTEMS. Parrot 2.3.0 will be a longer-term supported release and work will be based off that release.

July 13 – August 9 2010 (patches)

Documentation and tests for both Parrot and RTEMS will be provided throughout the entire period. I hope to have a cross-compilation branch merged into Parrot trunk so that a set of patches against a Makefile are no longer necessary.

August 10 – August 23 2010 (cleanup)

Clean up code, improve documentation, and package any patches or extra files for delivery.

In the even that I’m ahead of schedule:

If I am ahead of schedule during and Parrot is running cleanly on x86 I will begin to extend support to other BSPs. This will involve someone at RTEMS who already has a large number of BSPs built on their system and using the development build farm. While Parrot 2.3.0 will be a longer supported release, it would be ideal if RTEMS+Parrot could track Parrot’s ongoing development.  Programs will be developed that will allow regular automated testing of RTEMS+Parrot.

Continued Involvement

Assuming that the project is a success, I can continue to maintain and support and improve Parrot on RTEMS. Since my work will most likely take me into the guts of RTEMS, I will be contributing to RTEMS for some time to come.

Future Improvements

1. Remove compiler and linker warnings
2. Fix broken tests
3. Support more BSPs

Major Challenges foreseen

My understanding of Makefiles is a bit shallow and cross-compilation is very new to me. I’ll need some expert help in determining how to best modify both Parrot and RTEMS build chains.

Possible Mentors

As Chris Johns has previous work on RTEMS+Parrot, he would be most helpful in mentoring me.  Dr. Joel Sherrill has been extremely helpful on IRC. Jonathan Leto (with Parrot) has worked on the Parrot side of this equation and would be helpful as well.

References

Chris’ previous work: http://www.rtems.org/ftp/pub/rtems/people/chrisj/parrot/
Parrot ticket for cross-compilation: http://trac.parrot.org/parrot/ticket/1516
Branch on parrot for cross-compilatoin: http://trac.parrot.org/parrot/browser/branches/tt1516_builddir_option

Relevant Background Experience

I have been a Parrot developer since the end of last Google’s Summer of Code and have commit access. Last year I worked with the Perl Foundation to develop Math::Primality and my work can be seen at http://github.com/bubaflub/math–primality.

Personal

I am a 24 year old grad student living in Champaign-Urbana, IL. My undergraduate degree was in Math and Computer Science with course work focusing on Number and Coding Theory, Numerical Analysis, and Algorithms. I work part-time as a web developer.

How did you learn about RTEMS?

From parrot people.

Anything else that makes you special:

I have a strange light-blue birthmark towards the top of my forehead.

Experience

Free Software Experience/Contributions:
I am a committer to parrot, have a module on CPAN (Math::Primality from last year’s Google Summer of Code), and have committed bug fixes to various open source projects (e.g. CGI.pm and Math::GMPz).

Language Skill Set
My part-time job focuses on PHP and Ruby. My open source work takes me mostly towards Perl, and have been working with Perl in either web development or system administration for 4+ years. Previously I worked in C/C++ and Java.

Related Research and Work Experience (if any):
Besides some parrot hacking and playing around with ANTLR for parser generation, nothing significant.

Reference Links and Web URLs (optional):

blog.bobkuo.com
IRC handle: bubaflub
github.com/bubaflub

IPv4, IPv6, MACADDR Native Column Types

Student:

Robert Kuo

Abstract:

Removing the UNSIGNED column type from Drizzle forced IP addresses to be stored as BIGINTs which is inefficient. This project will implement the IPv4, IPv6, and MACADDR native column types for Drizzle and the functions INET_NTOA() and INET_ATON(). This project will include inline documentation, patches to the official documentation, and test cases that verify correct operation and error handling.

Content:

Project Title
IPv4, IPv6, MACADDR Native Column Types

Name:

Bob Kuo

Email:

(removed from here but in my official proposal sent to Google)

IRC:
bubaflub

Synopsis

This project will implement the IPv4, IPv6, and MACADDR native column types as well as the functions INET_NTOA() and INET_ATON() as well as the documentation and test cases to support this new functionality.

Benefits to the Drizzle Community

According to a message on the mailing list this feature has already been requested by the community. As well as a felt need, this would bring Drizzle closer to MySQL functionality and compatibility.

I’ve worked with MySQL for more than 4 years and would love to get involved with Drizzle but haven’t had the time or resources to do so. I love the idea of a MySQL that is lean, community-driven, and cruft-free and Drizzle seems to fit the bill.

Deliverables

• The INET_NTOA() and INET_ATON() functions that convert between dotted-quad and an integer network address.
• The IPv4, IPv6, and MACADDR native column types so that one could type: CREATE TABLE sessions ( id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY , ip_address IPv4 NOT NULL );
• Test cases that prove correct operation and error handling of the functions
• Documentation in whatever format is suitable

Project Details

This project will focus on the functions and then use them internally to convert IP addresses to a space-efficient binary format – e.g., binary(4) for IPv4 and binary(16) for IPv6.

Project Schedule

Before the start date (April 20th – May 23rd)

• Meet Drizzle community
• Familiarize myself with Drizzle internals and build process
• Familiarize myself with bzr (I usually use git)

First half (May 23rd – July 13th)

• Spec out functionality with documentation and write failing tests
• Write INET_NTOA() and INET_ATON()
• Implement the IPv4 native column type

Second half (July 13th – August 10th)

• Finish documentation and test cases
• Implement the IPv6 native column type
• Implement the MACADDR native column type

In the event that I finish early:

• More documentation and tests
• Any optimizations that profiling or benchmarking reveals
• Any other utility functions related to these column types
• Work on the testing framework

Bio/Experience

My name is Bob Kuo and I am a 24 year old currently pursuing my masters and living in Champaign-Urbana, IL. My undergrad was in Math and Computer Science at the University of Illinois at Chicago with my classwork focusing on algorithms, coding and number theory, and numerical analysis. I have used MySQL for more than 4 years in both a developer and DBA role. I participated in last year’s Google Summer of Code working for The Perl Foundation and successfully finished the project ahead of schedule. My project code is available on the CPAN and at http://github.com/bubaflub/math–primality. I currently work part-time as a web developer for a local company and work in PHP and Ruby and Rails and can work in Perl, PHP, Ruby, C/C++ and with both MySQL and PostgreSQL.

All comments, critiques, or feedback are most welcome.

Thanks,

Bob Kuo

Implementation of algorithm to infer gene duplications in BioRuby

Student:

Robert Kuo

Abstract:

This project will implement an algorithm to detect gene duplications in BioRuby described in Zmasek and Eddy, 2001, “A simple algorithm to infer gene duplication and speciation events on a gene tree”, Bioinformatics, 17, 821-828. The project will include full documentation, tests, and examples.

Content:

Name: Bob Kuo
(the following have been removed from here but they are in my official proposal to Google)
Address:
Email Address:
Mobile Phone:
IRC Handle: bubaflub

I am 24 and currently pursuing my masters and living in Champaign-Urbana, IL. My undergraduate degree was in Math and Computer Science from the University of Chicago at Illinois, with my coursework focusing on number and coding theory, numerical analysis, and algorithms. I am currently employed part-time as a web developer working in PHP and Ruby on Rails. I participated in last years’ Google Summer of Code by working with the Perl Foundation (http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/dukeleto/t124022226790) which was successfully completed ahead of schedule.

I am interested in BioRuby because I have always had an interest in combining the sciences with programming and am interested in learning more about evolutionary biology. I believe I am well-suited for this project not because I am the best Ruby programmer in the world, but because I am willing to learn and work through algorithms step-by-step. There is a reference specification written in Java from which I can test my Ruby project. My work as a web developer has required me to be multi-lingual and am comfortable reading and writing Java, Perl, PHP, and Ruby.

You can see my open source work at http://github.com/bubaflub, and would recommend looking through http://github.com/bubaflub/math–primality, my work from last years’ Google Summer of Code with The Perl Foundation which has significant in-line documentation and many tests.

All code developed for this project would be released on GitHub and available under the same terms as BioRuby itself.

Plan:
Before the start date (April 20th):

1. Meet the BioRuby and Open Bioinformatics community
2. Familiarize myself with the BioRuby package and the phyloXML format
3. Read necessary papers (such as http://bioinformatics.oxfordjournals.org/cgi/content/abstract/17/9/821)
4. Familiarize myself with the Java implementation of the algorithm (http://www.phylosoft.org/forester/applications/sdi/)
5. Discuss and set expectations of code – dependencies, code style, tests, documentation, etc.

First half (May 23rd to July 13th):

1. Decide on which libraries
2. Spec out all necessary components with documentation and failing unit tests
3. Write integration tests that cover the entire algorithm (i.e. if I input “A” I should get “B”)
4. Begin implementing the algorithm

Second half (July 13th to August 10th):

1. Finish implementing the algorithm
2. Finish documentation and tests

In the even that I finish early:

1. Profiling and speeding up existing code
2. Extra documentation and tests
3. More examples
4. Extend the algorithm to use non-binary species and gene trees.

Obligations
I will continue to work part-time during the summer and may take summer classes. Last year these obligations were not problematic and did not affect my performance. During the day though I will be at work I will be available via email, IM, and IRC.
I welcome any feedback, comments, and critiques to this proposal.

Thanks,
Bob Kuo