Monday, January 12, 2009

SciPy 0.7 coming soon . . .

After 16 months of hard work the next stable release of scipy is almost ready to be tagged. This is the most significant scipy release in several years. It contains many new features, numerous bug-fixes, improved test coverage, and better documentation.

The scipy developer community started porting scipy to numpy at the end of 2005. Most of the work during 2006-2007 was focused on porting and bug-fixes and culminated in scipy 0.6. However, since the 0.6 release, the development effort has shifted from porting and maintenance to a much greater focus on infrastructure, architecture, and functionality.

One of the most important developments during the last year has been the extensive work on the testing and documentation framework. The improvements to testing and documentation first appeared in the numpy 1.2 release.

Our new testing infrastructure is based on the nose testing framework. This testing framework makes writing tests much easier than previously. The simplified testing framework as well as an increasing recognition of the importance of unit testing among the scipy developers has led to a doubling of the number of tests since the last stable release.

This release also brings immense documentation improvements. You can now view the scipy reference manual online or download it as a PDF file. The new reference guide was built using the popular Sphinx tool. We have also updated the scipy tutorial, which hadn't been touched for several years. Both the reference manual and the tutorial are easily editable using our web-based documentation editor. If you find want to improve the documentation, please register a user name in our web-based editor and correct the issues.

In addition to huge improvements in the testing and documentation infrastructure, this release cleans up a number of annoyances and removes old cruft. There have been a number of deprecations and well-documented API changes in this release. This is also the first stable release since the sandbox was removed.

We also did a fairly extensive review of the code to make sure that all the code is correctly licensed. About a year ago, I noticed that there was code not licensed under the revised BSD license. I was trying to "correct" the license information for the scipy Fedora package, but was told that the Fedora packaging policy required including all licenses found in a package. After doing some grepping and looking at svn commit logs, it was easy to figure out what code needed to be relicensed and who had committed the code. Fortunately I was able to track everyone down and everyone kindly agreed to relicense their code. There was also some code included in scipy derived from "Numerical Recipes in C" code. Unfortunately, the "Numerical Recipes in C" code doesn't permit redistribution. The scipy developers quickly reimplemented the offending code from scratch. With the 0.7 release, scipy only includes code licensed under the revised BSD.

The 0.7 release should be out by the end of the month and we have all ready started working on the next feature release. During the development of the 0.7 release, there has been a rapid increase in community involvement and numerous infrastructure improvements to lower the barrier to contributions (e.g., more explicit coding standards, improved testing infrastructure, better documentation tools). I look forward to seeing this trend continue and invite everyone to become more involved. To learn more about the changes in the 0.7 release, please see the release notes.

2 comments:

Steve said...

Jarrod:

Congratulations to the whole SciPy community for this mammoth effort!

Many many Python users will benefit from your work, and once again Python will be brought to the attention of the scientific community of a compelling way.

Good luck with the final release. Let me know if there's anything the PSF can do to help.

Unknown said...

s this greatest effort by him will be fruitful to all linux users...........