PXE Bulk Install Proposal

by Max Winderbaum


Abstract:


My objective is to enable a common NetBSD user to configure and execute a bulk install of NetBSD on any number of networked machines using PXE booting. The user will be able to create custom defined machine configurations for application to different machines.
The PXE Bulk Install system has already been created, but is in a rough, virtually undocumented state. My work will involve simplifying the process of configuration as well as writing extensive documentation.


Content:


About Me: Please view my online resume at http://maxhart.thruhere.net/ for information on myself and my accomplishments. I am extremely interested in developing open source software one day, and believe the completion of this project will prove more than worthwhile for the NetBSD community.

A crucial definition: Throughout this proposal I will be referencing “classes” of computers. A class is simply a group of computers that are configured identically due to a similar purpose or usage.

What I hope to accomplish: By the end of the summer my mentor and I will have submitted, as part of NetBSD, a well-organized and well-documented package. The final system will consist of a collection of source scripts as well as an executable program that will reside in the NetBSD /src/distrib/ directory, and will not be compiled automatically as part of NetBSD. A user wishing to configure and run a bulk PXE install will run make in the directory of the system (lets call it 'bulk'), which will install a tree with subdirectories along the lines of /src/distrib/bulk/Root/architecture-version/netroot/distro (an explanation of the meaning of these directories follows). The user will then run the execuable included in the final system, which will automatically create customized classes for various architectures, distributions, etc. specified by the user.

Based on the architecture for each class, the final system will create a 'netroot' to be mounted as an NFS master root on the remote machine. This 'netroot' will contain the directories necessary to run NetBSD while the remote machine is prepared in addition to a directory ('distro') that is class specific and contains all source files, packages, etc. for its class. The remote machine will then mount its local hard drive as part of the remote file system, format it, and use the highly customizable 'distro' folder to unpack and install the assigned class (for more information on how the 'distrib' folder and classes could be customized, please review the main proposal).

This package will allow a user to easily set up and configure a bulk install of NetBSD onto one or more (think thousands) machines using PXE Booting. The user will begin by creating one or more installation “classes.” A “class” consists of a custom set of packages, configuration files, RSA keys, etc. that will be used in a fresh installation of NetBSD, in addition to the standard OS. The package will then, through the automatic detection of other computers on the network, present these machines to the user, and the user will be able to apply the already created classes to the machines as he or she pleases. This will be a user-friendly interface, involving as much automation as possible and as little confusing configuration as possible.

PXE booting is currently available as part of NetBSD as well as most other UNIX operating systems, but the aim of this project is to automate as much of the tedious administrative work as possible by creating exactly what the user would like to see on each machine based on its usage with the press of a button.

The part I will play in achieving this: I would like to accomplish my goals as part of a collaboration with Dr. Philip Nelson of Western Washington University (phil@cs.wwu.edu), a known contributor to NetBSD. Dr. Nelson has already created a system of scripts and configuration files that accomplish the bulk installation of NetBSD and allow him to create his own classes. I will simply be adding onto his existing work to make it viable for release to the NetBSD community.

My work will consist of:

I will be adding the following features and capabilities to Dr. Nelson's working version:

  1. Add a simple way to select and prepare packages for installation: The current version of the Bulk Install system has a simple configuration file with the names of the packages to be installed. I will add a more user-friendly interface to this configuration file, possibly involving the NetBSD repositories. After the user has selected the packages needed for a specific class, I will automate the preparation of those packages for fast installation.

  2. Add a simple, interactive way to create new classes: My class creation configuration will consist of a series of interactive questions put forth to the user that will fill in the necessary scripts and files for each class. The user will be exposed only to a simple visual representation of the packages, configuration files, and/or RSA keys he or she has selected.

  3. Add autodetection of network computers to the system: By pinging and arping machines attached to the PXE server automatically to detect whether they are online and prepared to recieve TFTP messages, I will cut down severely on the headache of ensuring each machine is ready to go every time.

  4. Add a quick way to group network machines into classes: I will allow the user to visually choose and then confirm which machines will be installed with which classes as part of my interactive configuration process.

  5. Extend the system to deal with multiple releases and architectures: At this time the system can only deal with one release for one architecture at a time: each new release/architecture combination takes an entirely new system that is maintained differently. I will expand PXE Bulk Install to include automation for different NetBSD releases and different architectures, including but not limited to sparc-5.0 , i386-4.0, i386-5.0, and shark-5.0. PXE Bulk Install systems for different release/architecture pairs will be maintained under a single root directory as part of my program, and a “class” will include a release and architecture for the new machine in addition to its other data.

  6. Make the configuration of a system easier for a new NetBSD distrobution: The current version of the Bulk Install system becomes extremely tedious and finicky when changing to a new distrobution of NetBSD due to an extreme lack of automation in the configuration and scripting processes. I will add the automation needed to cut down on the time and headache involved in upgrading many networked machines to a new distrobution.

  7. Create documentation on the use of the system: The current version of the Bulk Install system contains a few extremely cryptic notes by Dr. Nelson on the administration and configuration of the system that are not immediately obvious to the average NetBSD user (an example can be found at http://maxhart.thruhere.net/pxebulk/curr_doc.txt ). The goal is to create extensive documentation whereby a typical NetBSD user can implement this system.

  8. Submit the completed package to the NetBSD repositories: By the end of the summer I will have a completed work that will be ready for release to the NetBSD community.

Schedule for completion: I intend to complete my project in three months time. I will create goals for myself by allocating slightly less than two weeks for each step above. At the midway mark, I will have committed my code up to step four, i.e. the new system will interactively prompt the user to create complete classes and then assign these classes to machines on the network that are ready to recieve a PXE install, all in a user-friendly fashion. The next half of my time will be spent finishing my project, with the submission of the completed work taking place in the final days.

What I need to accomplish my goal: First and foremost, I will need the aid of Dr. Nelson, as I don't understand the details of how his system works to the degree I will require. I am moderately versed in scripting for Unix as well as creating clearly worded language that explains what can be complicated ideas or processes. In addition, I feel I am more than capable of writing networking code in C for NetBSD, (please see resume for more) which will prove necessary as I delve further into the project.

As for hardware requirements, Dr. Nelson has pledged several machines from the Western Washington University server banks for testing that span a wide range of architectures (for use in step five above) and I personally own four i-386 machines I could use for testing at home.

Why I am the best person to accomplish this: I have experience working by myself and with mentors as well as a great track record of accomplishing what I set out to do. I have experience in leadership positions, and am extremely motivated to do well given my aspirations of involvement in the open source community. I have the resources, ability, drive, and help needed to complete this project in a timely and organized manner. Thank you for considering my proposal.