Archive for the ‘linux’ Category

How a Nonstandard OS Led to Unix Standards

October 11, 2019

From 1988 through 1992, I worked as a system administrator for the Environmental Defense Fund in Washington, DC. As a nonprofit, it was too expensive for them to provide each employee with a PC, and Windows and PC networking were primitive back then, so their offices around the US used Wyse 50 terminals connected to a single computer in the copier room. When I arrived, that computer was a 68020-based Charles River Data Systems Universe 68, running their UNOS operating system.

1988-04 044-13 CRDS Universe 68-35T edfdc-Dfine-close


Command-Line Utilities on UNOS

UNOS was sort of like Unix System V if you squinted, but it changed many of the details, so porting software to it ran into all kinds of incompatibilities. The command line utilities were maddeningly deficient and different from the 4.2 and 4.3BSD Unix I had used at college. Apparently CRDS was trying to make Unix more user-friendly or more like some other operating systems; their version of cp was named copy, dd was called debe (a pun on girls’ names?), etc. and the command line options were multiple characters long. Completely useless for running shell scripts written for Unix.

I started looking around for a way to replace them. I found the source code for half-finished versions of some utilities in the GNU source tree at MIT, where I had access as a volunteer thanks to my friendship with Mike Haertel, a former roommate who wrote GNU grep and diff. I finished them and wrote some more missing utilities, and thus started the GNU fileutils, textutils, and shellutils, which were later all rolled up into the coreutils. I also ported GNU Emacs to UNOS, including its tricky “undump” method of restoring a RAM image from a file for faster startup. EDF paid me to spend some of my time on this work, because they wanted more usable computer systems themselves.

So a good deal of GNU/Linux code was written to make up for the deficiencies of UNOS. After a few years, EDF switched to timeshared 486 computers running SCO Xenix, then SCO Unix, and we ran the GNU utilities on them, too, because they were better than the SCO versions. EDF finally went to networked Windows PCs for everyone, but they’re still using GNU/Linux for their web presence, at least, and using descendants of the utilities they funded.

Kermit on UNOS

Long before SSH, Kermit was a popular file transfer protocol which worked over both serial connections, like modems, and network connections, like TCP/IP. It was often paired with terminal emulators. It was implemented for nearly every type of computer made in the 1980s and 1990s, in many programming languages. Yes, it was named after the muppet, with permission.

EDF needed a good way to transfer files to and from its UNOS systems (besides UUCP), so I ported C-Kermit (the version for systems that use the C programming language) to UNOS. For that, I had to deal with many quirks in UNOS system calls.

Standards Because of UNOS

Mike O’Dell, Internet pioneer, related on an email list that the incompatibilities of UNOS led to the creation of the UNIFORUM association and the POSIX standards, so the US government wouldn’t have a sole source for Unix products that were interoperable. Thus, the obscure and quirky OS from CRDS contributed to creating both the official standards for Unix and what is now the most-used implementation of them.

Why Do Long Options Start with Two Dashes?

August 2, 2019

Around 1990, Richard Stallman (RMS) and I were writing the GNU C library getopt() and he wanted to extend it to support long (multi-character) option names for user-friendliness. He considered Unix inferior in this regard to other operating systems such as TOPS-20 which supported long options (that could be abbreviated). He wanted GNU to be better than Unix while still compatible. There were a few programs that ran on Unix systems and used long option names starting with either - or no prefix at all, such as find, but those syntaxes were not compatible with Unix getopt() and were parsed by ad-hoc code.

Long options needed a prefix that wouldn’t clash with the Unix conventions, so programs could support both types of options without ambiguity. Richard chose +, since logically if - (for a small number mathematically) is for short options then + would be for long options, and it’s no additional typing. We created an extended interface called getopt_long() to support specifying long options.

But when the IEEE POSIX shell and utilities standard was published in 1992, the + syntax was disallowed. GNU developers discussed what to do over email. We considered -+ as the long options prefix, but that was hard to type, so we settled on --, which wouldn’t violate POSIX or Unix compatibility and wasn’t hard to type.

For a few months, GNU getopt() supported both + and -- to allow time for people to transition their scripts. I’m pretty sure support for + during that transition could be disabled by setting the environment variable POSIX_ME_HARDER (RMS’s exasperation with standards showing there), later changed to the more polite POSIXLY_CORRECT, which also disables recognizing options intermixed with positional arguments.

Because GNU software was popular and the solution was logical, everyone else adopted the -- prefix and implemented variations of it in their own argument parsers. Perl was probably one of the first.

Do What You’re Good At

August 8, 2008

In 1993, I unexpectedly did some good when I was hired to do something I wasn’t good at.

I was going to college part-time and working to pay my way. For almost five years, I had been writing and maintaining various utilities that were part of the GNU operating system (now used mainly with Linux). For a couple of those summers I’d actually been employed as a programmer by the Free Software Foundation, the nonprofit organization that coordinates the GNU Project.

I was looking for year-round part-time work (full-time in the summer), so I got in contact with Cygnus Support, a company founded to do for-profit work on GNU code. (It’s now part of Red Hat.) It seemed like a logical fit. After a telephone interview, I was hired. I spent the summer at the beautiful Cygnus offices in Mountain View, California; here is what the front entrance looked like:

Cygnus Support entrance

Cygnus was looking for someone to maintain the GNU linker/loader ld, part of the binutils compiler toolchain that their employees had written. The guy who wrote GNU ld, Steve Chamberlain, is a brilliant programmer several years older than I am. He had written it for a contract with an insane deadline, and then had to move immediately to another project with another insane deadline in order to keep bringing in enough income to keep the company afloat. In the meantime, he hadn’t had time to do much documenting of his code. And he didn’t have time to explain it to me, either.

Although I was working on a CS degree, I hadn’t taken a compilers class yet. When I started looking at the source code to ld, I was horrified to discover that it was all a hand-written parser for a complex linker language derived from System V Unix (but extended so it was even more complicated). I’d chase function calls and pointers from here to there but could never figure out what the control flow was. It was way more complicated than I’d expected. I never did understand most of the code.

I got so frustrated that weekly I’d rush into my boss’s office in tears saying I couldn’t do it. She’d reassure me and I’d go stare at the code some more. I did learn enough about the code to fix some isolated bugs, but the hard ones I’d always find ways to pass off onto more experienced coders on the team, even though they weren’t supposedly working on the linker.

While I was struggling to understand the code, I learned a significant amount about the BFD library that the Cygnus binutils are based on, and there I started to make a difference. Some things I am good at are making things more consistent and making things more user-friendly. Cleanup work. I’d done that a lot on other GNU utilities for the FSF. I started improving the BFD library; I added some missing functionality, documented it better, and wrote a new utility to make it easier to analyze files built with it.

While doing that, I also took a close look at the configuration scripts that the binutils used. They were designed to support many computer architectures, but a lot of the configuration had to be done by hand. For the FSF utilities, I had written a system called Autoconf to generate automatic configuration scripts, but those couldn’t handle CPU architecture selection. I decided to merge the two systems into a best-of system (Autoconf 2), and I spent much of the summer of ’93 doing that and converting all of Cygnus’s utilities to use it. It got an enthusiastic reception and I believe it’s still in use 15 years later.

While improving and documenting¬†Autoconf, I examined the Texinfo documentation system that GNU and Cygnus used. There were some scripts for printing out manuals in various formats, which involved some tricks with indexing and Postscript. I made some improvements to the documentation tools that summer, but I’ve forgotten exactly what they were.

After a few more months, I left the job at Cygnus to work as a programmer and system administrator for the University of Maryland. Cygnus deserved someone more suitable to work on the linker. But though I was just treading water as a linker maintainer, Cygnus got some valuable improvements to their software surrounding the linker that no one else probably would have done. And I enjoyed doing that.

It seems like this happens at most of the jobs I’ve had. I end up redefining the job description to be things I’m good at, and everyone’s pretty happy with how it turns out.