Pubnix Access Systems: A Pioneering Web Hosting Service

October 11, 2019

The Context

It was the early 1990s.

The Internet was beginning its transition from supporting only U.S. government and university research to supporting ordinary businesses. The main applications on the Internet so far were email, Usenet news, and anonymous FTP. The World Wide Web was still a primitive research project. Gopher and WAIS were obscure and of little consequence.

Most homes had nothing faster than a 28.8kbps dial-up modem with which to access the Internet or other online services like bulletin boards or AOL.

Microsoft Windows was still only a fragile shell over MS-DOS. Linux was still an obscure and immature hobby project. Neither was suitable for running a server on. MacOS X was almost a decade away. But BSD Unix, a favorite operating system at universities, had recently become available on the PC platform.

The Beginning

A group of friends who had worked together for the University of Maryland, College Park as student programmers and system administrators were trickling out into the real world. Many of them reunited in Fairfax, Virginia at UUNET Technologies, the first commercial Internet service provider.

When Kurt Lidl left UMCP for UUNET (with a stop along the way at SURAnet), he had an inspiration. Most dialup services of the time, such as AOL, were oriented toward home computers and were limited to their own little world, with no connection to the larger, growing world of the Internet. Kurt wanted to create a public-access Unix system that people could dial up to, login to a shell account, and connect to the Internet and use email. He came up with the name “Pubnix” for this public Unix service. Although Pubnix started out as a separate venture, Kurt soon sold its assets to UUNET, and it became a small semi-autonomous division of UUNET with Kurt managing it.

Soon Josh Osborne and Dave MacKenzie joined Kurt at UUNET and helped him start developing Pubnix. They wanted it to be as automated as possible, so they created a system to generate configuration files with Perl programs using an SQL database, University Ingres (the only DBMS at the time that ran on BSD Unix). Hours were spent at whiteboards designing the database schema and provisioning systems for users and servers. Within a few months, the service was up and running on a few DEC Pentium PCs running BSDI BSD/OS in UUNET’s offices at Fairview Park in Fairfax. The BSDI company had started in the UUNET offices, so there was a close connection from the beginning.

There were jokes about creating public terminal rooms and combining them with a microbrewery to create a “brewpubnix.”

Additions

As they were designing Pubnix and building the technology to run it, the Internet was changing. The World Wide Web gained enough capabilities to be a useful tool for companies to use, not just researchers. Kurt realized that they needed to change directions, as shell account systems were becoming obsolete, and that the Pubnix shell accounts technology could form the foundation of a shared web hosting platform, one of the first in the world. So they adapted their configuration system to provision a version of the NCSA httpd customized to support virtual hosts, and Josh modified the kernel of the BSD/OS operating system to support IP rate limiting by virtual IP address, so each customer could have a certain amount of bandwidth. By late 1994, UUNET’s virtual web hosting product had customers. Unlike GeoCities, which also started in 1994, each customer’s web site had its own domain name. Around that time, Chris Ross came to UUNET from UMCP and joined the group.

As customers were added, the Pubnix platform grew in capabilities. University Ingres was replaced by a port of the XDB SQL database which Kurt commissioned. The NCSA httpd was replaced by the more flexible Apache web server. SSL connections using the Netscape web server and Real Media hosting on Linux were added.

Dave MacKenzie had the idea to add virtual FTP hosting, and in a few days he modified the WU FTPD to support virtual hosting, added database and Perl script support, and presented the completed product to UUNET’s marketing group to add to the product price list and start selling. In those days as a small pioneering company, it was that easy to come up with a new product. This may have been the world’s first virtual FTP hosting service. It gradually took over from UUNET’s original scheme of selling customers a subdirectory of ftp.uu.net. Now they could have their own FTP host name.

UUNET started a Windows NT based web hosting service as well, which was independent of the Pubnix effort and seemed to have learned none of the lessons from it. Provisioning and monitoring were done manually, as NT was hard to configure programmatically, so the staff to customer ratio for the NT service was much higher. The independence from being an independent division of UUNET also proved to have a downside, in that the rest of the company began to marginalize and ignore the Pubnix product line, even though it was probably the most profitable product line the company had.

Multiplication

More people joined the Pubnix group, including Andy Crerar, Russell Street, Joey McDonald, Li Glover, and Peter Davis. Eventually, as UUNET grew, Kurt left the Pubnix group to start an R&D team. It was also based on the BSD/OS platform, and became a technology supplier to the Pubnix group in a symbiotic relationship, as the R&D systems were configured by the same SQL database. New technologies were incorporated as they became available, including Kerberos v5, Apache Stronghold for SSL,  ProFTPd, and Postgres. The service was running hundreds of web sites on a mixture of dedicated and shared servers.

UUNET was bought by MFS (Metropolitan Fiber Systems), who were soon bought by WorldCom, who then also bought MCI, and Pubnix became a tiny part of a huge conglomerate. Along the way, the former AOL web hosting division was acquired, and the Pubnix staff was moved into their offices in Reston, Virginia. The AOL web hosting ran on customized Solaris servers with little centralized or automated management. With input from Dave MacKenzie, they developed an Oracle database called FARMS to drive configuring their systems.

The End

As MCI WorldCom was going bankrupt in a financial scandal, most of the Pubnix and R&D staff left or volunteered to be laid off. At the end of 2002, the product line was sunsetted. Kurt Lidl tried to buy back the BSD/OS hosting business from them, but got stonewalled by MCI WorldCom’s legal department until it was too late.

For more context, see this history of UUNET’s product lines, which mentions Unix-based web hosting.

Advertisements

Who Got Perl from Me?

August 2, 2019

Unexpectedly, I made the de-facto distribution tar files for source code releases of the Perl programming language for a year or two in the early 1990s. Up until sometime in the Perl 4 release cycle, its author Larry Wall distributed Perl by posting a set of several dozen shar (shell archive) files on the Usenet group comp.sources.unix or comp.sources.misc, so people using dialup modems could download them without having to restart if the phone disconnected. In between major releases, he posted patches which were much smaller than a full distribution.

I was volunteering for the GNU project, which distributed Perl as a courtesy since it was popular free software that used the GNU license. The GNU FTP site prep.ai.mit.edu distributed source code in compressed tar files, so each time Larry released a new Perl version, I downloaded the shars and created a single tar file which I uploaded to prep. Dozens of organizations mirrored the GNU FTP server and did their own redistribution, so it was amusing to notice that most of the Perl 3 tar files out there had my user name in them if you did a tar -tvf to list them. They seem to have fallen off the Internet by now, as the last Perl 4 release is the oldest version that’s easy to find.

I finally stopped when Larry started making tar files of Perl releases himself. Larry last posted Perl 4 patch shars to comp.sources.misc in June 1992. I don’t remember whether he started making tar files concurrently, or only after he stopped posting the shars.

Why Do Long Options Start with Two Dashes?

August 2, 2019

Around 1990, Richard Stallman (RMS) and I were writing the GNU C library getopt() and he wanted to extend it to support long (multi-character) option names for user-friendliness. He considered Unix inferior in this regard to other operating systems such as TOPS-20 which supported long options (that could be abbreviated). He wanted GNU to be better than Unix while still compatible. There were a few programs that ran on Unix systems and used long option names starting with either - or no prefix at all, such as find, but those syntaxes were not compatible with Unix getopt() and were parsed by ad-hoc code.

Long options needed a prefix that wouldn’t clash with the Unix conventions, so programs could support both types of options without ambiguity. Richard chose +, since logically if - (for a small number mathematically) is for short options then + would be for long options, and it’s no additional typing. We created an extended interface called getopt_long() to support specifying long options.

But when the IEEE POSIX shell and utilities standard was published in 1992, the + syntax was disallowed. GNU developers discussed what to do over email. We considered -+ as the long options prefix, but that was hard to type, so we settled on --, which wouldn’t violate POSIX or Unix compatibility and wasn’t hard to type.

For a few months, GNU getopt() supported both + and -- to allow time for people to transition their scripts. I’m pretty sure support for + during that transition could be disabled by setting the environment variable POSIX_ME_HARDER (RMS’s exasperation with standards showing there), later changed to the more polite POSIXLY_CORRECT, which also disables recognizing options intermixed with positional arguments.

Because GNU software was popular and the solution was logical, everyone else adopted the -- prefix and implemented variations of it in their own argument parsers. Perl was probably one of the first.