QW/QWE: History, Significance, and Content Summary

Available QW/QWE Documentation
Material about QW/QWE is organized as follows:

• QW/QWE Consolidated Conference Material Inventory
  This is the main access point for all of the QW/QWE documents. This table is organized as follows:
  • Conference Name / [theme]:
    Each conference had a particular theme, except the early ones. If there is a website for that conference here is a link to it.
  • When Held / Where Held / [#attending]:
    The name of the city and the name of the hotel where the conference was held.. Including either one or two tutorial days, the conferences ran either three or four days or, at the end, an entire week. [The number of attendees is the number of registered attendees; exhibitor staff and support staff are not included in this count.]
  • Program Committee / [# members]:
    A list of (or a link to) the members of the program committee (after 1991)
  • Promotion / Call, Mailer, Brochure:
    Various items used in the conference promotion activity.
  • Tutorials / [# pages]:
    Complete tutorial notes and page count. Pre-Conference and Post-Conference Tutorial Notes are included.
  • Proceedings / [# papers] [# pages]:
    Complete conference technical proceedings, paper count, and page count.

  Note: This inventory has been constructed from the surviving Conference CDs (approximately the last four years of QW/QWE) and from the archival hard copies that were scanned using a double-sided scanner from Xerox Systems. Not every supporting document from every event was found, but all 20 Proceedings documents, and most of the tutorial notes, are included in the inventory.

• QW/QWE Author/Affiliation/Title/Date Index: 1988 -- 2002:
  This index shows, for each each QW/QWE paper:
  • Author Name
  • Author Affiliation
  • Title
  • Year of Event `

• Unique Board Member Name Index:
  The name, affiliation, and country of 130 individuals who served on the QW/QWE Program Committee.

Intercepting The Flow of Technology
The issues of software quality were known from the early 1970's [the NATO conferences, the GOTO controversy, the Structured Programming revolution, etc.] but by 1988 there was hardly any consensus on which way the field would go or grow.

The early Testing And Validation (TAV) conferences starting in the 1960's and early 1970's coalesced at the Florida Testing Workshop, December 1978.

Even so, prior to the mid-1980's "quality assurance" and "software testing" were not necessarily viewed as a serious discipline, but by the end of the QW/QWE conference series it was clear that the situation had changed: it was QUITE a serious business!

• The Context, The Background, The Beginnings, The Competition...
  Edward Miller had organized and chaired the Florida Workshop on Software Testing and Test Documentation 18-20 December 1978. (sponsored by NBS and chaired by Miller). Miller and his company, Software Research, Inc., (SR), had also sponsored the TAV2 Workshop, a followon to the Florida Workshop, chaired by Bill Howden and Susan Gerhart [who was then an SR employee] on Catalina Island.

  The annual Pacific Northwest Quality Conference (PNWQC), begun in 1982 and now in its 40th year, was another academic-oriented conference focused on software quality.

  Commercial organizations like QAI (Mr. Bill Perry), and USPDI, organized industry-oriented conferences, Dave Gelperin and Bill Hetzel launched SQE and its STAR conference in 1992, in Las Vegas, NV.

  In Europe, EuroSTAR was begun in 1993; it discouraged academic, esoteric "research oriented" papers. Other European organizations included CapGemini (CGS), CDI, Cii-Honeywell-Bull, and InfoTech, among others.

  In the midst of all of this, the goal of QW/QWE aimed at the "sweet spot" between the "industrial" and the "academic" conferences. By bringing together theorists and researchers with practitioners and their new tools and approaches, the attendees would get a balanced picture of what is practical and what might become practical. Here is a set of testimonials (dating from 1996): What They Say About The Quality Week Conferences. These comments capture the essential character of the QW/QWE events.

• Leading Up To 1988
  As a technology speaker, Miller had a relatively successful technical seminar/workshop series, starting with a 2-day presentation to Boeing in 1977 on "software testing technology".

  Later, this seminar was presented at the CompCon and CompSAC conferences, and let to publication (with W. E. Howden) of a tutorial book on Program Testing. While initially sponsored by the IEEE and ACM (with published IEEE Tutorial books to go along with the seminars), invitations arrived from various commercial firms (e.g. USPDI, CGI, Infotech, etc.) \ for public and private presentations of the material, in venues located all over the world (including Finland, Sweden, Norway, Denmark, Germany, Netherlands, England, France, Italy, Hungary, and Japan).

  Overall, Miller made over 100 presentations of these seminars, the one on Software Testing primarily, worldwide, between 1977 and 1987 (the year before the first QW/QWE event). Miller's presentation of the "Advanced Software Test Methods" at QW1988 was the last time this material was used.

• The First Conference, 1988
  The initial program was assembled by Miller, with inputs from Dan Zimmerman, Rita Bral, and others in the community. Speakers included SR clients and Miller's industrial contacts. Invitations were also made to speakers from other contemporaneous conferences such as PNWQC, QAI, USPDI, etc.

• The Early Days, 1989-1991: Before A Program Committee Was Formed
  In this period the program was assembled from suggestions by several professionals in the field who proposed themselves or others as having "something interesting to say." SR staff members, including Miller and Bral, made the final selections often in consultation with other technologists well known to them.

• Middle Period, 1992-1996 -- Software Research Institute and the Program Committee
  The official "Program Committee" (Advisory Board) was formed in 1992, at the primary behest of Dr. Boris Beizer, who served on that group for all of the subsequent conferences. Having a Program Committee made selecting papers easier. Each Committee member voted (on line and/or by email), the scores were tabulated, and Miller assembled a trial program that was send around to the Committee for comments, revisions, and was then finalized.

• Professional Affiliations Developed
  Beginning with QW1992 QW partnered with a number of professional organizations and media units in addition to the sponsorships by various industry partners. In many cases this led to increased publicity; the organizations were compensated with free attendance of their representatives. Here is a partial list:
  • Professional Organizations: IEEE Computer Society; ACM; ESSI (Europe); CNSS; SATC (Software Assurance Technology Center): NASA/GSFC, among others.
  • Media Partners: Software Development Magazine; Application Development Trends; C/C++ Users Journal; Dr. Dobbs Journal; SD Times; Software Business; Windows Developers' Journal, among others.
  • Professional/Industry Partners: BAQA; ASW: Compuware: Vanteon; VeriTest, among others.

• QWE Added in 1997
  A growing number of attendees and speakers from Europe had been suggesting for some time to start up a European version of Quality Week. Ms. Bral, who is originally from Belgium and is able to speak the important European languages (English Dutch, French, German and Italian), made the process of starting Quality Week Europe relatively painless.

  The QWE Program Committee emphasized European orientation and prior speakers from QW in the Bay Area were invited to propose presentations for QWE. Brussels was the ideal location, central to the EU government and the seat of NATO.

• The 9/11 Event and QWE2001
  Originally scheduled for November 2001, the 11 September 2001 attacks in New York and Washington shifted everything. After careful consideration of all of the factors involved, the decision was taken to postpone QWE for six months (and subsequently also to postpone QW to the Fall, rather than the Spring. The change announcement was made on 03 October 2001: Announcement of Postponement of QWE2001 to Spring 2002 by email to the QW/QWE and SR entire mailing list, as well as online on the QW/QWE Website and in various news groups and forums.

• Final QW/QWE Events, and Beyond 2002... QWE2001 became QWE2002, but there were some changes to the technical program necessitated by the new date (and by everyone's concerns about travel).

  QW2002, now set for September 2002, had a reduced program and a reduced Vendor Show, the consequence of budget cuts and travel concerns throughout the USA. The reasons for curtailing the conference series were many, and among them were these factors:

  • QW/QWE attendance was off -- conference-going was no longer as stylish as it used to be, and in addition travel was very difficult after Fall 2001.
  • The need for face-to-face conferences seemed to be vanishing with the growth of webinars replacing them.
  • The "Dot-Com" boom was largely over and many companies were struggling to survive -- and conference attendance was a cost that could be eliminated.
  • The Internet and Web -- the WWW -- had expanded significantly, and many of the "old" quality issues were not seen to be quite as pressing as before and technology solutions were widely available.
  • Proposals to QW/QWE from "academic organizations" diminished very rapidly -- probably due to the WWW providing sufficient outreach.
  • The new issue of "web quality" was only then being articulated as the major concern, but the web itself provided more than adequate resources to address those issues.

Production Notes
Here are some comments about how this material was assembled:

• Source Material
  The basic material described here is taken from the 20-30-year-old hard-copy and computer archive records of Software Research. Due to the inevitable changes in computers and support equipment (at least three generations of machines are involved) sadly, much of the promotional material prior to 1997 is lost. Only a single hard copy of early events' documentation survived.

  Many of the pages in document found herein are labeled "Software Research Institute," which was the educational arm of Software Research, Inc., under the leadership and direction of Rita Bral. Some pages are also labeled "eValid, Inc." but that organization was later incorporated into SR proper.

• Computer Support
  In the late 1990's we used Z80/Z8000 based minicomputers, built by Onyx Systems, that were running a version of UNIX, with 1 or 2 80 MB hard drives each. They were connected to ADM3a's -- "glass TTYs" or "dumb terminals" -- on RS232 serial connections.

  Later, by 1996, SR had some SUN SPARC machines and an ethernet LAN and with ADM3a's and XTerminals of various kinds that could connect to the Onyx and Sun/SPARC "mainframes" in the office. The office LAN also connected to numerous PCs running Windows; they could connect vi PuTTY sessions to the various UNIX boxes.

  The first websites presented to the public were hosted on dedicated Zenix machines hosted commercially by Verio, from ~1994 (the hosting machines were named "Verio", "Verio1" and "Verio2" successively).

• Technology
  Much of the logistics of the QW/QWE events was done by relatively primitive methods (in comparison with what is available twenty years later). Event promotion in 1988 was primarily by USPS mail, usually via a printed "glossy" conference brochure and only to a limited extent by email and by telephone calls. In addition, the publicity included ads in various magazines and periodicals.

  In the mid-1990's SR largely did email promotions using in-house computers and its public-facing commercial web servers. Ultimately there were ~90,000 email addresses stored on SR's internal database, which was kept current by SR staff members.

• Software Research Institute
  SR's TestWorks (1981-1999) and eValid (2000-2009) product development and sales effort proceeded in parallel with QW/QWE, and this presented an internal conflict of interest. How could a product company credibly sponsor a conference that was not merely a company sales pitch and marketing vehicle? The solution -- the ethical basis -- was creation of Software Research Institute under the leadership of Rita Bral, Executive Director. The Program Committee reviewed and rank-ordered all submitted papers and presentation and tutorial proposals. It is worth noting that the Program Committee voted against SR's presentation proposals time and time again.

  While SR was always represented in the vendor show and usually had a vendor-track presentation, and so did many of SR's competitors in the "product space." Throughout it all, the two activities were sufficiently distanced to maintain the quality reputation.

• Abuse Of Mailing Lists -- A Problem
  One conflict grew in ~2000 when certain vendors discovered that conference speakers and attendees always were given the complete mailing list of attendees. The trick that a vendor would use was to get a paper OK'd for the technical conference, attend for free (speakers never paid), and walk away with the complete contact details (including email) of everyone there, speakers, attendees, and other vendor staff. This was a major tactical error: we should have found a way to prevent vendors from gaining that advantage for free.

• Dot-Com Collapse
  It was the "dot-com collapse", combined with the 9/11 event, that created the greatest difficulty for the QW/QWE conferences. Also, academic paper proposals had essentially dried up; nobody had travel funds (and many speakers declined to travel).

  By 2002 it was clear that nothing new was showing up from the R&D community, and with the then very widespread of internet access the need for face-to-face meetings nearly vanished. The competing conferences were always located near Disneyland (East or West) and were always scheduled to be near spring break or other school vacation periods.

  Which it not to say that San Francisco was not an amusement park of a kind all its own, nor that Brussels didn't have its own unique charms. But after 15 years, it was time.

Acknowledgments
Most of the credit for QW/QWE goes to Rita Bral, Executive director of SR/Institute. She master-planned all of the QW/QWE events, organized all of the promotion activity and material, and supervised her support staff "hands-on" in actually running the events. Her language skills and European background were essential in developing a truly "international" conference series.

Credit also goes to Mr. Dennis Ng, who in Fall 2021 and Winder 2022 persisted in the effort to scan in the material that was not on CDs (only 8 proceedings were on CDs), probably 15,000 pages in total.

Apart from Miller's contribution as General Chair, the greatest technical influence on the events was undoubtedly Dr. Boris Beizer. He guided the Advisory Board and always had positive (and caustic, and humorous) suggestions for improvement.

QW/QWE Links
Here are links to the main resources (all mentioned above) available for this body of material.

1. QW/QWE Public Website
2. QW/QWE Document Inventory
3. QW/QWE Author/Title/Affiliation Index: 1988 -- 2002
4. Unique Board Member Name Index
5. Yearly Board Member Index
6. Boris Beizer (1934-2018) -- Remembered