Creating Automation Tools for Writing Teams

A Programmer’s Perspective

Senior Programmer-Writer

Tools will always be the weak link for writers. Not the high-level ones such as Flare or Framemaker, but low-level tools, ones that individual writers need to finish tasks. These tasks may be unique to each group or perhaps to each writer, from reformatting long lists of error codes, to collecting acronyms within a document, special or conditional formatting on tables, or repeating onerous sequences — a representative list is impossible to create. So, given the extreme diversity among writer’s procedures, it is unlikely we will get a comprehensive tool suite. The technical communicators community has not created these tools for us. That means we have to write them ourselves.

We’ve always been told that the goal is never far off. Most high-level tools have at least some support for this. Most notably, Microsoft Word has a built-in macro language (VBA), a macro recorder, code editor, and complete programmatic access to its commands. But these tools are still very much in a programmer’s realm, out of reach for many technical communicators, and development groups rarely loan out programmers. But what if they did? What results are possible?

I was fortunate to be a developer assigned to three writing groups at the NASA Johnson Space Center in Houston. The mission was simple: To improve these teams’ efficiency. I had considerable leeway in my methods but mostly through the creation of custom tools and procedural changes. My qualifications seemed fitting. I was a programmer, writer, and programmer-writer, and thus familiar with the writers and the writing process.

The teams were grateful for the attention and willing to help although there were several conditions that had to be addressed.

Initial mistrust. There was serious mistrust about automation, with team members imagining that it would eliminate their jobs. I understand that. The traditional approach is that management streamlines processes with the intent to let people go. So of course, no one’s going to cooperate. The truth is, it’s about freeing team members to do what they’re good at. Start on a project they are comfortable with and earn their trust. Let them set requirements, and you provide frequent updates, show progress, and offer options all alleviate initial fears.

User education. Most team members are Word power users and capable of solving Word problems themselves. Automation doesn’t minimize their skills; they still have to tell you how to construct the documents and requirements. Over time, they’ll learn what is possible with automation. The best ideas come from the team members themselves but they have to know what can be automated, and comfortable enough to ask. My favorite part of any project is when they start asking “Can we do this?…”

Bureaucracy. NASA is a governmental agency and despite the perception of being cutting edge, it is ultimately still a bureaucracy. Given that, the reliance on international partners, interdepartmental coordination, and natural resistance to change, it was not always possible to make the right change. We had to work within what we could change. Don’t be afraid to compromise or, conversely, overstep boundaries a little as needed. Remember, it’s easier to keep a project once it exists, rather than sell just an idea for a new one.

Development. Typical automation projects tend not to be large or take a long time, perhaps two or three weeks. They have very specific goals, often just a single task. One project took less than two hours. In a way, it may not be much more than rapid prototyping. Working directly with the clients provides immediate feedback and shortens development time.

There were two deployment options. If the project were simple enough, it could be written as a series of macros and deployed through a common template. Otherwise, it’d be deployed as a Windows application. Web deployment was not an option because Microsoft Office then couldn’t be used over an intranet.

The importance of Being Embedded

The traditional approach keeps the programmer separate from the client. However, in programming for writers, it is critical to be embedded with the team. Merely sitting with them has advantages of overhearing the conversations, learning the jargon, seeing their tasks, and noting how they’re being done. For example, in one instance I noticed the team member repeatedly hitting the same key for more than a minute. She was transferring files and her application didn’t accept long file names, so she had to skip those. Furthermore, there were at least 5000 files in the archive, so that task would have taken an excessive amount of time, including manually changing the file names.

By intervening, we crafted an application in less than thirty minutes that checked all the file names directly in the Windows depository and corrected the long ones. A day-long task only took less than two hours (with testing), and could be used by other team members. While she was an experienced team member who had worked with me before, she didn’t realize that the task could be automated at the Windows level.

An extreme example happened later when I was not just embedded with a team, but transferred to become part of them. They are a meeting support team that organizes routine formal management and engineering meetings, provided transcription, voice recordings, managed agendas, and the invitation list. This group was largely invisible and so, had been long-ignored for technology and process improvements. The mission was essentially process improvement, changing procedures as needed, and writing efficiency tools. They had processes that hadn’t changed in ten years and little automation. “Nothing is off the table,” according to my supervisor. This was truly an exciting and evolutionary step. In the nine-month assignment, the first three months were to learn and to do that job. After that, changes could be made. And the insight that period provided was revolutionary. In all, we added 14 tools. They ranged from converting email forms to meeting logs, automatically adding and modifying agenda items, and queued long lists of presentations (from PowerPoint, Word, and Excel) into a single PDF and print job. Procedures were changed too, such as pooling team members to help with other meetings based on their availability, to something as simple as combining Windows locations into a single directory.

Project Examples

The following are three examples of automation projects I worked on but they give a view of the diversity some groups have.

Image-based Application
This is an example of controlling two applications, and merging data from one to another. The source document was an enumerated list of parts and equipment going up to the space station. Tables, pictures, and text had to be manually copied to Photoshop, pasted and resized into the target. The source document was often more than 80 pages, and the handbook rated this procedure at 45 hours. In other words, it took one person more than a week of repetitious, error-prone processing per iteration, and they may have had five to eight documents a year. In addition, a quality assurance (QA) team member checked each page, ensuring completeness and no duplicates.

This was an ideal automation project. The steps were well defined, there were no exceptions to the business logic, and it had a repetitive nature. Deployed as a Windows application, the team member started by selecting the source document, and placing the cursor in the target document where the first image was to be placed. Complete processing was about 15 minutes. It was a two-week development project, and saved 44 hours and 45 minutes per cycle, for a total of 355.6 hours per year, or about nine weeks. More importantly, it freed up one team member.

Acronyms

The most onerous task the government required was that documents required an acronym list. Requirements included identifying the acronym, cataloging them in a list, complete with the definition, making sure it’s spelled out on first occurrence, and provided a revision tracking table to show newly added, deleted, or changed terms. This was rated as a 32-hour task.

The automation was in three steps because each required verification and validation.

1. Find and list all the acronyms.
2. Produce the table, complete with revision tracks and style formatting applied. Upon approval, it was this table that was pasted into the target document.
3. Create a first occurrences report that team members used to verify that the term was properly spelled out on first use.

The new automation took between 20 minutes and an hour, depending on the complexity of the document, and with verification documents could be completed in less than four hours. As a process improvement, this step was eventually moved from the QA group to the book creators, allowing them to more quickly correct terms.

It’s interesting to note that an acronym isn’t limited to what you’d expect. It could actually be anything from a three-letter acronym (USA or ARS [Air Revitalization System]), to terms (Log [Logistics], ATT [Attitude]), or almost anything else (rack loc [rack location], Rqmt [Requirement], or U.S. Lab). As a result, we could not exclusively use traditional pattern finding, so we developed a master list maintained by the team to explicitly list all the valid terms.

This project was not typical automation that could have been written as a series of macros. For instance, because of the sheer number of internal data iterations, it had to be optimized for speed, and there were a series of involved searches. These searches included not only finding the master list terms, but also a more generalized pattern matching search to find everything that could be considered an acronym (like a word with more than three letters and at least one capitalized other than the initial letter). Finally, Word’s internal misspelling dictionary was used to identify all other terms, assuming an acronym is considered a misspelling. We achieved near 100% accuracy.

Failed Projects: Table Number 3

Not all projects, even those considered good automation ones, always succeed. Table Number 3 in one of our documents was a notable failure. It was a standard Word table, in this case, launch, manifest, and return information for all the flights (manned and supply missions) to the space station during a given period. The application proposed flights and docking information. The docking of craft for the space station is a complicated dance, often moving craft around afterward.

The problems started when we discovered the business logic, that is, the rules applied to formatting and notes presentation, changed after each release. This was partly due, ironically, to our success, as managers wanted more business logic. But largely because the rules just changed outright, as flight engineers redefined them. In addition to just writing new rules, the business logic became almost incomprehensibly complex, including exceptions themselves having a series of exceptions. Due to the time and effort required to maintain it, management decided to revert the table to manual formatting. The automation was rolled back to an earlier phase of creating and formatting the table for its initial use.

Summary

Automation is an extremely powerful tool and can be applied in innumerable ways. For one writing team, the document turnaround time dropped from the contractually required twenty days to just three. In addition to saving time and money, these tools reduce stress. Their development doesn’t have to be complex. Word encourages creating macros just by recording your steps. However, adding a programmer can unlock additional features, such as all of .NET, regular expressions, and internal Word capabilities. It is unlikely though you would be assigned your own programmer, so consider approaching the IT department with a specific request for a single feature. As a compromise, suggest building complex tasks progressively. A case can be made that a small investment from a programmer could save the writing team literally weeks each year.