Software deliverables on a research contract

The code is a deliverable, so deliver it like one

On most research contracts the report gets the attention and the code gets zipped up the week before the period of performance ends. Then a program analyst tries to rerun a table eighteen months later, finds three folders named final, and learns that the one person who knew which script produced Figure 4 has moved on. If the statement of work lists software, analysis code or a data-collection app, that software is a deliverable with its own acceptance. Treat it with the same care as the report, and the report becomes something the agency can defend.

This is not a call for heavyweight process. A research team does not need a full product release pipeline. It needs a small set of artifacts that let a stranger install the code, run it, check that it works and connect its outputs to the numbers in the report. Each artifact is cheap when built along the way and expensive when reconstructed at the end.

Know which kind of software you are delivering

Research contracts carry two different kinds of software, and they need different packages.

Analysis code turns input data into the estimates, tables and figures in the report. Its users are analysts and reviewers. What matters is that it runs end to end on the delivered data, produces the published numbers, and is documented well enough that someone can change an assumption and rerun it. Speed and polish matter much less than traceability.

Data-collection software is anything participants or field staff use: a survey app, a field form, a sensor logger with a configuration interface, a small web tool for coding records. Its users are people outside the research team. What matters is that it works on the devices it will meet, protects what it collects, is accessible to the people who must use it, and can be maintained or retired after the contract ends.

Many contracts include both. Write down which pieces are which in the first month, because the acceptance evidence differs and the reviewer should not have to guess.

A versioned release, not a folder

A release is a fixed, named state of the code that can be retrieved exactly. In practice that means a tag in version control, a version number, and an archive of that tag delivered alongside the report. A shared drive folder is not a release, because nothing stops it from changing after delivery and nothing records what changed.

Use a simple numbering scheme and say what it means. A common convention is three numbers: a major version that changes when outputs or interfaces change in ways a user must know about, a minor version for added features, and a patch for fixes that do not change results. The convention matters less than writing it down and following it.

Tie each release to a contract event. The draft report is produced from one release; the final report from another; any revised tables after government comments from a third. The release identifier appears in the report itself, in a short methods note or appendix, so a reader holding the report knows which code produced it.

Release notes that list every change

Release notes are the record the government reads when deciding whether a new version needs fresh review. They should be complete, not promotional. Every change goes in, grouped so the reader can find what matters.

• Changes to results. Anything that moves a reported number, with the old and new values for the affected tables. This section comes first.
• Changes to inputs. New data versions, new sources, changed filters or exclusions, with the reason.
• Method changes. A new model specification, a changed weighting rule, a corrected formula.
• Fixes. Bugs found and fixed, including how they were found and which outputs they touched.
• Interface changes. For apps: screens, fields, workflows. For code: new parameters, renamed functions, changed file formats.
• Known issues. Problems found but not fixed in this release, with their effect on outputs.

The known issues section earns trust out of proportion to its length. A reviewer who sees a bug disclosed with its effect quantified will believe the rest of the notes. A reviewer who finds an undisclosed one will doubt all of them.

Test records a reviewer can read

A test record is evidence, not a claim. "All tests pass" in a release note is a claim. A saved log of the test run against the tagged release, with the date, the environment, and the count of tests run and passed, is evidence.

For analysis code, the most useful tests are not unit tests of individual functions, though those help. They are checks on the outputs: row counts at each stage of the pipeline, totals that must reconcile with a source, known values that a hand calculation confirms, and a regression test that compares every table in the report against the numbers the release produces. When a regression test fails, either the code changed a result or the report is out of date, and both are worth knowing before delivery.

For data-collection software, test records cover the paths a user can take: each screen, each validation rule, what happens offline, what happens when a session is interrupted, and how submitted data arrives at the other end. Record the devices and browsers tested. If field staff will use a particular tablet model, test on that model.

An installation guide written for a stranger

Write the installation guide for a competent person who has never seen the project and has no one to call. Then give it to someone who fits that description and watch them follow it. Every place they stop is a gap in the guide.

A good guide states the operating systems it was tested on, the language and runtime versions, every dependency with its version, how to obtain the input data and where to put it, the exact commands to run in order, how long a full run takes on stated hardware, and what the outputs look like when it worked. It ends with a short section on common errors and what each means.

Pin dependency versions in a lock file or environment specification delivered with the release. A guide that says "install the latest version" of a statistics package will produce different numbers on the day someone follows it, and nobody will know why. Where the environment is complicated, deliver a container definition as well, but treat it as an addition to the written guide, not a replacement for it.

Data dictionaries and data lists

The data dictionary is the deliverable reviewers most often find missing and most often need. It describes every field in every delivered dataset: name, plain-language definition, type, units, allowed values or range, how missing values are coded, and where the field comes from. For derived fields, it gives the derivation in words and points to the code that computes it.

The data list is the companion artifact one level up. It names every dataset the project used or produced, its version or extraction date, its source, its size, where it is stored, who may access it, and whether it is delivered, withheld for privacy, or available only from the original provider. When data cannot be delivered because of a use agreement, the data list is how the government knows what it would need to request to rerun the work.

Both should be files, not pages in a report appendix. A dictionary in a spreadsheet or plain text file can be read by a program, compared between releases and kept in version control next to the code it describes.

Accessibility for anything a person uses

Software the government will use or put in front of the public must meet federal accessibility requirements under Section 508. Data-collection apps that participants or staff use are squarely in scope. So are any dashboards or interactive tools delivered with the report.

Plan for it in the build, not the final week. Test with an automated checker, then test the main workflows with a keyboard only and with a screen reader, and record the results with the release. Common problems in research-built tools are form fields without labels, error messages shown only in red, charts with no text alternative, and custom controls that a keyboard cannot reach. Each is easy to fix early and tedious to fix after field staff are trained on the screens.

Handoff: repository, licenses and dependencies

Handoff is when the government stops depending on your team to use what it bought. The contract's data rights clause, typically FAR 52.227-14 on civilian contracts and DFARS 252.227-7013 and 252.227-7014 on defense contracts, governs what rights the government receives in delivered software. Read it before the first line of code, because it affects which components you can use and what you must mark.

Repository access. Agree early where the code will live at the end: transferred into an agency repository, delivered as an archive with full history, or both. Full history matters. It is the record of when each change happened and why, and it is often the only way to answer a question about an old result.

License inventory. List every third-party component with its license. Open source licenses vary in what they require when software is redistributed, and some agencies have policies on which they accept. A software bill of materials in a standard format is the cleanest way to deliver this, and some agencies ask for one by name.

Credentials and services. List every external service the software depends on: hosted databases, mapping APIs, survey platforms, cloud storage. Say who holds each account and what happens to it at contract end. An app that stops working because a personal API key expired is a failure of handoff, not of code.

Secrets. No passwords, keys or tokens in the repository or its history. Scan the history before transfer, not just the current state.

Acceptance criteria and the evidence for each

Acceptance criteria belong in the contract or the work plan, written as tests both sides can run. "Delivers working analysis code" is not a criterion. "Running the delivered release on the delivered data reproduces every table in the final report" is. The table below pairs common deliverable types with the evidence that should come with them.

Deliverable	Acceptance criterion	Evidence delivered
Analysis code release	Reproduces every reported table and figure from delivered inputs	Tagged release, regression test log, output-to-report crosswalk
Data-collection app	Completes each user workflow on target devices and stores data as specified	Test record by workflow and device, sample export, data flow diagram
Installation guide	An independent person installs and runs it without help	Dated walkthrough record noting who ran it and on what system
Data dictionary and data list	Every delivered field and dataset described	Machine-readable files checked against the delivered data
Accessibility	Meets the applicable Section 508 requirements	Automated scan results, keyboard and screen reader test notes
Handoff package	Government can operate without contractor accounts	Repository transfer record, license inventory, service and account list

Ask for the acceptance test to be run jointly on a draft release before the final one. It turns acceptance from a judgment at the end into a rehearsal in the middle, and it surfaces the installation problems while there is still time to fix them.

Every number in the report traces to a release

The goal of all this is one property: any number in the report can be traced to the code and data that produced it. The mechanism is a crosswalk, a short table that maps each table and figure in the report to the script that generates it, the release that produced the published version, and the input datasets it used.

If a reviewer cannot get from a number in the report to the line of code that produced it, the software deliverable is not finished. Build the crosswalk as you go. Have the code write output files with names that match the report's table numbers, and have it write the release identifier and run date into each output file. Then producing the crosswalk at delivery is a matter of listing the outputs, not reconstructing history. When a comment comes back asking why Table 6 changed between the draft and the final, the release notes and the crosswalk answer it in minutes.

This also protects the work after you are gone. A program office that can rerun the analysis with a new year of data, using the delivered code and following the delivered guide, gets value from the contract long after the final invoice. That is the difference between a report and a capability.

The release package checklist

Use this as the cover sheet for every software delivery. If an item does not apply, say why rather than leaving it out.

• Release identifier and tag, with an archive of the tagged code and its full history.
• Release notes listing every change since the last delivery, results changes first, known issues last.
• Test record from a run against this exact release, with date, environment and pass counts.
• Installation guide with pinned dependencies and a dated record of an independent walkthrough.
• Data dictionary and data list as machine-readable files, matched to the delivered data.
• Output crosswalk mapping every report table and figure to script, release and inputs.
• Accessibility results for any interface a person will use.
• License inventory or software bill of materials for every third-party component.
• Service and account list with the owner of each and the plan at contract end.
• Secrets scan of the full repository history, with the result.

Frequently asked questions