The curious flaws in the OIG’s geographic mail volume report

The USPS Office of Inspector general has issued a report claiming to show wide geographic variations in the decline of first class mail volume since 1995:

The recent volume declines in First-Class Mail vary widely across communities in the United States. Even in those areas with the largest declines, it appears that volumes may be converging towards a lower limit representing a new base level of demand.

Unfortunately, the report bases that conclusion on data that appears somewhat questionable.

The OIG analyzed ODIS-RPW data gathered over the years by the USPS. So far so good- those numbers come from automated piece counts and statistical sampling of mail as it exits the system, broken down by 3-digit ZIP areas. When you look at the summary the OIG has prepared from the data, however, there are obvious anomalies, starting with the city showing the largest volume drop in the country, Worcester, Massachusetts. The OIG says first class mail has virtually disappeared in Worcester- it’s down 96.4%! If you then move to the other end of the rankings, you’ll find a half dozen places where first class volume has actually gone up. The second highest increase, +59.9%, is in a previously unknown Massachusetts city that the OIG says is called “Central”.

If you happen to be from the Worcester area, as I am, you know that originating mail around here is processed at the Central Massachusetts processing center in Shrewsbury. The plant first opened in 1991, taking on work that had formerly been done at the Worcester PO. At first, mail was still cancelled “Worcester MA”- later, the cancellation was changed to “Central Mass”- but the three digit area served by the plant remained the same: 014-017. Obviously the OIG didn’t properly combine the volumes.

There are similar anomalies elsewhere- the 21% increase shown for “Southern” Connecticut (another non-existent “city”) is probably due to mail processing being moved from New Haven and Bridgeport (which show declines of 92.9% and 90.1% respectively) to the Southern Connecticut processing plant in Wallingford. The whopping 128% increase in Brockton is likely the result of the 1998 realignment of the 023/024 ZIP code areas between Brockton and Boston, complicated by mail processing shifts between Brockton, Cape Cod (-75%) and Providence (-52%).

To be fair, the OIG adds this note:

we suspect that a few geographic areas (particularly those with significant volume increases – most notably Brockton, MA and Central [sic], MA) have possible data issues or experienced significant changes to mail processing operations over time. Nonetheless, we do not expect this to affect the report’s overall conclusions.

But it isn’t just those two locations- most areas of the country have been affected by plant consolidations as well as the “AMPing” of mail on Saturdays. None of that appears to have been taken into account in the OIG report.

Given that the whole point of the OIG report was to document geographic differences in mail volume trends, the unreliability of the geographic breakdown makes it hard to give the conclusions much credence. Another issue is the lack of actual volumes: a commenter on the OIG blog had this to say:

How many of the Plants in the low decline group absorbed processing done previously by another Plant? How many of the Plants in the high decline groups lost processing to another Plant? For instance, Muncie and Lafayette Indiana lost their outgoing distribution to Kokomo Indiana prior to 2013. Because Kokomo’s processing volumes were relatively small, the gain when expressed as a number would look small, but when expressed as a percentage would look large. Bloomington and Terre Haute also lost processing to Indianapolis. But because the volumes handled in Indy were large, the change in the percentage would be lesser than the change in a small Plant like Kokomo.

The commenter’s point is a valid one- percentages are meaningless without the underlying volumes. A related question is exactly how such “cities” as Show Low, Arizona and Truth or Consequences, New Mexico ended up on the OIG’s list.

The ODIS/RPW system, the source of the data in the OIG report, was originally designed to track point to point mail volumes by 3 digit ZIP, transit time, and class of mail. It does a reasonably good job at that task. Using ODIS data to track geographic and demographic trends in mail volumes was always going to be a complicated proposition, and it doesn’t look like the OIG has succeeded.