Automation Tests Are Nonnegotiable - Beware the "Death Spiral"

🕵️‍♂️ Background

I'm a software engineer at Ad Astra - a higher education SaaS company focused on course scheduling & student success that services 500+ mascots around the country. We are hiring for several positions. We got some feedback that our current section details page was confusing and too busy with fields they didn't care as much about.

🏆 Our Goal

• Optimize the section details page for departments by displaying section data at the top level with all related meetings visible without switching pages.
• Highlight important fields & hide less-important fields below the "fold" with the ability to reveal those fields if needed.

The old view only displays one section meeting (child) at a time & the section details (parent) were displayed in a drawer & many users found this positioning to be confusing.

Here is the final product after the redesign. It's much easier to edit multiple meetings and see the fields you care about at a glance.

😬 The "Deadline"

Our Engineering team typically tries to avoid having deadlines. We aspire to get customer feedback on a story within 2 weeks of building and we "estimate" when our stories will be done. These estimates fluctuate with any new information we gather throughout the building process.

Things were a little different for this redesign because we had some pressure to implement the redesign in time for one of our mascots to begin onboarding. The mascot wanted to go live sooner than we had thought, which left us with 4 weeks to do the overhaul.

🚫 Testing

Early on in the overhaul, a large number of the original tests failed due to the structure of the app changing, not so much the functionality changing. With the pending deadline and fear of maintaining the tests during the refactor, we ignored coverage & tests for the redesigned page (NewSectionDetails) until we had more of the functionality implemented.

🐛 A Swarm of Bugs

Naturally we introduced a lot of bugs when introducing multiple meetings to the section details page. The complexity of the refactor had become worse than we had originally anticipated.

How It Started 😁

The number of bugs we introduced far exceeded all of our expectations. To keep track of all the bugs to go fix later, we started adding checklist items to our stories.

How It's Going 🤬

As the list of bugs we found grew, we ran out of allowed checklist items on out two stories. We had to create a "Bug Pack" to record all the bugs we found.

And another one...