Adam Niederer

Versioning Code at Scale, Part 1: Growing & Breaking

Published 2024-05-01 by Adam Niederer

Forty components, hundreds of consumers, fully hot-deployed. In this series, I'll explain how I made it work at Chatham without inciting our devs to pillory me.

That's right, I commit something to our design system, devs are coding against it ten minutes later, none the wiser. We have a way of opting out, but nobody uses it. The secret? Shamelessly steal from Rich Hickey's Spec-ulation Keynote from Clojure/conj 2016. Over the next few articles, I aim to build on his thoughts on versioning and how to successfully grow software without breaking it.

To quickly summarize some of Hickey's talk, there are two ways you can change software: growing, and breaking. Growing changes ask for less and provide more, breaking changes do the opposite.

Basically, growing involves doing this:

1 file changed, 3 insertions(+)
modified   src/my-library.ts
@@ -14,1 +14,4 @@
+ export function myNewFunction(firstArg: string, secondArg: string) {
+   // ...
+ }

and breaking involves doing this

1 file changed, 3 insertions(+)
modified   src/my-library.ts
@@ -14,1 +14,4 @@
- export function myOldFunction(firstArg: string, secondArg: string) {
-   // ...
- }

In the first example, we provide a new function that our users can use. In the second example, we provide less. This applies to functions in namespaces, keys in maps, environment variables, and the props and events of components; you can add more data in a returned or event value, or ask for fewer in an argument or prop.

Breaking is the opposite. Asking for more, such as requiring a new key in a map, or providing less, such as removing an event, is generally considered an incompatible change.

Type Compatibility

We can generalize much of this using type logic - When you're accepting data, it is not breaking to replace a subtype with one of its supertypes, and breaking otherwise. When you're providing data, it is the opposite - not breaking to replace a supertype with one of its subtypes.

In TypeScript, supertyping and subtyping is a bit more complicated than in other languages, because TypeScript has robust support for union and intersection types, as well as first-class map and list types. Here's a quick primer of how subtyping relationships play out with these operators:

You may have noticed that the latter point does not appear to be the case in practice, as the following code will not compile, as of TypeScript 5.3:

const fn = (x: {x: number}) => x
fn({x: 2, y: 2}) // does not compile

However, this is because of limitations in TypeScript's type inference logic; an explicitly typed or nonliteral argument will compile.

const arg = {x: 2, y: 2};
const fn = (x: {x: number}) => x
fn(arg) // compiles
fn({x: 2, y: 2} as {x: number, y: number}) // compiles

This, of course, will be cold comfort to all of your developers who need to update their types after you remove y from the map in that function's signature. We can, however, make this work for literal typing.

Any set can be represented by an intersection of its subsets. Maps' keys are sets, so the same rule applies. {a: number, b: number, c: number} is a functionally equivalent type to {a: number} & {b: number} & {c: number}. We can also represent a map containing the set of all map keys with Record<K, V>. Therefore, we can make our argument T & Record<K, V>, as both T and every other map is a supertype of it.

This is effectively saying nothing mathematically, like how the intersection of the sets of integers and real numbers is just the set of integers, but it makes TypeScript's type inference work because it can infer literals which don't match T to be Record<K, V> while maintaining the same type constraints within the function, making supertyping and subtyping compatible as described above.

type AtLeast<T> = T & Record<string | symbol | number, unknown>;
const Foo = (arg: AtLeast<{num: number}>) => Math.sqrt(arg.num)
Foo({num: 2, str: ''})

Thankfully, there is no such limitation for function return types, so the rules can be applied directly without tricks.

const cmp = {x: 0};
const ret = {x: 0, y: 0};
const fn: () => {x: number, y: number} = () => ret
fn() === cmp;

Requirements & Optionality

Subtyping and supertyping can not capture the full nuance of what can and cannot be considered breaking, however. The following change, for example, is also non-breaking, despite changing the argument to a subtype of its previous type:

- export function fn(arg: AtLeast<{num: number}>) {
+ export function fn(arg: AtLeast<{num: number, str?: string}>) {

This is because in these cases, the argument being added or removed is optional. The aforementioned rules around super- and subtyping and compatibility apply only to required arguments; when considering breakage, optional keys don't affect the sub-/supertyping relation. Accepting a new optional key is compatible, but requiring an existing optional key is not.

A counterargument for the above point may take the form of the following breaking change:

- export function format(arg: AtLeast<{date: Date, locale?: string}>) {
-     return new Intl.DateTimeFormat(locale ?? 'en-US').format(date);
+ export function format(arg: AtLeast<{date: Date}>) {
+     return new Intl.DateTimeFormat('en-US').format(date);
  }

However, removing the optional key here is not what caused the breakage. What is causing the breakage here is the change in behavior; if the new code read return new Intl.DateTimeFormat(readCallersMind()).format(date);, and readCallersMind always returned the correct locale for any given situation, the change would not be breaking.

Meaning and Behavior

The above example can be extended further to show another type of breaking change: a violation in behavioral contract. Breaking changes can be made without any change to what the function accepts or provides. For example, this change would be considered breaking in most situations, despite the function signature not changing:

- export function format(arg: AtLeast<{date: Date, locale?: string}>) {
-     return new Intl.DateTimeFormat(locale ?? 'en-US').format(date);
+ export function format(arg: AtLeast<{date: Date, locale?: string}>) {
+     return new Intl.DateTimeFormat('en-US').format(date);
  }

There's a lot of nuance to unpack here, so I'll explain how to deal with behavioral contracts in the next section. However, keep this in mind in the upcoming sections on applying this knowledge.

Namespace Typing

Namespaces can be treated in the exact same way as components; a namespace can be considered a map of symbols or names to JS objects, and all of the rules about providing data apply. Adding a function or class to a namespace can be considered replacing a namespace with its "subtype", and therefore compatible. Removing one can be considered replacement with a "supertype" and therefore breaking, and so on.

These typing rules also for renames; renamimg MY_CONST to MY_CONST_2 replaces the namespace with a namespace that has neither a super- or subtyping relationship, and is therefore not compatible.

Component Typing

All of this theory can also be applied to components: regardless of how they're specified in your framework, a component's properties are fundamentally a map. I will use Web Components as examples, because they're a standard and most broadly applicable.

Consider the following component, which has two properties.

/*
 * Renders data to an unordered list
 * Props:
 * @required `title: string`: The title of the list
 * @required `data: T[]`: Elements to render to their own items, mapped to strings by...
 * Events:
 * `userInitiatedEvent({detail: void})`: Fires when a user...
 */
class UnorderedListComponent<T> extends HTMLElement {
  private title: string;
  private data: T[];

  set title(prop: string) { ...; render(); }
  set data(prop: T[]) { ...; render(); }

  connectedCallback() { ... }
  disconnectedCallback() { ... }
  render() { ... }

  userInitiatedCallback() {
    this.dispatchEvent(new CustomEvent("userInitiatedEvent", Math.random())
  }
}

Its properties can be described as the map {title: string, data: T[]}. Therefore, we can apply all of the rules about accepting data from before to determine what is and isn't compatible. From a typing perspective, Adding new required properties is breaking, but new optional ones are compatible. Removing either required or optional properties is also compatible. The compatibility rules can also apply to the types of the props; changing the type of title to number is breaking, but string | number is compatible.

Its events can also be described as the map {userInitiatedEvent: number}, and the rules about providing data apply. Adding new events is compatible, but removing them is not. Adding keys to the provided data is compatible, removing them is not.

However, we are no longer in the world of pure functional programming; our component is littered with side effects, and those side effects are used to uphold the illusion of functional purity in the inherently stateful environment that is the DOM. In just that simple component, setting a property will cause the component to rerender, and removing the property setter will remove that side effect. These changes of behavior and more will be covered in the next section.

Conclusion

Some may point out that there are programs which may be broken by many of the aforementioned changes. For example, a program can check for the presence of a currently-unused property or environment variable, and, if detected, crash. This idea is often cited as Hyrum's Law, which I will tackle in the next part.

Furthermore, the short list of non-breaking changes provided above does not apply to much of what would be considered common work on a library. For example, performance improvements, bugfixes, and especially on the frontend, some classes of new features. This is where the nuance starts to creep in, which I'll cover in the part after next.