Before I criticize SDKs, I’ll give them a proper eulogy: SDKs have taken us a long way. Having packaged, reusable code to handle lower level mechanics of HTTP, JSON parsing, error handling, and more has changed software development. If we had to write our own code to construct every payload, initiate every exchange, handle every error, and process every result, we would be spending too much time flipping bits instead of making useful systems.
But it’s time for SDKs to die.
Fundamentally, SDKs address a simple situation: We want to make our systems and devices easier to use. In the best of times, they make interfaces more consistent, handle errors and interactions for us, and map them into the languages we want to use. But built into that are a painful number of assumptions and choices.
Shipping SDKs Today
First, the team has to document the underlying APIs effectively enough that their team – and eventually their users – can understand and build SDKs.
Then the team has to decide which languages and frameworks to support because no one has the time or expertise to support all of them.
Then the team has to document and build examples using each of those SDKs to ensure that developers can understand how to use them.
Now the team can distribute their SDKs through the language-specific package managers.
And finally, as new API capabilities ship, the team will update and version the SDKs providing clear upgrade paths with minimal pain.
For bonus points, great teams will also have:
- An engaged community of developers providing feedback, examples, and test cases.
- A test suite exercising the SDKs’ key capabilities, validating payloads, and confirming success & errors where you expect.
- An automated build process that takes care of every step consistently.
And you have to duplicate this effort for a number of frameworks in a number of languages. If you’re lucky, your customers & community may build some libraries for you but now you have to establish endorsement and support policies for each.
Fundamentally, good SDKs take a ton of work and that’s before we consider the risks.
The Risks of Today’s SDKs
As we think about building and using SDKs, there are three risks that overwhelm everything else.
Companies have to choose which languages to support. Ideally, they choose the best languages for their audience and get adoption. More realistically, developers in other languages chose your API and really only needed good docs and examples, not an SDK.
Then we have the developer side where we’re inserting libraries and their dependencies (and their dependencies and their dependencies) into our systems. While we say we’ve reviewed them, we haven’t. While we say they’re from trusted sources, we don’t know. While we think they’re okay license-wise, we haven’t checked. Our supply chain could be compromised but we have to count on tooling to tell us.
When we combine the first two risks, we get our third: SDKs change. As the API grows, gets better, or focuses on certain use cases, the SDKs must change. As the underlying language and frameworks change, the SDKs must change.
As each of these systems change and get more complex, these risks only multiply making our system brittle and more frustrating over time. How many times have you tried to update one library only to download a dozen supporting libraries in the process? At best, it’s annoying. At worst, it’s thousands of lines of unknown code with new oddities and bugs.
What if it didn’t have to be that way?
Fixing SDKs from Hello World
Our original goal with SDKs was noble:
We want to make our systems and devices easier to use.
We can and should build better systems but that’s a worthless answer because “better” is subjective based on your team, their average skill level, tooling, and more.
What if our goal was right but our approach was wrong?
Shipping SDKs Tomorrow
First, let’s make sure our team publishes good machine-readable documentation. For web/http APIs specifically, that’s probably OpenAPI but could be any sort of parseable documentation. Most likely you’re doing this in some form already.
Then let’s make sure we have good examples clearly showing well-formed correct requests and the expected output. For bonus points, we can include some bad requests and show the resulting errors.
Now as consumers of the API, we use the documentation and examples, combine them with a generative pre-trained transformer (GPT), and specify the language and framework we need. The output is an interface layer custom to both the language and the API.
The first version will have some rough edges but later versions will only get better.
From here, consumers can train the models using their existing codebase and those they (or anyone) has vetted as being high quality, aspirational, and appropriately licensed. Consumers can add their team’s coding standards to ensure it follows patterns and practices of the rest of their project. The most diligent teams will still build integration tests but as long as the original examples are accurate, we already have them in another form.
The Risks of Tomorrow’s SDKs
As a company, we’re no longer shipping SDKs via a package manager. Instead, we’re shipping machine-readable documentation with expressive examples so we need to spend much more time and effort here. We’re also not making bets up front on which SDKs are best. Instead, we have to choose which tools and models we recommend.
As a team, the interface layer matches their language, framework, and team idioms making it easier to use and integrate. Further, they’ve reduced their dependencies to things they trust or can be made trustable. Finally, they’ve taken control of their SDK update cycle choosing when and how to update.
But since nothing is without tradeoffs, we have two new risks:
First, the company producing the API loses a central point of coordination and control within different language ecosystems and language-specific examples become almost impossible to reuse.
Then, as a result, Support becomes more challenging because the Javascript snippet I generated won’t work for you. Both the company and the team need to understand how to debug and understand what’s happening in the layers of the system.
Kill your SDKs Now
The days of SDKs are numbered. While they were useful in the pre-GenAI internet era as APIs moved from bizarre ideas to mission critical services, they’ve outgrown their usefulness. But in order to move to the next stage of the internet, we’re going to need good machine-readable documentation and good data to train models on.
If you’re not documenting your APIs in OpenAPI, you’re already late but you can still catch up. If you don’t, the tools, systems, and eventually teams simply won’t find your API and know what to do with it.
I’ve talked about this as a less well-formed idea in “How ChatGPT will solve all API problems.. except yours.” For further reading, my friend, co-author, and occasional colleague James Higginbotham has talked about this and more in his book “Principles of Web API Design: Delivering Value with APIs and Microservices.”
