In May of 2018 WeTransfer released our Public API, with the goal of enabling everyone to share anything from everywhere, to create spaces for anything they can imagine, and to build things we couldn’t even dream of making.
In May of 2018 WeTransfer released our Public API, with the goal of enabling everyone to share anything from everywhere, to create spaces for anything they can imagine, and to build things we couldn’t even dream of making. Drawing from my experiences as a backend engineer on the team responsible for bringing this piece of tech into the world, the following offers some strategies for doing right by your users when making a Public API. If you want to get building on our API right away, might I suggest you check out our collaboration with the brilliant folks at Glitch.com, which you can use and remix however you’d like here, and/or I’ll embed the actual project later on in the article. For now though, let’s dig in…
Who are we? What is this? Why is this? Are you that money transfer thing?
For its almost ten years of existence, WeTransfer has been a file transfer service. Anyone could go to wetransfer.com, upload files, then send them where they needed to go. Done and dusted. In early 2018 our entire product structure got turned on its head: no longer would we be “that file transfer thing”, we would instead become a platform that would enable you to do multiple things with the content you make/consume/dream about. Part of that transformation involved forming a Public API team to allow people outside the company to build on our platform.
For those who will read this and not know what an API is, I recommend you read this very helpful article by our very own support team before going forward. It’ll contextualize what I’m about to talk about, provide you with links and details for finding out more / contacting us and even includes a video of me talking about the API (which I’ll also link down below) if you’d like to see the man behind the medium.
So: how has it gone so far?
There are — in my humble opinion — four big goals for a Public API: is it (1) secure, (2) stable, (3) collaborative, (4) a community? Security and stability form a foundation to build a collaborative tool on, and only with enough collaboration can you dream of a community. All four elements impact the lives of all of our users, whether they are users who build on the API or those who use the things that are built on the API.
Security
Security is ever-ongoing, but we’ve taken some steps to double-down on security efforts for our users. We won’t be allowing email transfers, this way we can try to prevent the creation of ✨amazing ✨ automated spambots. Additionally, we’re using an AWS service called API Gateway (which I’ll talk more about later) in part because it abstracts a layer of verification from our own systems. When we set out to develop features we want to consider security on both implementation and toolchain levels. We’ve also hired external auditors to run a security audit on the API, the developer portal, and (one of) the backend systems we use to power the API.
Stability
Maintaining stability is also an ongoing process, but we’ve made decisions from the start to support a stable service. API Gateway gives us the ability to respond to changes other teams make to our backend systems without affecting you, the API consumer. We can also make changes to the endpoints (new mappings, clarify error messages etc.) without negatively impacting the user experience. One thing we’re yet to solve is how to make sure we’re properly communicating if the underlying systems experience issues. We’re often the first to know when these things happen — we’ll notice increased latency or an uptick in errors — but communicating and interacting around that is still an issue.
Collaboration and Community
We’ve been working on a couple of glitchy things 😉 for collaboration and community but to be honest we are still iterating over what the API should and can be. So this is very much a work in progress. That being said we haveseveral community-contributed and community maintained SDKs (software development kits), and we participated in the Amsterdam ROSS Conference, an open source hackathon.
People have also made a wide range of applications using the API:
It’s also being used to collect a massive variety of things and creatures:
We’ve also heard stories of users working with the API to transmit data from low-bandwidth archaeological digs and other research sites, applications for playing card games, systems for integrating with a wide range of applications and devices, and somewhat notoriously our lead frontend engineer made this cat detector. If all of this sounds too exciting for words and you just want to start using the thing feel free to remix our projects on Glitch.
Some of the thoughts that went into making the thing
Since we’re (always) talking about users: it’s important to follow the example of Anthony Bourdain when it comes to making a public API. (Stay with me here).
Bourdain (rest in power) presents a powerful example of meeting people where they are — he could easily have spent his post-fame life shooting documentaries in only the highest of high-end kitchens, but instead, he would meet his subjects in their homes, the places they hang out, and so on. He would cook with them, play with them, get blind-drunk with them. As developers, we can do the same thing — meet our users where they are (no matter their level of skill), craft systems that can be used for use cases far beyond their original purpose, and relish in enabling people to create whatever their bizarre little hearts desire. (I mean that last bit as positively as possible.)
How can we do this practically? I would suggest five basic principles:
- Make your docs as friendly as possible — avoid phrasing like “as you can see”, “simple”, “easily”, etc. This is particularly difficult for a company like WeTransfer, where ‘simplicity’ and ‘ease of use’ are part of our core product. We have often had to make a distinction between “simplicity” — it doesn’t take much practical effort to accomplish a goal — and “simple” — the effort itself requires little comprehension of complex topics.
- Where possible be as explicit as you can. Get people who are not used to the systems you are exposing to review your documentation. Are you using internal terms or terms that make sense because you deal with them every day? For our first implementation of the API we were using a backend system that creates “boards” while skinning them as classic “transfers” — but ultimately this caused confusion because the copy would refer to a board being prepared rather than a transfer.
- Use consistent/reliable/useful error messages. We use AWS’ API Gateway to provide the actual API interface and it has a rather strange error condition: if you hit an non-existent endpoint on a Gateway powered API it will return the error message “Missing Authentication Token”, even if you provide the token in the request. This proved confusing (internally!) so we re-mapped it to properly return a “Not Found” error message.
- Don’t assume a particular level of skill. You never know who’s going to want to use your API, or what kind of history with programming, they’ll bring to the table. Our API involves multi-part-put file uploads, a process that can be and has been new to many users of wildly varying programming experience, especially thinking of files as (if I may compress a lot of detail here) strings of data, that can be cut up and reassembled at will. It’s an ongoing conversation about how we can make that make the most sense.
Explain concepts as well as procedures! You can have the best and most complete endpoint listing
ever, filled with necessary headers, error conditions etc. but if you don’t explain the concepts at play this can knock users out of their flow (no matter their level of experience) and limit the uptake of your product. This is also something we’re constantly working on — we’ve, etc. If you know the perfect way to do this, our. 😉
Okay, that’s all well and good, but what did we actually build to support all these wonderful users? As it stands the WeTransfer Public API is composed of two main parts: our developer portal and the API layer itself. The portal is your ‘classic’ Rails app + OAuth setup. Currently, we only provide GitHub and Google OAuth, but we’re hoping to change that soon. One of the lovely things about a rails app is that it dramatically reduces the “bus factor” of the project — if something needs to be developed/ updated/changed, anyone on the team can work on it. (Our lead frontend engineer wrote a significant amount of the Rails portal!)
This is especially beneficial because the API itself is an API Gateway x Lambda setup, configured and deployed using terraform. We don’t have any ops people on the team, so a lot of development effort goes into making sure this is working as smoothly as possible. The more we are able to divide tasks and share the workload, the more time we can spend focusing on creating and shipping really cool stuff for our users. Hashtag agile etc.
What’s next?
More of the same, in that we’ll continue to apply the principles described earlier towards continuing API development. We released V2 of the API (!) in late September 2018 and we’re eager to see what people make. We’ll also be working on ways of providing support (right now we just provide an email address: developers@wetransfer.com and a non-interactive but lovely help-desk section), more options for login, an improved onboarding experience, and of course we are always looking forward to seeing anything you build.
Anthony Bourdain once said “You have to be a romantic to invest yourself, your money, and your time in cheese” and I’d like to think the same applies to a really good Public API.
Did this make you want to craft some delicious cheese — er, make some amazing things on our lovely API? Well, head right on over to: https://developers.wetransfer.com. Let me know in the comments, on Twitter (twitter.com/additionaltext), or send us an email (developers@wetransfer.com) and keep us posted on how it goes! Ditto if you plan to implement any of the ideas I’ve discussed in your own projects. 🙂
