jump to navigation

Why I Don”t Like REST (The Shorter Version) April 22, 2016

Posted by PythonGuy in REST, Uncategorized.
add a comment

I posted earlier why I don’t like REST and avoid it at all costs. I’ve been trying to find a way to summarize it in a succinct idea, or find the one thing that everyone would agree is a “killer” feature, in that it would “kill” the idea of REST altogether.

I think I have it.

REST is bad because REST does not map to any language well. Or rather, any RPC would map much better to any language.

That is, you always have to write boiler-plate code on top of REST just to make it useful in your language. Whereas, with an RPC, boiler-plate code can be auto-generated for any language.

There are 2 reasons why this is:

One, REST only supports 8 operations. They are:

  • GET a list of stuff.
  • POST a list of stuff, replacing the list.
  • PUT a list of stuff, adding to the list.
  • DELETE an entire list of stuff.
  • GET a single item.
  • POST a single item, creating it brand new.
  • PUT a single item, replacing it.
  • DELETE a single item.

If you want more than these 8 operations, or if you want to remove some of these operations (and you typically do), then you’ll need to extend the REST API to handle additional things. There is no simple definition on how these extensions would even work, but that’s not the issue. The issue is that REST as an API is either too much or too little, and rarely, just exactly enough.

Two, all languages are function-based, or rather, procedure based. Every language has at its core something to process individual commands, and something to describe how to execute or run commands with further commands. These commands, which are called procedures or functions, are the fundamental unit of programming. It only makes sense that any layer between two programs should also be defined in terms of functions.

RPCs don’t need any operations by default. Some may have some default operations that define or characterize the service, or maybe some directory operation that tells the client what operations are available and what their parameters are.

To summarize, consider the fellow who has to write the API client code. He is going to put all the code into a module or package of some sort. He is going to introduce a new data structure into the language that represents the connection to the service, along with whatever authentication tokens are necessary.

Then he is going to define the operations that the service allows. He has to give them some sort of name to abstract away the RESTful nature of the service, or else he has to have a leaky abstraction that bubbles up features of the REST service to the ultimate end user of the client library. He has to pick and choose which to keep and which to not implement, and then decide what to do if someone tries to do one of the missing operations. It’s all a terrible mess when using REST.

RPC, on the other hand, is very clear cut. You give your methods the same name as the RPC methods, using the same parameters, adapted to the language of course. It’s so trivial to write a client library that any programmer writing one would be thinking of ways to automate the process using a template. Indeed, most languages would probably tolerate a data-driven architecture for defining the client API.

If you want to build tools, your architecture is too complicated March 8, 2016

Posted by PythonGuy in REST.
add a comment

It struck me, reading about Flasgger and Swagger and such, one of the reasons I detest REST API. (I can write a lot about why you should never, ever do REST, but I doubt you will be interested.)

This reason is the same reason I detest Java and pretty much any IDE. It’s a rather simple reason: Complexity. Or rather, giving up in the face of it.

If you’re building a system that is so complicated that it requires a computer to explain it to you, then you’re doing it wrong. If the computer can make your job easier, just let the computer do your job for you. Go up a higher level of abstraction by writing a library that will solve the problem once and for all, and then move on with your life.

REST APIs are artificially and unnecessarily complex. Every other API, or at least the good ones, follow a simple paradigm: Publish a function that takes parameters and returns results. This function-centric model is apparent in all but the most ambitious object-oriented frameworks. Even then, what you’re really looking at is a model where certain functions, such as object instantiation and get/set attributes, are assumed or hidden rather than exposed.

The truth is that I have little need for things like Sphynx. Wonderful as they are, you should get more out of reading my code first-hand with a plain old text editor than you will ever get by looking at a bunch of beautifully formatted HTML pages. And I try to minimize my comments. I subscribe to the philosophy that if you have to explain it, you’ve failed. This idea comes from The Design of Everyday Things. Things should declare what they are and what they do by their very shape and existence. It should be so simple that a child can use it. People shouldn’t be scared of doing the wrong things with it because doing the wrong thing should be very difficult to do.

This isn’t always possible, of course. When faced with complexity which you cannot seemingly overcome, you have a couple of choices. For me, it always boils down to (1) make it simpler by more problem solving and a deeper understanding of the problem, of (2) document it and ship it. As you can probably tell, I keep choosing (1) until the project is due. When I am forced to choose (2), there is usually not much left to document so the path to deployment is pretty clear.

When people write software which takes complicated things and automatically documents it, I cringe inside. We shouldn’t be dealing with those sorts of things, especially when they are trivial enough that a computer can understand it and create documentation for us. We should be dealing at a higher level of abstraction. We can bury complexity behind abstractions. Humans handle abstractions really well. We should find the simplest abstractions that still work and make that our interface with humans.

So please, stop writing REST APIs. They are needlessly complex. Instead, write functions that get exposed.

Why I Don’t Use REST October 16, 2015

Posted by PythonGuy in HTTP, Networking, REST, Ruby, TCP/IP.
1 comment so far

It seems REST is pretty entrenched nowadays. Somewhere along the line, someone decided that all web developers must use REST for their API because there’s really no better way to do it.

I disagree.

I’ll begin with my criticism of REST. I end with my proposal for a replacement, a proposal that is not new nor unique.

My first criticism is that REST is emphatically NOT a standard. There is no paper or publication that I can find that proposes a REST standard, let alone one that has been agreed upon by many vendors. As far as I can tell, the best definition of REST is “the way Ruby on Rails does it”. The only reason that definition wins is because Ruby on Rails is more popular than the numerous other platforms that provides something they call REST. This criticism is often ignored, but I believe wrongly so. Without a standard, how can one learn let alone comply with it? Someone somewhere should’ve documented by now what REST means, and the fact that no one has means that probably no one can. Or, in my mind, that once they set their keyboard to documenting it, they realize how flawed and horrible it really is.

The second criticism is that it doesn’t acknowledge or even comply with a critical component of the internet known as network layers. I can’t blame new programmers and developers for their ignorance, but I can blame experienced people for not knowing the basics of networking. It seems in developing the internet, its architects made it too seamless and easy to use such that people who have no business expanding it are fully capable of doing so.

Let me explain this to you in a nutshell. If you are a developer, you should read OSI Model at Wikipedia to get the bigger picture.

You may have heard of something called HTTP and TCP/IP. These aren’t just fancy acronyms, they are critical components that make the web possible. See, when two computers attempt to communicate with one another, there is a lot that needs to happen. At the hardware level, pulses of light or electrical potential are sent down cables, or radio waves are emitted. Those pulses are received, oftentimes distorted, by another bit of hardware. What is important is that both computers not only agree on what kind of wire or cable or radio waves are used, but what those pulses mean and how they are to be interpreted. These pulses contain all the information you care about — a photo of your grandmother, a message from your boss, a request to load a customer record from a server. However, at this level, the Physical Layer, the architects who design the protocols and the engineers who hook it all up don’t care. All they care is that bit of data are sent back and forth according to those protocols. You may have heard of some of these protocols but only because you’ve had to connect the wires or buy the hardware.

Above this layer sits the Data Link Layer. Now the data sent by the Physical Layer has meaning in terms of which computer is speaking and which is supposed to listen. This layer consists of acronyms you’ve never heard of unless you work at an ISP.

Above that layer sits the “IP” of “TCP/IP”. It is the Network Layer. This layer handles routing, the conveying of a message from one computer to another through a chain of other computers. You may have heard of IPv4 and IPv6. These are two protocols that can be described as envelopes with a “to” address and a bunch of data inside. It is important to note that, like the other layers, it doesn’t care what is in the packet or how the packets are flying from one machine to another. In fact, in the journey of a packet from your computer to the server, it is likely that several different kinds of Physical and Data Link Layers are used, and IP doesn’t care. This layer is where we get IP addresses and ports from. Everything below this doesn’t even know an IP address exists.

The “TCP” in “TCP/IP” refers to the Transport Layer. This is how data that cannot fit into a single packet is conveyed between machines. Above TCP are a few other layers, some you may have heard of.

On the top of the hill of the OSI model is the Application Layer. Here sits HTTP, the granddaddy of them all. This is where you can finally tell another computer to send you an HTML document. But this is not the end. Above HTTP sits your web browser or your web server, and ultimately, your application. When the user types in your website’s URL, they are interacting at a level where they don’t even know HTTP exists.

This is a long detour, but I did it for a purpose. See, at each of these layers, there is an abstraction going on that allows all the complexities of the lower layers to be bottled up. This frees the architects, engineers and programmers working at higher levels to write the simplest code possible that is still robust and efficient. If you had to worry about what kind of internet connection the client has at the HTTP level, for instance, sending different data if they were on WiFi or if they had an ethernet cable, or if they were communicating over a cable modem or DSL, why, you job would be impossible.

When I see people using HTTP as if it were part of the app that they were writing, I want to scream. What happens when HTTP is replaced with something better that doesn’t have HTTP codes in it? I mean, we could be using IP packet headers to send our error codes, but we don’t, because we know that IP can change (and indeed, it is changing, from IPv4 to IPv6!) Or rather, I believe REST developers don’t use packet headers because the authors of the software that implement IP have packaged everything up so nicely that they don’t even know it exists.

Now, sitting atop HTTP, REST developers see the nuts and bolts and say, “Gee, I’d like to use some of that for my own purposes.” No, this is bad behavior. Just because you have knobs and levers doesn’t mean you should pull them. You should use the simplest subset of features you can get away with, and leave the knobs and levers to be pulled by those who truly understand what they are for and how they work.

The final criticism of REST is that it is object-centric. To programmers in languages like Java, this seems natural, even elegant. But to everyone else, it is horrifying and difficult. See, it has been proven that the fundamental element of programming is the function. Meaning, if you have functions alone, you have everything you need to write any type of program. Objects, on the other hand, are not sufficient. Only when you wrap functions into objects (which we call methods) do they become powerful enough to write any type of program. You can create objects with functions, but you don’t have useful objects without them.

I understand why REST is object-centric. HTTP was originally written to be a document store, and documents are a kind of object. We have since repurposed HTTP to be far more than a document store, and browsers to be far more than document fetchers and readers, but that is what it was built to be. As we’ve repurposed HTTP, we’ve left behind certain features that a document store might need. But a document store is not enough to build a full-fledged application. It is only part of an application.

Had we done things right, we might have created a new protocol for building apps on the internet. Indeed, you could say that internet architects are hard at work developing that protocol right now. Unfortunately, they are hamstrung because they have to build it on top of HTTP, because there are so many people who built their application not on top of HTTP, but deeply integrated with it. I can explain why this happened, but that is another story. There was a time when people were inventing new Application Layers to suit their needs, a time when SMTP and HTTP and more were brand new. If we want a new way to communicate over TCP/IP, we should be inventing new protocols, not torturing existing protocols.

Here is an important example of why having an object-centric API doesn’t work. Consider how you might fetch a customer record. You would access the /customers/234 for the 234 customer, right? Well, what should that return?

  • The customer might want to see all of their information, even the bits which he wouldn’t want to share with anyone else, including the company.
  • A Customer Service Representative (CSR) would like to see the history of all communications with that customer, as well as all actions taken on that customer account. Maybe there is an additional REST URL like /customers/234/history? How would the CSR filter out which history he’d like to see? /customers/234/history?from_date=20150101&type=csr_interactions maybe.
  • The person who ships the package to the customer is only interested in their name and address. They might want to know if there are any special flags, such as holds on shipping to the customer, or maybe flags that indicate there should be an email or text message sent when the package is shipped.

As you can see, the idea of “customer” is different for different people in different roles. Some would like some data, others would like other data. How do you specify which data you need? You have to add additional parameters to the REST protocol, and things start to get really messy. What happens when you want to move all the queries for full customer information to one class of server, and the queries for customer interactions to a different cluster? Do you setup a fleet of servers just to reverse proxy the requests based on the URL, or do you setup different services entirely?

REST does not give an answer to this, and indeed, trying to make things “RESTful” only makes the complicated situations far more complicated than they would have otherwise been.

Now, my proposals.

We live in a world where HTTP is the de-facto communication standard. People run their apps in browsers, and the browsers only know how to speak HTTP. Web Sockets exist (which are basic TCP/IP connections) but for the same reasons that people were forced into HTTP, Web Sockets are unlikely to succeed. It seems that we have to plug all of our application into a single port (443) and it seems we have to stick to a single protocol because too many people do not understand the full power of the internet. (Maybe games programmers can save us. They’re the last hope to building an internet on HTTP.) So my first proposal is to stop using HTTP except as a document store. We should make “web server” synonymous with some other protocol, and leave “HTTP server” to be a special kind of server that only stores, retrieves, and updates documents.

However, even though we cannot decouple our apps from HTTP yet, we CAN start writing apps that are HTTP-unaware. Here’s what such an app would look like.

  • The app code would be retrieved from some static source. This would contain all the instructions on how to start the app, including any libraries needed to bootstrap the app. It needs to be static so that it can be cached. As much of the app as possible must be static for this reason.
  • App data would be retrieved over an API layer.

That’s pretty much it.

Now, HTTP fits the bill for retrieving the static app data to a T. It is the perfect solution for this, provided that the data is truly static. Given the demand for things like CDN, this is rapidly coming true. If you can’t store your app in a CDN, it’s not static enough and you need to make it that way. But keep in mind, that if someone came up with a better way to store and retrieve static documents, you should be able to quickly and easily port to that system because you only had static files in your HTTP servers! And don’t think it isn’t coming!

The benefits of having a static file be the base of your app is that it can be cached, distributed, shared, etc… at no cost to you. We’ve seen things like Bittorrent excel at things like this. If your app can’t be distributed through Bittorrent, it’s no good to you or anyone else. When you wake up one morning and half the world wants your app, if it’s not static and distributed, you’re going to have a very bad day. This need is going to drive us away from HTTP one day. We’re already publishing our apps by uploading them to the Google and Apple store. HTTP is going to disappear just like so many protocols before it disappeared!

The API, on the other hand, should exhibit the following features:

  • Function, not object, based.
  • Functions take any kind of parameters in any configuration. (Python is a good model for this. Many languages are similar.)
  • Functions may return a result (a blob of any kind of data) or raise an exception (which is also any kind of data.)
  • A session which would contain the global or dynamic context in which the function should operate.

That’s pretty much all the API has to do. Anything on top of this is unnecessary complexity.

Now, some of the API calls return static data. Those should be called documents and stored in HTTP, and retrieved not through the API but through HTTP. So don’t put static documents in your API, keep it separate and have your API point the client to them.

WebSockets can be the foundation for this API. We can fall back to communicating over HTTP, but we must be very careful not to tie our APIs to HTTP. Meaning, we should be able to switch the API to use WebSockets, raw TCP/IP, or HTTP, with the flick of a switch.

So, build your application on HTTP today, but build it in such a way that it doesn’t have to run on HTTP. That way, when the future comes, you’ll be ready. And you’ll also be building something that others can build upon reliably.

In conclusion, I wanted to show you why REST is not a good solution. I wanted to teach you what we can replace REST with. I don’t give any specific guidance on what the future should look like, just broad pointers on where it can and probably should be.

I do appreciate comments, positive or negative. Keep in mind that ad hominem, red herring and other logical fallacies should not be employed. If you don’t know what those are and why it is bad to use them, you should go study up on logic and logical fallacies before commenting on anything ever again.