RESTful API has gained wide adoption in API community as well as within Sophos. It was first introduced by Roy Felding in Chapter five of his dissertation on network based software architectures. The key principles of REST (REpresentational State Transfer) involve separating your API into logical resources. These resources are manipulated using HTTP requests where the method (GET, HEAD, POST, PUT, PATCH, DELETE) has specific meaning.

REST is not a standard, rather an architecture style with a set of constraints. It is not tied to HTTP, but associated most commonly with it. REST works well with a number of content types, JSON, XML, Plain text, etc. JSON is widely used particularly for the following reasons,

Ubiquity
Readability
Flexibility
Simplicity
Scalability

This document outlines the standards we should follow. And most of them are widely adopted by popular APIs, such as Facebook, Twilio.

The Standards

Resources and URL Structure

There should be only two types of resources, collections and instance resource.

REST.1: Must keep URL simple and intuitive

REST.2: Must not have verbs in the URLs

REST.3: Must use HTTP operations on the collections and instance resource

REST.4: Must use concrete names rather than abstract names

For example, use /v1/cats instead of /v1/animals to represent cat resource.

REST.5: Must use plural rather than singular nouns

REST.6: Must keep IDs (identifiers) globally unique and opaque

REST.7: Must avoid Using sequence numbers for IDs, recommend to use UUIDs or Url64 numbers

Resource	POST (create)	GET (read)	PUT (update)	DELETE (delete)	PATCH (partial update)
/books (collections)	create a new book	retrieve all books	not recommended s it's not practical to update a collection of resources	delete all books	not recommended s it's not practical to update a collection of resources
/books/a1b2c3	could be used for partial update but not recommended as we should use PUT/PATCH for updating resource	retrieve the specific book	if exists update the book if not, error All the fields must be provided	delete the specific book	if exists update the book if not, error Only providing the fields needed to be updated

Relationships and Associations

Resources almost always have relationships to other resources. Once you have the primary key for one level, you usually don't need to include the levels above because you have already got your specific object.

REST.8: Don't make the URL deeper than needed, maintain a structure of /resources/identifier/resource

/library/x6y7z8/books - represents all the books for library identified by x6y7z8
/library/x6y7z8/books/a1b2c3 - represent one book identified by a1b2c3 for library identified by x6y7z8

Response Filtering, Sorting and Searching

It's best to keep the base resource URLs as simple as possible. Complex result filters, sorting requirements and advanced searching (when restricted to a single type of resource) can all be easily implemented as query parameters on top of the base URL. Suggesting that query parameter uses snake case and be case sensitive while the value be case insensitive.

REST.9: Use query parameters for response filtering, sorting and searching

GET /library/x6y7z8/books?author=BobSmith - retrieve all the books written by Bob Smith from the library identified by x6y7z8

REST.9.1: Return 400 if any query parameter or value is unknown. Of course, the same applies to unknown fields in the payload. Must never silently ignore invalid fields and values in the payload.

REST.10: Return an empty array ([]) in response when there is empty data

REST.11: Return full representation in response when feasible for POST and PUT/PATCH. When this is not feasible, use include_payload query param to indicate if the full response should be returned or not.

PUT /books/x6y7z8?include_payload=true

REST.12: Use "fields" to control partial representation in response

GET /books/x6y7z8?fields=title, author - retrieve only the title and author for the book identified by x6y7z8

When design a service end point, consider how a REST API client would use it, NOT just solely focus on GUI case.

REST.13: Use limit query parameter to specify the number of results returned per page of output,. Define a default value

REST.14: Provide a link to next page of results so clients don't have to manipulate URLs in code to get to next page

{

  "meta": {

        "pagination": {

           "next_link": "/v2/customers?next=c3RhcnRBdD0zJmxpbWl0PTI"

        }

  },

  "results": [

    {

      "id": 196,

      "status": "ACTIVE",

      "fax": "318-978-7575",

  ...

}

Versioning

An API is never going to be completely stable. Change is inevitable. What's important is how that change is managed. Well documented and announced multi-month deprecation schedules can be an acceptable practice for many APIs. It comes down to what is reasonable given the industry and possible consumers of the API.

REST.15: Never release an API without a version and make the version mandatory
REST.16: Version identifier should be in the URL as far to the left as possible, and only use integers (e.g. v1, v2, v3, etc.). Do not use point release in URL, such as v1.2, or v2.3.1, etc.

Data and Time Format

REST.17: Must follow ISO 8601 standard to encode dates and time stamps in JSON requests and responses.

An example, "2013-01-23T13:48:44.108Z" and for Java apps, may use joda-time to help formatting and parsing the date and time.

Content Negotiation and Character Set

REST.18: Must use Request Headers "Accept" (For specifying the content types for GET) and "Content-Type" (For PUT/PATCH and POST) to specify what formats the response is in.

REST.19: The proper and only valid "Content-Type" for JSON is "application/json"

JSON Formatting

JSON (JavaScript Object Notation) by nature follows JavaScript convention hence it should use camelCase. However many popular JSON APIs use snake_case, and parsers such as Jackson for Java expect underscores.

REST.20: Use snake_case for property names, e.g. use first_name instead of firstName. REST.21: Boolean values must be lower case and can only be of the values true, false, or null

REST.22: If a property requires quotes, double quotes must be used. All property names must be surrounded by double quotes. Property values of type string must be surrounded by double quotes.

REST.23: Do not include the Object name in the JSON for a single resource or for a collection of the same resources.

For example use the following to represent a book

{

  "title":"RESTful Web APIs",

  "author":"Leonard Richardson",

  "isbn":"978-1449358068",

  ...

}

Instead of

{

  "book" : {

      "title":"RESTful Web APIs",

      "author":"Leonard Richardson",

      "isbn":"978-1449358068",

      ...

  }

}

Response Status Codes

HTTP defines a bunch of meaningful status codes that can be returned from your API. These can be leveraged to help the API consumers route their responses accordingly. There are in three categories,

200s - everything is good
400s - the application did something wrong - client error
500s - the API did something wrong - server error

REST.24: Must keep it simple and use well understood HTTP status codes.

Status Code	Meaning	Used For
200	Ok	Response to a successful GET, PUT, PATCH. Can also be used for a POST that doesn't result in a creation.
201	Created	Response to a POST that results in a creation. Should be combined with a Locatio n header pointing to the location of the new resource
202	Accepted	Response to a POST that resulted in a job being accepted and scheduled to be processed asynchronously. The response should contain a link to check/pull the status of the job.
204	No Content	Response to a successful request that won't be returning a body (like a DELETE request)
300s	Redirect	There are several 300 codes, use them carefully. Most of the time they are used to deprecate older versions of the API, and redirect to new ones; or used when maintenance is being performed.
400	Bad Request	The request is malformed, such as if the body does not parse
401	Unauthorized	This is actually for authentication failure, when no or invalid authentication details are provided
403	Forbidden	This is actually for authorization failure, when authentication succeeded but authenticated user doesn't have the right to access to the resource
404	Not Found	When a non-existent resource is requested, e.g, anything in the url is wrong or non-exist
405	Method Not Allowed	For resources that don't support a specific HTTP action.
406	Not Acceptable	The generated response entities which have content characteristics are not acceptable according to the accept headers sent in the request
409	Conflict	For trying to create a resource which is already existing
410	Resource Gone	Indicates that the resource at this end point is no longer available. Useful as a blanket response for old API versions
415	Unsupported Media Type	If incorrect content type was provided in the request headers
500	Internal Error	Internal Server Error (System is broken, such as RunTimeException occurred)
501	Not Implemented	The server does not support the functionality required to fulfill the request.
503	Service Unavailable	Service Temporarily Unavailable (System busy or down for maintenance)
504	Gateway Timeout	Internal server is busy and can’t process the request within predefined timeout limit

Refer to the comprehensive definition of HTTP status code for more details.

Error Handling

An API should provide a useful error message in a known consumable format. The representation of an error should be no different than the representation of any resource, just with its own set of fields.

REST.25: Must use an array of errors to include multiple errors in a single response.

[

   {

      "error_key":"customer.status.invalid",

      "error_message":"Invalid status value. Valid values for customer status field are ACTIVE or PENDING"

   },

   {

      "error_key":"customer.name.invalid",

      "error_message":"Invalid customer name provided. It can not contain special characters"

   }

]

REST.26: Must provide a "key" of sorts along with an error message.

For example, customer.email_address.invalid. This way, it allows for clients to come up with their own error messaging and not just parroting back our messages to their users or worse - parsing our error string.

REST.27: Error messages must never include internal system information such as class names or stack traces.

Documentation

An API is only as good as its documentation. Most developers will check out the docs before attempting any integration effort. The docs should show examples of complete request/response cycles. Preferably, the requests should be pastable examples - either links that can be pasted into a browser or curl examples that can be pasted into a terminal.

REST.28: The documentation must be easy to find and publicly accessible.

For internal REST API, Swagger is suggested tool to use.

Pages

Saturday, November 27, 2021

REST API