apinomics: January 2016

The other day, I was asked to provide some fundamentals on API design in a Java environment. So, I tried to compile both, the obvious basics and some additional, less present aspects. Looking forward to your feedback. While our company is implementing a lot in Java, most of the principles in this post are more or less technology-agnostic.

Best practices – the basics

Resources

The resource is the fundamental concept behind RESTful APIs. A resource is an object of a particular type, carrying associated data, typically having relationships to one or more other resources, offering a series of methods to operate on it. As this document is intended to be a best practice guide, you’ll have to refer to the list of books and sources at the end if you want to dive deeper into the details of the REST concept.

Resource types

There are two basic types of resources:

instance resources and
collection resources bundling together a number of resources

Note: resources can of course also be nested

Sample collection resource

/service/applications

Sample instance resource

/service/applications/app8762343

Resource type names

Resource names shall always be nouns, not verbs (nouns indicate “real” resources, verbs falsely suggest an RPC – remote procedure call – mechanism).
Always use the plural form for resource names (you’ll typically have multiple resources of the same type).

Resource identifiers

Resource identifiers should rather be UUIDs than simple numbers. Adding this type of opacity is less an invitation to playing around than potentially sequential numbers.

HTTP verbs

Make use of the well-known HTTP verbs to indicate the intended behavior.

Verb	Behavior	Description
GET	read	Retrieve a representation of the specified resource
HEAD	read	The HEAD verb is used to have the server act like in a GET operation but then have it only return the headers and not the body. It can be used to basically retrieve meta information (via the headers returned) about an entity without transferring the entity body. It is typically used to check hyperlinks for validity, accessibility, and/or recent changes. The headers returned have to be the same as if a GET operation with the same path and query parameters would have been requested.
DELETE	delete	Delete the specified resource
PUT	create or update	In any case, PUT must be idempotent. Use PUT to create a resource when the identifier of the resource is known/defined by the client. Use PUT to update an existing resource when the representation of the resource provided in the request body constitutes a full replacement.
POST	create or update	POST is not idempotent. Use POST on a parent resource to create a new instance. Use POST on an instance resource to perform a non-idempontent update.
PATCH	partial update	Use PATCH to make a partial update to a resource (e.g. if you want to change just one field or property). Obviously, PATCH is not idempotent.

Please pay special attention to the differentiation between create/update via PUT and POST respectively. HEAD and PATCH are rather optional in nature and will probably not occur as often as the other verbs.

Media types

In the realm of web services, XML and JSON clearly have become the predominant formats for data provided in the request and response body.

XML
XML has been around for a longer period of time than JSON, it is more human-readable, and there are various standardized ways to validate XML data (e.g. schemata).

JSON
JSON on the other hand has gained more and more popularity over the last couple of years as it comes with a smaller payload in comparison to XML.

Which format to select?
If possible, process and provide both formats as a baseline, depending upon the requirements of the attached clients. Most JAX-RS implementations provide built-in support for both formats, or additional providers can be easily plugged in. If providing multiple formats is not an option, the format generated by your service should at least be based on one of the two (JSON, XML), preferably JSON due to the smaller payload (without losing readability) and the increasing popularity.

Format specifications
Then go ahead and use a combination of format specification and parsing rules, e.g. application/vnd.sb.tariff+json. This provides a more specific idea of what you are processing/producing. Ideally, use the vendor tree for format specification names (as in the example above). See the “content negotiation” section later on how to use the Accept and Content-Type headers in this context.

Always accept and return objects, not arrays

XML always has its mandatory root element. Apply a similar principle to the JSON data you accept and produce. This means: use JSON objects, not arrays. So a “good” payload would rather look like this

{
    “items”: [ 1, 2, 3]
}

and not like this

[ 1, 2, 3]

The latter one does not provide any contextual information, and it is not well suited for later enhancements. The JSON object approach however can be easily extended with no hassle for the existing clients:

{
    “snapshotTimestamp”: “2015-11-12T12:00:00.000Z”,
    “items”: [ 1, 2, 3]
}

camelCase for parameter names

Always use camelCase for parameter names, underscores are unconventional in this context.

{
    “userName”: “Tom”,
    “dateCreated”: “2015-09-01T23:17:02.006Z”
}

Date/time/timestamp

Use the ISO 8601 standard for date/time/timestamp formatting.

{
    “dateCreated”: “2015-09-01T23:17:02.006Z”
}

Strings vs. numbers

A number implies that a mathematical operation can be applied to it. If this is the case for a particular element in one of your payload objects, then use the number format. In all other cases, use (quoted) strings. A “good” example would be:

{
    “deviceId”: “98498”,
    “numberOfCoinStores”: 5
}

In the above example, the device id is an identifier, nothing more. So it definitely shall be a string. The second property however (numberOfCoinStores) is something that might be incremented or decremented, hence this is a good candidate for a number.

Enumerations

Make sure to model enum values as strings, not numerical codes. Everyone will immediately know what this object is about:

{
    “cardType”:  “monthly”,
    “agency”: “Houston Transit Authority”
}

So, don’t do anything like this:

{
    “cardType”:  42,
    “agency”: 17
}

Even though a machine wouldn’t bother, a human reading this will depend on telling names.

Response body

When responding to a POST request, return the representation in the response body (whenever possible).

# Request
POST /accounts/983274
Content-Type: application/vnd.sb.account+json
Accept: application/vnd.sb.account+json

{ “accountType”: “BASIC”, “userName”: “Tom”, …}

# Response
HTTP/1.1 201 CREATED
Content-Type: application/vnd.sb.account+json

{ “href”: “/accounts/983274”, “accountNumber”: “983274”, “accountType”: “BASIC”, “userName”: “Tom”, … }

Content negotiation

Accept header
Use/process the Accept header for content negotiation. Please keep in mind that a client can also offer multiple format specifications. In this case, the header values will be comma-delimited in order of precedence.

# Request
GET /accounts/983274
Accept: application/vnd.sb.account+json, application/vnd.sb.account+xml

# Response
HTTP/1.1 200 OK
Content-Type: application/vnd.sb.account+json
{ “href”: “/accounts/983274”, “accountNumber”: “983274”, “accountType”: “BASIC”, … }

Use the @Consumes annotation in your code to indicate the formats accepted / processed by your service. Use the @Produces annotation to list formats available for the response body of a particular request.
Resource extensions

When your API is likely to become available to 3rd parties, consider to also allow resource extensions as format indicators. By convention, they override the Accept header:

GET /accounts/983274.json HTTP/1.1
GET /accounts/983274.xml HTTP/1.1

HTTP status codes

2xx – Success

200	OK	Standard for a successful request
201	CREATED	The request was successful, and a new resource has been created (details in the response body).
202	ACCEPTED	The request has been accepted, but not yet fully processed. Might be used in case of first accepting jobs and then later on sending a “done” notification in a separate request once the job has been completed.

3xx – Redirection

304

NOT MODIFIED

Indicates that the resource has not been modified since the version specified by the request header If-Modified-Since. This means that there is no need to retransmit the resource, since the client still has a previously-downloaded copy. Can be useful to cut down on transfer volume.

4xx – Client Error

400	BAD REQUEST	The request is syntactically invalid/malformed.
401	UNAUTHORIZED	This should actually read “UNAUTHENTICATED” as the client needs to authenticate itself.
403	FORBIDDEN	This should be the real “UNAUTHORIZED”, because this error means that client is not allowed to request this resource.
404	NOT FOUND	The requested resource could not be found.
405	METHOD NOT ALLOWED	A request was made of a resource using a request method not supported by that resource. E.g. a PUT or POST request made for a read-only resource. Most popular JAX-RS implementations automatically generate this for you in case an unsupported verb is used in a client request.
406	NOT ACCEPTABLE	The requested resource cannot be provided in any of the formats specified in the Accept header of the request. It is up to you whether to then not provide any data in the body or ignore the Accept header and provide the resource in the format preferred by the server. Most popular JAX-RS implementations automatically generate this for you in case an unsupported format is requested (as long as you are making use of the @Produces annotation).

5xx – Server Error

500	INTERNAL SERVER ERROR	A generic error message. Make sure to provide additional information in the response body.
503	SERVICE UNAVAILABLE	The server is currently unavailable (overloaded, down for maintenance, etc. )

Errors

A client must be able to tell by the HTTP status code returned whether the request has been successful or not. In case of an error, the status code will be one from the 4xx or 5xx group. The response body shall then provide additional information. At a bare minimum, you should provide the HTTP status code, the application specific error code, and an error message generated by the application

HTTP/1.1 400 BAD REQUEST
{
    “status”: 400,
    “errorCode”: “20001”,
    “errorMessage”: “the provided object is not a valid device object (application/vnd.sb.device+json)”
}

Security Aspects

Whenever you are exposing a web service that is used by others or across the boundaries of a datacenter, usage of the secure version of the HTTP protocol is mandatory. Also for internal communication, it at least is highly recommended.

Avoid sessions

Avoid sessions. Authentication shall be done on a per-request basis to keep your design state-less.

URL based authorization

Authorization shall be based on resource representations, not URLs.

Use existing protocols

Don’t re-invent the wheel. Use existing protocols such as e.g. OAUTH. Offer Basic authentication only via HTTPS.

Client certificates

For situations where there is a stable set of clients consuming your service, client certificates can be used to make sure that the service is receiving requests from known, authorized clients.

API keys

If you decided to not use OAUTH, then for automatic clients, use API keys instead of username/passwords. API keys shall be put into a custom header. Do not define them to be a query parameter in the request URI. The rational behind that is the fact that URLs are often exchanged via email, they are visible to many intermediaries and you would hence expose your API key via those channels, too. The name of the custom header holding the API key shall be X-API-KEY.

GET  /accounts/4344/users
Accept: application/json
X-API-KEY: 9kjh7sd8ishfku67

JWT

A more recent alternative to the traditional API key approach are JSON web tokens (JWT).

Best practices – more advanced topics

Hypermedia, relationships

Please always keep in mind what REST is about. It is not just another synonym for HTTP web services, and it is definitely not an RPC-style mechanism to suggest method invocation where something totally different is happening. It is about resources and their representation which can undergo state transfers. So, always remember the classical REST constraint, the HATEOAS: Hypermedia as the Engine of Application State. Besides the actual application model related data processed and produced by your API, there will typically also be meta data describing the relationship of one resource to others. Every resource that can be accessed via your API, should have a unique URL speficied in the href attribute. The href attribute can be interpreted as a hyperlink that can be used retrieve this particular resource via a GET request. Doing this must be guaranteed to be free of any side-effects.

{
    “meta”: { “href”: “/billing/accounts/acc972578”},
    “id”: “acc972578”,
    “name”: “Tom Sawyer”,
    “accountType”: “individual”
}

Please note that in the above example, the meta data of the resource are provide in a meta object. This might look overloaded here, but you’ll see that there can be additional meta data to go here.

Relationships to other resources

Resources typically do not live in isolation. So, also provide links to associated resources. There are two types of relationships to other resources:

a direct link to an associated object / resource (using href only)
a link object where the nature of the relationship is defined via the additional rel attribute and the link itself is provided through a corresponding href

Lost you? Maybe, the following example helps.

{
    “meta”: { “href”: “//accounts/acc9579”},
    “id”: “acc9579”,
    “name”: “Addams Family”,
    “accountType”: “family”,
    “mailBox": { “meta”: { “href”: “/billing/accounts/acc9579/boxes/box5634”}},
    “link”: { “meta”: { “href”: “/billing/accounts/acc9579/members”, “rel”: “collection/members”}}
}

In the above example, you’ll find a direct link the account’s mailbox and a link relation to the members of this family account (with the rel attribute defining the semantics).

Versioning

This is an often discussed topic. Originally, the default approach to versioning an API used to be versioning the URL of the endpoint via which the service was exposed:

# Initial API endpoint
/someServer/serviceName/v1/resourceName

# New endpoint after creating a new version
/someServer/seviceName/v2/resourceName

The (preferred) alternative to this is using the request/response headers to provide version information. That way, the actual endpoint of your API remains stable across versions. When selecting the right approach to versioning, it is important to first look at what really has changed. There are basically three scenarios to consider:

Adding fields to an existing resource representation
Changes to a representation that break the clients
Semantical changes (that break the clients, too)

Adding fields to an existing resource representation

Whenever you’re simply adding content to an existing representation, this should not have any impact on your versioning strategy. It shall always be the implicit understanding that all clients consuming your services will ignore what they don’t know or understand and work with the well-known. If adding one or more new data fields to a representation breaks a client, then it’s the client that needs to be fixed, not your service.
So, if your resource representation changes from this

{
    “name”: “Tom”
}

into this

{
    “name”: “Tom”,
    “hobbies”: [ “painting”, “music”]
}

this should not be a concern for any of the existing clients.

Breaking changes to a representation

If the changes you are making to your API involve removal of content or renaming of content, this will most likely break the clients consuming it. While the clients have been designed to ignore what they don’t know, you are now taking away what they already understand. This is definitely a situation where you have to consider versioning your representation. The approach to this is using the content negotiation mechanism built into HTTP. Use the Content-Type header to indicate representation, version, and format information. This mechanism works for both, requests (in conjunction with the POST and PUT verbs) as well as responses (typically to GET, POST and PUT requests).

Breaking the semantics

In the first two scenarios, changes were made to particular representations (non-breaking in the first one, breaking in the second one). The actual resources and corresponding resource identifiers themselves fortunately remained stable along the lines of those changes. There might however also be – hopefully not too often – situations where you have to change the actual meaning of resources. In this case, you would be breaking the contract between your API and its clients, with the approach being to then version the resource identifier itself.

Idempotent requests

There might be situations where you would like to let clients safely retry the same request towards a resource without accidently performing an operation twice. For example, if a request to record a payment transaction fails due to a network error (resulting in the request being processed but the response not being received by the client), a client could simply submit the same request again without running the risk of duplication. In order to do this, the API will have to process a custom header Idempotency-Key that would be populated with a unique key (generated by the client). The server will have to make sure that all requests to a representation with that same idempotency key will lead to the exact same response. Also, the server will have to always include the Idempotency-Key header in the response.

# Request
POST /accounts/4344/payments
Content-Type: application/json
Idempotency-Key: 046b6c7f-0b8a-43b9-b35d-6489e6daee91
Accept: application/json
{
    “accountNumber”: “4344”,
    “amount”: 100,
    “description”: “T-shirt”,
    “transactionDateTime”: “2015-01-02T16:17:34.001Z”
}


# Response (always the same no matter how often the request is being sent with the same key)
HTTP/1.1 201 CREATED
Content-Type: application/json
Idempotency-Key: 046b6c7f-0b8a-43b9-b35d-6489e6daee91
{
    “href”: “/accounts/4344/payments/9834973”,
    “transactionId”:  “9834973”,
    “processorResponse”: “ACCEPTED”
}

Conclusion

A robust and strong API design is a critical success factor and the best way to avoid refactoring costs and other related risks. None of your future APIs will be perfect from the get-go. So challenge yourself and have others challenge your work during the design phase to at least ensure a solid version 1.0 to release.

RESTful APIs - Implementation Aspects

JAX-RS

The implementation of your RESTfully designed API has to be conducted based on JAX-RS, the official Java API for RESTful Web Services. In the current xMS40 runtime environment, we are using the RESTEASY framework that comes with Wildfly and JBOSS. This should however not be of any concern for you as a developer as the RESTEASY framework is a certified implementation of the official JAX-RS interface.

Annotations

JAX-RS defines a set of annotations that greatly help to render your web service code readable for you and others. So, make use of them. The most popular annotations in this context are listed in the table below. Do not make use of implementation specific annotations.

Annotation	Comments
@ApplicationPath	Defines the base URL for your parent application, it can be used as an alternative to doing this in the web.xml. Example: @ApplicationPath(“myawesomeapplication”)
@Path	Defines the path to your service Example: @Path(“myawesomeservice”) Note: in this example, your service will then be accessible via /myawesomeapplication/myawesomeservice
@GET	Indicates that the following method will be responding to a HTTP GET request
@PUT	Indicates that the following method will be responding to a HTTP PUT request
@POST	Indicates that the following method will be responding to a HTTP POST request
@DELETE	Indicates that the following method will be responding to a HTTP DELETE request
@HEAD	Indicates that the following method will be responding to a HTTP HEAD request
@PATCH	Indicates that the following method will be responding to a HTTP PATCH request
@Produces	Use this annotation to indicate which MIME type(s) this method offers to deliver to the requestor. Examples: @Produces( “application/xml”) @Produces({MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON})
@Consumes	Use this annotation to indicate which MIME type(s) the method can consume/process. @Consumes( “application/xml”) @Consumes({MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON})

@PathParam

This annotation is used to extract “dynamic” parameters from the URL path (such as e.g. a resource identifier) as a parameter to your method.

Example:

@GET

@Produces( MediaType.APPLICATION_JSON)

@Path(“accounts/{accountId}”)

public Response getAccountInstance(

@PathParam( “accountId”) String accId) {

// you can now work with accId to retrieve resource instance

…

}

@QueryParam

Use this annotation to get access to query parameters in the request URI.

Example request:

/finance/accounts?type=individual&status=valid

Above request has two embedded query parameters: type and status. You can access them via the @QueryParam annotation like this:

@GET

@Produces( MediaType.APPLICATION_JSON)

@Path(“accounts”)

public Response getAccountList(

@QueryParam( “type”) String accountType,

@QueryParam( “status”) String accountStatus) {

// you can now work with accountType and accountStatus
// to do your filter magic…

…

}

@DefaultValue

The list of query parameters provided is obviously up to the discretion of the requesting client, so you never know if a particular query parameter is present or not. So, if in the @QueryParam example, the client request would have been

/finance/accounts? status=valid

, the accountType parameter would have a value of null. To reduce the number of “if (par != null)” checks in your code, you can use the @DefaultValue convenience annotation. That way, you can make sure that the query parameter always has a non-null value. The modified @QueryParam example would be:

@GET

@Produces( MediaType.APPLICATION_JSON)

@Path(“accounts”)

public Response getAccountList(

@DefaultValue(“corporate”) @QueryParam( “type”) String accountType,

@DefaultValue(“active”) @QueryParam( “status”) String accountStatus) {

// you can now work with accountType and accountStatus
// to do your filter magic…

…

}

In the example, the accountType parameter would have a value of “corporate”, even though the requesting client didn’t specifiy it:

/finance/accounts? status=valid

Jan 23, 2016

API gateway options

Mac OS X: launching an application from the terminal window

Jan 18, 2016

Linux: find which process is listening on a port

Jan 17, 2016

Design of RESTful APIs - a best practices guide

Best practices – the basics

Resources

Resource types

Resource type names

Resource identifiers

HTTP verbs

Media types

Always accept and return objects, not arrays

camelCase for parameter names

Date/time/timestamp

Strings vs. numbers

Enumerations

Response body

Content negotiation

HTTP status codes

2xx – Success

3xx – Redirection

4xx – Client Error

5xx – Server Error

Errors

Security Aspects

Avoid sessions

URL based authorization

Use existing protocols

Client certificates

API keys

JWT

Best practices – more advanced topics

Hypermedia, relationships

Relationships to other resources

Versioning

Adding fields to an existing resource representation

Breaking changes to a representation

Breaking the semantics

Idempotent requests

Conclusion

RESTful APIs - Implementation Aspects

JAX-RS

Annotations

Recommended reading

Books

REST in Practice

RESTful Web Services Cookbook

RESTful Web Services

Jan 16, 2016

Recommended Reading

NodeJS on Ubuntu 12