At my work, we use JSON-RPC based APIs very heavily, in particular with our PHP JSON-RPC library php-json-rpc. While JSON-RPC is not as wide spread as REST, it fits our needs quite nicely. In particular, it is protocol independent and can be used over HTTP, SSH or as local CLI. With our library and its numerous extensions (HTTP, SSH, authentication, validation, request-to-class mapping and logging), development is super fast and incredibly easy.
In this post, I’d like to demonstrate how to set up a PHP based JSON-RPC API, with authentication, validation and logging.
Content
- 0. Code available on GitHub
- 1. Demo API: Managing devices (e.g. phones, computers)
- 2. JSON-RPC core library (php-json-rpc)
- 3. Method and parameter mapping (php-json-rpc-simple)
- 4. Validation of parameters (php-json-rpc-validator)
- 5. Authentication (php-json-rpc-auth)
- 6. Logging with “php-json-rpc-log”
- 7. Advanced usage
0. Code available on GitHub
All the code for this post is available in a JSON-RPC Demo Project on GitHub. Feel free to poke around there and/or steal bits and pieces from it.
To try the examples, check out the code and run this:
|
1 2 3 4 |
git clone https://github.com/binwiederhier/json-rpc-demo cd json-rpc-demo composer install php -S localhost:8888 -t web |
1. Demo API: Managing devices (e.g. phones, computers)
For this blog post, we’ll create a simple Device Management API to list and add devices (e.g. phones, computers, and such) using two endpoints called devices/add and devices/listAll. The endpoints will be simple stub implementations, but we’ll still see how easy it is to create and manage endpoints.
Here’s an example JSON-RPC API request and its response to list all devices:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
// 2. Request to list all devices, sorted by id { "jsonrpc": "2.0", "id": 1, "method": "devices/listAll", "params": { "sortBy": "id" } } // The corresponding response returns a list { "jsonrpc": "2.0", "id": 1, "result": [ {"name": "Philipp PC", "id": 1, "type": "pc"}, {"name": "Phil Phone", "id": 2, "type": "phone"}, {"name": "Phil Washing Machine", "id": 3, "type": "other"} ] } |
2. JSON-RPC core library (php-json-rpc)
The core JSON-RPC library php-json-rpc is written by Spencer Mortensen. It’s a great piece of software that implements the core specification of JSON-RPC 2.0.
2.1. Sample code
At the center of it, the JsonRpc\Server class provides a way to encode and decode messages according to JSON-RPC standards. All you have to do is provide your own JsonRpc\Evaluator implementation to use it:
|
1 2 3 4 5 6 7 |
$evaluator = new DevicesEvaluator(); $server = new JsonRpc\Server($evaluator); header('Content-Type: application/json'); $message = file_get_contents('php://input'); echo $server->reply($message); // {"jsonrpc":"2.0","id":1,"result":3} |
An Evaluator basically translates JSON-RPC methods and parameters to PHP methods/arguments and executes the method. Your implementation must only provide a method evaluate($method, $arguments). It might (but should not) look as simple as this:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
class DevicesEvaluator implements JsonRpc\Evaluator { // ... public function evaluate($method, $arguments) { if ($method === 'devices/add') { return $this->devices->add($arguments['id'], $arguments['name'], $arguments['type']); } else if ($method === 'devices/listAll') { return $this->devices->listAll($arguments['sortBy']); }else { throw new Exception\Method(); } } } |
2.2. Usage
Once we’ve implemented the endpoints/methods, we’ll be able to add and list devices via HTTP(S). Here’s an example via curl:
|
1 2 3 4 5 |
# Adding a device $ curl \ -d '{"jsonrpc":"2.0","id":1,"method":"devices/add","params":{"name":"Philipp PC","type":"pc","id":1}}' \ http://localhost:8888/example0/api.php {"jsonrpc":"2.0","id":1,"result":{"id":1,"name":"Philipp PC","type":"pc"}} |
Of course you can also use your own JSON-RPC client, or you can use JavaScript/jQuery (see below). I’m using curl here because it’s super easy.
2.3. More examples
Find a working demo of the php-json-rpc library in the examples folder of the library or in example 0 of the JSON-RPC demo project.
3. Method and parameter mapping (php-json-rpc-simple)
While we can certainly implement all use cases with the core library, having to provide the mapping yourself (usually with a giant switch-case) can be tedious. The php-json-rpc-simple library provides a way to do that mapping for you. It offers a simple implementation of the JsonRpc\Evaluator interface for you and maps JSON-RPC method and params to a corresponding PHP class, method and arguments.
3.1. Sample code
To use the mapping magic of the php-json-rpc-simple library, alter the API server logic to automatically map JSON-RPC methods to a PHP class/method inside the Demo\Api\Endpoint namespace (see example1/api.php):
|
1 2 3 4 5 6 7 |
$evaluator = new Simple\Evaluator(new Simple\Mapper('Demo\\Api\\Endpoint\\')); $server = new JsonRpc\Server($evaluator); header('Content-Type: application/json'); $message = file_get_contents('php://input'); echo $server->reply($message); |
Since the mapping is done for you, the endpoint methods only contain the actual logic and/or library calls (see src/Api/Endpoint/Devices.php):
|
1 2 3 4 5 6 7 8 9 10 11 |
namespace Demo\Api\Endpoint; class Devices { public function listAll($sortBy = 'name') { return array(..); } // ... } |
3.2. Usage
The usage is not very different from the example above. This time, however, the mapping is performed by the library:
|
1 2 3 4 5 |
# Listing all devices $ curl \ -d '{"jsonrpc":"2.0","id":1,"method":"devices/listAll","params":{"sortBy":"id"}}' \ http://localhost:8888/example1/api.php {"jsonrpc":"2.0","id":1,"result":[{"id":1,"name":"Marian-PC","type":"pc"},{"id":1,"name":"Philipp PC","type":"pc"}]} |
That also means, of course, that invalid or missing parameters will be detected by the library and a JSON-RPC error is returned:
|
1 2 3 4 5 |
# Invalid parameters return an error $ curl \ -d '{"jsonrpc":"2.0","id":1,"method":"devices/add","params":{"INVALID":"1"}}' \ http://localhost:8888/example1/api.php {"jsonrpc":"2.0","id":1,"error":{"code":-32602,"message":"Invalid params"}} |
3.3. More examples
Find a working demo of the php-json-rpc-simple library in example 1 of the JSON-RPC demo project.
4. Validation of parameters (php-json-rpc-validator)
Another tedious thing in API development is parameter validation. User input can be invalid or even malicious, an API (or any web page really) must validate the input parameters. The php-json-rpc-validator library offers annotation-based validation of parameters.
4.1. Sample code
Normally, a publicly available endpoint method must check the arguments that are passed to it. Code like this is not uncommon:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
class Devices { public function listAll($sortBy = 'name') { if ($sortBy !== 'name' && $sortBy !== 'type' && $sortBy !== 'id') { throw new Exception('Illegal argument'); } // Actual API code // ... } // ... } |
Instead of this annoying and polluting validation code, we can now define the constraints of each method argument using the @Validate annotation. This annotation is based on Symfony’s @Collection annotation and supports a variety of constraints. Examples include @Assert\NotBlank, @Assert\NotNull, @Assert\EqualTo, @Assert\Regex, @Assert\GreaterThan, and may more.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
class Devices { /** * @Validate(fields={ * "sortBy" = { @Assert\Regex("/^(id|name|type)$/") } * }) */ public function listAll($sortBy = 'name') { // Actual API code // ... } } |
To add validation support to your API server file (in our examples: api.php), all you need to do is wrap the existing JsonRpc\Evaluator in a Validator\Evaluator. After that, the API server logic looks like this (see example2/api.php):
|
1 2 3 4 5 6 7 8 |
$mapper = new Simple\Mapper('Demo\\Api\\Endpoint\\'); $evaluator = new Validator\Evaluator(new Simple\Evaluator($mapper), $mapper); $server = new JsonRpc\Server($evaluator); header('Content-Type: application/json'); $message = file_get_contents('php://input'); echo $server->reply($message); |
4.2. Usage
If the input is valid, the request and response look no different than above. However, if the parameter input is invalid, the API now responds with a JSON-RPC error:
|
1 2 3 4 5 |
# Invalid parameter values are rejected by validation $ curl \ -d '{"jsonrpc":"2.0","id":1,"method":"devices/listAll","params":{"sortBy":"INVALID"}}' \ http://localhost:8888/example2/api.php {"jsonrpc":"2.0","id":1,"error":{"code":-32602,"message":"Invalid params"}} |
4.3. More examples
Find a working demo of the php-json-rpc-validator library in example 2 of the JSON-RPC demo project.
5. Authentication (php-json-rpc-auth)
What would an API be without authentication? Probably reckless, in most cases.
The php-json-rpc-auth library offers a simple framework to implement any kind of authentication and authorization for your API. Instead of implementing all the different auth mechanisms (HTTP Basic Auth, Digest, OAuth, SAML, Cookies, …), it merely provides a simplistic Auth\Authenticator class to consult a user-provided set of Auth\Handlers. The actual implementation of these handler class(es) must be provided by the developer.
5.1. Sample code
Assuming that we want to implement HTTP Basic Auth for our API, we only need to implement an Auth\Handler (see src/Api/Auth/BasicAuthHandler.php):
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
class BasicAuthHandler implements Auth\Handler { public function canHandle($method, $arguments) { return isset($_SERVER['PHP_AUTH_USER']); } public function authenticate($method, $arguments) { // Don't use '===', as that's vulnerable to timing attacks! return $_SERVER['PHP_AUTH_USER'] === 'user' && $_SERVER['PHP_AUTH_PW'] === 'pass'; } } |
The canHandle() method tells the authenticator whether the given request can be authenticated with this handler. The authenticate() method is only called if canHandle() returned true and then authenticates the actual request.
As for the API server code, we can simply wrap an Auth\Evaluator around our existing evaluator (see example3/api.php).
|
1 2 3 4 5 6 7 8 9 10 11 12 |
$authenticator = new Auth\Authenticator(array( new BasicAuthHandler() )); $mapper = new Simple\Mapper('Demo\\Api\\Endpoint\\'); $evaluator = new Auth\Evaluator(new Validator\Evaluator(new Simple\Evaluator($mapper), $mapper), $authenticator); $server = new JsonRpc\Server($evaluator); header('Content-Type: application/json'); $message = file_get_contents('php://input'); echo $server->reply($message); |
This may look a bit complicated, but it merely nests different Auth\Evaluator implementations. In the example above, an incoming request is first passed to the Auth\Evaluator, then (if authorized) to the Validator\Evaluator, and finally (if validated) to the Simple\Evaluator. This is a very classic implementation of the decorator pattern.
5.2. Usage
Trying to access API without authentication will lead to a “Missing auth” error message. In this example, please note that curl‘s -u parameter (basic auth user/pass) is not being passed:
|
1 2 3 4 |
$ curl \ -d '{"jsonrpc":"2.0","id":1,"method":"devices/listAll"}' \ http://localhost:8888/example3/api.php {"jsonrpc":"2.0","id":1,"error":{"code":-32651,"message":"Missing auth."}} |
In this example, the -u parameter is passed, i.e. the Authorization HTTP header is sent, but with invalid credentials. We’re basically trying to access API with invalid Basic Auth credentials:
|
1 2 3 4 5 |
$ curl \ -u invaliduser:pass \ -d '{"jsonrpc":"2.0","id":1,"method":"devices/listAll"}' \ http://localhost:8888/example3/api.php {"jsonrpc":"2.0","id":1,"error":{"code":-32652,"message":"Invalid auth."}} |
Only if we provide the correct credentials, we’re allowed to access the API via HTTP (authenticated via HTTP Basic Auth):
|
1 2 3 4 5 |
$ curl \ -u user:pass \ -d '{"jsonrpc":"2.0","id":1,"method":"devices/listAll"}' \ http://localhost:8888/example3/api.php {"jsonrpc":"2.0","id":1,"result":[{"name":"Phil Phone","id":2,"type":"phone"},{"name":"Phil Washing Machine","id":3,"type":"other"},{"name":"Philipp PC","id":1,"type":"pc"}]} |
5.3. More examples
Find a working demo of the php-json-rpc-auth library in example 3 of the JSON-RPC demo project.
6. Logging with “php-json-rpc-log”
Logging requests and responses of an API can be a very important tool for troubleshooting or statistics. The php-json-rpc-log library offers a very simplistic way to log JSON-RPC equests and responses.
6.1. Sample code
We’re mostly using Monolog, but the php-json-rpc-log library supports any kind of PSR-3 logger. Simply use the Logged\Server class instead of the original JsonRpc\Server class, and pass a Psr\Log\LoggerInterface to it:
|
1 2 3 4 5 6 7 8 |
$evaluator = // see above $logger = new Logger('demo.api', array(new SyslogHandler('api'))); $server = new Logged\Server($evaluator, $logger); header('Content-Type: application/json'); $message = file_get_contents('php://input'); echo $server->reply($message); |
6.2. Usage
In the case above, we passed a Monolog\Logger with a SyslogHandler to it, meaning that requests that we send will be logged to /var/log/syslog. The usage is the same as in the other examples, but we’ll see output like this when we tail syslog:
|
1 2 3 4 |
Jan 15 01:24:21 platop api[13157]: demo.api.INFO: Message received: {"jsonrpc":"2.0","id":1,"method":"devices/add"} [] [] Jan 15 01:24:21 platop api[13157]: demo.api.INFO: Sending reply: {"jsonrpc":"2.0","id":1,"error":{"code":-32602,"message":"Invalid params"}} [] [] Jan 15 01:24:46 platop api[13157]: demo.api.INFO: Message received: {"jsonrpc":"2.0","id":1,"method":"devices/listAll"} [] [] Jan 15 01:24:46 platop api[13157]: demo.api.INFO: Sending reply: {"jsonrpc":"2.0","id":1,"result":[{"id":0,"name":"Philipp PC","type":"pc"}]} [] [] |
6.3. More examples
Find a working demo of the php-json-rpc-log library in example 4 of the JSON-RPC demo project.
7. Advanced usage
There are a million ways to combine these components, and I can surely not list them all here. However, here’s a short list of very useful applications.
7.1. Using the API from the shell or via SSH (with ‘root’-only access)
Unlike REST-based APIs, we can easily expose a JSON-RPC based API via other protocols, such as SSH or raw TCP sockets. We can even just use it locally by piping the JSON-RPC request to a PHP script.
In example 5 of the JSON-RPC demo project, we extend the api.php file to support requests from STDIN (if the file is called from the CLI). The API can still be used via HTTP, but the script now checks via php_sapi_name() if we’re in CLI mode.
In addition to that, the demo implements another auth handler (see src/Api/Auth/CliAuthHandler.php) to check if the user is root, and disallows access if she isn’t:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
$authenticator = new Auth\Authenticator(array( new CliAuthHandler(), new BasicAuthHandler() )); // ... $isCLI = php_sapi_name() === 'cli'; if ($isCLI) { $message = file_get_contents('php://stdin'); } else { header('Content-Type: application/json'); $message = file_get_contents('php://input'); } echo $server->reply($message); |
With this insanely simple addition, we can now use the API locally by piping the request to the api.php script:
|
1 2 3 4 5 6 7 |
# Access locally; or NOT, since we're not 'root' (access denied by CliAuthHandler) $ echo '{"jsonrpc":"2.0","id":1,"method":"devices/listAll"}' | php web/example5/api.php {"jsonrpc":"2.0","id":1,"error":{"code":-32652,"message":"Invalid auth."}} # Let's try again as 'root' $ echo '{"jsonrpc":"2.0","id":1,"method":"devices/listAll"}' | sudo php web/example5/api.php {"jsonrpc":"2.0","id":1,"result":[{"id":0,"name":"Philipp PC","type":"pc"}]} |
Now, since we can simply pipe, that means using the API across machines via SSH is trivial:
|
1 2 3 4 |
# And via SSH ... ! $ echo '{"jsonrpc":"2.0","id":1,"method":"devices/listAll"}' \ | ssh root@myserver php /opt/demo/web/example1/api.php {"jsonrpc":"2.0","id":1,"result":[{"id":0,"name":"Philipp PC","type":"pc"}]} |
7.2. Using the API with JavaScript and cookie-based auth
I’ve been using curl in all the examples, but of course the primary use case of an API is often the use inside a web application. Since JSON-RPC requests are merely POST requests with JSON payload, you can use jQuery’s $.ajax() method — for instance like this:
|
1 2 3 4 5 6 7 8 9 10 11 12 |
$.ajax({ url: '/example6/api.php', type: 'POST', contentType: 'application/json', dataType: 'json', data: JSON.stringify({ jsonrpc: '2.0', method: 'devices/listAll', params: {} }), // ... }); |
In example 6 of the JSON-RPC demo project, we implement a small web page to display (and auto-refresh) the list of devices using the API, but only if we’re logged in via a cookie:
Without being logged in (no cookie):
|
When logged in (with cookie):
|
To achieve this, we implement yet another auth handler (CookieAuthHandler, see src/Api/Auth/CookieAuthHandler.php), and toggle a cookie when the “Login” button is pressed.
All that’s left is to call the API in regular intervals and update the UI. We implemented a small helper class called Datto.API.Client (see web/example6/js/client.js) to interact with the API from our web app:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
var api = new Datto.API.Client('api.php'); api.call('devices/listAll', {}, function (data) { // Success function $devices.empty(); $(data).each(function (idx, device) { $devices.append($('<li>').addClass(device.type).text(device.name)); }); }, function (data) { // Failure function $devices .empty() .append($('<li>').addClass('error').text('ERROR: ' + data.error.message)); }); |
That’s it. I hope this was helpful. Please let me know what you think in the comments!
Looks good.
I assume the transport encoding is UTF-8 (or UTF-16 or UTF-32 if configurable).
Now if 2 applications are to communicate using this library but they both use cp1251 encoding internally instead of UTF-8, where can that be set in this library or will transcoding occur automatically (e.g. using mb_internal_encoding() or ini_get(‘default_charset’)) ?
The libraries currently only support UTF-8.
Howdy Philipp,
“At Datto, we use JSON-RPC based APIs very heavily…”
I know it’s been a while since you wrote this article
but it interests me why you choose the JSON-RPC instead of
(as some claim better, easier, etc) REST/CRUD API?
What is the objective/technical reason for such a choice ?
Thanks,
Dav.
Hey Dav, sorry for the delayed response. Had a baby 2 weeks ago, hence I’m a little busy.
We chose JSON-RPC and still heavily use it over REST because it is not tied to a protocol (i.e. HTTP). The message itself contains everything needed for the request, as opposed to REST where HTTP verb, path and body are relevant. With JSON-RPC, you can do things like echo '{"jsonrpc":"2.0","id":1,...' | ssh user@myserver apicli, or you can queue requests and batch them properly.
It is much more versatile than REST, although I must say that lots of people are used to REST. Another drawback is that large binary content (images, files) is not easily transported over JSON-RPC, because it has to be encoded with base64 or something like that, which blows up its size.
I think it was a good choice, we still mostly use JSON-RPC.
Howdy Philipp,
Wow, congratulations then, and it is totally understandable.
Thank you for your answer and I hope it won’t be much of a hassle if I dwell on this issue a bit more.
“…encoded with base64 or something like that, which blows up its size..”
I think any protocol that uses JSON as its framing (or any nonbinary e.g. XML) format is essentially affected by this issue (the REST included).
It seems to me (and please correct me if I’m wrong) that the main benefit of the JSON-RPC is its independence from the transport protocol. Which combined with the completeness of the data frame/ message is what makes this protocol truly portable and universal.
But how to convince the people, so accustomed to the HTTP (and its subset REST) or in the broader sense the prevalent (and annoying to some) webification of the IT, that the JSON-RPC is actually the better protocol to implement the m2m communication especially if a machine/node can be managed without the HTTP overhead?
> But how to convince the people,
Well, that is the million dollar question, isn’t it? I think if you find a good use case in your org that isn’t easily solvable with REST then you can easily convince people.
I will say this: To this day we still have these discussions in our org. People are very very very used to REST.
> without the HTTP overhead
You’ll still have a transport overhead of some sort. For the most part, we still do JSON-RPC over HTTP, so the this argument is a little flawed, depending how you use it of course. We mainly use it over HTTP, SSH and just over local unix sockets.
I’ve recently started looking at binary protocols such as gRPC, but of course then you are bound to a transport again. If you’re looking for universally portable and binary-compatible, but transport independent, you could also do straight up protobuf to encode the message and then pick your own transport. That is certainly more cumbersome for devs though.
Howdy Philipp.
>You’ll still have a transport overhead of some sort….
Yes, true, but it would span in between non-existent (UDS), insignificant (TCP), up to slight/expected (SSH/TLS).
Me personally, I’m impressed, how you decided to use just regular SSH to transport the RPC calls. It seems so unorthodox and yet very much suited for this role considering e.g. that one could configure the server side to start some specialized cli as a shell alternative for such connections.
And as we are talking here about the transport layer,
I’d like to take this opportunity and ask you about the ZeroMQ?
Have you ever used it?
What properties/features would you look for in it, for you to consider it as a replacement for your current transport technology?
>I’ve recently started looking at binary protocols…
Have you ever considered bjson (http://bjson.org/) ?