Overview

JAMES is a web service for creating and interpreting charts of child growth and development. The current version

  1. provides access to high-quality growth charts used by the Dutch youth health care;
  2. interchanges data coded according to the Basisdataset JGZ 4.0.1;
  3. screens for abnormal height, weight and head circumference;
  4. converts developmental data into the D-score;
  5. predicts future growth and development.

JAMES is a RESTful API that runs on a remote host. The following sections illustrate how a client can make requests to JAMES using various client languages. In principle, any HTTP client will work with JAMES. The document highlights some applications of the service and provides pointers to relevant background information.

The service aids in monitoring and evaluating childhood growth. JAMES is created and maintained by the Netherlands Organisation for Applied Scientific Research TNO. Please contact Stef van Buuren <stef.vanbuuren at tno.nl> for further information.

Primary JAMES user functionality

Verb API end point Description Maps to james function
POST /data/upload/{dfm} Upload child data upload_data()
POST /charts/draw/{ffm} Draw child data on growth chart draw_chart()
POST /charts/list/{dfm} List available growth charts list_charts()
POST /charts/validate/{dfm} Validate a chart code validate_chartcode()
POST /screeners/list/{dfm} List available growth screeners list_screeners()
POST /screeners/apply/{dfm} Apply growth screeners to child data apply_screeners()
GET /site Request empty site request_site()
POST /site/request/{dfm} Request personalised site request_site()
POST /blend/request/{sfm} Obtain a blend from multiple end points request_blend()
POST /version/{dfm} Obtain version information version()
GET /{session}/{info} Extract session details
GET /{2}/{1}/man Consult R help help({1}_{2})

The table lists the defined API end points and the mapping to each end point to the corresponding R function.

The current OpenAPI definition of JAMES is at https://app.swaggerhub.com/apis/stefvanbuuren/james/1.5.4. Note that this definition may evolve over time.

Output formats

JAMES is built on top of the OpenCPU API, a powerful and flexible way for online deployment of R functionality. Although it is possible to use JAMES without knowledge of OpenCPU, it is useful to browse the OpenCPU features.

OpenCPU offers multiple output formats out of the box. JAMES supports a subset of these through the /{session}/{info} end point. Here session is a special session code generated by OpenCPU that identifies the server location containing the results of a request. The output format info is one of the following:

{info} Description
json Function result as JSON
print Function result as formatted as print
csv Function result as comma-separated format
tab Function result as tab-delimited format
md Function result as markdown format
svglite Function result as SVG graph
warnings Warnings from the R execution
messages Messages, especially data validation info
console Console print out, useful for locating errors
stdout Standard output
info Overview of JAMES deployment
parent Parent directory of all OpenCPU session output

In addition, the user can request the function result in a particular form. JAMES distinguishes the following groups of formats.

Format group Description
dfm Data format: json, csv, tab, md, print, parent
ffm Figure format: svglite, print, parent
sfm System format: json, print, parent

In general, the user can specify the desired format by appending the format name to the URL. See https://www.opencpu.org/api.html#api-formats for examples.

Objective

This document provides a quick introduction into the main JAMES features, and how these can be assessed from R and from the command line.

Features

/version: Obtain version information

Let us first check whether JAMES is running. The following code makes a simple request to JAMES to see whether it is alive and to return the version number of the underlying james R package. We illustrate both requests in R and in bash.

R

We first need to install and load packages.

install.packages(c("remotes", "httr", "jsonlite"))
remotes::install_github("growthcharts/jamesclient")
remotes::install_github("growthcharts/jamesdemodata")
remotes::install_github("growthcharts/bdsreader")
library(jamesclient)
library(httr)
library(jsonlite)

Let’s find out the JAMES version number. In this document, we define the server that hosts JAMES as follows.

host <- "https://james.groeidiagrammen.nl"
# host <- "http://localhost"
system(paste0("echo ", host, " > .host"))

We first illustrate a method that makes two requests to the server. The following commands call the /version/json end point in the JAMES API.

r <- james_post(host = host, path = "version/json")

We added the /json to the pathname to extract the JSON representation of the result of the R function james::version(). The function result is an object of class james_post and consists of various components.

names(r)
##  [1] "url"          "status_code"  "headers"      "all_headers"  "cookies"     
##  [6] "content"      "date"         "times"        "request"      "handle"      
## [11] "request_path" "parsed"       "warnings"     "messages"     "session"
r$url
## [1] "https://james.groeidiagrammen.nl/version/json"

Most of the element are documented in the response object in the httr package. For example, we could use the call httr::status_code(r) to obtain the status code. The function james_post() adds the last five elements:

  • r$request_path echoes the endpoint, here /version/json;
  • r$parsed is a parsed version of the element r$content. Here it is a list of elements like names of the package, its date, and so on. In case of an error of the server function, we find the error message here;
  • r$warnings contain any warnings thrown during execution;
  • r$messages contain any messages, e.g. data reading errors;
  • r$session (like x06601655f3e2f1) is a unique session code.

The jamesclient::james_post() function wraps the basis workhorse httr::POST() that does the actual server request. For illustration, we may obtain equivalent content by the POST function directly.

path <- "version/json"
url <- parse_url(host)
url <- modify_url(url, path = file.path(url$path, path), query = "auto_unbox=true")
r <- POST(url)
fromJSON(content(r, type = "text", encoding = "UTF-8"))
## $package
## [1] "james"
## 
## $packageVersion
## [1] "0.68.0"
## 
## $packageDate
## [1] "2023-10-19"
## 
## $Rversion
## [1] "4.3.1"
bash

We use the curl Linux command. If needed, on Ubuntu install curl as

sudo apt update
sudo apt -y install curl

Let’s find out the JAMES version number. We first illustrate a method that makes two requests to the server.

The following bash commands call the /version API end point

curl -sX POST $(cat .host)/version > resp

The response to the request consists of a set of URLs created on the server, each of which contains details on the response.

cat resp
## /ocpu/tmp/x0a8ee96b4ff7f2/R/.val
## /ocpu/tmp/x0a8ee96b4ff7f2/R/version
## /ocpu/tmp/x0a8ee96b4ff7f2/stdout
## /ocpu/tmp/x0a8ee96b4ff7f2/source
## /ocpu/tmp/x0a8ee96b4ff7f2/console
## /ocpu/tmp/x0a8ee96b4ff7f2/info
## /ocpu/tmp/x0a8ee96b4ff7f2/files/DESCRIPTION

The path element following tmp/ is a unique session key. See https://www.opencpu.org/api.html for the interpretation of the OpenCPU API.

The next snippet constructs the URL of a JSON representation of the result and downloads the contents of the URL as a file value1.

curl -s $(cat .host)$(head -1 resp)/json?auto_unbox=true > value1
cat value1
## {
##   "package": "james",
##   "packageVersion": "0.68.0",
##   "packageDate": "2023-10-19",
##   "Rversion": "4.3.1"
## }

The above sequence makes two requests to the server. The following code compacts both steps into one.

curl -sX POST $(cat .host)/version/json?auto_unbox=true > value2
cat value2
## {
##   "package": "james",
##   "packageVersion": "0.68.0",
##   "packageDate": "2023-10-19",
##   "Rversion": "4.3.1"
## }

/data/upload: Upload child data

JAMES understands data that conform to the Basisdataset JGZ 4.0.1 coded as JSON according to a JSON schema. This section explains how we create, validate and upload child data to JAMES.

R

Let us assume that we have already have child data in R stored as a data.frame or tibble. Here we copy the longitudinal demo data maria.json from the bdsreader package into the working directory.

success <- file.copy(system.file("examples/maria.json", package = "bdsreader"), 
                     "maria.json", overwrite = TRUE)

The contents of the file is json format, ready for upload:

{
  "OrganisatieCode": 1234,
  "Referentie": "fa308134-069e-49ce-9847-ccdae380ed6f",
  "ClientGegevens": {
    "Elementen": [
      {
        "Bdsnummer": 19,
        "Waarde": "2"
      },
      {
        "Bdsnummer": 20,
        "Waarde": "20181011"
      },
      {
        "Bdsnummer": 82,
        "Waarde": "189"
      },
      {
        "Bdsnummer": 91,
        "Waarde": "2"
      },
      {
        "Bdsnummer": 110,
        "Waarde": "990"
      },
      {
        "Bdsnummer": 238,
        "Waarde": "1670"
      },
      {
        "Bdsnummer": 240,
        "Waarde": "1900"
      }
    ],
    "Groepen": [
      {
        "Elementen": [
          {
            "Bdsnummer": 63,
            "Waarde": "19950704"
          },
          {
            "Bdsnummer": 71
          },
          {
            "Bdsnummer": 62,
            "Waarde": "01"
          }
        ]
      },
      {
        "Elementen": [
          {
            "Bdsnummer": 63,
            "Waarde": "19901202"
          },
          {
            "Bdsnummer": 71
          },
          {
            "Bdsnummer": 62,
            "Waarde": "02"
          }
        ]
      }
    ]
  },
  "Contactmomenten": [
    {
      "Tijdstip": "20181011",
      "Elementen": [
        {
          "Bdsnummer": 245,
          "Waarde": "990"
        }
      ]
    },
    {
      "Tijdstip": "20181111",
      "Elementen": [
        {
          "Bdsnummer": 235,
          "Waarde": "380"
        },
        {
          "Bdsnummer": 245,
          "Waarde": "1250"
        },
        {
          "Bdsnummer": 252,
          "Waarde": "270"
        }
      ]
    },
    {
      "Tijdstip": "20181211",
      "Elementen": [
        {
          "Bdsnummer": 235,
          "Waarde": "435"
        },
        {
          "Bdsnummer": 245,
          "Waarde": "2100"
        },
        {
          "Bdsnummer": 252,
          "Waarde": "305"
        }
      ]
    }
  ]
}

There are four ways to upload the data to JAMES:

  1. Upload the file "maria.json";
  2. Convert to a string and upload;
  3. Convert to a JSON object and upload;
  4. Read the JSON file from a URL.

The /data/upload API end point handles these cases as follows:

# upload as file
fn <- "maria.json"
r1 <- james_post(host = host, path = "data/upload/json", txt = fn)
status_code(r1)
## [1] 201
# upload as string
js <- read_json_js(fn)
r2 <- james_post(host = host, path = "data/upload/json", txt = js)
status_code(r2)
## [1] 201
# upload as JSON object
jo <- read_json_jo(fn)
r3 <- james_post(host = host, path = "data/upload/json", txt = jo)
status_code(r3)
## [1] 201
# upload as URL
url <- file.path(host, "ocpu/library/bdsreader/examples/maria.json")
r4 <- james_post(host = host, path = "data/upload/json", txt = url)
status_code(r4)
## [1] 201

If the status is 201, the data are uploaded to JAMES and processed. For example, the processed data after file upload is available as an R data frame under element r1$parsed.

r1$parsed
## $psn
##   id  name        dob       dobm       dobf  src    sex gad ga smo  bw hgtm
## 1 -1 Maria 2018-10-11 1990-12-02 1995-07-04 1234 female 189 27   1 990  167
##   hgtf
## 1  190
## 
## $xyz
##       age xname yname zname                  zref       x     y      z
## 1  0.0849   age   hgt hgt_z nl_2012_hgt_female_27  0.0849 38.00 -0.158
## 2  0.1670   age   hgt hgt_z nl_2012_hgt_female_27  0.1670 43.50  0.047
## 3  0.0000   age   wgt wgt_z nl_2012_wgt_female_27  0.0000  0.99  0.190
## 4  0.0849   age   wgt wgt_z nl_2012_wgt_female_27  0.0849  1.25 -0.203
## 5  0.1670   age   wgt wgt_z nl_2012_wgt_female_27  0.1670  2.10  0.015
## 6  0.0849   age   hdc hdc_z nl_2012_hdc_female_27  0.0849 27.00 -0.709
## 7  0.1670   age   hdc hdc_z nl_2012_hdc_female_27  0.1670 30.50 -0.913
## 8  0.0000   age   bmi bmi_z nl_1997_bmi_female_nl  0.0000    NA     NA
## 9  0.0849   age   bmi bmi_z nl_1997_bmi_female_nl  0.0849  8.66 -5.719
## 10 0.1670   age   bmi bmi_z nl_1997_bmi_female_nl  0.1670 11.10 -3.767
## 11 0.0000   hgt   wfh wfh_z   nl_2012_wfh_female_      NA  0.99     NA
## 12 0.0849   hgt   wfh wfh_z   nl_2012_wfh_female_ 38.0000  1.25 -0.001
## 13 0.1670   hgt   wfh wfh_z   nl_2012_wfh_female_ 43.5000  2.10  0.326

The session details, including the uploaded data, will remain available for a limited time. After 30 minutes the session is wiped. The session key is your entrance to the resource within the 30-minute window. The key can be retrieved as r1$session. For example, to see the result of the file upload session in markdown use

(session <- r1$session)
## [1] "x07d3fd0a12b19b"
resp <- james_get(host = host, path = file.path(session, "md"))
cat(resp$parsed)
## 
## 
##   * **psn**:
## 
##     -------------------------------------------------------------------------------
##      id   name       dob          dobm         dobf      src    dnr    sex     gad
##     ---- ------- ------------ ------------ ------------ ------ ----- -------- -----
##      -1   Maria   2018-10-11   1990-12-02   1995-07-04   1234   NA    female   189
##     -------------------------------------------------------------------------------
## 
##     Table: Table continues below
## 
## 
##     -----------------------------------------------------------------------------------
##      ga   smo   bw    hgtm   hgtf   agem   etn   pc4   blbf   blbm   eduf   edum   par
##     ---- ----- ----- ------ ------ ------ ----- ----- ------ ------ ------ ------ -----
##      27    1    990   167    190     NA    NA    NA     NA     NA     NA     NA    NA
##     -----------------------------------------------------------------------------------
## 
##   * **xyz**:
## 
##     ----------------------------------------------------------------------------------
##       age     xname   yname   zname           zref              x        y       z
##     -------- ------- ------- ------- ----------------------- -------- ------- --------
##      0.0849    age     hgt    hgt_z   nl_2012_hgt_female_27   0.0849    38     -0.158
## 
##      0.167     age     hgt    hgt_z   nl_2012_hgt_female_27   0.167    43.5    0.047
## 
##        0       age     wgt    wgt_z   nl_2012_wgt_female_27     0      0.99     0.19
## 
##      0.0849    age     wgt    wgt_z   nl_2012_wgt_female_27   0.0849   1.25    -0.203
## 
##      0.167     age     wgt    wgt_z   nl_2012_wgt_female_27   0.167     2.1    0.015
## 
##      0.0849    age     hdc    hdc_z   nl_2012_hdc_female_27   0.0849    27     -0.709
## 
##      0.167     age     hdc    hdc_z   nl_2012_hdc_female_27   0.167    30.5    -0.913
## 
##        0       age     bmi    bmi_z   nl_1997_bmi_female_nl     0       NA       NA
## 
##      0.0849    age     bmi    bmi_z   nl_1997_bmi_female_nl   0.0849   8.657   -5.719
## 
##      0.167     age     bmi    bmi_z   nl_1997_bmi_female_nl   0.167    11.1    -3.767
## 
##        0       hgt     wfh    wfh_z    nl_2012_wfh_female_      NA     0.99      NA
## 
##      0.0849    hgt     wfh    wfh_z    nl_2012_wfh_female_      38     1.25    -0.001
## 
##      0.167     hgt     wfh    wfh_z    nl_2012_wfh_female_     43.5     2.1    0.326
##     ----------------------------------------------------------------------------------
## 
## 
## <!-- end of list -->

Troubleshooting data upload: JAMES executes checks on the conversion and ranges of the data. To gain efficiency, it does not automatically validate the input data against the specified JSON schema. JAMES writes diagnostic, sometimes cryptic, messages to the directory {session}/messages if it finds a problem. The user can rerun the data upload with two additional flags that request extra diagnostic output.

Example: Suppose we compromise the data by removing the required "clientDetails" and the optional "nestedDetails" sections. The mangled input data look like:

{"Format":"3.0","organisationCode":12345,"reference":"Maria's mangled data","clientMeasurements":[{"bdsNumber":235,"values":[{"date":"20181111","value":380},{"date":"20181211","value":435}]}]}

Everything appears normal if we read this data by the default:

fn <- "maria-mangled.json"
r5 <- james_post(host = host, path = "data/upload/json", txt = fn)
r5$parsed
## $psn
##   id                 name   src
## 1 -1 Maria's mangled data 12345
## 
## $xyz
##   xname yname zname            zref  y
## 1   age   hgt hgt_z nl_1997_hgt__nl 38

If we upload with the additional validate = TRUE flag, JAMES runs the validation of the uploaded JSON against the JSON schema:

r6 <- james_post(host = host, path = "data/upload/json", txt = fn, validate = TRUE)
mess <- james_get(host = host, path = file.path(r6$session, "messages"))
cat(mess$parsed)
## must have required property 'clientDetails'

which now indicates that the required JSON element "clientDetails" is missing.

We may drill down further by setting the intermediate = TRUE flag. This writes five JSON files that document the data flow into {session}/files/{*}.json.

For example, we can ask for the input data as read by JAMES by

r7 <- james_post(host = host, path = "data/upload/json", txt = fn, validate = TRUE, intermediate = TRUE)
url <- file.path(host, r7$session, "files/input.json")
url
## [1] "https://james.groeidiagrammen.nl/x05e444ef5f070e/files/input.json"

With browseURL(url) we may view the file contents in the browser. The files directory contains five JSON files:

  1. files/input.json: the JSON input data;
  2. files/bds.json: a data frame with info per BDS number;
  3. files/ddi.json: result of recoding BDS into GSED item names;
  4. files/psn.json: known fixed child covariates;
  5. files/xy.json: time-varying variables.

Inspection of these files may uncover any problems with JAMES’s understanding of the data. If needed, study the underlying R source code at https://raw.githubusercontent.com/growthcharts/bdsreader/master/R/read_bds.R.

The validate and intermediate flag are useful for development and debugging. In production, we recommend leaving them at their default value (FALSE) and monitor any messages written to {session}/messages.

bash

We start from child data in the file maria.json that we wish to process with JAMES. For testing purposes, you may change the values, but keep the general structure intact. The following curl commands uploads the file and processes the data.

curl -sF '[email protected]' -D headers $(cat .host)/data/upload/json > content
head content
## {
##   "psn": [
##     {
##       "id": -1,
##       "name": "Maria",
##       "dob": "2018-10-11",
##       "dobm": "1990-12-02",
##       "dobf": "1995-07-04",
##       "src": "1234",
##       "sex": "female",

Alternatively, we may read the file into a JSON string, and upload as follows:

JS=$(jq '.' maria.json | jq -sR '.')
curl -s $(cat .host)/data/upload/json -d "txt=$JS" > content

Finally, if the data are located at a URL, use

URL=$(cat .host)/ocpu/library/bdsreader/examples/maria.json
curl -s $(cat .host)/data/upload/json -d "txt='$URL'" > content

/charts/draw: Draw child data on growth chart

R

Maria is a preterm born at 27 weeks of gestational age. We already uploaded her data. We may now plot her growth data on the A4 chart for preterms as follows:

r5 <- james_post(host = host, 
                 path = "/charts/draw/svglite", 
                 session = r1$session,
                 chartcode = "PMAAN27", selector = "chartcode",
                 query = list(height = 29.7/2.54, width = 21/2.54))
mysvg <- tempfile(pattern = "chart", fileext = rep(".svg", 3))
writeLines(r5$parsed, con = mysvg[1])