| Title: | Normalized HTTP Transport with Circuit Breaking and Caching |
| Version: | 0.1.2 |
| Description: | Web service calls return a normalized result value instead of raising a condition, so a caller branches on data rather than on an error handler. Transport failure, a non-success status code, and an unreadable response body are reported as three distinct outcomes. Per-host circuit breaking, retry with a transient-failure predicate, optional throttling, redacted request headers, and a success-only cache come as defaults. Many questions to one source can be asked as a single batch, where only the entries the cache is missing reach the network. Service-specific knowledge is left to the client packages built on top. The circuit breaker is the pattern described in Nygard (2018, ISBN:9781680502398). |
| License: | MIT + file LICENSE |
| URL: | https://github.com/samuelbharti/biohttp, https://www.samuelbharti.com/biohttp/ |
| BugReports: | https://github.com/samuelbharti/biohttp/issues |
| Encoding: | UTF-8 |
| Language: | en-US |
| Depends: | R (≥ 4.0) |
| Imports: | cachem, curl, httr2, jsonlite, rlang, tools |
| Suggests: | knitr, rmarkdown, testthat (≥ 3.0.0), webfakes, withr |
| VignetteBuilder: | knitr |
| Config/testthat/edition: | 3 |
| Config/roxygen2/version: | 8.0.0 |
| NeedsCompilation: | no |
| Packaged: | 2026-08-23 03:19:46 UTC; Samuel |
| Author: | Samuel Bharti |
| Maintainer: | Samuel Bharti <samuelbharti.io@gmail.com> |
| Repository: | CRAN |
| Date/Publication: | 2026-09-03 11:50:41 UTC |
biohttp: Normalized HTTP Transport with Circuit Breaking and Caching
Description
Web service calls return a normalized result value instead of raising a condition, so a caller branches on data rather than on an error handler. Transport failure, a non-success status code, and an unreadable response body are reported as three distinct outcomes. Per-host circuit breaking, retry with a transient-failure predicate, optional throttling, redacted request headers, and a success-only cache come as defaults. Many questions to one source can be asked as a single batch, where only the entries the cache is missing reach the network. Service-specific knowledge is left to the client packages built on top. The circuit breaker is the pattern described in Nygard (2018, ISBN:9781680502398).
Details
Start at get_json(), post_json(), or get_text() for a call that
assembles, performs, and caches itself. Drop to perform() when you need to
build the request yourself, and pass it through req_defaults() first.
Every one of them returns an envelope() rather than raising, so a client
branches on res$status and writes no tryCatch() of its own.
On the jsonlite import
jsonlite is imported but never called here, because httr2 calls it on this
package's behalf: httr2::resp_body_json() does
check_installed("jsonlite") at runtime, and httr2 keeps it in Suggests
rather than Imports. Declaring it is what stops get_json() failing on a
clean install. The import below is what makes that declaration honest to
R CMD check.
Author(s)
Maintainer: Samuel Bharti samuelbharti.io@gmail.com (ORCID) [copyright holder]
Authors:
Samuel Bharti samuelbharti.io@gmail.com (ORCID) [copyright holder]
See Also
Useful links:
Report bugs at https://github.com/samuelbharti/biohttp/issues
The seven envelope statuses
Description
Ordered from best outcome to worst.
Usage
STATUS_LEVELS
Format
A character vector of length seven.
Details
okThe call succeeded and
dataholds the parsed result.no_dataThe source was reached and has nothing for this query, for example a 404 or an empty body. This is an answer, not a fault.
staleA cached value served past its freshness window.
rate_limitedThe source asked the caller to slow down, a 429.
timeoutThe request did not return in time.
skippedNot attempted, because the host's breaker is open or a caller-side budget was hit.
errorAnything else: a 5xx, a malformed body, a transport failure.
Convert an envelope to the old four-field shape
Description
Deprecated. Scheduled for removal in 0.3.0.
Translates an envelope() into the four-field shape a hand-rolled HTTP layer
commonly returns:
list(ok = TRUE, status = 200L, data = <parsed>, error = NULL)
detail is carried through as a fifth field, because call sites that log it
would otherwise break on adoption.
Usage
as_legacy_envelope(res)
Arguments
res |
An envelope from |
Value
A list with ok, status (the HTTP code), data, error, and
detail.
Why this is deprecated on arrival
The old shape cannot express two things the new one can. It has no way to say "skipped because the host's breaker was open" as distinct from "failed", and it collapses the user-facing sentence and the log detail into one field. Both are load-bearing, and neither can be reconstructed once a call site is written against the old shape.
Note the collision in the word status: here it is the HTTP code, an
integer, and in an envelope() it is the enum. That is exactly the ambiguity
the new contract removes.
Removal
Scheduled for removal in 0.3.0. Use it to land a migration with a reviewable diff, then delete the call sites and delete this.
Warns once per session, not once per call, so a migration loop stays readable.
Examples
old <- as_legacy_envelope(status_ok(data = list(n = 1), source = "MyGene"))
old$ok
old$status
The parsed body, or NULL
Description
For a client that wants the body and does not care why a call failed. Every other field is still there for a client that does care, so reach for this when the failure is genuinely not actionable, not to avoid reading the envelope.
Usage
body_or_null(res)
Arguments
res |
An envelope from |
Details
Returns the body for ok and for stale. A stale envelope carries real
data that is merely past its freshness window, so returning NULL for it
would throw away the one thing the caller asked for. Branch on res$status
if the difference matters. Note that res$ok is FALSE for stale, which
is what keeps cached() from storing it.
Value
res$data when the status is ok or stale, otherwise NULL.
Examples
body_or_null(status_ok(data = list(n = 1)))
body_or_null(status_stale(data = list(n = 1)))
body_or_null(status_error(source = "gnomAD"))
Is a host's breaker open
Description
Open means too many consecutive transport failures and still inside the
cooldown. While open, perform() short-circuits to a skipped envelope
instead of waiting on a request that is likely to fail anyway.
Usage
breaker_open(host, now = as.numeric(Sys.time()))
Arguments
host |
A hostname, as returned by parsing a request URL. |
now |
The current time in seconds since the epoch. Exposed so a test can drive the clock rather than sleep. |
Value
A single logical.
Examples
breaker_reset()
breaker_open("gnomad.broadinstitute.org")
Record a call outcome against a host
Description
reachable means an HTTP response came back at all, whatever its status. A
reachable call clears the count; the Kth consecutive unreachable one trips
the breaker for the cooldown window.
Usage
breaker_record(host, reachable, now = as.numeric(Sys.time()))
Arguments
host |
A hostname. |
reachable |
Whether an HTTP response arrived. |
now |
The current time in seconds since the epoch. |
Details
Pass reachable = TRUE for any response in hand, including a 5xx and
including a 2xx whose body will not parse. Only a genuine transport failure
is FALSE.
Value
The host's new breaker state, invisibly.
Examples
breaker_reset()
breaker_record("example.org", reachable = FALSE)
breaker_record("example.org", reachable = TRUE)
breaker_open("example.org")
Clear all breaker state
Description
Forgets every host's failure count and cooldown. For tests, and for a manual "try again" once a source is known to be back.
Usage
breaker_reset()
Value
NULL, invisibly.
Examples
breaker_reset()
The cache store
Description
The cachem object backing cached(). Memory-only unless
BIOHTTP_CACHE_DISK asks for the disk tier, in which case the two are
layered. Built on first use.
Usage
cache()
Details
It is safe to hold the returned object. cache_reset() clears it in place
rather than replacing it, unless a cache setting has changed, in which case a
differently configured store genuinely has to be built. See cache_reset().
Value
A cachem cache object.
Examples
cache_reset()
class(cache())[1]
Build a cache key
Description
A stable lowercase-hex hash. cachem keys may contain only lowercase
letters and numbers, so a URL cannot be one directly and the inputs are
hashed instead. rlang::hash() returns lowercase hex, which satisfies the
rule.
Usage
cache_key(source, key, params = NULL)
Arguments
source |
A friendly label for the service. |
key |
Something identifying the call, usually the method and URL. |
params |
Anything else that changes the answer, such as a POST body. |
Details
The salt and the schema version are inside the hash, so changing either
yields a fresh keyspace. Set BIOHTTP_CACHE_SALT per deployment: a shared
disk cache would otherwise collide across app versions, and it would leak
which queries were run to anyone able to probe it.
Put anything that changes the answer into params, credentials
included. The wrappers pass their headers through, because two callers
hitting the same URL with different tokens can legitimately get different
responses, and keying on the URL alone would serve one caller's data to the
other. The cost is that rotating a token misses the cache once, which is the
right trade.
Value
A single string.
Examples
cache_key("gnomAD", "GET https://example.org/v1/gene/BRCA1")
cache_key("gnomAD", "POST https://example.org/graphql", list(q = "BRCA1"))
Empty the cache
Description
Removes every entry. Which of two things happens underneath depends on whether the cache settings have changed since the store was built:
Usage
cache_reset()
Details
- Settings unchanged
The existing store is cleared in place, so a reference taken from
cache()earlier stays valid and keeps pointing at the live cache. This is the ordinary case.- Settings changed
The store is dropped so the next
cache()builds one from the new settings. Tests rely on this after pointingBIOHTTP_CACHE_DIRat a fresh tempdir or togglingBIOHTTP_CACHE_DISK.
The in-place branch exists because dropping unconditionally silently orphans any held reference: writes through it go somewhere nothing else can see, and the only symptom is a hit rate quietly falling to zero. A reference held across a genuine settings change is still orphaned, but nothing changes cache settings mid-run except a test suite, and those do not hold references.
Value
NULL, invisibly.
Examples
held <- cache()
cache_reset()
identical(held, cache())
Serve from cache, or fetch and cache a success
Description
Returns a cached result for key if there is one, otherwise runs fetch()
and stores the result only when it succeeded.
Usage
cached(key, fetch)
Arguments
key |
A key from |
fetch |
A function of no arguments returning an envelope. |
Details
A failure is never cached. This is the single most important rule in the package. A cache that stores an error fallback poisons itself for the life of the R process, and every later lookup then serves the stored failure instead of retrying. Storing only successes means a transient outage resolves itself the moment the source comes back.
Value
The envelope, from the cache or from fetch().
Examples
cache_reset()
key <- cache_key("demo", "GET /x")
cached(key, function() status_ok(data = list(n = 1), source = "demo"))
# A failure runs fetch() again every time.
bad <- cache_key("demo", "GET /y")
cached(bad, function() status_error(source = "demo"))
Classify a transport condition onto the status enum
Description
For a failure with no HTTP response at all. A DNS failure or a refused
connection is an error; only an actual timeout is a timeout, because the
two read very differently to a user deciding whether to retry.
Usage
classify_condition(cond)
Arguments
cond |
A condition caught from a failed request. |
Value
One of STATUS_LEVELS.
Examples
classify_condition(simpleError("Timeout was reached"))
classify_condition(simpleError("Could not resolve host"))
Classify an HTTP status code onto the status enum
Description
Only the codes the transport treats as an outcome in themselves are
classified. A 2xx is the caller's to interpret, since a 200 carrying an empty
result may still be no_data once parsed.
Usage
classify_http(http)
Arguments
http |
An HTTP status code. |
Value
One of STATUS_LEVELS.
Examples
classify_http(404L)
classify_http(429L)
classify_http(503L)
Build a result envelope
Description
The single return shape for the whole package. ok is derived from status
rather than passed in, so the two can never disagree.
Usage
envelope(
status,
data = NULL,
source = "API",
http = NA_integer_,
error = NULL,
detail = NULL
)
Arguments
status |
One of STATUS_LEVELS. |
data |
The parsed body on success, |
source |
A friendly label for the service, used in the user-facing
message. For example |
http |
The HTTP status code, or |
error |
One sentence fit to show a user. Never carries technical detail. |
detail |
The technical cause, for a log. Never shown to a user. |
Details
A client that only wants the parsed body can use body_or_null() instead of
reading the fields directly.
Value
A list with ok, status, http, data, source, error,
detail, and ts.
Examples
envelope("ok", data = list(symbol = "BRCA1"), source = "MyGene", http = 200L)
envelope("error", source = "gnomAD", http = 503L)
GET a JSON endpoint
Description
Assembles the request, applies req_defaults(), performs it, and caches the
result if it succeeded.
Usage
get_json(
base_url,
path = NULL,
query = list(),
source = "API",
timeout = 15,
max_tries = 3,
headers = NULL,
throttle = NULL,
secret_query = NULL
)
Arguments
base_url |
The service's base URL. |
path |
An optional path appended to |
query |
A named list of query parameters. Blank values are dropped. |
source |
A friendly label for the service. |
timeout |
Seconds before the request is abandoned. |
max_tries |
Total attempts, including the first. |
headers |
A named list of headers, all marked sensitive. |
throttle |
An optional throttle spec. See |
secret_query |
A named list of query-string credentials, for a service
that has no header form. Deliberately not part of the cache key, so it
suits a credential that raises a rate limit and not one that changes the
response. See |
Value
An envelope. See envelope().
Examples
# Mocked so the example runs offline. A real call drops the
# with_mocked_responses() wrapper.
cache_reset()
breaker_reset()
httr2::with_mocked_responses(
list(httr2::response(
status_code = 200,
headers = list(`content-type` = "application/json"),
body = charToRaw('{"hits":[{"symbol":"BRCA1"}]}')
)),
get_json(
"https://mygene.info/v3",
path = "query",
query = list(q = "BRCA1", species = "human"),
source = "MyGene"
)$data$hits[[1]]$symbol
)
GET many JSON endpoints as one batch
Description
The batched counterpart to get_json(). Assembles one request per entry in
queries, serves whatever the cache already holds, performs only the rest,
and caches the successes.
Usage
get_json_many(
base_url,
path = NULL,
queries = list(),
source = "API",
timeout = 15,
max_tries = 3,
headers = NULL,
throttle = NULL,
max_active = 6,
progress = FALSE,
secret_query = NULL
)
Arguments
base_url |
The service's base URL. |
path |
A path appended to |
queries |
A list of named lists, one per request. Blank values are dropped from each. |
source |
A friendly label for the service, used in the user-facing
message. For example |
timeout |
Seconds before a request is abandoned. |
max_tries |
Total attempts, including the first. Note that httr2 does not honor this under parallel performance. |
headers |
A named list of headers, all marked sensitive. |
throttle |
A throttle spec. See |
max_active |
Maximum requests in flight at once. |
progress |
Passed to |
secret_query |
A named list of query-string credentials, applied to
every request in the batch at dispatch. See |
Details
Read the sections on perform_many() first. The same two rules apply: pass
requests for one host, and supply a throttle.
Value
A list of envelopes, the same length and order as queries.
Examples
cache_reset()
breaker_reset()
httr2::with_mocked_responses(
function(req) {
httr2::response(
status_code = 200,
headers = list(`content-type` = "application/json"),
body = charToRaw('{"hits":[]}')
)
},
length(get_json_many(
"https://mygene.info/v3",
path = "query",
queries = list(list(q = "BRCA1"), list(q = "TP53")),
source = "MyGene"
))
)
GET a text endpoint
Description
For a source that ships a bulk flat file, CSV or TSV, rather than a
per-record JSON API. Same timeout, retry, caching, and header redaction as
get_json(); the body comes back verbatim as a string in data.
Usage
get_text(
base_url,
path = NULL,
query = list(),
source = "API",
timeout = 30,
max_tries = 3,
headers = NULL,
throttle = NULL,
secret_query = NULL
)
Arguments
base_url |
The service's base URL. |
path |
An optional path appended to |
query |
A named list of query parameters. Blank values are dropped. |
source |
A friendly label for the service. |
timeout |
Seconds before the request is abandoned. |
max_tries |
Total attempts, including the first. |
headers |
A named list of headers, all marked sensitive. |
throttle |
An optional throttle spec. See |
secret_query |
A named list of query-string credentials, for a service
that has no header form. Deliberately not part of the cache key, so it
suits a credential that raises a rate limit and not one that changes the
response. See |
Details
The success-only cache means the file is fetched once per URL and every later lookup against it is served in process.
Value
An envelope whose data is a single string on success.
Examples
cache_reset()
breaker_reset()
httr2::with_mocked_responses(
list(httr2::response(
status_code = 200,
body = charToRaw("gene\tclassification\nBRCA1\tDefinitive\n")
)),
get_text(
"https://search.clinicalgenome.org",
path = "kb/gene-validity/download",
source = "ClinGen"
)$data
)
Catch a GraphQL query error inside a 200
Description
GraphQL reports query errors in the body of an HTTP 200, as a top-level
errors array, so a 2xx status is not enough to call a call successful.
Usage
graphql_error(res, source = "API")
Arguments
res |
An envelope from |
source |
A friendly label for the service. |
Details
Returns an error envelope for either kind of failure, transport or query, so a GraphQL client collapses the two checks into one:
res <- post_json(url, body, source = "gnomAD") bad <- graphql_error(res, "gnomAD") if (!is.null(bad)) return(bad)
Value
The failing envelope, or NULL when the call genuinely succeeded.
Examples
clean <- status_ok(data = list(data = list(gene = "BRCA1")), source = "G")
graphql_error(clean, "G")
queried <- status_ok(data = list(errors = list(list(message = "bad"))))
graphql_error(queried, "G")$status
A user-facing sentence for a failed call
Description
Works out which status the failure is and asks status_message() for the
wording, so it honours biohttp.status_message like everything else. Pass
http for a response that arrived, or condition for a transport failure
where none did.
Usage
http_error_message(source, http = NA_integer_, condition = NULL)
Arguments
source |
A friendly label for the service, for example |
http |
An HTTP status code, or |
condition |
A caught condition, used only when |
Value
A single string.
Examples
http_error_message("gnomAD", http = 404L)
http_error_message("gnomAD", http = 503L)
http_error_message("gnomAD", condition = simpleError("Timeout was reached"))
Is a value blank
Description
TRUE for NULL, a zero-length value, NA, or a string that is empty once
trimmed. Used to drop empty query parameters so a request never sends
param= with nothing after it.
Usage
is_blank(x)
Arguments
x |
Any value. |
Value
A single logical.
Examples
is_blank(NULL)
is_blank(" ")
is_blank("gnomAD")
Is a response worth retrying
Description
A 429 and the standard 5xx codes are transient. Everything else is a settled answer, and retrying it just spends someone else's capacity.
Usage
is_transient(resp)
Arguments
resp |
An httr2 response. |
Value
A single logical.
Examples
is_transient(httr2::response(status_code = 503))
is_transient(httr2::response(status_code = 404))
Perform a request and normalize the result
Description
Never raises on an HTTP or a parse failure. Returns an envelope for every
outcome, so a caller branches on res$status and writes no tryCatch() of
its own.
Usage
perform(req, source = "API", secret_query = NULL)
Arguments
req |
An httr2 request, prepared with |
source |
A friendly label for the service, used in the user-facing
message. For example |
secret_query |
A named list of query-string credentials. Appended here
rather than by the caller, so |
Details
Short-circuits to a skipped envelope without sending anything when the
host's breaker is open. See breaker_open().
The request must have been through req_defaults(), which disarms httr2's
error raising. Without that, req_perform() throws on a non-2xx and a
perfectly reachable host is recorded as a transport failure.
Value
An envelope. See envelope().
Examples
breaker_reset()
req <- req_defaults(httr2::request("https://mock.test/x"))
httr2::with_mocked_responses(
list(httr2::response(
status_code = 200,
headers = list(`content-type` = "application/json"),
body = charToRaw('{"symbol":"BRCA1"}')
)),
perform(req, "MyGene")$data$symbol
)
Perform many requests as one batch
Description
For many questions to one source. Returns a list of envelopes the same length
as reqs and in the same order, so a caller zips the results back onto
whatever it asked about by position.
Usage
perform_many(
reqs,
source = "API",
max_active = 6,
progress = FALSE,
secret_query = NULL
)
Arguments
reqs |
A list of httr2 requests, each prepared with |
source |
A friendly label for the service, used in the user-facing
message. For example |
max_active |
Maximum requests in flight at once. |
progress |
Passed to |
secret_query |
A named list of query-string credentials, applied to
every request in the batch at dispatch. See |
Details
Each request must already have been through req_defaults(), the same as for
perform().
Value
A list of envelopes, the same length and order as reqs. See
envelope().
Same host, not a mix
httr2 applies req_throttle() and req_retry() across the whole list rather
than per request, so a throttled request to one host makes an unthrottled
request to another wait behind it. Requests are therefore grouped by host and
each group is dispatched separately, which keeps each host's throttle bucket
honest but means host groups run one after another.
Pass requests for one host. To query many different services at once, use process-level concurrency in the application instead.
Supply a throttle
httr2's own advice is never to perform in parallel without req_throttle(),
because it is otherwise very easy to flood a source with simultaneous
requests. req_defaults() takes a throttle argument and defaults its realm
to the request's host. Use it. Public biological data sources are typically
run on a research budget.
The breaker acts between batches, not inside one
A host with an open breaker is never dispatched to, and every one of its
requests comes back skipped. But once a batch is in flight, a failure on the
first request cannot short-circuit the rest, because they have already been
sent. Transport failures within a batch are recorded, so they take effect on
the next call rather than the current one.
Examples
breaker_reset()
reqs <- lapply(
c("BRCA1", "TP53"),
function(symbol) {
req_defaults(httr2::request(paste0("https://mock.test/gene/", symbol)))
}
)
httr2::with_mocked_responses(
function(req) {
httr2::response(
status_code = 200,
headers = list(`content-type` = "application/json"),
body = charToRaw('{"ok":true}')
)
},
vapply(perform_many(reqs, "MyGene"), function(res) res$status, character(1))
)
Perform a request and return the body as text
Description
The non-JSON counterpart to perform(), for a source that ships a bulk flat
file rather than a per-record JSON API. The body lands in data as a single
string, so the envelope shape is the same either way and a caller does not
branch on which wrapper it used.
Usage
perform_text(req, source = "API", secret_query = NULL)
Arguments
req |
An httr2 request, prepared with |
source |
A friendly label for the service, used in the user-facing
message. For example |
secret_query |
A named list of query-string credentials. Appended here
rather than by the caller, so |
Details
The breaker rule is the same one: only a transport failure counts against the host.
Value
An envelope whose data is a single string on success.
Examples
breaker_reset()
req <- req_defaults(httr2::request("https://mock.test/genes.tsv"))
httr2::with_mocked_responses(
list(httr2::response(
status_code = 200,
body = charToRaw("gene\tscore\n")
)),
perform_text(req, "ClinGen")$data
)
Pull a value out of a nested list
Description
Walks a key path and returns default if any level is missing or NULL,
rather than erroring. Parsed JSON is deeply nested and frequently missing
branches, so this is the difference between a client that reads cleanly and
one buried in if (!is.null(...)).
Usage
pluck_at(x, ..., default = NULL)
Arguments
x |
A list, usually a parsed JSON body. |
... |
Keys to follow, outermost first. |
default |
Returned when any level of the path is missing or |
Value
The value at the end of the path, or default.
Examples
body <- list(data = list(gene = list(symbol = "BRCA1")))
pluck_at(body, "data", "gene", "symbol")
pluck_at(body, "data", "variant", "id", default = NA_character_)
POST a JSON body
Description
For a JSON or GraphQL POST. Caches on the URL and the body together, so two different queries to the same endpoint do not collide.
Usage
post_json(
url,
body,
source = "API",
timeout = 20,
max_tries = 3,
headers = NULL,
throttle = NULL,
secret_query = NULL
)
Arguments
url |
The endpoint URL. |
body |
A list, serialized as the JSON request body. |
source |
A friendly label for the service. |
timeout |
Seconds before the request is abandoned. |
max_tries |
Total attempts, including the first. |
headers |
A named list of headers, all marked sensitive. |
throttle |
An optional throttle spec. See |
secret_query |
A named list of query-string credentials, for a service
that has no header form. Deliberately not part of the cache key, so it
suits a credential that raises a rate limit and not one that changes the
response. See |
Value
An envelope. See envelope().
Examples
cache_reset()
breaker_reset()
httr2::with_mocked_responses(
list(httr2::response(
status_code = 200,
headers = list(`content-type` = "application/json"),
body = charToRaw('{"data":{"gene":{"gene_id":"ENSG00000012048"}}}')
)),
post_json(
"https://gnomad.broadinstitute.org/api",
body = list(query = "{ gene(gene_symbol: \"BRCA1\") { gene_id } }"),
source = "gnomAD"
)$data$data$gene$gene_id
)
POST many JSON bodies as one batch
Description
The batched counterpart to post_json(), for a JSON or GraphQL endpoint
answering many queries. Keyed on the URL and the body together, so two
different queries to the same endpoint do not collide.
Usage
post_json_many(
url,
bodies = list(),
source = "API",
timeout = 20,
max_tries = 3,
headers = NULL,
throttle = NULL,
max_active = 6,
progress = FALSE,
secret_query = NULL
)
Arguments
url |
The endpoint URL. |
bodies |
A list of request bodies, one per request. Each is serialized as JSON. |
source |
A friendly label for the service, used in the user-facing
message. For example |
timeout |
Seconds before a request is abandoned. |
max_tries |
Total attempts, including the first. Note that httr2 does not honor this under parallel performance. |
headers |
A named list of headers, all marked sensitive. |
throttle |
A throttle spec. See |
max_active |
Maximum requests in flight at once. |
progress |
Passed to |
secret_query |
A named list of query-string credentials, applied to
every request in the batch at dispatch. See |
Details
Read the sections on perform_many() first.
Value
A list of envelopes, the same length and order as bodies.
Examples
cache_reset()
breaker_reset()
httr2::with_mocked_responses(
function(req) {
httr2::response(
status_code = 200,
headers = list(`content-type` = "application/json"),
body = charToRaw('{"data":{}}')
)
},
length(post_json_many(
"https://gnomad.broadinstitute.org/api",
bodies = list(list(query = "{ a }"), list(query = "{ b }")),
source = "gnomAD"
))
)
Remove secret values from a string
Description
Replaces each secret wherever it appears, in both the form it was passed and the percent-encoded form a URL carries. Used on the messages built from a transport failure, because a curl error normally carries the URL that failed and that URL may hold a query-string credential.
Usage
redact_secrets(text, secrets = NULL)
Arguments
text |
A single string. |
secrets |
A named list of secret values, as passed to |
Details
Matching on the value rather than on the parameter name is deliberate: the same secret can reach a message through a URL, a header dump, or a proxy error, and only the value is common to all three.
Both forms have to be matched because a secret never reaches a URL in the
form it was passed. httr2::req_url_query() percent-encodes it, so a key
holding +, /, or =, which is an ordinary shape for a base64 key,
arrives as %2B, %2F, and %3D. Matching only the raw value would let
exactly those keys through into a logged detail.
Value
text, with every secret value replaced by <redacted>.
Examples
redact_secrets(
"Could not resolve host: example.org/?api_key=abc123",
list(api_key = "abc123")
)
# The encoded form is caught too, which is the one a URL actually carries.
redact_secrets(
"Could not resolve host: example.org/?api_key=ab%2Bcd",
list(api_key = "ab+cd")
)
Apply the shared request options
Description
Timeout, bounded retry on transient failures, an attributable user agent, and optionally a throttle and headers.
Usage
req_defaults(
req,
timeout = 20,
max_tries = 3,
headers = NULL,
throttle = NULL,
user_agent = default_user_agent()
)
Arguments
req |
An httr2 request. |
timeout |
Seconds before the request is abandoned. |
max_tries |
Total attempts, including the first. |
headers |
A named list of headers. All of them are marked sensitive. |
throttle |
An optional list with |
user_agent |
The User-Agent string. See |
Details
req_error() is disarmed here on purpose. httr2 would otherwise raise on a
non-2xx, and this package normalizes every outcome into an envelope instead,
so perform() needs the response object rather than a condition.
Headers are attached with .redact set to their names, so a token never
prints in an inspected request, a log line, or an error message. Header auth
also keeps the secret out of the URL, where it would end up in access logs.
Value
The request, with the options applied.
Examples
req_defaults(httr2::request("https://example.org"), timeout = 5)
# A key-gated source. The token is redacted from anything printable.
req_defaults(
httr2::request("https://example.org"),
headers = list(Authorization = "Bearer secret-token")
)
Envelope constructors, one per status
Description
Shorthands for envelope() that fill in the user-facing error sentence for
their status. error is what a user reads; detail is what a log records.
Usage
status_ok(data, source = "API", http = 200L)
status_no_data(source = "API", http = NA_integer_, detail = NULL)
status_stale(data, source = "API", detail = NULL)
status_rate_limited(source = "API", http = 429L, detail = NULL)
status_timeout(source = "API", detail = NULL)
status_skipped(source = "API", detail = NULL)
status_error(source = "API", http = NA_integer_, error = NULL, detail = NULL)
Arguments
data |
The parsed body on success, |
source |
A friendly label for the service, used in the user-facing
message. For example |
http |
The HTTP status code, or |
detail |
The technical cause, for a log. Never shown to a user. |
error |
One sentence fit to show a user. Never carries technical detail. |
Details
The sentence comes from status_message(), so setting
biohttp.status_message changes what these produce too.
Value
An envelope list, as described in envelope().
Examples
status_ok(data = list(n = 1), source = "MyGene")
status_no_data(source = "gnomAD", http = 404L)
status_skipped(source = "gnomAD", detail = "gnomad.broadinstitute.org open")
The user-facing sentence for an outcome
Description
Every error field in the package is produced here, so the seven statuses
cannot drift apart and an application can replace all of them at once.
Usage
status_message(
source = "API",
status = "error",
http = NA_integer_,
condition = NULL
)
Arguments
source |
A friendly label for the service, for example |
status |
One of STATUS_LEVELS. |
http |
An HTTP status code, or |
condition |
A caught condition, used when no response arrived. |
Value
A single string, or NULL for a status that carries no message.
Supplying your own wording
Set biohttp.status_message to a function of source, status, http and
condition. It is called instead of the built-in, and it is the way to keep
an app's own voice, or to localize, while adopting the transport:
options(biohttp.status_message = function(source, status, http, condition) {
if (identical(status, "timeout")) {
paste0(source, " took too long. Please try again shortly.")
} else {
NULL # fall through to the built-in for everything else
}
})
Returning anything other than a single non-empty string falls back to the built-in, so covering one case and leaving the rest is expected rather than an error.
An override cannot break the return-a-value contract. It runs inside a
tryCatch(), and a function that raises is treated as though it returned
nothing. This matters because the override runs on the failure path, which is
precisely where the package promises never to raise.
Examples
status_message("gnomAD", "no_data")
status_message("gnomAD", "error", http = 503L)
old <- options(
biohttp.status_message = function(source, status, http, condition) {
paste0(source, " says: ", status)
}
)
status_message("gnomAD", "timeout")
options(old)
Build an attributable User-Agent string
Description
Public sources ask callers to identify themselves. A caller identity plus a contact route let a source operator reach you before they rate-limit you.
Usage
user_agent(
identity = env_chr("BIOHTTP_CALLER_IDENTITY", "biohttp"),
version = as.character(getNamespaceVersion("biohttp")),
url = env_chr("BIOHTTP_CONTACT_URL", ""),
email = env_chr("BIOHTTP_CONTACT_EMAIL", "")
)
Arguments
identity |
The calling application's name. Defaults to
|
version |
The calling application's version. |
url |
A URL an operator can look up, usually the app's repository.
Defaults to |
email |
A contact address. Defaults to |
Details
The package deliberately does not ship an identity of its own. A consumer
passes its own, or sets BIOHTTP_CALLER_IDENTITY, BIOHTTP_CONTACT_URL, and
BIOHTTP_CONTACT_EMAIL so a hosted deployment can set them without touching
code.
Value
A single string.
Examples
user_agent("my-shiny-app", "1.2.0", email = "ops@example.org")
user_agent("my-client-package", "0.4.0")