Robustness release following a code audit of the compiled reader and of the download layer. No new dependencies.
* flag) are dropped instead of
returned as all-NA rows.<cache>/<source>/<file_type>/<period>/<file_name>);
existing caches re-download once.timeout in datasus_download() /
datasus_fetch() defaults to Inf.datasus_download() gains success and
error columns; datasus_fetch() gains
refresh and overwrite arguments.encoding, year, month and
uf are validated and invalid values error.* flag) are dropped.
They used to come back as all-NA rows. n_max
now counts live records only.R_UnwindProtect() whenever an R error or interrupt unwinds
out of the reader.-2147483648, R’s
NA sentinel) are inferred as double or
returned as NA instead of a wrong number.trim_ws now only affects character output (with
trim_ws = FALSE, padded numbers used to become
NA).20240231 are now NA
instead of rolling over into the next month.encoding is validated: "latin1" (and the
latin-1 / ISO-8859-1 aliases),
"UTF-8" and "unknown" are accepted; anything
else errors instead of being silently treated as Latin-1.select and col_types names are matched
case-insensitively against the DBF field names, and select
names that match no column raise a warning instead of silently returning
fewer columns.n_max is floored to a whole number of rows.SIM/DO and
SINASC/DN without uf now list all
27 UFs (previously zero files were returned).<cache>/<source>/<file_type>/<period>/<file_name>.
Final and preliminary releases of the same file
(e.g. DOPE2022.dbc) used to share one path, which made
datasus_download() fail with “Duplicate destfiles” or
served a stale preliminary file forever. Existing caches will
re-download once..part name and
renamed only on success; partial files are removed.
datasus_download() gains success and
error columns and downloaded reflects the real
outcome. datasus_fetch() skips failed files with a warning
and errors if every download failed.datasus_fetch() prefers final over preliminary
data: when the same file (matched case-insensitively, and
treating SINAN_P / ESUSNOTIFICA_P as
preliminary aliases of SINAN / ESUSNOTIFICA)
is available in both trees only the final copy is read, so rows are not
duplicated. A final candidate that could not be confirmed on the FTP
does not suppress a preliminary copy known to exist.
datasus_fetch() also gains refresh and
overwrite arguments (the vignette documented
refresh but it was not accepted).timeout no longer caps the whole
transfer. It used to be passed to libcurl as the total time
allowed per file (240 s), which aborted large files on the slow DATASUS
FTP. It now defaults to Inf; stalled transfers are detected
with a connect timeout and a low-speed limit instead.exists = NA, instead of being silently reported
as absent.datasus_build_path() no longer recycles a vector
year against the SINASC path templates (which produced a
warning and wrong paths).SINAN_P was advertised in
datasus_sources() but had no file types and could never be
fetched; its file types are now registered.year, month and uf are
validated before any network access.datasus_download(use_cache = FALSE) without
dest_dir writes to a fresh subdirectory of
tempdir() instead of the working directory, and
overwrite = FALSE is honoured in that mode.datasus_get_territory() copes with upper-case member
names inside the territorial ZIP.tests/testthat) covering the
DBF parser with synthetic files and the catalog / cache / download logic
with file:// URLs.DESCRIPTION now consistently single-quotes external
software/API names (‘DATASUS’, ‘DBC’, ‘DBF’, ‘FTP’, ‘C’, ‘PKWare DCL’,
‘blast’, ‘zlib’) and unquotes the in-package function reference
datasus_fetch(). The description field also now references
the upstream sources (DATASUS file transfer site and Adler 2003 for the
bundled blast decompressor).\dontrun{} to
\donttest{}. Network- dependent examples are wrapped in
tryCatch() and write to tempdir() so that
running them never touches the user’s home filespace.tempdir() instead of
tools::R_user_dir("datasusr", "cache"). To opt in to a
persistent cache across sessions, set the
DATASUSR_CACHE_DIR environment variable, the
datasusr.cache_dir R option, or pass cache_dir
explicitly.datasus_fetch(): high-level
convenience function that lists, downloads, and reads DATASUS files in a
single call. Supports column selection, type specification, and
row-binding of multiple files.DATASUSR_CACHE_DIR environment
variable: the cache directory now respects the
DATASUSR_CACHE_DIR env var in addition to the existing
datasusr.cache_dir option.format_bytes() is now exported for general use.DOEXT,
DOINF, and DOMAT are served from the same
DOFET FTP directory as DOFET (verified against
the live FTP); the templates register that directory for each of
them.datasus_build_path(), datasus_file_types(),
and datasus_list_files(). Column names like
source and file_type no longer clash with
function arguments thanks to proper use of .env$
pronouns.datasus_download() instead of proceeding with zero
iterations..data$ pronoun usage
inside dplyr verbs. Columns are now referenced with bare
(tidy evaluation) names, and external variables use .env$
where needed. This follows current tidyverse best practice.pmap_dfr() replaced with
map2() |> list_rbind().purrr::map_lgl() in cache
functions with base vapply() for fewer
dependencies in hot paths.cli::cli_abort() with structured hints
(i bullets) instead of bare stop(). Progress
messages are more concise and informative.rlang is now an explicit dependency (was already an
indirect dep via dplyr), making the %||%
import explicit.rlang (>= 1.0.0) and
dplyr (>= 1.1.0) minimum version constraints..o and .so files excluded from
the source bundle via .Rbuildignore..data$ usage inside mutate pipelines and
simplified tidyverse expressions.read_datasus_dbc() so row
and column counts print on separate lines.datasus_file_types(source = ...) filtering
consistency.cli spacing and summary formatting in key
functions.datasus_list_files().cli.datasus_download().verbose control to FTP, listing, and
cache helpers.datasus_cache_dir(),
datasus_cache_list(), datasus_cache_info(),
datasus_cache_clear(), and
datasus_cache_prune().datasus_download().datasus_sources(),
datasus_modalities(), datasus_file_types(),
and datasus_ufs().datasus_ftp_ls(),
datasus_build_path(), datasus_list_files(),
and datasus_download().guess_types, col_types, and
parse_dates to read_datasus_dbc().