# User Guide

### Package ‘photobiologyInOut’ 0.4.23

#### Pedro J. Aphalo

#### 2021-10-11

## Introduction

Package ‘photobiologyInOut’ defines functions for importing spectral data from different instruments, data repositories and simulation models, and for data exchange with R packages ‘hyperSpec’, ‘colorSpec’ and ‘pavo’ by reading or writing objects of classes defined in them (see Tables at the start of each section).

As of version 0.4.21 packages ‘hyperSpec’ and ‘pavo’ need to be installed only if the corresponding functions will be used. This avoids bloat when these functions are not needed. If the vignette is built or examples attempted to run in a system where these packages are not installed, some output will be missing but no error will be triggered.

All functions that read data from files attempt to decode and store as metadata the information present in file headers. In addition, in most cases the unchanged header of the file is stored unaltered as a comment in the constructed objects.

It should be remembered, though, that this package has been developed based on the example files I had access to. Files from the same instruments with different hardware configurations, different firmware versions, or even settings may differ substantially. In many cases the output is produced by software in a host computer rather by the instrument itself, adding further uncertainties and possible differences due to for example the operating system of the host computer. A further complication is that in some cases the format of dates, times and numbers depends on the locale settings in use at the time of data acquisition, or analysis. For all those reasons, do expect to have to do some debugging, and most importantly always validate the imported data against the original file (remembering to run a new validation each time there is a software or firmware update) or update of this package as I test each version before release only with the example files I have available, which are not many.

## Preliminaries

When loading suggested packages flags are set based on success and later used to selectively evaluate code chunks.

    library(photobiology)
    library(photobiologyWavebands)
    library(photobiologyInOut)
    library(lubridate)
    library(ggspectra)
    library(readr)
    library(colorSpec)
    if (eval_pavo) {library(pavo)}
    if (eval_hyperSpec) {library(hyperSpec)}

    # plot defaults
    theme_set(theme_bw()) # ggplot2
    set_annotations_default(annotations = c("+", "title:what:when"))
    # decrease lines printed
    options(tibble.print_max = 5,
            tibble.print_min = 3,
            photobiology.strict.range = NA_integer_)

## Spectrometer output files

In the examples we use function system-file to reliably locate the example files included in the package. For reproducing the examples with these files, this call is needed. When using user’s files only the path as a character string needs to be passed as argument.

Functions for importing measured spectral data.

R function | Instrument | Program | `class` of value
--- | --- | --- | ---
read_oo_ssirrad() | Ocean Optics spectrom. | SpectraSuite | source_spct
read_oo_ssdata() | Ocean Optics spectrom. | SpectraSuite | raw_spct
read_oo_jazirrad() | Ocean Optics Jaz | instrument | source_spct
read_oo_jazpc() | Ocean Optics Jaz | instrument | filter_spct
read_oo_jazpc() | Ocean Optics Jaz | instrument | reflector_spct
read_oo_jazdata() | Ocean Optics Jaz | instrument | raw_spct
read_oo_pidata() | Ocean Optics spectrom. | STS DK (Raspbian) | raw_spct
read_wasatch_csv() | Wasatch Phot. spectrom. | Enlighten | raw_spct
read_wasatch_csv() | Wasatch Phot. spectrom. | Enlighten | filter_spct
read_wasatch_csv() | Wasatch Phot. spectrom. | Enlighten | reflector_spct
read_wasatch_csv() | Wasatch Phot. spectrom. | Enlighten | source_spct
read_avaspec_csv() | Avantes spectrom. | instrument? | source_spct
read_macam_file() | Macam | instrument | source_spct
read_li180_txt() | LI-COR LI-180 | instrument | source_spct
read_m_li180_txt() | LI-COR LI-180 | instrument | source_mspct
read_licor_prn() | LI-COR LI-1800 | PC1800 (MS-DOS) | source_spct
read_licor_prn() | LI-COR LI-1800 | PC1800 (MS-DOS) | filter_spct
read_licor_prn() | LI-COR LI-1800 | PC1800 (MS-DOS) | reflector_spct
read_m_licor_prn() | LI-COR LI-1800 | PC1800 (MS-DOS) | source_mspct
read_m_licor_prn() | LI-COR LI-1800 | PC1800 (MS-DOS) | filter_mspct
read_m_licor_prn() | LI-COR LI-1800 | PC1800 (MS-DOS) | reflector_mspct

### Ocean Optics Jaz

#### Raw detector counts

Reading a raw data file generated by Ocean Optics’ Jaz spectrometer. The light source was the Jaz PX pulsed Xenon light module.

The first few lines of the file look like this, with W for wavelength, D for dark, R for reference, S for sample and P for processed (all spectral data values are raw detector counts):

    Jaz Data File
    ++++++++++++++++++++++++++++++++++++
    Date: Mon Apr 25 12:49:11 2016
    User: jaz
    Dark Spectrum Present: Yes
    Reference Spectrum Present: Yes
    Processed Spectrum Present: Yes
    Spectrometers: JAZA3098
    Integration Time (usec): 748000 (JAZA3098)
    Spectra Averaged: 1 (JAZA3098)
    Boxcar Smoothing: 0 (JAZA3098)
    Correct for Electrical Dark: No (JAZA3098)
    Strobe/Lamp Enabled: Yes (JAZA3098)
    Correct for Detector Non-linearity: No (JAZA3098)
    Correct for Stray Light: No (JAZA3098)
    Number of Pixels in Processed Spectrum: 2048
    >>>>>Begin Processed Spectral Data<<<<<
    W   D   R   S   P
    190.313904  0.000000    0.000000    0.000000    0.000000
    190.695511  0.000000    0.000000    0.000000    0.000000
    191.077087  1138.953125 1123.134277 1102.795898 228.570541
    191.458633  1184.149658 1227.086426 1059.859131 -289.473419
    191.840149  1175.110352 1193.188965 1132.173584 -237.500336
    ...

    jaz.raw.file <-
      system.file("extdata", "spectrum.jaz",
                  package = "photobiologyInOut", mustWork = TRUE)
    jazraw.spct <- read_oo_jazdata(file = jaz.raw.file)
    jazraw.spct <- trim_wl(jazraw.spct, range = c(250, 900))

Plotting the spectrum.

    autoplot(jazraw.spct)

The metadata stored in attributes can be accessed with functions. It is clear, that not all settings can be recovered from the file. However, we store the record will all the fields which would have been filled if the data had been acquired directly from R using package ‘ooacquire’.

    getWhenMeasured(jazraw.spct)

    ## [1] "2016-04-25 12:49:02 UTC"

    getInstrDesc(jazraw.spct)

    ## Data acquired with 'Jaz' s.n. JAZA3098
    ## grating 'NA', slit 'NA'

    getInstrSettings(jazraw.spct)

    ## integ. time (s): 0.748
    ## total time (s): 0.748
    ## counts @ peak (% of max):

#### Spectral energy irradiance

Reading an “Absolute Irradiance File” (sic) generated by Ocean Optics’ Jaz spectrometer results in a `source_spct` object. In this example, the light source measured was a `white’ fluorescent tube.

The first few lines of the file look like this:

    Jaz Absolute Irradiance File
    ++++++++++++++++++++++++++++++++++++
    Date: Tue Feb 03 09:44:41 2015
    User: jaz
    Dark Spectrum Present: Yes
    Processed Spectrum Present: Yes
    Spectrometers: JAZA1065
    Integration Time (usec): 193000 (JAZA1065)
    Spectra Averaged: 3 (JAZA1065)
    Boxcar Smoothing: 5 (JAZA1065)
    Correct for Electrical Dark: Yes (JAZA1065)
    Strobe/Lamp Enabled: No (JAZA1065)
    Correct for Detector Non-linearity: Yes (JAZA1065)
    Correct for Stray Light: No (JAZA1065)
    Number of Pixels in Processed Spectrum: 2048
    Fiber (micron): 3900
    Collection Area: 0.119459
    Int. Sphere: No
    >>>>>Begin Processed Spectral Data<<<<<
    W   D   S   P
    188.825226  0.000000    0.000000    0.000000
    189.284851  0.000000    0.000000    0.000000
    189.744415  -89.659378  -90.917900  -0.000000
    190.203964  -106.165916 -96.419785  0.000000
    ...

    jaz.s.irrad.file <-
      system.file("extdata", "spectrum.JazIrrad",
                  package = "photobiologyInOut", mustWork = TRUE)
    jaz.spct <- read_oo_jazirrad(file = jaz.s.irrad.file)
    jaz0.spct <- jaz.spct
    jaz.spct <- trim_wl(jaz.spct, range = c(290, 800))

Plotting the spectrum.

    autoplot(jaz.spct)

### Cleaning spectral data

We can see that the data have problems. We get a warning because the data include negative values for spectral irradiance. We will use some methods from package ‘photobiology’ to correct the problem. As the data are noisy we cannot just shift the scale so that the most negative value becomes zero. Neither can we replace all negative values with zeros, as this would create bias.

In the following code chunk we will use a region of the spectrum in which spectral irradiance is known to be equal to zero as reference to shift the scale zero. Afterwards we discard data ``known’’ to be zero, and for which the instrument calibration is not valid, and finally we plot the spectrum.

    jaz.spct <- fshift(jaz0.spct, range = c(255, 290), f = "mean")
    jaz.spct <- trim_wl(jaz.spct, range = c(290, 800))
    autoplot(jaz.spct)

We can next try to smooth the spectrum as it is very noisy outside the visible region.

    jaz.spct <- smooth_spct(jaz.spct)

    ## 413 possibly 'bad' values in smoothed spectral response

    autoplot(jaz.spct)

Photon and energy irradiances.

    e_irrad(jaz.spct, PAR())       # W m-2

    ##    E_PAR
    ## 26691.58
    ## attr(,"time.unit")
    ## [1] "second"
    ## attr(,"radiation.unit")
    ## [1] "total energy irradiance"

All in one statement.

    autoplot(read_oo_jazirrad(file = jaz.s.irrad.file))

As above but limiting the wavelength range plotted.

    autoplot(read_oo_jazirrad(file = jaz.s.irrad.file),
         range = c(250,850))

Adding our custom ``adaptive’’ smoothing.

    autoplot(smooth_spct(read_oo_jazirrad(file = jaz.s.irrad.file)),
         range = c(250,850))

    ## 629 possibly 'bad' values in smoothed spectral response

### Other modular spectrometers from Ocean Optics

Now a file from an Ocean Optics’ Q6500 spectrometer, with data processed with the Spectra Suite software.

Format of the header is similar, but not identical. The first few lines of the file look like this:

    SpectraSuite Data File
    ++++++++++++++++++++++++++++++++++++
    Date: Mon May 06 15:13:40 CEST 2013
    User: User
    Dark Spectrum Present: Yes
    Reference Spectrum Present: No
    Number of Sampled Component Spectra: 1
    Spectrometers: QEB1523
    Integration Time (usec): 100000 (QEB1523)
    Spectra Averaged: 1 (QEB1523)
    Boxcar Smoothing: 0 (QEB1523)
    Correct for Electrical Dark: No (QEB1523)
    Strobe/Lamp Enabled: No (QEB1523)
    Correct for Detector Non-linearity: No (QEB1523)
    Correct for Stray Light: Yes (QEB1523)
    Number of Pixels in Processed Spectrum: 1044
    >>>>>Begin Processed Spectral Data<<<<<
    199.08  0.0000E00
    199.89  0.0000E00
    200.70  0.0000E00
    ...

    q.raw.file <-
      system.file("extdata", "spectrum.SSIrrad",
                  package = "photobiologyInOut", mustWork = TRUE)
    autoplot(read_oo_ssirrad(file = q.raw.file))

### Array spectrometers from Wasatch Photonics

Enlighten can save spectral data in a variety of file formats. The function `read_wasatch_csv()` reads CSV files with spectral data in columns. It is designed so that it can read any variation of this file format. In Enlighten it is possible to select which columns are included in the file so their number can vary. However, the header is rich in information and this allows in many but not all cases to guess based on the “Technique” used in Enlighten the type of data being imported. This means that in some cases the user needs to pass an argument to parameter `s.qty`. This is also the case when the column to be extracted is not that with heading “Processed” in the CSV file. The metadata is as for Ocean Insight/Ocean Optics spectrometers copied to attributes in the returned object.

Format of the header is simple and rather easy to parse. The header of the file and the first few lines look like this:

    ENLIGHTEN Version,2.2.7
    Measurement ID,20211003-134004-612972-WP-00591
    Serial Number,WP-00591
    Model,WP-UV-VIS-C-S-25
    Label,13:40:04 WP-00591
    Declared Match,
    Declared Score,0
    Scan Averaging,1
    Boxcar,0
    Technique,Scope
    Baseline Correction Algo,
    ROI Pixel Start,0
    ROI Pixel End,1023
    Slit Width,25
    Vignetted,False
    Interpolated,False
    Raman Intensity Corrected,False
    Deconvolved,False
    Integration Time,60
    Timestamp,2021-10-03 13:40:04.612972
    Note,Dark substracted
    Temperature,-15.327480412352088
    CCD C0,247.9385986328125
    CCD C1,0.5131465792655945
    CCD C2,-0.00012270470324438065
    CCD C3,7.728250039917839e-08
    CCD Offset,0
    CCD Gain,1.9
    Laser Wavelength,0.0
    Laser Enable,False
    Laser Power,100
    Laser Temperature,0
    Pixel Count,1024

    Pixel,Wavelength,Processed
    0,247.94,175.00
    1,248.45,178.00

    file.name <-
        system.file("extdata", "enlighten-wasatch-scope.csv",
                    package = "photobiologyInOut", mustWork = TRUE)

    wasatch.raw.spct <-
        read_wasatch_csv(file = file.name, extra.cols = "drop")

    summary(wasatch.raw.spct)

    ## Summary of raw_spct [1,024 x 2] object: wasatch.raw.spct
    ## Wavelength range 247.94 to 709.67 nm, step 0.43 to 0.52 nm
    ## Label: File: enlighten-wasatch-scope.csv
    ## Measured on 2021-10-03 13:40:04 UTC
    ## Data acquired with 'WP-UV-VIS-C-S-25' s.n. WP-00591
    ## grating 'default', slit '25'
    ## integ. time (s): 0.06
    ## total time (s): 0.06
    ## counts @ peak (% of max): NA
    ##     w.length         counts
    ##  Min.   :247.9   Min.   :  173
    ##  1st Qu.:372.4   1st Qu.: 1604
    ##  Median :487.6   Median :27621
    ##  Mean   :484.8   Mean   :25832
    ##  3rd Qu.:598.8   3rd Qu.:43249
    ##  Max.   :709.7   Max.   :61251

    autoplot(wasatch.raw.spct)

### Modular spectrometers from Avantes

Avantes’ two column .csv files can also be imported.

    ava.raw.file <-
      system.file("extdata", "spectrum-avaspec.csv",
                  package = "photobiologyInOut", mustWork = TRUE)
    autoplot(read_avaspec_csv(file = ava.raw.file),
         range = c(280, 900), unit.out = "photon")

### Scanning spectrometer from Macam

Macam’s single column DTA files can also be imported.

The first few lines of the file look like this with all data in a single column with alternate rows for wavelengths (in nm) and irradiances, and a very terse header:

    @19/5/1997
    @17:44:58
    #No Title
     2.5000000000E+02
     0.0000000000E+00
     2.5100000000E+02
     0.0000000000E+00
     2.5200000000E+02
     0.0000000000E+00
    ...

    macam.raw.file <-
      system.file("extdata", "spectrum.DTA",
                  package = "photobiologyInOut", mustWork = TRUE)
    autoplot(read_macam_dta(file = macam.raw.file))

### LI-180 array dector spectrometer from LI-COR

The LI-COR LI-180 is a portable, self-contained instrument, with relatively low spectral resolution. This instrument measures spectral irradiance in the range 380 nm to 780 nm. There is only one hardware configuration but firmware updates have been released.

This instrument can save the spectral data in different formats. The “XXX” format includes a rather long header, followed by spectral data, then followed by a file footer. The long header can contain various summaries computed from the spectrum, as well as date and time, and information on the instrument.

The top of the file we will read looks like this:

    Model Name  LI-180
    Serial Number   A18M0157
    Time    2021/03/02_09:24:26
    PPFD    129.879440
    PFD 171.178452
    PFD-UV  2.336031
    PFD-B   33.417435
    PFD-G   46.992531
    PFD-R   49.469433
    PFD-FR  38.963074
    Custom1(655~665nm)  5.057602
    Custom2(725~735nm)  4.957998
    Custom3(650~670nm)  10.137251
    Custom4(720~740nm)  9.965228
    UV% 1.364676
    B%  19.521980
    G%  27.452354
    R%  28.899324
    FR% 22.761662
    Custom1%    2.954579
    Custom2%    2.896391
    Custom3%    5.922037
    Custom4%    5.821544
    R:B 1.480348
    R:FR    1.269649
    R:G 1.052708
    B:G 0.711122
    UV:B    0.069905
    UV:FR   0.059955
    B:G:R   0.000000
    B:R:FR  0.000000
    UV:B:G:R:FR 0.000000
    Ratio1(Custom1:Custom2) 1.020090
    Ratio2(Custom3:Custom4) 1.017262
    Ratio3  0.000000
    Ratio4  0.000000
    LambdaP 495.000000
    LambdaPV    106.211479
    LambdaD 554.000000
    LUX 7366.741699
    IRR 35.163498
    fc  684.641418
    I-Time  92.000000
    380nm   27.503876
    381nm   27.968113
    382nm   28.452974
    383nm   28.665405
    ...

And the footer containing chromaticity data:

    ...
    778nm   82.020149
    779nm   82.078781
    780nm   82.042412
    CCT 5465.000000
    Duv 0.004667
    x   0.333318
    y   0.351069
    u'  0.203671
    v'  0.482666
    deltax  0.000063
    deltay  0.009303
    deltau' -0.003491
    deltav' 0.004648
    Purity  5.337072
    CRI 98.779282
    R1  98.665459
    R2  99.470703
    R3  98.974945
    R4  98.248100
    R5  98.830368
    R6  99.473335
    R7  98.902351
    R8  97.668961
    R9  94.101921
    R10 99.232346
    R11 98.246521
    R12 99.336227
    R13 98.899147
    R14 99.324699
    R15 97.831963

Function `read_licor_espd()` will automatically extract the spectral data, date and time, and serial number.

    licor_espd.file <-
      system.file("extdata", "LI-180-irradiance.txt",
                  package = "photobiologyInOut", mustWork = TRUE)
    li180.spct <- read_li180_txt(file = licor_espd.file)

In all cases as much information as possible is decoded, and the data file headers are preserved as comments in the source.spct objects.

    li180.spct

    ## Object: source_spct [401 x 2]
    ## Wavelength range 380 to 780 nm, step 1 nm
    ## Label: File: LI-180-irradiance.txt
    ## Measured on 2021-03-02 09:24:26 UTC
    ## Time unit 1s
    ##
    ## # A tibble: 401 x 2
    ##   w.length s.e.irrad
    ##      <dbl>     <dbl>
    ## 1      380    0.0275
    ## 2      381    0.0280
    ## 3      382    0.0285
    ## # ... with 398 more rows

    cat(comment(li180.spct))

    ## LICOR LI-180 file 'LI-180-irradiance.txt' imported on 2021-10-10 21:01:08 UTC
    ## Model Name   LI-180
    ## Serial Number    A18M0157
    ## Time 2021/03/02_09:24:26
    ## PPFD 129.879440
    ## PFD  171.178452
    ## PFD-UV   2.336031
    ## PFD-B    33.417435
    ## PFD-G    46.992531
    ## PFD-R    49.469433
    ## PFD-FR   38.963074
    ## Custom1(655~665nm)   5.057602
    ## Custom2(725~735nm)   4.957998
    ## Custom3(650~670nm)   10.137251
    ## Custom4(720~740nm)   9.965228
    ## UV%  1.364676
    ## B%   19.521980
    ## G%   27.452354
    ## R%   28.899324
    ## FR%  22.761662
    ## Custom1% 2.954579
    ## Custom2% 2.896391
    ## Custom3% 5.922037
    ## Custom4% 5.821544
    ## R:B  1.480348
    ## R:FR 1.269649
    ## R:G  1.052708
    ## B:G  0.711122
    ## UV:B 0.069905
    ## UV:FR    0.059955
    ## B:G:R    0.000000
    ## B:R:FR   0.000000
    ## UV:B:G:R:FR  0.000000
    ## Ratio1(Custom1:Custom2)  1.020090
    ## Ratio2(Custom3:Custom4)  1.017262
    ## Ratio3   0.000000
    ## Ratio4   0.000000
    ## LambdaP  495.000000
    ## LambdaPV 106.211479
    ## LambdaD  554.000000
    ## LUX  7366.741699
    ## IRR  35.163498
    ## fc   684.641418
    ## I-Time   92.000000

    getInstrDesc(li180.spct)

    ## Data acquired with 'LI-180' s.n. A18M0157
    ## grating 'NA', slit 'NA'

    getInstrSettings(li180.spct)

    ## integ. time (s): 0.092
    ## total time (s): NA
    ## counts @ peak (% of max): NA

    autoplot(li180.spct, unit.out = "photon")

### LI-1800 scanning spectrometer from LI-COR

The LI-COR LI-1800 was developed in the early 1980’s and remained available for many years. Some units are still in use although the technology has been superceeded. If re-calibrated these instruments are still useful.

Spectral photon irradiance output files generated by LI-COR’s PC1800 program for the LI-1800 spectroradiometer. These files have a relatively detailed header, but it lacks year information. Files can contain either energy or photon based spectral irradiances, and this is signalled in the header. In this example photon (= quantum) spectral irradiance is returned. The first few lines of the file look like this:

    "FILE:FL2"
    "REM: TLD 36W/865       (QNTM)"
    "LIMS: 300- 900NM"
    "INT:  1NM"
    "DATE:08/23 16:32"
    "MIN:  300NM  1.518E-04"
    "MAX:  546NM  7.491E-01"
     300  1.518E-04
     301  3.355E-04
     302  2.197E-04
     303  3.240E-04
    ...

Function `read_licor_prn` will automatically detect whether the data is energy or photon based.

    licor.file <-
      system.file("extdata", "spectrum.PRN",
                  package = "photobiologyInOut", mustWork = TRUE)
    licor.spct <- read_licor_prn(file = licor.file)

In all cases as much information as possible is decoded, and the data file headers are preserved as comments in the source.spct objects.

    licor.spct

    ## Object: source_spct [601 x 2]
    ## Wavelength range 300 to 900 nm, step 1 nm
    ## Label: File: spectrum.PRN
    ## Measured on 0000-08-23 16:32:00 UTC
    ## Time unit 1s
    ##
    ## # A tibble: 601 x 2
    ##   w.length s.q.irrad
    ##      <dbl>     <dbl>
    ## 1      300  1.52e-10
    ## 2      301  3.36e-10
    ## 3      302  2.20e-10
    ## # ... with 598 more rows

    cat(comment(licor.spct))

    ## LICOR LI-1800 file 'spectrum.PRN' imported on 2021-10-10 21:01:09 UTC
    ## "FILE:FL2"
    ## "REM: TLD 36W/865       (QNTM)"
    ## "LIMS: 300- 900NM"
    ## "INT:  1NM"
    ## "DATE:08/23 16:32"
    ## "MIN:  300NM  1.518E-04"
    ## "MAX:  546NM  7.491E-01"

    autoplot(licor.spct, unit.out = "photon")

It is also possible to use the same function to import reflectance, and transmittance spectra acquired by the LI-1800.

And a spectral reflectance output file generated by LI-COR’s PC1800 program for the LI-1800 spectroradiometer is used next.

The first few lines of the file look like this:

    "FILE:RGD1"
    "REM: REFL GREEN AD 1 "
    "LIMS: 350- 800NM"
    "INT:  2NM"
    "DATE:05/30 13:50"
    "MIN:  358NM  4.628E-02"
    "MAX:  776NM  4.693E-01"
     350  5.135E-02
     352  4.713E-02
     354  5.324E-02
     356  4.740E-02
    ...

Function `read_licor_prn` cannot automatically detect the spectral quantity in the file, and when the irradiance default is not correct, users need to override it with an explicit argument for parameter `s.qty`.

    licor.file <-
      system.file("extdata", "reflectance.PRN",
                  package = "photobiologyInOut", mustWork = TRUE)
    licor.spct <- read_licor_prn(file = licor.file, s.qty = "Rfr")

In all cases as much information as possible is decoded, and the data file headers are preserved as comments in the source.spct objects.

    licor.spct

    ## Object: reflector_spct [226 x 2]
    ## Wavelength range 350 to 800 nm, step 2 nm
    ## Label: File: reflectance.PRN
    ## Measured on 0000-05-30 13:50:00 UTC
    ## Reflectance of type 'total'
    ##
    ## # A tibble: 226 x 2
    ##   w.length    Rfr
    ##      <dbl>  <dbl>
    ## 1      350 0.0514
    ## 2      352 0.0471
    ## 3      354 0.0532
    ## # ... with 223 more rows

    cat(comment(licor.spct))

    ## LICOR LI-1800 file 'reflectance.PRN' imported on 2021-10-10 21:01:10 UTC
    ## "FILE:RGD1"
    ## "REM: REFL GREEN AD 1 "
    ## "LIMS: 350- 800NM"
    ## "INT:  2NM"
    ## "DATE:05/30 13:50"
    ## "MIN:  358NM  4.628E-02"
    ## "MAX:  776NM  4.693E-01"

    autoplot(licor.spct)

## Data from loggers

### Campbell Scientific

Campbell Scientific is a well know supplier of data loggers for commercial and research applications. Function `read_csi_dat()` defined in this package has been tested with a recent datalogger model, the CR6, and using recent versions of programs PC400 and PC200W to download the data. The currently used format of .DAT files is easy to decode and our function can automatically detect the number and type of columns and the number of rows.

    cs.day.file <-
      system.file("extdata", "cr6-day.dat",
                  package = "photobiologyInOut", mustWork = TRUE)

Executing the statement below displays the 10 top lines of the DAT file as is, one character string per line.

    # not run
    read_lines(yoctopuce_hour.file, n_max = 10)

    day.dat <- read_csi_dat(file = cs.day.file)
    day.dat

    ## # A tibble: 2 x 33
    ##   TIMESTAMP           RECORD PAR_Den_Avg PAR_BF_tot_Avg
    ##   <dttm>               <dbl>       <dbl>          <dbl>
    ## 1 2016-07-27 00:00:00      0        20.9           401.
    ## 2 2016-07-28 00:00:00      1       526.            592.
    ## # ... with 29 more variables: PAR_BF_diff_Avg <dbl>,
    ## #   PAR_Den_Min <dbl>, PAR_Den_TMn <dttm>,
    ## #   PAR_Den_Max <dbl>, PAR_Den_TMx <dttm>,
    ## #   PAR_BF_tot_Min <dbl>, PAR_BF_tot_TMn <dttm>,
    ## #   PAR_BF_tot_Max <dbl>, PAR_BF_tot_TMx <dttm>,
    ## #   PAR_BF_diff_Min <dbl>, PAR_BF_diff_TMn <dttm>,
    ## #   PAR_BF_diff_Max <dbl>, PAR_BF_diff_TMx <dttm>, ...

All information is preserved in the returned `tibble::tibble` object, which is derived from `data.frame`.

    cs_hour.file <-
      system.file("extdata", "cr6-hour.dat",
                  package = "photobiologyInOut", mustWork = TRUE)
    hour.dat <- read_csi_dat(file = cs_hour.file)
    ggplot(hour.dat, aes(TIMESTAMP, PAR_Den_Avg)) + geom_line()

### YoctoPuce

Yocto Puce Sarl sells numerous different USB modules. Those capable of data input can log the data to memory and these data can be downloaded as a CSV file. These files can be easily read into R using base functions but function `read_yoctopuce_csv()` makes this even a bit simpler.

    yoctopuce_hour.file <-
      system.file("extdata", "yoctopuce-data.csv",
                  package = "photobiologyInOut", mustWork = TRUE)

Executing the statement below displays the 10 top lines of the CSV file as is, one character string per line.

    # not run
    read_lines(yoctopuce_hour.file, n_max = 10)

Here we import and plot the data.

    hour.dat <- read_yoctopuce_csv(file = yoctopuce_hour.file)
    ggplot(hour.dat, aes(ISO.time, temperature.avg)) + geom_line()

## Output from simulation models

Functions for importing simulated spectral data.

R function | Simulation model | Version | `class` of value
--- | --- | --- | ---
read_tuv_usrout() | TUV (S. Madronich) | version 5.0 | source_spct
read_tuv_usrout2mspct() | TUV (S. Madronich) | version 5.0 | source_mspct
read_qtuv_txt() | TUV (S. Madronich) | version 5.2 | source_spct
read_uvspec_disort() | libRadtran | irradiance | source_spct
read_uvspec_vesa() | (T. & V. Kotilainen) | irradiance | source_spct
read_fmi_cum() | (A. Lindfors) | daily cumulated | source_spct
read_m_fmi_cum() | (A. Lindfors) | daily cumulated | source_mspct

### TUV

The output from the TUV model can be imported either by editing it before import, or by making a simple edit to the output routine of TUV. This function is known to work with TUV version 5.0 output. The output from TUV can contain a variable number of spectra in ‘’parallel’’ columns, which are melted into a single column, with a factor with letters as levels, a numeric variable with the zenith angle and a POSIXct column with times. A date needs to be always supplied as the output file from TUV has only time of day information.

    tuv.file <-
      system.file("extdata", "usrout.txt",
                  package = "photobiologyInOut", mustWork = TRUE)
    tuv.spct <- read_tuv_usrout(file = tuv.file,
                                date = ymd("2014-03-21"))
    summary(subset(tuv.spct, spct.idx == "A"))

    ## Summary of source_spct [482 x 5] object: anonymous
    ## containing  8  spectra in long form
    ## Wavelength range 280.5 to 761.5 nm, step 1 nm
    ## Label: TUV spectral simulation File: usrout.txt
    ## Measured on 2014-03-21 12:00:00 UTC
    ## Measured on 2014-03-21 13:00:00 UTC
    ## Measured on 2014-03-21 14:00:00 UTC
    ## Measured on 2014-03-21 15:00:00 UTC
    ## Measured on 2014-03-21 16:00:00 UTC
    ## Measured on 2014-03-21 17:00:00 UTC
    ## Measured on 2014-03-21 18:00:00 UTC
    ## Measured on 2014-03-21 19:00:00 UTC
    ## Time unit 1s
    ##
    ##     w.length       spct.idx           s.e.irrad
    ##  Min.   :280.5   Length:482         Min.   :0.000
    ##  1st Qu.:400.8   Class :character   1st Qu.:1.216
    ##  Median :521.0   Mode  :character   Median :1.483
    ##  Mean   :521.0                      Mean   :1.322
    ##  3rd Qu.:641.2                      3rd Qu.:1.680
    ##  Max.   :761.5                      Max.   :1.947
    ##      angle            date
    ##  Min.   :1.829   Min.   :2014-03-21 12:00:00
    ##  1st Qu.:1.829   1st Qu.:2014-03-21 12:00:00
    ##  Median :1.829   Median :2014-03-21 12:00:00
    ##  Mean   :1.829   Mean   :2014-03-21 12:00:00
    ##  3rd Qu.:1.829   3rd Qu.:2014-03-21 12:00:00
    ##  Max.   :1.829   Max.   :2014-03-21 12:00:00

    tuv.spct

    ## Object: source_spct [3,856 x 5]
    ## containing 8 spectra in long form
    ## Wavelength range 280.5 to 761.5 nm, step 1 nm
    ## Label: TUV spectral simulation File: usrout.txt
    ## Measured on 2014-03-21 12:00:00 UTC
    ## Measured on 2014-03-21 13:00:00 UTC
    ## Measured on 2014-03-21 14:00:00 UTC
    ## Measured on 2014-03-21 15:00:00 UTC
    ## Measured on 2014-03-21 16:00:00 UTC
    ## Measured on 2014-03-21 17:00:00 UTC
    ## Measured on 2014-03-21 18:00:00 UTC
    ## Measured on 2014-03-21 19:00:00 UTC
    ## Time unit 1s
    ##
    ## # A tibble: 3,856 x 5
    ##   w.length spct.idx s.e.irrad angle date
    ##      <dbl> <chr>        <dbl> <dbl> <dttm>
    ## 1     280. A         3.04e-15  1.83 2014-03-21 12:00:00
    ## 2     282. A         1.16e-13  1.83 2014-03-21 12:00:00
    ## 3     282. A         1.82e-12  1.83 2014-03-21 12:00:00
    ## # ... with 3,853 more rows

It is possible to extract individual spectra with subset, or as done here plot them in different panels.

    autoplot(tuv.spct, annotations = c("colour.guide")) +
      facet_wrap(~date, ncol = 2)

The output is a single `source_spct} object that can be easily converted into a`source_mspct} object containing the individual spectra as members of the collection.

    tuv.mspct <- subset2mspct(tuv.spct)
    tuv.mspct

    ## Object: source_mspct [8 x 1]
    ## --- Member: A ---
    ## Object: source_spct [482 x 4]
    ## Wavelength range 280.5 to 761.5 nm, step 1 nm
    ## Label: TUV spectral simulation File: usrout.txt
    ## Measured on 2014-03-21 12:00:00 UTC
    ## Measured on 2014-03-21 13:00:00 UTC
    ## Measured on 2014-03-21 14:00:00 UTC
    ## Measured on 2014-03-21 15:00:00 UTC
    ## Measured on 2014-03-21 16:00:00 UTC
    ## Measured on 2014-03-21 17:00:00 UTC
    ## Measured on 2014-03-21 18:00:00 UTC
    ## Measured on 2014-03-21 19:00:00 UTC
    ## Time unit 1s
    ##
    ## # A tibble: 482 x 4
    ##   w.length s.e.irrad angle date
    ##      <dbl>     <dbl> <dbl> <dttm>
    ## 1     280.  3.04e-15  1.83 2014-03-21 12:00:00
    ## 2     282.  1.16e-13  1.83 2014-03-21 12:00:00
    ## 3     282.  1.82e-12  1.83 2014-03-21 12:00:00
    ## # ... with 479 more rows
    ## --- Member: B ---
    ## Object: source_spct [482 x 4]
    ## Wavelength range 280.5 to 761.5 nm, step 1 nm
    ## Label: TUV spectral simulation File: usrout.txt
    ## Measured on 2014-03-21 12:00:00 UTC
    ## Measured on 2014-03-21 13:00:00 UTC
    ## Measured on 2014-03-21 14:00:00 UTC
    ## Measured on 2014-03-21 15:00:00 UTC
    ## Measured on 2014-03-21 16:00:00 UTC
    ## Measured on 2014-03-21 17:00:00 UTC
    ## Measured on 2014-03-21 18:00:00 UTC
    ## Measured on 2014-03-21 19:00:00 UTC
    ## Time unit 1s
    ##
    ## # A tibble: 482 x 4
    ##   w.length s.e.irrad angle date
    ##      <dbl>     <dbl> <dbl> <dttm>
    ## 1     280.  1.31e-15  13.2 2014-03-21 13:00:00
    ## 2     282.  5.42e-14  13.2 2014-03-21 13:00:00
    ## 3     282.  9.04e-13  13.2 2014-03-21 13:00:00
    ## # ... with 479 more rows
    ## --- Member: C ---
    ## Object: source_spct [482 x 4]
    ## Wavelength range 280.5 to 761.5 nm, step 1 nm
    ## Label: TUV spectral simulation File: usrout.txt
    ## Measured on 2014-03-21 12:00:00 UTC
    ## Measured on 2014-03-21 13:00:00 UTC
    ## Measured on 2014-03-21 14:00:00 UTC
    ## Measured on 2014-03-21 15:00:00 UTC
    ## Measured on 2014-03-21 16:00:00 UTC
    ## Measured on 2014-03-21 17:00:00 UTC
    ## Measured on 2014-03-21 18:00:00 UTC
    ## Measured on 2014-03-21 19:00:00 UTC
    ## Time unit 1s
    ##
    ## # A tibble: 482 x 4
    ##   w.length s.e.irrad angle date
    ##      <dbl>     <dbl> <dbl> <dttm>
    ## 1     280.  4.52e-17  28.2 2014-03-21 14:00:00
    ## 2     282.  2.51e-15  28.2 2014-03-21 14:00:00
    ## 3     282.  5.41e-14  28.2 2014-03-21 14:00:00
    ## # ... with 479 more rows
    ## --- Member: D ---
    ## Object: source_spct [482 x 4]
    ## Wavelength range 280.5 to 761.5 nm, step 1 nm
    ## Label: TUV spectral simulation File: usrout.txt
    ## Measured on 2014-03-21 12:00:00 UTC
    ## Measured on 2014-03-21 13:00:00 UTC
    ## Measured on 2014-03-21 14:00:00 UTC
    ## Measured on 2014-03-21 15:00:00 UTC
    ## Measured on 2014-03-21 16:00:00 UTC
    ## Measured on 2014-03-21 17:00:00 UTC
    ## Measured on 2014-03-21 18:00:00 UTC
    ## Measured on 2014-03-21 19:00:00 UTC
    ## Time unit 1s
    ##
    ## # A tibble: 482 x 4
    ##   w.length s.e.irrad angle date
    ##      <dbl>     <dbl> <dbl> <dttm>
    ## 1     280.  3.08e-20  43.2 2014-03-21 15:00:00
    ## 2     282.  3.27e-18  43.2 2014-03-21 15:00:00
    ## 3     282.  1.23e-16  43.2 2014-03-21 15:00:00
    ## # ... with 479 more rows
    ## --- Member: E ---
    ## Object: source_spct [482 x 4]
    ## Wavelength range 280.5 to 761.5 nm, step 1 nm
    ## Label: TUV spectral simulation File: usrout.txt
    ## Measured on 2014-03-21 12:00:00 UTC
    ## Measured on 2014-03-21 13:00:00 UTC
    ## Measured on 2014-03-21 14:00:00 UTC
    ## Measured on 2014-03-21 15:00:00 UTC
    ## Measured on 2014-03-21 16:00:00 UTC
    ## Measured on 2014-03-21 17:00:00 UTC
    ## Measured on 2014-03-21 18:00:00 UTC
    ## Measured on 2014-03-21 19:00:00 UTC
    ## Time unit 1s
    ##
    ## # A tibble: 482 x 4
    ##   w.length s.e.irrad angle date
    ##      <dbl>     <dbl> <dbl> <dttm>
    ## 1     280.  2.25e-26  58.2 2014-03-21 16:00:00
    ## 2     282.  7.75e-24  58.2 2014-03-21 16:00:00
    ## 3     282.  8.15e-22  58.2 2014-03-21 16:00:00
    ## # ... with 479 more rows
    ## --- Member: F ---
    ## Object: source_spct [482 x 4]
    ## Wavelength range 280.5 to 761.5 nm, step 1 nm
    ## Label: TUV spectral simulation File: usrout.txt
    ## Measured on 2014-03-21 12:00:00 UTC
    ## Measured on 2014-03-21 13:00:00 UTC
    ## Measured on 2014-03-21 14:00:00 UTC
    ## Measured on 2014-03-21 15:00:00 UTC
    ## Measured on 2014-03-21 16:00:00 UTC
    ## Measured on 2014-03-21 17:00:00 UTC
    ## Measured on 2014-03-21 18:00:00 UTC
    ## Measured on 2014-03-21 19:00:00 UTC
    ## Time unit 1s
    ##
    ## # A tibble: 482 x 4
    ##   w.length s.e.irrad angle date
    ##      <dbl>     <dbl> <dbl> <dttm>
    ## 1     280.  1.93e-27  73.2 2014-03-21 17:00:00
    ## 2     282.  5.71e-25  73.2 2014-03-21 17:00:00
    ## 3     282.  5.20e-23  73.2 2014-03-21 17:00:00
    ## # ... with 479 more rows
    ## --- Member: G ---
    ## Object: source_spct [482 x 4]
    ## Wavelength range 280.5 to 761.5 nm, step 1 nm
    ## Label: TUV spectral simulation File: usrout.txt
    ## Measured on 2014-03-21 12:00:00 UTC
    ## Measured on 2014-03-21 13:00:00 UTC
    ## Measured on 2014-03-21 14:00:00 UTC
    ## Measured on 2014-03-21 15:00:00 UTC
    ## Measured on 2014-03-21 16:00:00 UTC
    ## Measured on 2014-03-21 17:00:00 UTC
    ## Measured on 2014-03-21 18:00:00 UTC
    ## Measured on 2014-03-21 19:00:00 UTC
    ## Time unit 1s
    ##
    ## # A tibble: 482 x 4
    ##   w.length s.e.irrad angle date
    ##      <dbl>     <dbl> <dbl> <dttm>
    ## 1     280.  4.72e-28  88.2 2014-03-21 18:00:00
    ## 2     282.  1.38e-25  88.2 2014-03-21 18:00:00
    ## 3     282.  1.25e-23  88.2 2014-03-21 18:00:00
    ## # ... with 479 more rows
    ## --- Member: H ---
    ## Object: source_spct [482 x 4]
    ## Wavelength range 280.5 to 761.5 nm, step 1 nm
    ## Label: TUV spectral simulation File: usrout.txt
    ## Measured on 2014-03-21 12:00:00 UTC
    ## Measured on 2014-03-21 13:00:00 UTC
    ## Measured on 2014-03-21 14:00:00 UTC
    ## Measured on 2014-03-21 15:00:00 UTC
    ## Measured on 2014-03-21 16:00:00 UTC
    ## Measured on 2014-03-21 17:00:00 UTC
    ## Measured on 2014-03-21 18:00:00 UTC
    ## Measured on 2014-03-21 19:00:00 UTC
    ## Time unit 1s
    ##
    ## # A tibble: 482 x 4
    ##   w.length s.e.irrad angle date
    ##      <dbl>     <dbl> <dbl> <dttm>
    ## 1     280.         0  103. 2014-03-21 19:00:00
    ## 2     282.         0  103. 2014-03-21 19:00:00
    ## 3     282.         0  103. 2014-03-21 19:00:00
    ## # ... with 479 more rows
    ##
    ## --- END ---

A file can be directly read into a collection using `read_tuv_usrout2mspct()` which is a simple wrapper.

With the default of `lubridate::today()` date times are ‘mapped’ to the current local date using the time zone of the computer as visible to R.

    tuv_nd.spct <- read_tuv_usrout(file = tuv.file)
    tuv_nd.spct

    ## Object: source_spct [3,856 x 5]
    ## containing 8 spectra in long form
    ## Wavelength range 280.5 to 761.5 nm, step 1 nm
    ## Label: TUV spectral simulation File: usrout.txt
    ## Measured on 2021-10-10 12:00:00 UTC
    ## Measured on 2021-10-10 13:00:00 UTC
    ## Measured on 2021-10-10 14:00:00 UTC
    ## Measured on 2021-10-10 15:00:00 UTC
    ## Measured on 2021-10-10 16:00:00 UTC
    ## Measured on 2021-10-10 17:00:00 UTC
    ## Measured on 2021-10-10 18:00:00 UTC
    ## Measured on 2021-10-10 19:00:00 UTC
    ## Time unit 1s
    ##
    ## # A tibble: 3,856 x 5
    ##   w.length spct.idx s.e.irrad angle date
    ##      <dbl> <chr>        <dbl> <dbl> <dttm>
    ## 1     280. A         3.04e-15  1.83 2021-10-10 12:00:00
    ## 2     282. A         1.16e-13  1.83 2021-10-10 12:00:00
    ## 3     282. A         1.82e-12  1.83 2021-10-10 12:00:00
    ## # ... with 3,853 more rows

### Quick TUV calculator

The files output by the online calculator based on the TUV model, contain at most one spectrum, and arguments to only some parameters can be set by users. However, it is convenient to use when we only need a few simple simulations.

Function `read_qtuv_txt()` can extract spectra and the corresponding metadata from these files. These files do contain date time information and geolocation data when they are supplied as arguments to the calculator interface, otherwise only zenith angle is available.

    qtuv.file <-
      system.file("extdata", "qtuv.txt",
                  package = "photobiologyInOut", mustWork = TRUE)
    qtuv.spct <- read_qtuv_txt(file = qtuv.file)
    summary(qtuv.spct)

    ## Summary of source_spct [140 x 7] object: qtuv.spct
    ## Wavelength range 280.5 to 419.5 nm, step 1 nm
    ## Label: Quick TUV spectral simulation File: qtuv.txt
    ## Measured on 2015-06-30 UTC
    ## Time unit 1s
    ##
    ##     w.length       s.e.irrad      s.e.irrad.dir
    ##  Min.   :280.5   Min.   :0.0000   Min.   :0.00000
    ##  1st Qu.:315.2   1st Qu.:0.1930   1st Qu.:0.05189
    ##  Median :350.0   Median :0.4600   Median :0.16540
    ##  Mean   :350.0   Mean   :0.4439   Mean   :0.18938
    ##  3rd Qu.:384.8   3rd Qu.:0.6097   3rd Qu.:0.26338
    ##  Max.   :419.5   Max.   :1.0390   Max.   :0.53710
    ##  s.e.irrad.diff.down s.e.irrad.diff.up     angle
    ##  Min.   :0.0000      Min.   :0.00000   Min.   :45
    ##  1st Qu.:0.1387      1st Qu.:0.01930   1st Qu.:45
    ##  Median :0.2900      Median :0.04600   Median :45
    ##  Mean   :0.2545      Mean   :0.04439   Mean   :45
    ##  3rd Qu.:0.3467      3rd Qu.:0.06097   3rd Qu.:45
    ##  Max.   :0.5109      Max.   :0.10390   Max.   :45
    ##       date
    ##  Min.   :2015-06-30
    ##  1st Qu.:2015-06-30
    ##  Median :2015-06-30
    ##  Mean   :2015-06-30
    ##  3rd Qu.:2015-06-30
    ##  Max.   :2015-06-30

    qtuv.spct

    ## Object: source_spct [140 x 7]
    ## Wavelength range 280.5 to 419.5 nm, step 1 nm
    ## Label: Quick TUV spectral simulation File: qtuv.txt
    ## Measured on 2015-06-30 UTC
    ## Time unit 1s
    ##
    ## # A tibble: 140 x 7
    ##   w.length s.e.irrad s.e.irrad.dir s.e.irrad.diff.~
    ##      <dbl>     <dbl>         <dbl>            <dbl>
    ## 1     280.  1.34e-14      3.89e-15         9.49e-15
    ## 2     282.  4.33e-13      1.24e-13         3.09e-13
    ## 3     282.  5.86e-12      1.66e-12         4.2 e-12
    ## # ... with 137 more rows, and 3 more variables:
    ## #   s.e.irrad.diff.up <dbl>, angle <dbl>, date <dttm>

### libRadtran

By default libRadtran’s uvspec writes only spectral irradiances to a text file as output. This is different from ‘TUV’ which by default includes an extensive header with the parameter settings used for the simulation. It is easy to read this simple output file with R’s functions. However, we provide functions, that simplify reading of the files. The output from uvspec varies depending on its input. The main source of differences is the solver routine used. We will provide a separate function for each solver.

For reading this simple output, no special function is needed. We can use `read.table` from base R. Here we read a file with two columns with wavelengths and global spectral energy irradiance (named “eglo” in libRadtran) in \(mW\,m^{-2}\,nm^{-1}\). The file was created with one of the ‘uvspec’ examples included with libRadtran, but reducing the output to two columns.

The first few lines of the file look like this:

      250.000  0.000000e+00
      251.000  0.000000e+00
      252.000  0.000000e+00
      253.000  0.000000e+00
    ...

    uvspec.2col.file <-
      system.file("extdata", "uvspec-plain-2col.dat",
                  package = "photobiologyInOut", mustWork = TRUE)
    lrt.df <- read.table(file = uvspec.2col.file,
                         col.names = c("w.length", "s.e.irrad"))
    uvspec.01.spct <- source_spct(w.length = lrt.df$w.length,
                                   s.e.irrad = lrt.df$s.e.irrad * 1e-3)
    summary(uvspec.01.spct)

    ## Summary of source_spct [3,751 x 2] object: uvspec.01.spct
    ## Wavelength range 250 to 4000 nm, step 1 nm
    ## Time unit 1s
    ##
    ##     w.length      s.e.irrad
    ##  Min.   : 250   Min.   :0.000000
    ##  1st Qu.:1188   1st Qu.:0.003808
    ##  Median :2125   Median :0.023999
    ##  Mean   :2125   Mean   :0.244545
    ##  3rd Qu.:3062   3rd Qu.:0.264351
    ##  Max.   :4000   Max.   :1.744596

    cat(comment(uvspec.01.spct))
    autoplot(uvspec.01.spct, range = c(250, 2500), unit.out = "photon")

An example using solver `disort` and our function read_uvspec_disort() follows.

The first few lines of the file look like this:

      290.000  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00
      291.000  1.046525e-11  4.800521e-06  1.674293e-07  2.374267e-12  5.342789e-07  2.664720e-08
      292.000  5.888299e-10  2.813865e-05  9.814190e-07  1.335888e-10  3.162808e-06  1.561977e-07
      293.000  4.383296e-09  5.764524e-05  2.010660e-06  9.944455e-10  6.522616e-06  3.200065e-07
    ...

    uvspec.disort.file <-
      system.file("extdata", "uvspec-disort.dat",
                  package = "photobiologyInOut", mustWork = TRUE)
    uvspec.02.spct <- read_uvspec_disort(uvspec.disort.file)

    ## Warning: 611 parsing failures.
    ## row col  expected    actual                                                                                                          file
    ##   1  -- 7 columns 8 columns 'C:/Users/Aphalo/AppData/Local/Temp/RtmpK4JXiQ/Rinst30b82209260b/photobiologyInOut/extdata/uvspec-disort.dat'
    ##   2  -- 7 columns 8 columns 'C:/Users/Aphalo/AppData/Local/Temp/RtmpK4JXiQ/Rinst30b82209260b/photobiologyInOut/extdata/uvspec-disort.dat'
    ##   3  -- 7 columns 8 columns 'C:/Users/Aphalo/AppData/Local/Temp/RtmpK4JXiQ/Rinst30b82209260b/photobiologyInOut/extdata/uvspec-disort.dat'
    ##   4  -- 7 columns 8 columns 'C:/Users/Aphalo/AppData/Local/Temp/RtmpK4JXiQ/Rinst30b82209260b/photobiologyInOut/extdata/uvspec-disort.dat'
    ##   5  -- 7 columns 8 columns 'C:/Users/Aphalo/AppData/Local/Temp/RtmpK4JXiQ/Rinst30b82209260b/photobiologyInOut/extdata/uvspec-disort.dat'
    ## ... ... ......... ......... .............................................................................................................
    ## See problems(...) for more details.

    summary(uvspec.02.spct)

    ## Summary of source_spct [611 x 4] object: uvspec.02.spct
    ## Wavelength range 290 to 900 nm, step 1 nm
    ## Label: libRadtran spectral simulation File: uvspec-disort.dat
    ## Time unit 1s
    ##
    ##     w.length     s.e.irrad.dir    s.e.irrad.diff
    ##  Min.   :290.0   Min.   :0.0000   Min.   :0.00000
    ##  1st Qu.:442.5   1st Qu.:0.2693   1st Qu.:0.01647
    ##  Median :595.0   Median :0.3781   Median :0.04204
    ##  Mean   :595.0   Mean   :0.3269   Mean   :0.06485
    ##  3rd Qu.:747.5   3rd Qu.:0.4440   3rd Qu.:0.11234
    ##  Max.   :900.0   Max.   :0.4928   Max.   :0.19479
    ##    s.e.irrad
    ##  Min.   :0.0000
    ##  1st Qu.:0.3237
    ##  Median :0.4236
    ##  Mean   :0.3918
    ##  3rd Qu.:0.5049
    ##  Max.   :0.6058

    cat(comment(uvspec.02.spct))

    ## libRadtran file 'uvspec-disort.dat' imported on 2021-10-10 21:01:14 UTC

    autoplot(uvspec.02.spct, unit.out = "photon")

The data contains also estimates of diffuse and direct spectral irradiance. Here we plot the total energy irradiance with a solid line and the diffuse component with a dashed line.

    ggplot(uvspec.02.spct) +
      geom_line() +
      geom_line(aes(y = s.e.irrad.diff), linetype = "dashed")

The uvspec file used to generate the spectrum read above is:

    data_files_path uvspec_home/data/
    atmosphere_file uvspec_home/data/atmmod/afglms.dat
    source solar uvspec_home/data/solar_flux/kurudz_1.0nm.dat
    rte_solver disort
    mol_abs_param lowtran
    deltam on
    number_of_streams 6
    wavelength 290 900
    day_of_year 287
    altitude 0.012
    albedo_library IGBP
    brdf_rpv_type 5
    mol_modify O3 288 DU
    mol_modify H2O 10 MM
    sza 69.4662
    sur_temperature 273

If we plan to save and reuse the spectral object, it is recommended to append the input file to the comment.

    uvspec.disort.inp.file <-
      system.file("extdata", "uvspec-disort.inp",
                  package = "photobiologyInOut", mustWork = TRUE)
    comment(uvspec.02.spct) <- paste(comment(uvspec.02.spct),
                                     read_file(uvspec.disort.inp.file),
                                     sep = "\n\n")
    cat(comment(uvspec.02.spct))

    ## libRadtran file 'uvspec-disort.dat' imported on 2021-10-10 21:01:14 UTC
    ##
    ## data_files_path uvspec_home/data/
    ## atmosphere_file uvspec_home/data/atmmod/afglms.dat
    ## source solar uvspec_home/data/solar_flux/kurudz_1.0nm.dat
    ## rte_solver disort
    ## mol_abs_param lowtran
    ## deltam on
    ## number_of_streams 6
    ## wavelength 290 900
    ## day_of_year 287
    ## altitude 0.012
    ## albedo_library IGBP
    ## brdf_rpv_type 5
    ## mol_modify O3 288 DU
    ## mol_modify H2O 10 MM
    ## sza 69.4662
    ## sur_temperature 273

We give two additional examples, which will most likely need some adjustment by users, as these are for output from libRadtran post-processed to add additional information. These are included in the package because myself and collaborators use these formats heavily. In fact users could develop shell scripts or Perl scripts using the same output format.

### Output enriched with time and date data

In this case the file to be read is similar as above, but including separate columns for direct and diffuse components of the spectral energy irradiance. In addition two columns, one with date strings in ISO format and one with times have been added. The file instead of containing a single spectrum, contains several spectra in long form.

The first few lines of the file look like this:

    290.000 2015-05-19 11_00_00 0.000000e+00 0.000000e+00
    291.000 2015-05-19 11_00_00 0.000000e+00 0.000000e+00
    292.000 2015-05-19 11_00_00 0.000000e+00 0.000000e+00
    293.000 2015-05-19 11_00_00 1.893645e-05 3.439497e-05
    294.000 2015-05-19 11_00_00 1.648530e-04 2.764368e-04
    ...

A function is included for reading data saved in a text file in this format. It also automatically converts \(mW\,m^{-2}\,nm^{-1}\) into \(W\,m^{-2}\,nm^{-1}\).

    uvspec.multi.file <-
      system.file("extdata", "uvspec-multi.dat",
                  package = "photobiologyInOut", mustWork = TRUE)
    lbr.multi.spct <- read_uvspec_disort_vesa(uvspec.multi.file)
    print(lbr.multi.spct, n = 5)

    ## Object: source_spct [3,055 x 5]
    ## containing 5 spectra in long form
    ## Wavelength range 290 to 900 nm, step 1 nm
    ## Label: libRadtran spectral simulation File: uvspec-multi.dat
    ## Measured on 2015-05-19 11:00:00 UTC
    ## Measured on 2015-05-19 11:01:00 UTC
    ## Measured on 2015-05-19 11:02:00 UTC
    ## Measured on 2015-05-19 11:03:00 UTC
    ## Measured on 2015-05-19 11:04:00 UTC
    ## Time unit 1s
    ##
    ## # A tibble: 3,055 x 5
    ##   datetime            w.length s.e.irrad.dir
    ##   <dttm>                 <dbl>         <dbl>
    ## 1 2015-05-19 11:00:00      290      0
    ## 2 2015-05-19 11:00:00      291      0
    ## 3 2015-05-19 11:00:00      292      0
    ## 4 2015-05-19 11:00:00      293      1.89e-11
    ## 5 2015-05-19 11:00:00      294      1.65e-10
    ## # ... with 3,050 more rows, and 2 more variables:
    ## #   s.e.irrad.diff <dbl>, s.e.irrad <dbl>

### Scripts developed by Anders Lindfors

A model for the simulation of the solar spectrum was developed at the Finnish Meteorological Institute (FMI) by Dr. Anders Lindfors and collaborators and uses functions from ‘libRadtran’ as its engine, but saves some additional metadata to the output file. The main addition is related to the estimation of the effect of clouds.

Functions `read_fmi_cum()` and `read_m_fmi_cum()` can be used to read text files output as daily integrated spectral irradiance. In other words cummulated daily data. Function `read_fmi2mspct()` reads spectral irradiance, extracting multiple sequential spectra from a single file.

The first few lines of the files with cummulated data look like this:

    # date number_of_scans start_scan stop_scan max_time_gap max_sza_gap warnings
    # 20140821 15 3:30:00 17:30:00 60 7.4
    # wavelength exposure(J/m2/nm)
    2900 0.00000000e+00
    2910 2.93132235e-05
    2920 7.23526379e-04
    ...

We can read an individual file into a `source_spct` object while adding some metadata read from the file header. In this case values are for daily global spectral energy exposures rather than irradiances. Wavelengths are expressed in Angstroms instead of nanometres.

    fmi.file <-
      system.file("extdata", "2014-08-21_cum.hel",
                  package = "photobiologyInOut", mustWork = TRUE)
    z.spct <- read_fmi_cum(fmi.file)
    class_spct(z.spct)

    ## [1] "source_spct"  "generic_spct"

    getWhenMeasured(z.spct)

    ## [1] "2014-08-21 UTC"

    z.spct

    ## Object: source_spct [511 x 2]
    ## Wavelength range 290 to 800 nm, step 1 nm
    ## Label: File: 2014-08-21_cum.hel
    ## Measured on 2014-08-21 UTC
    ## Time unit 86400s (~1 days)
    ##
    ## # A tibble: 511 x 2
    ##   w.length s.e.irrad
    ##      <dbl>     <dbl>
    ## 1      290 0
    ## 2      291 0.0000293
    ## 3      292 0.000724
    ## # ... with 508 more rows

With function `read_m_fmi_cum` with an ``m’’ in the name we can read several files each containing a single spectrum. The returned object is a collection of source spectra.

    fmi.files <-
      system.file("extdata", c("2014-08-21_cum.hel", "2014-08-21_cum.hel"),
                  package = "photobiologyInOut", mustWork = TRUE)
    z.mspct <- read_m_fmi_cum(fmi.files)
    class(z.mspct)

    ## [1] "source_mspct"  "generic_mspct" "list"

    getWhenMeasured(z.mspct)

    ## # A tibble: 1 x 2
    ##   spct.idx           when.measured
    ##   <fct>              <dttm>
    ## 1 2014_08_21_cum.hel 2014-08-21 00:00:00

    z.mspct

    ## Object: source_mspct [1 x 1]
    ## --- Member: 2014_08_21_cum.hel ---
    ## Object: source_spct [511 x 2]
    ## Wavelength range 290 to 800 nm, step 1 nm
    ## Label: File: 2014-08-21_cum.hel
    ## Measured on 2014-08-21 UTC
    ## Time unit 86400s (~1 days)
    ##
    ## # A tibble: 511 x 2
    ##   w.length s.e.irrad
    ##      <dbl>     <dbl>
    ## 1      290 0
    ## 2      291 0.0000293
    ## 3      292 0.000724
    ## # ... with 508 more rows
    ##
    ## --- END ---

Above we gave the names of the files explicitly, but as we show here, one can build on-the-fly a list of file names matching some pattern. The example below is not run, as the location of example files may vary. The string `"."` should be replaced with the path to the folder where the files to be read are located.

    fmi.files <- list.files(".", "*cum.hel")
    fmi.files <- paste(".", fmi.files, sep = "")
    z1.mspct <- read_m_fmi_cum(fmi.files)
    class(z1.mspct)
    getWhenMeasured(z1.mspct)
    z1.mspct

One also add a geocode at the time of import (or later).

    # because of Google's query limits call will frequently fail without a key
    # my.geocode <- ggmap::geocode("Kumpula, Helsinki, Finland", source = "google")
    my.geocode <- data.frame(lon = 24.96474, lat = 60.20911)
    z2.mspct <-
      read_m_fmi_cum(fmi.files,
                     geocode = my.geocode)
    class(z2.mspct)

    ## [1] "source_mspct"  "generic_mspct" "list"

    getWhenMeasured(z2.mspct)

    ## # A tibble: 1 x 2
    ##   spct.idx           when.measured
    ##   <fct>              <dttm>
    ## 1 2014_08_21_cum.hel 2014-08-21 00:00:00

    getWhereMeasured(z2.mspct)

    ## # A tibble: 1 x 4
    ##   spct.idx             lon   lat address
    ##   <fct>              <dbl> <dbl> <chr>
    ## 1 2014_08_21_cum.hel  25.0  60.2 <NA>

    z2.mspct

    ## Object: source_mspct [1 x 1]
    ## --- Member: 2014_08_21_cum.hel ---
    ## Object: source_spct [511 x 2]
    ## Wavelength range 290 to 800 nm, step 1 nm
    ## Label: File: 2014-08-21_cum.hel
    ## Measured on 2014-08-21 UTC
    ## Time unit 86400s (~1 days)
    ##
    ## # A tibble: 511 x 2
    ##   w.length s.e.irrad
    ##      <dbl>     <dbl>
    ## 1      290 0
    ## 2      291 0.0000293
    ## 3      292 0.000724
    ## # ... with 508 more rows
    ##
    ## --- END ---

Files with spectral irradiance contain data for multiple spectra stored as text. Each spectrum is delimited at the top by a header line with metadata and at the end by `"end"` in a line by itself.

The first few and last lines for each spectrum look like this:

    # 20130501 3:30:00 3:30:00 82.656
    210.00 2900 0
    210.00 2910 0
    210.00 2920 0
    ...
    210.00 8480 103.73018
    210.00 8490 104.63495
    210.00 8500 90.18384
    end
    ...

The number of spectra, range of wavelengths and the length of each spectrum can vary. Function `read_fmi2mspct()` uses matching to the delimiters to read all the data in all cases.

    fmi.file <-
      system.file("extdata", "2013-05-01.hel",
                  package = "photobiologyInOut", mustWork = TRUE)
    z3.mspct <- read_fmi2mspct(fmi.file)
    class(z3.mspct)[1:2]

    ## [1] "source_mspct"  "generic_mspct"

    getWhenMeasured(z3.mspct[[1]])

    ## [1] "2013-05-01 03:30:00 UTC"

    length(z3.mspct)

    ## [1] 15

    names(z3.mspct)

    ##  [1] "2013-05-01 03:30:00" "2013-05-01 04:30:00"
    ##  [3] "2013-05-01 05:30:00" "2013-05-01 06:30:00"
    ##  [5] "2013-05-01 07:30:00" "2013-05-01 08:30:00"
    ##  [7] "2013-05-01 09:30:00" "2013-05-01 10:30:00"
    ##  [9] "2013-05-01 11:30:00" "2013-05-01 12:30:00"
    ## [11] "2013-05-01 13:30:00" "2013-05-01 14:30:00"
    ## [13] "2013-05-01 15:30:00" "2013-05-01 16:30:00"
    ## [15] "2013-05-01 17:30:00"

    getWhenMeasured(z3.mspct[[1]])

    ## [1] "2013-05-01 03:30:00 UTC"

    getWhatMeasured(z3.mspct[[1]])

    ## [1] "File: 2013-05-01.hel"

## Online public repositories of spectral data

Functions for importing spectral data downloaded from repositories.

R function | Data repository | Version | `class` of value
--- | --- | --- | ---
read_FReD_csv() | Floral Reflectance db. | 2017-03-19 | reflector_spct
read_ASTER_txt() | ASTER spectral lib. | version 2.0 ASCII | reflector_spct

### FReD Floral Reflectance Database

The files downloaded from FReD do not contain a header, but the first column indicates the flower ID.

    157, 300, 0.0627119
    157, 301, 0.0654036
    157, 302, 0.0677941
    157, 303, 0.0670396
    ...

    fred.file <-
      system.file("extdata", "FReDflowerID_157.csv",
                  package = "photobiologyInOut", mustWork = TRUE)
    fred.spct <- read_FReD_csv(file = fred.file,
                               label = "Gazania heterochaeta",
                               geocode = data.frame(lat = -28.8751, lon = 17.2293))

In this case as there is no metadata present in the file, it needs to be supplied by the user.

    fred.spct

    ## Object: reflector_spct [401 x 3]
    ## Wavelength range 300 to 700 nm, step 1 nm
    ## Label: File: FReDflowerID_157.csv
    ## Gazania heterochaeta
    ## Reflectance of type 'total'
    ##
    ## # A tibble: 401 x 3
    ##   flower.id w.length    Rfr
    ##       <dbl>    <dbl>  <dbl>
    ## 1       157      300 0.0627
    ## 2       157      301 0.0654
    ## 3       157      302 0.0678
    ## # ... with 398 more rows

    cat(comment(fred.spct))

    ## FReD file 'FReDflowerID_157.csv' imported on 2021-10-10 21:01:16 UTC

    autoplot(fred.spct)

### ASTER spectral database

The files downloaded from ASTER contain a 25-lines-long header, but at the moment only the first field is decoded, as the whole header copied as a comment.

    Name: Dry grass
    Type:  Vegetation
    Class:  Grasses
    Subclass:  Dry grass
    Particle Size:  Solid
    Sample No.:  drygrass.doc
    Owner:  JHU
    Wavelength Range:  All
    Origin:  The entire spectral range was measured at Johns Hopkins University.

    Description:  Dry grass.  Spectra were assembled from two segments; the
    bidirectional VNIR and SWIR comprising segment one, and the hemispherical
    MWIR and TIR comprising segment two. The VNIR/SWIR spectrum was
    measured in the laboratory at JHU with a GER IRIS Mark IV, using a large piece
    of sod.  The grass was illuminated from directly above and measured at a
    reflectance angle of 60 degrees to avoid viewing the thatch.
    Measurement:  Bidirectional and directional hemispherical reflectance.
    First Column:  X
    Second Column:  Y
    X Units:  Wavelength (micrometers)
    Y Units:  Reflectance (percent)
    First X Value: 0.38049
    Last X Value: 14.011
    Number of X Values: 2559
    Additional Information:  None.

    0.38049 14.249
    0.38299 14.251
    0.38544 14.546
    0.38791 14.694
    ...

    aster.file <-
      system.file("extdata", "drygrass-spectrum.txt",
                  package = "photobiologyInOut", mustWork = TRUE)
    aster.spct <- read_ASTER_txt(file = aster.file)

The label and comment are set from the file header.

    aster.spct

    ## Object: reflector_spct [2,559 x 2]
    ## Wavelength range 380.49 to 14011 nm, step 0.8 to 38 nm
    ## Label: Name: Dry grass
    ## File: drygrass-spectrum.txt
    ## Reflectance of type 'total'
    ##
    ## # A tibble: 2,559 x 2
    ##   w.length   Rfr
    ##      <dbl> <dbl>
    ## 1     380. 0.142
    ## 2     383. 0.143
    ## 3     385. 0.145
    ## # ... with 2,556 more rows

    cat(comment(aster.spct))

    ## ASTER database file 'drygrass-spectrum.txt' imported on 2021-10-10 21:01:16 UTC
    ## Name: Dry grass
    ## Type:  Vegetation
    ## Class:  Grasses
    ## Subclass:  Dry grass
    ## Particle Size:  Solid
    ## Sample No.:  drygrass.doc
    ## Owner:  JHU
    ## Wavelength Range:  All
    ## Origin:  The entire spectral range was measured at Johns Hopkins University.
    ## Description:  Dry grass.  Spectra were assembled from two segments; the
    ## bidirectional VNIR and SWIR comprising segment one, and the hemispherical
    ## MWIR and TIR comprising segment two. The VNIR/SWIR spectrum was
    ## measured in the laboratory at JHU with a GER IRIS Mark IV, using a large piece
    ## of sod.  The grass was illuminated from directly above and measured at a
    ## reflectance angle of 60 degrees to avoid viewing the thatch.
    ## Measurement:  Bidirectional and directional hemispherical reflectance.
    ## First Column:  X
    ## Second Column:  Y
    ## X Units:  Wavelength (micrometers)
    ## Y Units:  Reflectance (percent)
    ## First X Value: 0.38049
    ## Last X Value: 14.011
    ## Number of X Values: 2559
    ## Additional Information:  None.
    ##

    autoplot(aster.spct)

## Other R packages

A general way of exchanging data with other R packages or for use with base R functions is to create a matrix from a collection of spectra with `as.matrix()`, or a collection of spectra from a matrix with one of the `as.xxxx_mspct()` methods such as `as.source_spct`. Such methods are defined in package ‘photobiology’ as well as method `join_mspct()` for conversion of collections of spectra into wide data frames. However, a matrix is only guaranteed to contain numeric data and a `"dim"` attribute, while conversion to a data frame preserves only part of the metadata. These generic conversions cannot be reversed without loss of information. When possible use the package specific functions as they automate much of the recovery and preservation of metadata.

Functions for exchanging data with foreign R packages.

R function | Foreign R package | Function | `class` of value
--- | --- | --- | ---
hyperSpec2spct() | ’hyperSpec’ | import | source_spct
spct2hyperSpec() | ’hyperSpec’ | export | hyperSpec
hyperSpec2mspct() | ’hyperSpec’ | import | source_mspct
mspct2hyperSpec() | ’hyperSpec’ | export | hyperSpec
colorSpec2spct() | ’colorSpec’ | import | source_spct
spct2colorSpec() | ’colorSpec’ | export | colorSpec
colorSpec2mspct() | ’colorSpec’ | import | source_mspct
mspct2colorSpec() | ’colorSpec’ | export | colorSpec
chroma_spct2colorSpec() | ’colorSpec’ | export | colorSpec
rspec2mspct() | ’pavo’ | import | source_mspct

### To ‘hyperSpec’

Can export to `''hyperSpec''` objects only collections of spectra where all members have identical `w.length` vectors, as objects of class `hyperSpec` store a single vector of wavelengths for the whole collection of spectra.

    z2.hspct <- mspct2hyperSpec(z2.mspct, "s.e.irrad")

    ## Warning in .local(.Object, ...): Spectra in data are
    ## overwritten by argument spc.

    class(z2.hspct)

    ## [1] "hyperSpec"
    ## attr(,"package")
    ## [1] "hyperSpec"

    # plot(z2.hspct)

### From ‘hyperSpec’

Can import only data with wavelength in nanometres. Other quantities and units are not supported by the ‘photobiology’ classes for spectral data. See package ‘hyperSpec’ vignette “laser” for details on the data and the conversion of the original wavelength units into nanometres.

    data(laser)
    class(laser)

    ## [1] "hyperSpec"
    ## attr(,"package")
    ## [1] "hyperSpec"

    laser

    ## hyperSpec object
    ##    84 spectra
    ##    3 data columns
    ##    36 data points / spectrum
    ## wavelength: lambda/nm [numeric] 404.5828 404.6181 ... 405.8176
    ## data:  (84 rows x 3 columns)
    ##    1. t: t / s [numeric] 0 2 ... 5722
    ##    2. spc: I / a.u. [matrix, array36] 164.650 179.724 ... 112.086
    ##    3. filename: filename [character] rawdata/laser.txt.gz rawdata/laser.txt.gz ... rawdata/laser.txt.gz

    plot(laser)

We assume here, that the quantity for the spectral emission of the laser is spectral energy irradiance, expressed in \(mW\,m^{-2}\,nm^{-1}\). This is likely to be wrong but for the sake of showing how the conversion takes place is irrelevant. The parameter `multiplier` can be passed a numeric argument to rescale the original data. The default multiplier is 1.

    wl(laser) <- list (
      wl = 1e7 / (1/405e-7 - wl (laser)),
      label = expression (lambda / nm)
    )
    laser

    ## hyperSpec object
    ##    84 spectra
    ##    3 data columns
    ##    36 data points / spectrum
    ## wavelength: lambda/nm [numeric] 411.7467 411.7473 ... 411.7677
    ## data:  (84 rows x 3 columns)
    ##    1. t: t / s [numeric] 0 2 ... 5722
    ##    2. spc: I / a.u. [matrix, array36] 164.650 179.724 ... 112.086
    ##    3. filename: filename [character] rawdata/laser.txt.gz rawdata/laser.txt.gz ... rawdata/laser.txt.gz

    plot(laser)
    laser.mspct <-
      hyperSpec2mspct(laser, "source_spct", "s.e.irrad", multiplier = 1e-3)
    ggplot(laser.mspct[[1]]) +
      geom_line() +
      stat_peaks(geom = "text", vjust = -1, label.fmt = "%.6g nm", color = "red")

### From ‘colorSpec’

    # bug that needs to be fixed
    fluorescent.mspct <- colorSpec2mspct(colorSpec::Fs.5nm)
    print(fluorescent.mspct, n = 3, n.members = 3)

    ## Object: source_mspct [12 x 1]
    ## --- Member: F1 ---
    ## Object: source_spct [81 x 2]
    ## Wavelength range 380 to 780 nm, step 5 nm
    ## Time unit 1s
    ##
    ## # A tibble: 81 x 2
    ##   w.length s.e.irrad
    ##      <dbl>     <dbl>
    ## 1      380      1.87
    ## 2      385      2.36
    ## 3      390      2.94
    ## # ... with 78 more rows
    ## --- Member: F2 ---
    ## Object: source_spct [81 x 2]
    ## Wavelength range 380 to 780 nm, step 5 nm
    ## Time unit 1s
    ##
    ## # A tibble: 81 x 2
    ##   w.length s.e.irrad
    ##      <dbl>     <dbl>
    ## 1      380      1.18
    ## 2      385      1.48
    ## 3      390      1.84
    ## # ... with 78 more rows
    ## --- Member: F3 ---
    ## Object: source_spct [81 x 2]
    ## Wavelength range 380 to 780 nm, step 5 nm
    ## Time unit 1s
    ##
    ## # A tibble: 81 x 2
    ##   w.length s.e.irrad
    ##      <dbl>     <dbl>
    ## 1      380      0.82
    ## 2      385      1.02
    ## 3      390      1.26
    ## # ... with 78 more rows
    ## ..........................
    ## 9 other member spectra not shown
    ##
    ## --- END ---

    colorSpec2mspct(colorSpec::Hoya)

    ## Object: filter_mspct [4 x 1]
    ## --- Member: R-60 ---
    ## Object: filter_spct [46 x 2]
    ## Wavelength range 300 to 750 nm, step 10 nm
    ## Transmittance of type 'total'
    ## Rfr (/1): NA, thickness (mm): NA, attenuation mode: NA.
    ##
    ## # A tibble: 46 x 2
    ##   w.length   Tfr
    ##      <dbl> <dbl>
    ## 1      300     0
    ## 2      310     0
    ## 3      320     0
    ## # ... with 43 more rows
    ## --- Member: G-533 ---
    ## Object: filter_spct [46 x 2]
    ## Wavelength range 300 to 750 nm, step 10 nm
    ## Transmittance of type 'total'
    ## Rfr (/1): NA, thickness (mm): NA, attenuation mode: NA.
    ##
    ## # A tibble: 46 x 2
    ##   w.length   Tfr
    ##      <dbl> <dbl>
    ## 1      300     0
    ## 2      310     0
    ## 3      320     0
    ## # ... with 43 more rows
    ## --- Member: B-440 ---
    ## Object: filter_spct [46 x 2]
    ## Wavelength range 300 to 750 nm, step 10 nm
    ## Transmittance of type 'total'
    ## Rfr (/1): NA, thickness (mm): NA, attenuation mode: NA.
    ##
    ## # A tibble: 46 x 2
    ##   w.length   Tfr
    ##      <dbl> <dbl>
    ## 1      300     0
    ## 2      310     0
    ## 3      320     0
    ## # ... with 43 more rows
    ## --- Member: LB-120 ---
    ## Object: filter_spct [46 x 2]
    ## Wavelength range 300 to 750 nm, step 10 nm
    ## Transmittance of type 'total'
    ## Rfr (/1): NA, thickness (mm): NA, attenuation mode: NA.
    ##
    ## # A tibble: 46 x 2
    ##   w.length     Tfr
    ##      <dbl>   <dbl>
    ## 1      300 0.00003
    ## 2      310 0.0058
    ## 3      320 0.081
    ## # ... with 43 more rows
    ##
    ## --- END ---

    fluorescent.spct <- colorSpec2spct(colorSpec::Fs.5nm)
    autoplot(fluorescent.spct, annotations = "")

    colorSpec2chroma_spct(colorSpec::xyz1931.5nm)

    ## Object: chroma_spct [81 x 4]
    ## Wavelength range 380 to 780 nm, step 5 nm
    ##
    ## # A tibble: 81 x 4
    ##        x      y      z w.length
    ##    <dbl>  <dbl>  <dbl>    <dbl>
    ## 1 0.0014 0      0.0065      380
    ## 2 0.0022 0.0001 0.0105      385
    ## 3 0.0042 0.0001 0.0201      390
    ## # ... with 78 more rows

### To ‘colorSpec’

    sun.cspec <- spct2colorSpec(sun.spct)
    plot(sun.cspec, col = "blue")

    spct2colorSpec(yellow_gel.spct)

    ##
    ## colorSpec object.   The organization is 'vector'.  Object size is 11632 bytes.
    ## the object describes 1 transparent materials, and the quantity is 'transmittance'.
    ## Wavelength range: 190 to 800 nm.  Step size is 1 nm.
    ##
    ## 1 spectra
    ## 611 data points / spectrum
    ##
    ##   Material   Min    Max LambdaMax Integral
    ## 1   spct_1 1e-05 0.9025     768.5 260.3194

    chroma_spct2colorSpec(beesxyzCMF.spct)

    ##
    ## colorSpec object.   The organization is 'matrix'.  Object size is 14560 bytes.
    ## the object describes a responder to light with 3 output channels, and the quantity is 'power->neural'.
    ## Wavelength range: 300 to 700 nm.  Step size is 1 nm.
    ##
    ## 3 spectra
    ## 401 data points / spectrum
    ##
    ##   Channel   Min Max LambdaMax E.response
    ## 1       x 0.006   1       560    135.358
    ## 2       y 0.000   1       435    103.696
    ## 3       z 0.000   1       340     68.365

### From ‘pavo’

In this example we convert an `rspec` object from package ‘pavo’ into a collection of spectra and then we plot it with `ggplot` methods from package `ggspectra' (an extension to`ggplot2’). The data are the spectral reflectance of the plumage from seven different individual birds of the same species, measured in three different body parts.

    data(sicalis)
    class(sicalis)

    ## [1] "rspec"      "data.frame"

    names(sicalis)

    ##  [1] "wl"     "ind1.C" "ind1.T" "ind1.B" "ind2.C"
    ##  [6] "ind2.T" "ind2.B" "ind3.C" "ind3.T" "ind3.B"
    ## [11] "ind4.C" "ind4.T" "ind4.B" "ind5.C" "ind5.T"
    ## [16] "ind5.B" "ind6.C" "ind6.T" "ind6.B" "ind7.C"
    ## [21] "ind7.T" "ind7.B"

We convert the data into a collection of spectra, and calculate summaries for three spectra.

    sicalis.mspct <- rspec2mspct(sicalis, "reflector_spct", "Rpc")
    summary(sicalis.mspct[[1]])

    ## Summary of reflector_spct [401 x 2] object: anonymous
    ## Wavelength range 300 to 700 nm, step 1 nm
    ##
    ##     w.length        Rfr
    ##  Min.   :300   Min.   :0.001798
    ##  1st Qu.:400   1st Qu.:0.008288
    ##  Median :500   Median :0.031709
    ##  Mean   :500   Mean   :0.052848
    ##  3rd Qu.:600   3rd Qu.:0.098775
    ##  Max.   :700   Max.   :0.114807

    summary(sicalis.mspct[[2]])

    ## Summary of reflector_spct [401 x 2] object: anonymous
    ## Wavelength range 300 to 700 nm, step 1 nm
    ##
    ##     w.length        Rfr
    ##  Min.   :300   Min.   :0.006783
    ##  1st Qu.:400   1st Qu.:0.030112
    ##  Median :500   Median :0.096994
    ##  Mean   :500   Mean   :0.105449
    ##  3rd Qu.:600   3rd Qu.:0.179691
    ##  Max.   :700   Max.   :0.183823

    summary(sicalis.mspct[[3]])

    ## Summary of reflector_spct [401 x 2] object: anonymous
    ## Wavelength range 300 to 700 nm, step 1 nm
    ##
    ##     w.length        Rfr
    ##  Min.   :300   Min.   :0.001191
    ##  1st Qu.:400   1st Qu.:0.022293
    ##  Median :500   Median :0.085235
    ##  Mean   :500   Mean   :0.116253
    ##  3rd Qu.:600   3rd Qu.:0.212554
    ##  Max.   :700   Max.   :0.224162

We convert the subset of the collection corresponding to the first individual into a single spectra object for plotting with `ggplot`.

    ggplot(rbindspct(sicalis.mspct[1:3])) +
      aes(linetype = spct.idx) +
      ylim(0,0.3) +
      geom_line()

Here we extract the ``crown’’ data from all individuals and plot these spectra in a single plot.

    print(sicalis.mspct[c(TRUE, FALSE, FALSE)])

    ## Object: reflector_mspct [7 x 1]
    ## --- Member: ind1.C ---
    ## Object: reflector_spct [401 x 2]
    ## Wavelength range 300 to 700 nm, step 1 nm
    ## Reflectance of type 'total'
    ##
    ## # A tibble: 401 x 2
    ##   w.length     Rfr
    ##      <int>   <dbl>
    ## 1      300 0.00759
    ## 2      301 0.00773
    ## 3      302 0.00829
    ## # ... with 398 more rows
    ## --- Member: ind2.C ---
    ## Object: reflector_spct [401 x 2]
    ## Wavelength range 300 to 700 nm, step 1 nm
    ## Reflectance of type 'total'
    ##
    ## # A tibble: 401 x 2
    ##   w.length     Rfr
    ##      <int>   <dbl>
    ## 1      300 0.00297
    ## 2      301 0.00233
    ## 3      302 0.00323
    ## # ... with 398 more rows
    ## --- Member: ind3.C ---
    ## Object: reflector_spct [401 x 2]
    ## Wavelength range 300 to 700 nm, step 1 nm
    ## Reflectance of type 'total'
    ##
    ## # A tibble: 401 x 2
    ##   w.length      Rfr
    ##      <int>    <dbl>
    ## 1      300 0.000595
    ## 2      301 0
    ## 3      302 0.00119
    ## # ... with 398 more rows
    ## --- Member: ind4.C ---
    ## Object: reflector_spct [401 x 2]
    ## Wavelength range 300 to 700 nm, step 1 nm
    ## Reflectance of type 'total'
    ##
    ## # A tibble: 401 x 2
    ##   w.length     Rfr
    ##      <int>   <dbl>
    ## 1      300 0.00375
    ## 2      301 0.00347
    ## 3      302 0.00413
    ## # ... with 398 more rows
    ## --- Member: ind5.C ---
    ## Object: reflector_spct [401 x 2]
    ## Wavelength range 300 to 700 nm, step 1 nm
    ## Reflectance of type 'total'
    ##
    ## # A tibble: 401 x 2
    ##   w.length     Rfr
    ##      <int>   <dbl>
    ## 1      300 0.00423
    ## 2      301 0.00536
    ## 3      302 0.00655
    ## # ... with 398 more rows
    ## --- Member: ind6.C ---
    ## Object: reflector_spct [401 x 2]
    ## Wavelength range 300 to 700 nm, step 1 nm
    ## Reflectance of type 'total'
    ##
    ## # A tibble: 401 x 2
    ##   w.length      Rfr
    ##      <int>    <dbl>
    ## 1      300 0.000633
    ## 2      301 0.000614
    ## 3      302 0.000193
    ## # ... with 398 more rows
    ## --- Member: ind7.C ---
    ## Object: reflector_spct [401 x 2]
    ## Wavelength range 300 to 700 nm, step 1 nm
    ## Reflectance of type 'total'
    ##
    ## # A tibble: 401 x 2
    ##   w.length     Rfr
    ##      <int>   <dbl>
    ## 1      300 0.00168
    ## 2      301 0.00104
    ## 3      302 0.00170
    ## # ... with 398 more rows
    ##
    ## --- END ---

    ggplot(rbindspct(sicalis.mspct[c(TRUE, FALSE, FALSE)])) +
      aes(linetype = spct.idx) +
      ylim(0,0.15) +
      geom_line() +
      ggtitle("'crown' reflectance spectra")

We calculate the mean reflectance in wavebands corresponding to ISO colors obtaining a data frame. We then add to this returned data frame a factor indicating the body parts.

    refl.by.band <- reflectance(sicalis.mspct, w.band = list(Red(), Green(), Blue(), UVA()))
    refl.by.band$body.part <- rep(c("crown", "throat", "breast"), 7)

    refl.red <- reflectance(sicalis.mspct, w.band = Red())
    names(refl.red)[2] <- "red.reflectance"
    refl.red$body.part <- rep(c("crown", "throat", "breast"), 7)
    ggplot(refl.red, aes(x = body.part, y = red.reflectance)) +
      stat_summary(fun.data = "mean_se", color = "red") +
      geom_point(alpha = 0.5)

## Dealing with odd and bad data

### Using locales

Most functions in this package have a parameter `locale`, that accepts `readr::locale` objects as arguments. At the moment only the time zone and decimal mark are respected. This allows files using comma for decimal marker be easily imported, or the dates and times in the input file be interpreted in a given time zone. Setting the correct time zone is very important to avoid errors. Time coordinates are always stored in the created objects using universal time coordinates (“UTC”).

    jaz.irrad.comma.file <-
      system.file("extdata", "spectrum-comma.JazIrrad",
                  package = "photobiologyInOut", mustWork = TRUE)
    my.locale <- locale(decimal_mark = ",", tz = "EET")
    jaz00.spct <- read_oo_jazirrad(file = jaz.irrad.comma.file,
                                   locale = my.locale)

    jaz00.spct

    ## Object: source_spct [2,048 x 2]
    ## Wavelength range 188.82523 to 1033.1483 nm, step 0.357056 to 0.459625 nm
    ## Label: File: spectrum-comma.JazIrrad
    ## Measured on 2015-02-03 07:44:41 UTC
    ## Time unit 1s
    ##
    ## # A tibble: 2,048 x 2
    ##   w.length s.e.irrad
    ##      <dbl>     <dbl>
    ## 1     189.     0
    ## 2     189.     0
    ## 3     190.    -0.909
    ## # ... with 2,045 more rows

### Overriding default metadata

We revisit now the Jaz irradiance data to show how the metadata can be changed by the user if needed (e.g. clock settings at the time of data acquisition were wrong).

A variable with the user supplied date and time data, or the date read from the header (the text itself) not the file date as the file date may not reflect the creation date and time.

    jaz.s.irrad.file <-
      system.file("extdata", "spectrum.JazIrrad",
                  package = "photobiologyInOut", mustWork = TRUE)

    jaz01.spct <- read_oo_jazirrad(file = jaz.s.irrad.file,
                                   date = NULL)
    getWhenMeasured(jaz01.spct)

    ## [1] "2015-02-03 09:44:41 UTC"

    jaz02.spct <- read_oo_jazirrad(file = jaz.s.irrad.file,
                                   date = ymd_hms("2015-11-15 12:00:00"))
    getWhenMeasured(jaz02.spct)

    ## [1] "2015-11-15 12:00:00 UTC"

    jaz03.spct <- read_oo_jazirrad(file = jaz.s.irrad.file,
                                   date = now())
    getWhenMeasured(jaz03.spct)

    ## [1] "2021-10-10 21:01:21 UTC"

### Adding additional metadata

When can add a geocode, either directly by giving latitude and longitude coordinates or by generating it from a Google maps search using function `ggmap::geocode()` as shown here.

    my.geocode <- data.frame(lon = 25.02006, lat = 60.22525)
    jaz04.spct <- read_oo_jazirrad(file = jaz.s.irrad.file,
                                   geocode = my.geocode)
    jaz04.spct

    ## Object: source_spct [2,048 x 2]
    ## Wavelength range 188.82523 to 1033.1483 nm, step 0.357056 to 0.459625 nm
    ## Label: File: spectrum.JazIrrad
    ## Measured on 2015-02-03 09:44:41 UTC
    ## Time unit 1s
    ##
    ## # A tibble: 2,048 x 2
    ##   w.length s.e.irrad
    ##      <dbl>     <dbl>
    ## 1     189.     0
    ## 2     189.     0
    ## 3     190.    -0.909
    ## # ... with 2,045 more rows

    getWhereMeasured(jaz04.spct)

    ## # A tibble: 1 x 3
    ##     lon   lat address
    ##   <dbl> <dbl> <chr>
    ## 1  25.0  60.2 <NA>