Commit e4a96457 by Paul Bouchequet

eeg

parent eecfb853
......@@ -15,3 +15,11 @@ R -e "blogdown::serve_site()"
```
R -e "blogdown::build_site()"
```
## Data sources
### Datadouanes
### ISRUC Sleep data
Download performed in the *About the ISRUC-Sleep dataset* article.
\ No newline at end of file
......@@ -11,24 +11,25 @@ csl: "../../sources/computer-methods-and-programs-in-biomedicine.csl"
The Insitute of Systems and Robotics from Coimbra, Portugal (ISRUC) publishes a full sleep dataset under open-access terms @Khalighi2016. A [dedicated website]( http://sleeptight.isr.uc.pt/ISRUC_Sleep/) lists available files and documentation for all to download and use freely. As for many sleep resources like SleepEDF @Kemp2000 or Sleepdata.org @Dean2016 @Zhang2018, signals, scoring and metadata are distributed in separate files and differents formats.
This article provides examples to download, read and visualise the ISRUC sleep dataset. Examples are implemented using the R programming language @Hornik2012.
This article provides an easy workflow to download, read and visualise the ISRUC sleep dataset. Examples are implemented using the R programming language @Hornik2012 and the SleepR library @Bouchequet2018 .
# Downloading data
Records archives contains signals in EDF format @Kemp2003a as long as scoring in Excel files. 3 recordings subgroups of differents size and characteristics are available:
3 recordings subgroups of differents size and characteristics are available:
* **Subgroup 1**: 100 records from 100 subjects, many with sleep apnea.
* **Subgroup 2**: 8 subjects with 2 records each, to study changes between records.
* **Subgroup 3**: 10 records from 10 healthy subjects.
Metadata, or *biodata*, are distributed in Excel files. A useful Sleepr function downloads and expands archives from the 3 subgroups.
Records archives contains signals in EDF format @Kemp2003a as long as scoring in Excel files. Metadata, or *biodata*, are distributed in Excel files.
To download all A useful Sleepr function downloads and expands archives from the 3 subgroups.
```{r download, message=FALSE, warning=FALSE}
# Set dataset target directory
target <- "../../sources/isruc-sleep-dataset/"
target <- "../../sources/isruc-sleep-dataset/raw/"
# Install latest Sleepr version.
# Install latest SleepR version.
devtools::install_github("boupetch/sleepr")
# Dowload ISRUC dataset
......@@ -36,17 +37,17 @@ sleepr::download_isruc(target)
```
# Reading sleep stages
# Reading and plotting sleep stages
Reading scored stages provides an easy way to take a first look at the data. For analysis purposes, sleeps records are traditionaly splited into 30 seconds epochs. These 30 seconds epochs are scored between 5 categories, following the American Association for Sleep Medicine (AASM) manual @Berry2013.
Reading scored stages provides an easy way to take a first look at the data. For analysis purposes, sleeps records are traditionaly splited into 30 seconds epochs. These 30 seconds epochs are scored between 5 stages, following the American Association for Sleep Medicine (AASM) manual @Berry2013.
* **AWA**: *Wake*, the wake stage.
* **REM**: *Rapid Eye Movement stage*, or paradoxical sleep.
* **REM**: *Rapid Eye Movement (REM) stage*, or paradoxical sleep.
* **N1**: *Non-REM Sleep 1*, a transitional stage between sleep and wake.
* **N2**: *Non-REM Sleep 2*, the most encountered sleep stage.
* **N3**: *Non-REM Sleep 3*, deep sleep.
A hypnogram visualize these stages through a night course.
A hypnogram visualize these stages through the course of the night. Traditionaly, REM sleep is colored in red. The following code chunk plot the hypnogram from the first record from the first subgroup of the ISRUC database. The patient here suffers from obstructive sleep apnea, hence a fragmented sleep with many wake epochs.
```{r hypnogram, message=FALSE, warning=FALSE, fig.cap="Hypnogram, subgroup 1, subject 1, record 1.",fig.height=2}
......@@ -60,9 +61,9 @@ sleepr::plot_hypnogram(hypnogram)
<!-- Many other events can be read from the scoring files. The [provided documentation](http://sleeptight.isr.uc.pt/ISRUC_Sleep/ISRUC_Sleep/Content.pdf) lists available data. -->
# Metadata
# Database metadata and statistics
3 metadata files, one for each subgroup, contain many informations about the subject, the record and the recording conditions.
3 metadata files, one for each subgroup, contain many informations about the subject, the record and its recording conditions.
```{r read-metadata, message=FALSE, warning=FALSE}
......@@ -70,7 +71,9 @@ metadata <- sleepr::read_isruc_metadata(target)
```
From the gloabal metadata files can be plotted availables stages, records durations and subjects age distributions across the whole database. These informations help assert the database quality.
From the global metadata files can be plotted availables stages, records durations and subjects age distributions across the whole database. These informations help assert the database quality.
## Stages distribution
```{r stages-distribution, echo=FALSE, message=FALSE, warning=FALSE, fig.cap='Sleep stages distribution by subgroup.'}
library(plotly,
......@@ -120,6 +123,10 @@ subplot(p1,p2,p3) %>%
```
A sleep database analysis starts by plotting stages distribution, or the number of epochs by sleep stages. Usually, *N2* counts for the most epochs. Moreover, a large number of wake epochs could imply too long records, requiring further investigation.
## Records duration
```{r record-duration, echo=FALSE, message=FALSE, warning=FALSE, fig.cap='Records duration by subgroup.'}
duration <- dplyr::mutate(metadata, Duration = (Epoches/2)/60)
......@@ -129,7 +136,11 @@ plot_ly(y = duration$Duration[duration$subgroup == 1], type = "box",name="Subgro
layout(showlegend = FALSE)
```
```{r subject-age, echo=FALSE, message=FALSE, warning=FALSE, fig.cap='Subject age by subgroup.'}
Records durations should be consistent across the database. Vizualizing the distribution highlights outliers requiring attention.
## Subjects age
```{r subjects-age, echo=FALSE, message=FALSE, warning=FALSE, fig.cap='Subject age by subgroup.'}
age <- metadata %>%
dplyr::mutate(Age = as.numeric(Age))
......@@ -140,4 +151,6 @@ plot_ly(y = age$Age[age$subgroup == 1], type = "box",name="Subgroup 1") %>%
layout(showlegend = FALSE)
```
Sleep analysis must take subjects age into account, as sleep evolves throughout lifespan @Ohayon2004. Over-representation of an age class will enlight class specific features. On the contrary, scattered ages can give an overbroad analysis.
# References
\ No newline at end of file
---
title: How to read sleep data
title: How to read sleep data from the ISRUC-Sleep dataset
author: ''
date: '2018-08-17'
slug: read-sleep-data
categories: []
tags: []
draft: true
bibliography: "../../sources/isruc-sleep-dataset/references.bib"
csl: "../../sources/computer-methods-and-programs-in-biomedicine.csl"
---
The Insitute of Systems and Robotics from Coimbra, Portugal (ISRUC) publishes a [full sleep dataset](https://www.frenchkpi.com/isruc-sleep-dataset/) under open-access terms @Khalighi2016.
A sleep dataset always come in three differents parts: signals, scoring and metadata. In polysomnographies, signals include electroencephalograms, electrocardiograms, electrooculograms and many more channels including respiratory flows. Frequently, publishers offer scoring data, a list of manualy scored events: sleep stages, apneas, eye movements... Finally, a metadata, or biodata file give more informations about patients or subjects, as long as recording conditions.
A sleep dataset always come in three differents parts: signals, scoring and metadata. In polysomnographies, signals include electroencephalograms (EEG), electrocardiograms (ECG), electrooculograms (EOG) and many more channels including respiratory flows. Frequently, publishers provide scoring data, a list of manually scored events: sleep stages, apneas, eye movements... Finally, a metadata, or biodata file give more informations about patients or subjects, as long as recording conditions.
Data quality varies according to the equipement, scorers skills and metadata completness. If signals are almost always published using the EDF standard, scoring and metadata format can change according to the publisher. In the ISRUC dataset, signals use EDF, scoring and metadata use a Microsfot Excel format.
Data quality varies according to the recording equipement, scorers skills and metadata completness. If signals are almost always published using the EDF standard @Kemp2003a, scoring and metadata format can change according to the publisher. In the ISRUC-Sleep dataset, signals are provided through EDF, scoring and metadata through Microsoft Excel format.
# Data formatting
```{r format, eval=FALSE}
#bibliography: "../../static/post/read-sleep-data_files/references.bib"
library(doParallel)
doMC::registerDoMC(parallel::detectCores()-1)
save.mdf <- function(
subgroup,subject,record,channels,
target = "../../sources/isruc-sleep-dataset/"){
if(subgroup == 2){
path <- paste0(target,subgroup,
"/",subject,"/",record,"/")
recPath <- paste0(path,record,".rec")
xlsxPath <- paste0(path,record,"_1.xlsx")
} else{
path <- paste0(target,subgroup,"/",subject,"/")
recPath <- paste0(path,subject,".rec")
xlsxPath <- paste0(path,subject,"_1.xlsx")
}
h <- edfReader::readEdfHeader(recPath)
s <- edfReader::readEdfSignals(h)
for (c in channels){
if (c %in% h$sHeaders$label){
cPath <- paste0(path,c)
unlink(cPath)
dir.create(cPath)
writeBin(s[[c]]$signal,
con=paste0(cPath,"/data.bin"),
endian="little",size=4)
jsonlite::write_json(
list("fs"=as.integer(s[[c]]$sRate[1]),
"start"=as.numeric(
s[[c]]$startTime[1])),
path = paste0(cPath,"/meta.json"))
stages <- readxl::read_excel(xlsxPath)
if("Epoch" %in% colnames(stages)){
jsonlite::write_json(
stages$Stage,
path=paste0(path,"stages.json"))
}
}
}
}
pops <- list(c(1:100),c(1:8),c(1:10))
records <- list(1,c(1:2),c(1))
channels <- c("C3-M2","C3-M2",
"F4-A1","C4-A1","C4-M1")
for(subgroup in c(1:length(pops))){
foreach::foreach(subject = pops[[subgroup]]) %dopar% {
for(subject in pops[[subgroup]]){
for(record in records[[subgroup]]){
save.mdf(subgroup,subject,record,channels)
}
}
```{r formatting, message=FALSE, warning=FALSE}
# Install latest SleepR library
devtools::install_github("boupetch/sleepr")
# Set ISRUC Sleep database directory
target <- "../../sources/isruc-sleep-dataset/"
# Set raw files directory (Subgroup 1)
raw_dir <- paste0(target,"raw/1")
# Set MDF files directory
mdf_dir <- paste0(target,"mdf/1")
if(!dir.exists(mdf_dir)) dir.create(mdf_dir)
for(record in list.dirs(raw_dir, recursive = F)){
id <- basename(record)
rec <- paste0(record,"/",id,".rec")
mdf <- paste0(mdf_dir,"/",id)
events <- sleepr::read_events_isruc(record,1)
channels <- c("F4-A1","C4-A1","C4-M1")
if(!dir.exists(mdf)){
sleepr::write_mdf(edfPath = rec,
mdfPath = mdf,
channels = channels,
events = events)
}
}
```
# Visualizing an EEG channel
```{python, engine.path = '/usr/local/bin/python3', eval=FALSE}
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import json
figPath = "../../static/post/" \
"read-sleep-data_files/spec1.png"
sigPath = "../../sources/" \
"isruc-sleep-dataset/1/1/F4-A1/data.bin"
stagesPath = "../../sources/" \
"isruc-sleep-dataset/1/1/stages.json"
fig, (fig1, fig2) = plt.subplots(nrows=2, figsize=(9,4))
# Spectorgram
c3m2 = np.fromfile(sigPath,
dtype= np.float32)
fig1.specgram(c3m2, NFFT=1024, Fs=200,
cmap="gist_ncar",noverlap=512)
fig1.tick_params(axis='x',which='both',bottom=False,
top=False,labelbottom=False)
# Hypnogram
h = json.loads(open(stagesPath).read())
h = [5 if (i == "W") else i for i in h]
h = [4 if (i == "R") else i for i in h]
h = [3 if (i == "N1") else i for i in h]
h = [2 if (i == "N2") else i for i in h]
h = [1 if (i == "N3") else i for i in h]
idx = [i*30 for i in range(0,len(h))]
fig2.plot(idx, h)
plt.xlim(0,max(idx))
fig2.set_yticklabels(["","N3","N2","N1","R","W"])
fig.savefig(figPath, bbox_inches='tight')
```{r hypnogram, message=FALSE, warning=FALSE, paged.print=FALSE, fig.height=2}
rec_path <- "../../sources/isruc-sleep-dataset/mdf/1/1/"
channel <- "C4-A1"
record <- sleepr::read_mdf(rec_path, channels = channel)
sleepr::plot_hypnogram(record[["events"]])
```
![Figure 1, Spectrogram and hypnogram, subgroup 1, subject 1](/post/read-sleep-data_files/spec1.png)
```{r spectrogram, message=FALSE, warning=FALSE, paged.print=FALSE, fig.height=3}
signal <- record[["channels"]][[channel]][["signal"]]
sRate <- record[["channels"]][[channel]][["metadata"]][["sRate"]]
Spectrogram + Hypnogram
sleepr::plot_spectrogram(signal = signal, sRate = sRate, show = T)
```
# Spectral powers
```{r spectral_powers, message=FALSE, warning=FALSE, paged.print=FALSE}
library(doMC)
doMC::registerDoMC(parallel::detectCores())
paths <- list.dirs(mdf_dir, recursive = F)
data_path <-
# bands <- foreach::foreach(
# path = paths,
# .combine = dplyr::bind_rows) %dopar% {
for(path in paths){
print(path)
record <- sleepr::read_mdf(path, "C4-M1")
sleepr::hypnogram_band_powers(record, "C4-M1")
}
```
# References
\ No newline at end of file
......@@ -95,6 +95,28 @@ title = {{The AASM Manual for the Scoring of Sleep and Associated Events}},
volume = {53},
year = {2013}
}
@article{Ohayon2004,
abstract = {OBJECTIVES: $\backslash$r$\backslash$n$\backslash$r$\backslash$nThe purposes of this study were to identify age-related changes in objectively recorded sleep patterns across the human life span in healthy individuals and to clarify whether sleep latency and percentages of stage 1, stage 2, and rapid eye movement (REM) sleep significantly change with age.$\backslash$r$\backslash$n$\backslash$r$\backslash$nDESIGN: $\backslash$r$\backslash$n$\backslash$r$\backslash$nReview of literature of articles published between 1960 and 2003 in peer-reviewed journals and meta-analysis.$\backslash$r$\backslash$n$\backslash$r$\backslash$nPARTICIPANTS: $\backslash$r$\backslash$n$\backslash$r$\backslash$n65 studies representing 3,577 subjects aged 5 years to 102 years.$\backslash$r$\backslash$n$\backslash$r$\backslash$nMEASUREMENT: $\backslash$r$\backslash$n$\backslash$r$\backslash$nThe research reports included in this meta-analysis met the following criteria: (1) included nonclinical participants aged 5 years or older; (2) included measures of sleep characteristics by "all night" polysomnography or actigraphy on sleep latency, sleep efficiency, total sleep time, stage 1 sleep, stage 2 sleep, slow-wave sleep, REM sleep, REM latency, or minutes awake after sleep onset; (3) included numeric presentation of the data; and (4) were published between 1960 and 2003 in peer-reviewed journals.$\backslash$r$\backslash$n$\backslash$r$\backslash$nRESULTS: $\backslash$r$\backslash$n$\backslash$r$\backslash$nIn children and adolescents, total sleep time decreased with age only in studies performed on school days. Percentage of slow-wave sleep was significantly negatively correlated with age. Percentages of stage 2 and REM sleep significantly changed with age. In adults, total sleep time, sleep efficiency, percentage of slow-wave sleep, percentage of REM sleep, and REM latency all significantly decreased with age, while sleep latency, percentage of stage 1 sleep, percentage of stage 2 sleep, and wake after sleep onset significantly increased with age. However, only sleep efficiency continued to significantly decrease after 60 years of age. The magnitudes of the effect sizes noted changed depending on whether or not studied participants were screened for mental disorders, organic diseases, use of drug or alcohol, obstructive sleep apnea syndrome, or other sleep disorders.$\backslash$r$\backslash$n$\backslash$r$\backslash$nCONCLUSIONS: $\backslash$r$\backslash$n$\backslash$r$\backslash$nIn adults, it appeared that sleep latency, percentages of stage 1 and stage 2 significantly increased with age while percentage of REM sleep decreased. However, effect sizes for the different sleep parameters were greatly modified by the quality of subject screening, diminishing or even masking age associations with different sleep parameters. The number of studies that examined the evolution of sleep parameters with age are scant among school-aged children, adolescents, and middle-aged adults. There are also very few studies that examined the effect of race on polysomnographic sleep parameters.$\backslash$r$\backslash$n},
author = {Ohayon, Maurice M. and Carskadon, Mary A. and Guilleminault, Christian and Vitiello, Michael V.},
doi = {10.1093/sleep/27.7.1255},
file = {:Users/paul/Nextcloud/Documents/Articles/2004.Ohayon.pdf:pdf},
isbn = {0161-8105 (Print)},
issn = {01618105},
journal = {Sleep},
keywords = {Meta-analysis,Moderator analysis,PSG,Psychiatric disorders,Sleep disorders},
number = {7},
pages = {1255--1273},
pmid = {15586779},
title = {{Meta-analysis of quantitative sleep parameters from childhood to old age in healthy individuals: Developing normative sleep values across the human lifespan}},
volume = {27},
year = {2004}
}
@misc{Bouchequet2018,
abstract = {Sleep Analysis with R},
author = {Bouchequet, Paul},
title = {{SleepR}},
url = {https://github.com/boupetch/sleepr},
year = {2018}
}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment