Challenge Open Access

You Snooze You Win: The PhysioNet/Computing in Cardiology Challenge 2018

Mohammad Ghassemi Benjamin Moody Li-wei Lehman Roger Mark Gari D. Clifford

Published: Feb. 21, 2018. Version: 1.0.0


When using this resource, please cite the original publication:

Ghassemi MM, Moody BE, Lehman LW, Song C, Li Q, Sun H, Mark RG, Westover MB, Clifford GD. You snooze, you win: the physionet/computing in cardiology challenge 2018. In 2018 Computing in Cardiology Conference (CinC) 2018 Sep 23 (Vol. 45, pp. 1-4). IEEE. doi: 10.22489/CinC.2018.049.

Please include the standard citation for PhysioNet: (show more options)
Goldberger, A., Amaral, L., Glass, L., Hausdorff, J., Ivanov, P. C., Mark, R., ... & Stanley, H. E. (2000). PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation [Online]. 101 (23), pp. e215–e220.

Introduction

At the end of last year, American scientists Jeffrey Hall, Michael Rosbash and Michael Young received a Nobel Prize in Physiology “for their discoveries of molecular mechanisms controlling the circadian rhythm"— the mechanism that regulates sleep (Osborn, 2017). The precise reasons why humans sleep (and even how much sleep we need) remains a topic of scientific inquiry. Contemporary theorists indicate that sleep may be responsible for learning and/or the clearing of neural waste products (Ogilvie and Patel, 2017).

While the precise reasons why we sleep are not perfectly understood, there is consensus on the importance of sleep for our overall health, and well-being. Inadequate sleep is associated with a wide range of negative outcomes including: impaired memory and learning, obesity, irritability, cardiovascular dysfunction, hypotension, diminished immune function (Harvard Medical School, 2006), depression (Nutt et al, 2008), and quality of life (Lee, 2009). Further studies even suggest causal links between quality of sleep, and important outcomes including mental health.

It follows that improving the quality of sleep could be used to improve a range of societal health outcomes, more generally. Of course, the treatment of sleep disorders is necessarily preceded by the diagnosis of sleep disorders. Traditionally, such diagnoses are developed in sleep laboratory settings, where polysomnography, audio, and videography of sleeping subject may be carefully inspected by sleep experts to identify potential sleep disorders.

One of the more well-studied sleep disorders is Obstructive Sleep Apnea Hypopnea Syndrome (or simply, apnea). Apneas are characterized by a complete collapse of the airway, leading to awakening, and consequent disturbances of sleep. While apneas are arguably the best understood of sleep disturbances, they are not the only cause of disturbance. Sleep arousals can also be spontaneous, result from teeth grinding, partial airway obstructions, or even snoring. In this year's PhysioNet Challenge we will use a variety of physiological signals, collected during polysomnographic sleep studies, to detect these other sources of arousal (non-apnea) during sleep.

Challenge Data

Data for this challenge were contributed by the Massachusetts General Hospital’s (MGH) Computational Clinical Neurophysiology Laboratory (CCNL), and the Clinical Data Animation Laboratory (CDAC). The dataset includes 1,985 subjects which were monitored at an MGH sleep laboratory for the diagnosis of sleep disorders. The data were partitioned into balanced training (n = 994), and test sets (n = 989).

The sleep stages of the subjects were annotated by clinical staff at the MGH according to the American Academy of Sleep Medicine (AASM) manual for the scoring of sleep. More specifically, the following six sleep stages were annotated in 30 second contiguous intervals: wakefulness, stage 1, stage 2, stage 3, rapid eye movement (REM), and undefined.

Certified sleep technologists at the MGH also annotated waveforms for the presence of arousals that interrupted the sleep of the subjects. The annotated arousals were classified as either: spontaneous arousals, respiratory effort related arousals (RERA), bruxisms, hypoventilations, hypopneas, apneas (central, obstructive and mixed), vocalizations, snores, periodic leg movements, Cheyne-Stokes breathing or partial airway obstructions.

The subjects had a variety of physiological signals recorded as they slept through the night including: electroencephalography (EEG), electrooculography (EOG), electromyography (EMG), electrocardiology (EKG), and oxygen saturation (SaO2). Excluding SaO2, all signals were sampled to 200 Hz and were measured in microvolts. For analytic convenience, SaO2 was resampled to 200 Hz, and is measured as a percentage.

Objective of the Challenge

The goal of the challenge is use information from the available signals to correctly classify target arousal regions. For the purpose of the Challenge, target arousals are defined as regions where either of the following conditions were met:

  • From 2 seconds before a RERA arousal begins, up to 10 seconds after it ends or,
  • From 2 seconds before a non-RERA, non-apnea arousal begins, up to 2 seconds after it ends.

Please note that regions falling within 10 seconds before or after a subject wakes up, has an apnea arousal, or a hypopnea arousal will not be scored for the Challenge.

We have pre-computed the target arousals for you. They are contained in a sample-wise vector (described below in “Accessing the Data”), marked by “1”. Regions that will not be scored are marked by a “-1”, and regions that will be penalized if marked by your algorithm are marked by “0”. You do not need to recompute these scores.

Accessing the Data

The Challenge data repository contains two directories (training and test) which are each approximately 135 GB in size. Each directory contains one subdirectory per subject (e.g. training/tr03-0005). Each subdirectory contains signal, header, and arousal files; for example:

  1. tr03-0005.mat: a Matlab V4 file containing the signal data.
  2. tr03-0005.hea: record header file - a text file which describes the format of the signal data.
  3. tr03-0005.arousal: arousal and sleep stage annotations, in WFDB annotation format.
  4. tr03-0005-arousal.mat: a Matlab V7 structure containing a sample-wise vector with three distinct values (+1, 0, -1) where:
    • +1: Designates arousal regions
    • 0: Designates non-arousal regions
    • -1: Designates regions that will not be scored

Table 1 lists functions that can be used to import the data into Python, Matlab, and C programs.

Table 1: Functions that can be used to import Challenge data.
File type Python Matlab C / C++
Signal (.mat) and header (.hea) files wfdb.rdrecord rdmat isigopen
Arousal annotation files (.arousal) wfdb.rdann rdann annopen
Arousal files (.mat) scipy.io.loadmat load libmatio

Submitting your Entry

Participants should use the provided signal and arousal data to develop a model that classifies test-set subjects. More specifically, for each subject in /test, participants must generate a .vec text file that describes the probability of arousal at each sample, such as:

0.001
0.000
0.024
0.051

The names of the generated annotation files should match the name of the test subject. For instance, test/te09-0094.mat should have a corresponding file named annotations/te09-0094.vec.

Entries must be submitted as a zip file containing:

  • All of the code and data files needed to train and run your algorithm
  • An AUTHORS.txt file containing the list of authors
  • A LICENSE.txt file containing the license for your code
  • The .vec files described above

To upload your entry, create a PhysioNet account (if you don't have one), and go to challenge.physionet.org. Entries must be uploaded prior to the deadline in order to be eligible.

Scoring

Your final algorithm will only be graded for its binary classification performance on target arousal and non-arousal regions (designated by +1 and 0 in teNN-NNNN-arousals.mat), measured by the area under the precision-recall curve. The area is defined as follows:

R j = n u m b e r o f a r o u s a l s a m p l e s w i t h p r e d i c t e d p r o b a b i l i t y ( j / 1000 ) o r g r e a t e r t o t a l n u m b e r o f a r o u s a l s a m p l e s R_j = \frac{number\,of\,arousal\,samples\,with\,predicted\,probability\,(j/1000)\,or\,greater}{total\,number\,of\,arousal\,samples}

P j = n u m b e r o f a r o u s a l s a m p l e s w i t h p r e d i c t e d p r o b a b i l i t y ( j / 1000 ) o r g r e a t e r t o t a l n u m b e r o f s a m p l e s w i t h p r e d i c t e d p r o b a b i l i t y ( j / 1000 ) o r g r e a t e r P_j = \frac{number\,of\,arousal\,samples\,with\,predicted\,probability\,(j/1000)\,or\,greater}{total\,number\,of\,samples\,with\,predicted\,probability\,(j/1000)\,or\,greater}

A U P R C = j P j ( R j R j + 1 ) AUPRC = \sum_{j} P_j(R_j -R_{j+1})

Note that this is the gross AUPRC (i.e., for each possible value of j, the precision and recall are calculated for the entire test database), which is not the same as averaging the AUPRC for each record.

Python (score2018.py) and Matlab/Octave (score2018.m) implementations of the scoring algorithm are available in the challenge files.

Sample Submission

Two simple example algorithms are provided and may be used as a template for your own submission:

Rules and Deadlines

Entrants may have an overall total of up to three submitted entries over both the unofficial and official phases of the competition (see Table 2). Following submission, entrants will receive an email confirming their submission and reporting how well their arousal annotations match those of the held-out test set.

All deadlines occur at noon GMT (UTC) on the dates mentioned below. If you do not know the difference between GMT and your local time, find out what it is before the deadline!

Table 2: Rules and deadlines.
Start at noon GMT on Entry limit End at noon GMT on
Unofficial Phase 15 February 1 13 April
[Hiatus] 13 April 0 22 April
Official Phase 23 April 2 1 September
* Wildcard submissions due 15 July

All official entries must be received no later than noon GMT on Saturday, 1 September 2018. In the interest of fairness to all participants, late entries will not be accepted or scored. Entries that cannot be scored (because of missing components, improper formatting, or excessive run time) are not counted against the entry limits.

To be eligible for the open-source award, you must do all of the following:

  1. Submit at least one open-source entry that can be scored before the Phase I deadline (noon GMT on Monday, 9 April 2018).
  2. Submit at least one entry during the second phase (between noon GMT on Monday, 16 April 2018 and noon GMT on Saturday, 1 September 2018). Only your final entry will count for ranking.
  3. Entering an Abstract to CinC: Submit an acceptable abstract (about 299 words) on your work on the Challenge to Computing in Cardiology no later than 15 April 2018. Include the overall score for your Phase I entry in your abstract. Please select “PhysioNet/CinC Challenge” as the topic of your abstract, so it can be identified easily by the abstract review committee. You will be notified if your abstract has been accepted by email from CinC during the first week in June.
  4. Wildcard submissions: For teams who did not submit an abstract in time, or whose abstracts were not accepted, the team who submits the highest-scoring entry before 15 July 2018 will have another chance to compete, if they submit a high-quality abstract and present their work at the CinC conference. We will contact the winners in July with more information.
  5. Submit a full (4-page) paper on your work on the Challenge to CinC no later than the deadline of conference paper submission.
  6. Attend CinC 2018 (23-26 September 2018) in Maastricht and present your work there.

Please do not submit analysis of this year’s Challenge data to other Conferences or Journals until after CinC 2018 has taken place, so the competitors are able to discuss the results in a single forum. We expect a special issue from the journal Physiological Measurement to follow the conference and encourage all entrants (and those who missed the opportunity to compete or attend CinC 2018) to submit extended analysis and articles to that issue, taking into account the publications and discussions at CinC 2018.

Attending the Conference

If your abstract is accepted, you must log in to the conference site and agree that you will attend. Then, you must submit a full article describing your results and mark it as a preprint (for others to read) by September 15th. (Don't forget that the competition deadline is noon GMT on the 1st September - this deadline will *not* be extended.)

After agreeing to attend, you must register for the conference, pay the conference fee (prices go up after July ends), and secure a visa if you need one. See the Computing in Cardiology site for more information.

If you need a visa, we strongly suggest you register this week and begin the process. Visas can take months to issue and attendance is mandatory - you cannot receive a prize if you do not attend because defending your work is part of the Challenge. The conference (not PhysioNet) will supply you with a letter for your visa. Please see the CinC 2018 site for details on how to obtain that letter and who to contact. If you have any questions about this process, or are concerned about paying the conference fee before securing a visa, please contact the conference organizers, not PhysioNet.

If your abstract is rejected, then you have one more chance! This year we are introducing a 'wildcard' submission. On July the 15th, the top scoring entry that has not so far been accepted to CinC will be offered the opportunity to submit another (or a new) abstract to the conference system (containing full results). If the team can submit a quality abstract (with performance results) and register for the conference then it's members will be eligible for a prize (assuming they also attend the conference and present a poster). Don't forget, your abstract was probably rejected because it didn't contain any useful results (even on training data) and/or did not describe your methods well. So please pay attention to the abstract when submitting - it won't be automatic. We strongly believe that if you are unable to explain what you did and why, then the code is of very limited value.

We hope this is a suitable encouragement for teams that are either late to the Challenge or failed to secure a place at the conference to continue with their efforts in the competition. It would be a shame not to see potentially great works at the conference.

Look out for future announcements via the community discussion forum.

After the Challenge

As is customary, we hope to run a special issue in Physiological Measurement with a closing date of 31 January 2019. We will therefore encourage competitors (and non-competitors) to submit updates and further reworks based on the Challenge after the award ceremony at the Computing in Cardiology Conference in Maastricht in September.

Obtaining complimentary MATLAB licenses

The MathWorks has kindly decided to sponsor Physionet’s 2018 Challenge providing licenses. The MathWorks is offering to all teams that wish to use MATLAB, complimentary licenses. User can apply for a license and learn more about MATLAB support through The Mathworks’ PhysioNet Challenge link. If you have questions or need technical support, please contact The MathWorks at studentcompetitions@mathworks.com.

Challenge Results

Official results, as well as a paper describing the Challenge, are now available. Top scores were achieved by

  1. Matthew Howe-Patterson, Bahareh Pourbabaee, and Frederic Benard (0.54)
  2. Guðni Fannar Kristjansson, Heiðar Már Þráinsson, Hanna Ragnarsdóttir, Bragi Marinósson, Eysteinn Gunnlaugsson, Eysteinn Finnsson, Sigurður Ægir Jónsson, Halla Helgadóttir, and Jón Skírnir Ágústsson (0.45)
  3. Runnan He, Kuanquan Wang, Yang Liu, Na Zhao, Yongfeng Yuan, Qince Li, and Henggui Zhang (0.43)

An unofficial entry from Hongyang Li and Yuanfang Guan (who unfortunately missed the deadline to submit an abstract) achieved a score of 0.55.

Papers

The following paper is an introduction to the challenge topic, with a summary of the challenge results and a discussion of their implications. Please cite this publication when referencing the Challenge.

Ghassemi MM, Moody B, Lehman L, Song C, Li Q, Sun H, Westover M, Clifford GD., "You Snooze, You Win: the PhysioNet/Computing in Cardiology Challenge 2018," 2018 Computing in Cardiology Conference (CinC), 2018, pp. 1-4, doi: 10.22489/CinC.2018.049.

Over 20 papers were presented at Computers in Cardiology 2018. These papers have been made available under the terms of the Creative Commons Attribution License 3.0 (CCAL). See this page for details. We wish to thank all of the authors for their contributions.


Share
Access

Access Policy:
Anyone can access the files, as long as they conform to the terms of the specified license.

License (for files):
Open Data Commons Attribution License v1.0

Corresponding Author
You must be logged in to view the contact information.

Files

Total uncompressed size: 266.6 GB.

Access the files
Folder Navigation: <base>/test
Name Size Modified
Parent Directory
te03-0024
te03-0031
te03-0032
te03-0037
te03-0041
te03-0046
te03-0048
te03-0050
te03-0054
te03-0062
te03-0076
te03-0080
te03-0088
te03-0091
te03-0104
te03-0106
te03-0112
te03-0125
te03-0132
te03-0137
te03-0145
te03-0162
te03-0193
te03-0195
te03-0211
te03-0213
te03-0221
te03-0225
te03-0230
te03-0231
te03-0240
te03-0276
te03-0284
te03-0286
te03-0293
te03-0313
te03-0317
te03-0319
te03-0324
te03-0335
te03-0368
te03-0371
te03-0383
te03-0407
te03-0416
te03-0422
te03-0436
te03-0440
te03-0441
te03-0445
te03-0460
te03-0466
te03-0475
te03-0478
te03-0480
te03-0491
te03-0492
te03-0496
te03-0499
te03-0505
te03-0513
te03-0524
te03-0525
te03-0552
te03-0557
te03-0577
te03-0586
te03-0593
te03-0603
te03-0604
te03-0608
te03-0611
te03-0616
te03-0620
te03-0621
te03-0632
te03-0660
te03-0676
te03-0704
te03-0717
te03-0730
te03-0747
te03-0750
te03-0776
te03-0782
te03-0798
te03-0800
te03-0805
te03-0811
te03-0819
te03-0827
te03-0839
te03-0848
te03-0856
te03-0858
te03-0864
te03-0880
te03-0889
te03-0892
te03-0910
te03-0927
te03-0936
te03-0944
te03-0948
te03-0950
te03-0954
te03-0960
te03-0975
te03-0978
te03-0980
te03-1007
te03-1009
te03-1016
te03-1025
te03-1039
te03-1051
te03-1055
te03-1059
te03-1067
te03-1072
te03-1079
te03-1086
te03-1093
te03-1098
te03-1104
te03-1105
te03-1109
te03-1110
te03-1117
te03-1118
te03-1133
te03-1156
te03-1178
te03-1213
te03-1216
te03-1230
te03-1291
te03-1308
te03-1318
te03-1320
te03-1325
te03-1330
te03-1334
te03-1342
te03-1347
te03-1348
te03-1364
te03-1365
te03-1372
te03-1377
te03-1380
te03-1398
te03-1400
te04-0012
te04-0037
te04-0071
te04-0080
te04-0092
te04-0101
te04-0102
te04-0103
te04-0107
te04-0125
te04-0130
te04-0141
te04-0149
te04-0153
te04-0154
te04-0172
te04-0203
te04-0215
te04-0234
te04-0235
te04-0241
te04-0253
te04-0257
te04-0268
te04-0272
te04-0280
te04-0303
te04-0329
te04-0346
te04-0350
te04-0353
te04-0361
te04-0367
te04-0373
te04-0378
te04-0384
te04-0395
te04-0398
te04-0406
te04-0416
te04-0424
te04-0449
te04-0467
te04-0477
te04-0479
te04-0483
te04-0487
te04-0489
te04-0496
te04-0497
te04-0510
te04-0521
te04-0529
te04-0543
te04-0564
te04-0574
te04-0581
te04-0582
te04-0596
te04-0602
te04-0624
te04-0641
te04-0651
te04-0664
te04-0665
te04-0669
te04-0677
te04-0697
te04-0707
te04-0721
te04-0729
te04-0753
te04-0758
te04-0761
te04-0775
te04-0779
te04-0797
te04-0807
te04-0811
te04-0813
te04-0827
te04-0832
te04-0838
te04-0842
te04-0845
te04-0849
te04-0857
te04-0858
te04-0880
te04-0885
te04-0889
te04-0895
te04-0897
te04-0899
te04-0901
te04-0906
te04-0932
te04-0953
te04-0962
te04-0970
te04-0976
te04-0990
te04-0992
te04-1018
te04-1031
te04-1035
te04-1046
te04-1047
te04-1065
te04-1086
te04-1118
te04-1130
te05-0038
te05-0043
te05-0073
te05-0082
te05-0088
te05-0108
te05-0126
te05-0130
te05-0131
te05-0151
te05-0165
te05-0173
te05-0177
te05-0194
te05-0209
te05-0255
te05-0271
te05-0272
te05-0280
te05-0283
te05-0290
te05-0314
te05-0339
te05-0347
te05-0351
te05-0365
te05-0369
te05-0371
te05-0388
te05-0401
te05-0402
te05-0421
te05-0430
te05-0433
te05-0450
te05-0451
te05-0457
te05-0473
te05-0518
te05-0540
te05-0541
te05-0551
te05-0555
te05-0567
te05-0571
te05-0584
te05-0606
te05-0616
te05-0626
te05-0628
te05-0658
te05-0660
te05-0661
te05-0684
te05-0693
te05-0695
te05-0697
te05-0700
te05-0708
te05-0717
te05-0722
te05-0731
te05-0740
te05-0747
te05-0786
te05-0804
te05-0810
te05-0830
te05-0832
te05-0836
te05-0842
te05-0862
te05-0886
te05-0887
te05-0901
te05-0908
te05-0922
te05-0942
te05-0950
te05-0953
te05-0997
te05-0998
te05-1001
te05-1016
te05-1019
te05-1036
te05-1038
te05-1044
te05-1049
te05-1062
te05-1069
te05-1073
te05-1075
te05-1079
te05-1087
te05-1124
te05-1141
te05-1143
te05-1151
te05-1155
te05-1164
te05-1204
te05-1246
te05-1249
te05-1251
te05-1254
te05-1267
te05-1281
te05-1296
te05-1331
te05-1334
te05-1335
te05-1352
te05-1378
te05-1380
te05-1390
te05-1416
te05-1420
te05-1421
te05-1427
te05-1436
te05-1439
te05-1470
te05-1484
te05-1495
te05-1500
te05-1505
te05-1517
te05-1525
te05-1529
te05-1530
te05-1544
te05-1548
te05-1552
te05-1557
te05-1559
te05-1578
te05-1581
te05-1587
te05-1588
te05-1592
te05-1594
te05-1605
te05-1621
te05-1623
te05-1626
te05-1627
te05-1628
te05-1633
te05-1634
te05-1640
te05-1647
te05-1658
te05-1660
te05-1665
te05-1667
te05-1672
te05-1680
te05-1685
te05-1697
te05-1701
te05-1703
te05-1724
te05-1827
te05-1828
te05-1830
te06-0001
te06-0002
te06-0016
te06-0021
te06-0031
te06-0032
te06-0033
te06-0034
te06-0038
te06-0040
te06-0045
te06-0047
te06-0067
te06-0071
te06-0072
te06-0088
te06-0105
te06-0110
te06-0123
te06-0131
te06-0143
te06-0149
te06-0162
te06-0179
te06-0208
te06-0209
te06-0235
te06-0237
te06-0248
te06-0252
te06-0260
te06-0266
te06-0273
te06-0287
te06-0293
te06-0305
te06-0321
te06-0338
te06-0342
te06-0357
te06-0367
te06-0395
te06-0396
te06-0412
te06-0413
te06-0428
te06-0442
te06-0445
te06-0455
te06-0457
te06-0461
te06-0482
te06-0484
te06-0493
te06-0518
te06-0519
te06-0540
te06-0543
te06-0546
te06-0581
te06-0585
te06-0597
te06-0613
te06-0614
te06-0623
te06-0629
te06-0635
te06-0641
te06-0642
te06-0645
te06-0666
te06-0667
te06-0669
te06-0681
te06-0692
te06-0695
te06-0699
te06-0713
te06-0717
te06-0731
te06-0744
te06-0748
te06-0766
te06-0770
te06-0793
te06-0796
te06-0813
te06-0816
te06-0828
te06-0840
te06-0848
te06-0849
te06-0857
te06-0867
te06-0872
te06-0873
te06-0874
te06-0886
te06-0887
te06-0896
te06-1008
te06-1110
te06-1118
te06-1130
te06-1177
te07-0004
te07-0006
te07-0007
te07-0009
te07-0014
te07-0020
te07-0051
te07-0052
te07-0057
te07-0060
te07-0062
te07-0063
te07-0070
te07-0096
te07-0117
te07-0122
te07-0128
te07-0146
te07-0149
te07-0158
te07-0172
te07-0175
te07-0177
te07-0185
te07-0218
te07-0220
te07-0226
te07-0233
te07-0234
te07-0252
te07-0270
te07-0271
te07-0287
te07-0289
te07-0311
te07-0316
te07-0328
te07-0331
te07-0348
te07-0356
te07-0361
te07-0365
te07-0378
te07-0405
te07-0417
te07-0433
te07-0447
te07-0476
te07-0480
te07-0493
te07-0495
te07-0496
te07-0510
te07-0515
te07-0518
te07-0547
te07-0550
te07-0552
te07-0556
te07-0573
te07-0597
te07-0613
te07-0617
te07-0644
te07-0671
te07-0674
te07-0677
te07-0682
te07-0683
te07-0692
te07-0693
te07-0694
te07-0699
te07-0704
te07-0706
te07-0715
te07-0740
te07-0742
te07-0746
te07-0749
te07-0754
te07-0778
te07-0779
te07-0785
te07-0797
te07-0799
te07-0813
te07-0816
te07-0831
te07-0838
te07-0839
te07-0840
te07-0844
te07-0851
te07-0898
te07-0903
te07-0917
te07-0965
te08-0019
te08-0100
te08-0102
te08-0163
te08-0207
te08-0210
te08-0215
te08-0223
te08-0225
te08-0258
te08-0262
te08-0278
te08-0281
te08-0317
te08-0320
te08-0321
te08-0325
te08-0352
te09-0001
te09-0003
te09-0032
te09-0035
te09-0046
te09-0084
te09-0094
te09-0117
te09-0118
te09-0121
te09-0123
te09-0133
te09-0137
te09-0145
te09-0149
te09-0151
te09-0156
te09-0167
te09-0171
te09-0174
te09-0193
te09-0222
te09-0228
te09-0231
te09-0248
te09-0272
te09-0277
te09-0285
te09-0289
te09-0300
te09-0301
te09-0310
te09-0312
te09-0315
te09-0343
te09-0380
te09-0386
te09-0391
te09-0407
te09-0415
te09-0429
te09-0458
te09-0459
te09-0468
te09-0481
te09-0502
te09-0521
te09-0538
te09-0548
te09-0549
te09-0550
te09-0554
te09-0566
te09-0582
te09-0585
te09-0589
te09-0601
te09-0605
te09-0634
te09-0645
te10-0041
te10-0062
te10-0075
te10-0107
te10-0143
te10-0145
te10-0168
te10-0180
te10-0186
te10-0208
te10-0230
te10-0244
te10-0262
te10-0276
te10-0279
te10-0299
te10-0337
te10-0374
te10-0397
te10-0428
te10-0451
te10-0490
te10-0503
te10-0532
te10-0552
te10-0560
te10-0574
te10-0605
te10-0631
te10-0637
te10-0649
te10-0657
te10-0679
te10-0684
te10-0738
te10-0755
te10-0759
te10-0766
te10-0774
te10-0791
te10-0819
te10-0859
te11-0005
te11-0014
te11-0026
te11-0045
te11-0047
te11-0057
te11-0066
te11-0067
te11-0075
te11-0077
te11-0093
te11-0112
te11-0117
te11-0131
te11-0141
te11-0164
te11-0167
te11-0170
te11-0173
te11-0189
te11-0196
te11-0313
te11-0321
te11-0330
te11-0336
te11-0340
te11-0346
te11-0348
te11-0351
te11-0353
te11-0360
te11-0369
te11-0374
te11-0384
te11-0389
te11-0390
te11-0394
te11-0409
te11-0412
te11-0420
te11-0425
te11-0444
te11-0477
te11-0480
te11-0482
te11-0488
te11-0489
te11-0506
te11-0511
te11-0517
te11-0526
te11-0538
te11-0554
te11-0557
te11-0559
te11-0583
te11-0588
te11-0601
te11-0605
te11-0618
te11-0624
te11-0647
te11-0651
te11-0664
te11-0667
te11-0675
te11-0684
te11-0695
te11-0697
te11-0702
te11-0725
te11-0728
te11-0731
te11-0738
te11-0753
te11-0768
te11-0779
te12-0039
te12-0048
te12-0063
te12-0072
te12-0087
te12-0100
te12-0112
te12-0146
te12-0156
te12-0164
te12-0180
te12-0186
te12-0188
te12-0226
te12-0239
te12-0246
te12-0256
te12-0257
te12-0305
te12-0307
te12-0311
te12-0331
te12-0351
te12-0378
te12-0381
te12-0383
te12-0388
te12-0418
te12-0422
te12-0430
te12-0443
te12-0444
te12-0447
te12-0458
te12-0459
te12-0466
te12-0482
te12-0486
te12-0487
te12-0488
te12-0498
te12-0507
te12-0508
te12-0514
te12-0538
te12-0544
te12-0547
te12-0550
te12-0553
te12-0556
te12-0562
te12-0565
te12-0584
te12-0585
te12-0586
te12-0590
te12-0612
te12-0628
te12-0636
te12-0641
te12-0651
te12-0653
te12-0660
te12-0664
te13-0062
te13-0066
te13-0072
te13-0102
te13-0116
te13-0123
te13-0135
te13-0137
te13-0169
te13-0172
te13-0190
te13-0200
te13-0209
te13-0242
te13-0248
te13-0249
te13-0253
te13-0263
te13-0272
te13-0275
te13-0281
te13-0283
te13-0286
te13-0301
te13-0357
te13-0360
te13-0371
te13-0377
te13-0390
te13-0402
te13-0409
te13-0415
te13-0418
te13-0428
te13-0430
te13-0442
te13-0446
te13-0456
te13-0458
te13-0459
te13-0470
te13-0481
te13-0484
te13-0486
te13-0516
te13-0520
te13-0530
te13-0559
te13-0574
te13-0597
te13-0602
te13-0609
te13-0622
te13-0629
te13-0633
te13-0634
te13-0636
te13-0639
te13-0670
te13-0678
te13-0682
te13-0821
te13-0823
te13-0827
te13-0848
te13-0851
te14-0015
te14-0017
te14-0032
te14-0038
te14-0100
te14-0104
te14-0108
te14-0115
te14-0123
te14-0124
te14-0146
te14-0159
te14-0168
te14-0176
te14-0177
te14-0194
te14-0199
te14-0211
te14-0221
te14-0225
te14-0226
te14-0230
te14-0241
te14-0245
te14-0257
te14-0264
te14-0271
te14-0286
MD5SUMS (download) 112.1 KB 2018-02-20
RECORDS (download) 10.6 KB 2018-02-20
SHA1SUMS (download) 127.5 KB 2018-02-20
SHA256SUMS (download) 173.9 KB 2018-02-20