Software Open Access
Random Search Toolbox
Published: Nov. 4, 2014. Version: 1.0.0
Goldberger AL, Amaral LAN, Glass L, Hausdorff JM, Ivanov PCh, Mark RG, Mietus JE, Moody GB, Peng C-K, Stanley HE. PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals (2003). Circulation. 101(23):e215-e220.
A major issue with many signal processing and machine learning algorithms is the lack of optimisation methods for determining the numerous hyper-parameters associated with the model as well as the knowledge of which hyper-parameters are relevant. These parameters are usually tunned by trial and error (manual search) or by grid search. In 2012 Bergstra et al. showed empirically that random search was performing similar or better than grid search while reducing the computational cost substantially.
RS-toolbox is contributed by Joachim Behar, Alistair Johnson, Julien Oster and Gari D Clifford. All authors are part of the institute of biomedical engineering at the University of Oxford. The source code is available in MATLAB rs-toolbox.zip
The RS-toolbox code provides functions for creating Efficiency Curves (EC) and Automatic Relevance Determination (ARD) plots. The functions can easily be run to reproduce the plots of this example. Start by simply launching the ESN_EfficiencyCurve.m script in order to produce a similar EC as Figure 1 of the example and then launch ESN_IntrinsicDimensionality.m in order to produce an ARD plot as Figure 2 of the example. This code together with its example of parameters search for training an Echo State Neural network should provide the users with a quick insight on how to use random search and produce EC and ARD from the searched parameters.
General remark on Random Search and ARD: be aware that the interval that you sample your variables from will have an impact on the ARD plots and the conclusions you will make from it!
Anyone can access the files, as long as they conform to the terms of the specified license.
License (for files):
Open Data Commons Attribution License v1.0
Total uncompressed size: 1.4 MB.