Faster statistical analysis of large data sets

Increasing digitization is generating ever larger volumes of data, for instance when surfing the Net, gathering scientific data from sensors or analyzing correlations for medical diagnostics. With statistical analysis researchers can for example make forecasts and deduce the right course of action. The normal procedures and steps of a process are not always optimally coordinated or cannot be implemented because of external factors such as limited processing power. The statistical research group, ‘Mathematical Statistics in the Information Age’, including speaker Prof. Dr. Angelika Rohde of the Institute of Mathematics at the University of Freiburg has recognized this problem and is aiming to make the statistical analysis of large data volumes more efficient. The German Research Foundation (DFG) is providing the group with a total of 2.1 million euros in funding over the next four years. The Austrian Science Fund (FWF) is additionally funding two doctorate posts.

The maximum quantity of statistical information at every step

“Our aim is to develop a comprehensive statistical methodology that meets the demands of modern data analysis,” says Rohde. To do this, the team is developing fast procedures for statistical data analysis and studying all successive data processing steps simultaneously, to produce the best possible statistical evidence. “Statistical guarantees are after all only valid if the distribution properties of the pre-processed data are taken into account in subsequent processing,” explains Rohde. At the same time, the researchers want to extract the maximum quantity of statistical information in each processing step. This would overall result in a minimal number of processing steps, making the statistical data analysis more efficient and also computationally realizable for large quantities.

Besides Rohde, the research group also involves Prof. Dr. Alexandra Carpentier from the University of Potsdam, Prof. Dr. Holger Dette from the Ruhr University Bochum, Prof. Dr. Johannes Moritz Jirak and Assistant Professor Dr. Lukas Steinberger from the University of Vienna, Austria, Prof. Dr. Alexander Meister from the University of Rostock, Prof. Dr. Axel Munk from the University of Göttingen and Prof. Dr. Markus Reiß from the Humboldt University, Berlin.

DFG’s Press Release