Validating Big Data, a Big Issue

Abstract

Big data consist of multiple fractions of small data. If you wish your big data to be valid, then you will, first, have to make sure, that the fractions are validated:

by the use of the scientific rules for clinical trials, and, in addition,

by the use of traditional diagnostic test validations.

Once this is all done well and good, only then you will be at the starting point of a serious big data analysis. Unfortunately, this is a pretty laborious scenario, and, although, currently, many data bases of big data do exist, most of them are, documentedly, of a poor quality and un-validated. Big data analyses tend to suffer from too many null-values, lack of experienced analysis teams, lacking validation tools, limited validation checklists. Big data tools are in expensive commercial software, and have not been judged by Academia. The best approach to big data analyses may be the use of large checklists, multiple analysis teams, and the use of multiple independent computers with simple programs rather than supercomputers with complex programs.