Preprocessing profiling model for visual analytics

Abstract

In the information age, we have evolved the ability to collect and store data, create sophisticated data mining methods, and generate rich visualizations to share the information resulting from the data analysis process. However, analyzing and managing raw data is still a challenging part of this process, mainly with regards to data preprocessing, which aims to transform this raw data into an appropriate format for subsequent analysis. Although we can find studies proposing design implications or recommendations for future visualiza- tion solutions in the data analysis scope, they do not focus on the challenges during the Preprocessing phase and on how visualization can support it. Likewise, the current Visual Analytics Models are not considering preprocessing an equally important phase in their process, such as Data, Models, Visualization, and Knowledge. Thus, with this study, we aim to contribute to the discussion of how we can use and combine methods of visualization and data mining to assist data analysts during the preprocessing activities. To achieve that, we are introducing the Preprocessing Profiling Model for Visual Analytics, which contemplates a set of features to inspire the implementation of new solutions. In turn, these features were designed considering a list of insights we obtained during an interview study with thirteen data analysts. The main contributions in our study are three: (a) the Preprocessing Profiling Model for Visual Analytics as a solution to assist during Preprocessing phase. (b) The list of ten insights, as a consolidated set of requirements for future visualization research studies applied to preprocessing and data mining. (c) The details on the profile of the data analysts, the main challenges they face, and the opportunities that arise while they are engaged in data mining projects in diverse organizational areas