{'fraction_non_alphanumeric': 0.027116729044559, 'fraction_numerical': 0.01031000635548337, 'mean_word_length': 4.824182603331277, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Towards Hybrid Profiling: Combining Digital Phenotyping with Validated Survey Questions to Balance Data Entry Effort with Predictive Power

Tailoring apps based on user traits has attracted tremendous interest in developing mHealth apps and understanding a users’ personality is a key challenge in that context. This challenge is typically addressed via classic surveys, which pose a regrettably high burden on app users. This study aims to reduce the response burden of personality tests by introducing a model for predicting the user personality based on digital footprints of app usage. At the same time, skipping surveys completely turns out to undermine prediction accuracy. Therefore, this paper conceptualizes a hybrid framework that utilizes user event data in combination with surveys that have fewer questions than conventionally. The proposed method demonstrates a promising trade-off between the simplicity of using user event data and the accuracy of the validated survey methods: when applying the hybrid method to a retrospective case study, the accuracy is higher than when using the event data exclusively. Also, the number of survey questions needed is significantly lower. Since this is a novel method, we expect that results will strengthen as larger data sets available over time.


INTRODUCTION
Recently, mHealth interventions have shown significant potential to empower its users to change their daily routines and adopt healthier lifestyles [10]. However, these digital support tools often fail to retain their impact in the long-term due to their users losing interest [12].
A promising strategy to retain users is by tailoring your intervention to the users' individual wants and needs [7]. Particularly, it was found that personalized mHealth interventions are more effective than interventions that adopt one-size-fits-all approaches [7].
User preferences are commonly derived from knowledge on a user's personality [11]. Therefore, to personalize an mHealth intervention, the tool first needs to understand the preferences a particular user has. Historically, personality traits are derived using questionnaires [22,23].
However, users often perceive questionnaires as cumbersome [23], and the quality of their answers decreases with every next question [2]. Hence, lengthy personality tests increase the risk of users dropping out, and as such, the aim to reduce dropouts by applying trait-based personalization may fail a priori.
Although efforts have been invested in shortening popular personality tests, and user experience design techniques have been employed to reduce the perceived burden of questionnaires (e.g., using blocks and progress bars) [13], it was found that filling out popular personality tests still requires between five to fifteen minutes [15].
When the primary aim of such surveys is to personalize an mHealth intervention then they are typically part of a digital onboarding process. In digital onboarding processes, dropout is already a challenging issue so adding five to fifteen minutes there is probably unacceptable from the dropout point of view.
Another strategy to gain understanding of a user's personality is to apply unobtrusive learning methods from the domain of Artificial Intelligence (AI). For example, user event data is automatically recorded by users performing digital actions on mHealth tools. These actions have a certain meaning that may, or may not be, iconic for people with a certain personality. Hence, these digital traces can be tracked to construct an image of one's personality. For example, AI has been deployed to automatically determine a user's personality based on: the apps someone has installed [26], someone's behavior on social media platforms [4], and someone's call and messages logs [6]. These unobtrusive approaches are typically categorized as "Digital phenotyping, behavioral sensing, or personal sensing" [17]. Unfortunately, the current state-of-the-art in digital phenotyping in mHealth is insufficiently accurate to predict one's personality reliably [18].
In this study, we aim to combine classical personality tests with digital phenotyping techniques in order to explore the personalities of users of an mHealth tool. Particularly, we aim to increase the accuracy of digital phenotyping techniques, by enriching these unobtrusive learning methods with user reported survey responses of highly trimmed down personality tests.
Our findings demonstrate the relevance of investigating more subtly how hybrid approaches can be developed which balance well the user efforts with the predictive power. Since specific recommendations are largely dependent also on the quality of the data set that one has available to train a predictive model, our study aims to contribute also a mathematical model for rationalizing the design process of these new hybrid systems.
We showcase and validate our method using the GameBus platform, but the techniques are applicable more broadly: our proposed method can be implemented in any mobile application which (1) offers pages that have a strong link to psychological traits, and (2) which offers functionalities for prompting the user with short surveys. In this case study, we use user event data and a multilayer perceptron (MLP) neural network to have an estimation of two probability trait profiles of each user. Then, based on these predictions a limited number of questions will be asked to predict the user's personality. Results show that the proposed hybrid approach is promising and deserves more research attention.

BACKGROUND 2.1 GameBus
Recently, mHealth tools have been changing the face of healthcare. The rapidly increasing number of mHealth apps available in the market, clearly reveals that mHealth plays an important role in increasing health literacy, health management and patient engagement. It does remain a scientifically challenging question how one can design mHealth tools in a manner that users remain engaged over long periods of time.
GameBus (www.gamebus.eu) is an mHealth platform that uses gamification techniques as classified by Hamari [16]: it provides elements such as 'challenges', 'points', 'goals', 'progress', 'leaderboards', and 'rewards' in non-game contexts for promoting health activities in a playful manner. The GameBus platform is designed to persuade its end-users to a healthier lifestyle. Depending on the challenges chosen to take, different types of healthy activities -from social, to mental, to physical -are rewarded with virtual points. Such points can be linked to individual or team-based competitions, but also non-competitive goal tracking is supported. Moreover, Social support elements such as newsfeeds, chats and likes and comments can be employed for behavior change. The academic platform has already been used to support various controlled experiments that aim to show statistically the effect of certain platform configurations on user engagement [19,20]. It turns out that results often depend on personality traits of the study participants [21]. Personalization based on users' traits in GameBus can be applied to persuasive messages, often called nudges [9]. It is also possible to personalize the content, offered challenges, and design of the App in accordance with their personality. Therefore, we are preparing a dynamic variant that tailors the platform configuration based on user traits. Such plasticity in user experiences based on personal traits has been described in the context of marketing [22] but it seems not explored for health promotion well and the current stateof-art approaches are not accurate enough in mHealth [18]. Trait modeling can be done via a variety of models. In a gamification context, for examples, one could use the HEXAD model [25]. In the following section, we clarify a more established model from general psychology.

Big-Five Framework for Modeling User Preferences
The most common personality framework is the Big-Five [14], and there are several mappings framework between users' personality trait and actual (game) content [1,25]. Personality testing in accordance with the Big-Five personality traits consists of five traits including extraversion, agreeableness, conscientiousness, neuroticism, and intellect representing personality at the broadest level of abstraction. Mini-IPIP [8] is one of the most commonly used short inventories measuring an individual on the Big Five dimensions of personality. Although Mini-IPIP with 20 questions provides a short-form IPIP survey delivering an acceptable proxy for a person's traits, it may still be felt like a burden since it typically takes 5 minutes to fill out. Although we contextualize the research using the GameBus app primarily because we have full data access for scientific research in compliance with the GDPR, this proposed method can be used by other mHealth platforms that collect similar data. Moreover, this hybrid approach is applicable to any psychological model.

METHODOLOGY
Construction of user's trait profile is considered as one of the main challenges in personalization [3]. User preference as one of the most important aspects of user satisfaction should map into personality characteristics. Therefore, the user's personality as the users' most representative of personality trait makes this as a classification problem. The main goal of this study is to evaluate users' preferences from their user event data in order to reduce their effort by answering fewer survey questions. The user event database is a collection of behavioral and analytics data that represents how users act in the app. Big-five survey results are considered as the ground truth for one's actual preferences. We collected such data for pilots that enable us to train and test machine learning models such that future users do not have to fill out such long surveys. Specifically, we start from a 20-item Mini-IPIP survey (Appendix B) and aim to further reduce the number of survey items without significantly compromising the predictive power of the questions that remain. Figure 1 is the graphical representation of the study design and general user journey when using the digital phenotype platform within mHealth applications.

Data Collection
Data for this study was reused from a previous study [19] which evaluated the impact of competitiveness in a mHealth setting for 12 weeks among first-year high-school students. In total, 313 unique participants -including 290 students, and 23 teachers -have joined the study. Consent was collected to use the pseudonymized data for scientific studies on mHealth behavior change. Participants also were invited to complete a Mini-IPIP survey personality test in accordance with the Big-Five personality traits leaving a total of 94 participants in the dataset. Regarding event data, a total of 35 input parameters were derived. These parameters are fully listed in Appendix A and some of them are mentioned below x 1 : number of days a user has been online in the given period (excluding the automatically tracked events) x 2 : number of days a user has been tracked using automatic trackers x 3 : number of times a user has viewed their personal activity log . . .
x 7 : number of times a user has viewed their profile . . .
x 32 : number of points a user has collected x 33 : number of unique challenge rules a user has performed x 34 : number of unique activity types a user has performed x 35 : number of unique activity types a user has performed that were rewarded.

Data Preparation
As clarified in the introduction, our proposed method aims to classify users in psychological profiles in order to personalize the user experience and optimize user engagement. When aiming to reduce dropout rates, that classification should happen as fast as possible, with adequate accuracy. Therefore, we analyzed empirically different scenario's (ranging from classifying already after one day to classifying only after multiple weeks). The study took place in 12 weeks and datasets as it is expressed in the form Eq. 1) they were prepared based on daily intervals T from users' data. Datasets include both the values of input parameters X i,j (user event data) and the actual degree of belonging to the Big-Five classes y i,l for all participants that completed the survey.
where i represents users (i = {1, 2, . . . , 94}) , j is the feature item number (j = {1, 2, . . . , 35}) , l denotes the Big-Five traits group (l = {1, 2, . . . , 5}) . In all datasets, two types of normalization were applied to each of the input variables x i,j . As it can be seen in Eq. 4) ; first, to improve the scalability of our analysis, input parameters (user event data) for each user within these datasets were divided by the number of days that person has been online (x i,1 ). Second, all user event data were transformed via z-score for better normalization and comparisons. In this equation n stands for the total number of users (n = 94).
To normalize the Big-Five survey data against data that represents Dutch society and categorize each user into one of the five trait groups, the score of each individual participant is compared with the average µ nl and standard deviation σ nl score of people in the Netherlands (see figure 1 and table 1 ) (i.e., average scores and standard deviations of personality traits for the Dutch populations, we made use of data of the LISS (Longitudinal Internet Studies for the Social sciences) panel administered by CentERdata (Tilburg University, The Netherlands) [24]) to the Big-Five personality test by computing the Z-score for each trait group l (Eq. 5) ). Consequently, user i belongs to trait group Y i which has higher value among other trait groups (Eq. 6) ).

Optimum Time Frame and Key Features.
One reason why user might abandon an app is we were not able to deliver personalized experiences that fit into their needs. The previous study shows most of the users abandon the app after only two weeks [19]. Also, analysis show users' activities reached their peak on the third day, but then started to decrease significantly. Hence, the decision came down to a trade-off between the accuracy and observation time.
Three days was considered as the optimum minimum time span required to have a good balance of the estimation accuracy and time. This time span can be considered as the golden time to engage people and make a personalized intervention. Feature selection is one of the most important concepts in Machine Learning techniques to eliminate redundant variables and keeps only the best subset of predictors in the model. In the first phase, after categorizing users in their trait groups, average and standard deviation values of the input variables xi,1 to xi,35 (features) for users in the same group were calculated. Based on the standard deviation and dispersion of features, the algorithm selects three features with a higher value of standard deviation between the average of that variable within groups. In another word, this filter method follows the reasoning scheme to choose variables which their value being close within their group but have a significant distance from the other group.

Inactive
Users. The second set of statistical analyses was focused on the evaluation of the users' activities. A respondent labeled as an inactive user if their key user event data have not changed during the study period of 12 weeks. It means these features have zero standard deviation during this period.   and softmax function for the output layer are supplied along with the number of nodes. Since this problem is a type of single-label multi-class classification, categorical cross-entropy was applied in its settings rather than binary cross-entropy as the loss function of the model. Therefore, the proposed model gives a dispersion of probabilities between all five traits groups. Two categories that receive the highest probability will be the output for the model for the next step.

Selective Questions.
By predicting two possible personality groups for each user from the previous step, users' mini-IPIP test scores on those two personality groups are considered. In practice, it is equivalent to just 8 questions out of 20 questions of the Mini-IPIP test that are asked from a user according to the predicted probable personality traits. In the next experiment with the aim to decrease the burdensome of the test, even more, we reduced the number of questions to 6 questions which includes 1 question with a positive keyed direction and 2 questions with negative keyed directions for each group. These questions were selected based on the confirmatory factor analysis (CFA) of the Mini-IPIP by choosing the highest values [5]. Confirmatory factor analysis is a popular powerful statistical tool in social research for evaluating latent variables. This analysis measures whether a prespecified factor model (hypothetical constructs) provides a good fit to the data or not. Consequently, questions number 6, 11 and 16 for extraversion, 2, 7 and 17 for agreeableness, 3, 8 and 18 for conscientiousness, 4, 9 and 19 for neuroticism, 5, 15 and 20 for intelligence were selected (table 2). As can be seen from table 3, in the next step, the number of questions is reduced to 4 questions by using confirmatory factors and choosing the highest positive and negative keyed directions in each group.

Cost function.
Since this study uses the standard mini-IPIP questions to serve as ground-truth representing users' scores on   each of the Big Five dimensions, cognitive load is not contributing to our decision and the only factor that may have the greatest impact on the respondent burden is the length of the questionnaire. Therefore, to better visualization, we defined users' effort and model accuracy as the cost function, to make a balance between users' effort and the accuracy of the model. In Eq. 7 n is the number of questions and a is the accuracy of the model.

RESULTS
As previously mentioned, the main objective of this paper is to predict users' digital phenotype as the first step of personalization to create an optimized mobile app user experience based on users' trait. To ensure that the accuracy of the model persists, a 5-fold cross-validation method is applied for all the experiments.

Features Selection
Three features are selected based on their scores in the first statistical analysis. These features, as presents in table 4, are x i,3 , x i,7 , x i,32 which respectively are the number of times a user has viewed their personal activity log, the number of times a user has viewed their profile, and the number of points a user has collected. Table 5 presents the average and the standard deviation of these features in all trait groups.

Active Users
Analyzing users based on the standard deviation of their key event data during these 12 weeks shows about 40% of participants who filled the survey (30 out of 94) were not active in that study at all.  By comparing users in each trait group, it appears that in three out of five categories we have fewer (active) users (see figure 3 and figure 4).

MLP Classifier
Initially, an MLP neural network was used to assess the feasibility of determining users' personality traits just using user event data. Considering the highest probability in groups as the predicted personality trait, the balance average accuracy of 42.7% (±7.7%) and the F-score 32.6% (±11.8%) for all 5-folds was obtained. Other scores of using MLP in predicting each trait group is summarized in table 6.
As discussed earlier, this paper proposes a model to combine user event data and ask some selective questions with the aim of reducing the response burden of the Mini-IPIP test and having a more accurate model for predicting user personalities. Therefore, by considering the two groups with the highest probability and rating those based on their scores to the questions for the predicted groups, the accuracy increases by 30% compared with using just MLP. Although the decline in the number of questions to 6 reduces the burden of the users by 10% more, the accuracy of the model slightly affected. Consequently, by using the defined cost function (Eq. 7) ) it is concluded asking 8 questions would be better. Table 7 shows the accuracy of proposed model for each trait group with 8 and 6 questions.

Evaluation of Model Performance
The values of accuracy are quite stable in all five folds of crossvalidation. Hence, promising results are achieved for the prediction  of the digital phenotype by the proposed framework. To test the effect of asking smart selective questions, the model was tested by random questions. It was ensured that at least one question from each group was asked about that type of personality. The accuracy in this scenario drops to the 33.7%(±5.6%) .

Performance Analysis
To analyze the performance of the proposed model, it was tested by a set of 14 users' data representing in table 8. Moreover, users' personality was defined by the Mini-IPIP test as the ground truth table. The results of this test appear in Tables 8 as well. In the first step, an MLP classifier is used to predict the test groups' traits. Figure 5 shows confusion matrix of using MLP classifier and as can be seen the accuracy of using user event data without asking any question is 43%. Also, precision, recall and F-score, in this case, are 25%, 43%, 31% respectively.
To follow our goals, instead of using binary values in classification with MLP, we use predicted probabilities that of to what extent a user belongs to each trait group. Therefore, Z-score for related questions to the two groups having the highest probabilities is calculated. In this method, the accuracy of using hybrid technique, as it is shown in table 9 and figure 6, is 71.43% which is about 30%higher than using the MLP neural network.
By reducing the number of questions to 6, the performance of the model dropped. Figure 7 shows the confusion matrix of that and table 10 clearly shows that when users are asked 6 questions, the result is worse than when they are asked 8 questions.

DISCUSSION AND CONCLUSIONS
Quickly and accurately learning the personality traits of mobile app users is challenging. Our results demonstrate the potential of combining validated survey methods with unobtrusive trait modeling based on digital footprints. Specifically, we demonstrated a significant reduction in the number of survey questions that users need to answer during an onboarding process while maintaining acceptable accuracy. Unfortunately, our retrospective case study does suffers from the lack of samples in three trait groups leading to low accuracy for those groups. Others applying our proposed method on their own data set may experience similar yet slightly different data set limitations. Therefore, we also contributed a formal approach to rationalizing the engineering process for hybrid systems.
Results of using MLP for users' trait classification show that for the two groups for which the retrospective data set has most active users, the accuracy and corresponding F-score are highest. Therefore, hopefully increasing in number of samples may help the model to obtain better results. Having Fewer samples in the three groups cannot be a tangible proof or adequate reason that most users have an extroverted or Intellect personality. Rather, it is quite probable that the users who did not participate in the survey can belong to those three groups whom for some reason we could not encourage them to fill out the survey.
To improve reliability and prediction accuracy for determining users' digital phenotype, a limited number of questions based on estimated personalities from users event data is asked. The proposed model has established a good interaction between the simplicity of using user event data and the accuracy of the conventional survey methods. In light of this, the optimal solution occurs when users are asked by 8 questions subsequently. It means we reduced the burden of the survey by 60% and we still have acceptable accuracy in our prediction. Even when users do not actively participate in the survey, by using user event data exclusively, we would reach only about 50% accuracy. Although diminishing the number of questions to 6 reduces the burden of the survey by 10%, it reduces the performance of the model by the same level as well.
The retrospective case study was based on a health gamification app. In that setting, digital traces included variables such as the number of times a user has viewed their personal activity log, the number of times a user has viewed their profile, and the number of points a user has collected. These variables aided to predict significant differences in users with different personality traits. On one hand, users in the Conscientiousness group are more loyal to the platform; on the other hand, the app has not engaged the interest of users with agreeableness and extraversion trait to use the app and they have a higher rate of dropout.
Running a trial on the impact of personalization of challenges and users' experience would be an opportunity to have a better understanding of the effect of it in the context of mHealth tools. More broadly, having a good proportion of samples in all traits' groups could improve the accuracy of the proposed model.

APPENDIX
A EVENT DATA x1: number of days a user has been online in the given period(excluding the automatically tracked events) x2: number of days a user has been tracked using automatic trackers x3: number of times a user has viewed their personal activity log x4: number of times a user has viewed their friends' activity log x5: number of times a user has viewed the details of an activity x6: number of times a user has viewed their newsfeed x7: number of times a user has viewed their profile x8: number of times a user has viewed their friends' profile x9: number of times a user has viewed the details of (one of their)circles (= teams) x10: number of times a user has viewed their overview ofchallenges x11: number of times a user has viewed the leaderboard of a particular challenge x12: number of times a user has viewed the leaderboard of a particular circle (= team) within a particular challenge x13: number of times a user has given support to others x14: number of unique players a user has supported x15: number of times a user has received support from other players (i.e., not including the number of times a user has supported herself) x16: number of unique players that have supported the user (i.e., not including the user herself, in case she has supported herself) x17: number of reactions on activities a user has made x18: number of unique players a user has reacted on (i.e., a reaction on their activities) x19: number of reactions a user has made on her own activities x20: number of reactions a user has received on her activities, not including the reactions she has made on her own activities x21: number of unique players a user has received reactions from on her reactions, not including herself (if she has made a reaction on her own activities) x22: number of circle (= team) chat messages send by a user x23: number of unique circles a user has sent a chat message to x24: number of circle chat messages a user has received, to including the chat messages she has received from herself x25: number of unique players that have sent the user a circle (= team) chat message, excluding the user herself (if she has sent any messages herself) x26: number of unique circles (=teams) that have sent the user a circle chat message x27: number of activities a user has performed (both manually and automatically registered) x28: number of activities that were automatically registered for a user (for example via Google Fit, Fitbit, or any other automatic tracker) x29: number of activities (both annually and automatically registered) that were rewarded by a challenge x30: number of automatically registered activities that were rewarded by a challenge x31: number of challenges a user was enrolled in x32: number of points a user has collected x33: number of unique challenge rules a user has performed x34: number of unique activity types a user has performed x35: number of unique activity types a user has performed that were rewarded