{'fraction_non_alphanumeric': 0.028228460793804453, 'fraction_numerical': 0.009409486931268151, 'mean_word_length': 4.98239518183924, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
If the Prospect of Some Occupations Are Stagnating With Technological Advancement? A Task Attribute Approach to Detect Employment Vulnerability

Two distinct trends can prove the existence of technological unemployment in the US. First, there are more open jobs than the number of unemployed persons looking for a job, and second, the shift of the Beveridge curve. There have been many attempts to find the cause of technological unemployment. However, all of these approaches fail when it comes to evaluating the impact of modern technologies on employment future. This study hypothesizes that rather than looking into skill requirement or routine non-routine discrimination of tasks, a holistic approach is required to predict which occupations are going to be vulnerable with the advent of this 4th industrial revolution, i.e., widespread application of AI, ML algorithms, and Robotics. Three critical attributes are considered: bottleneck, hazardous, and routine. Forty-five relevant attributes are chosen from the O*NET database that can define these three types of tasks. Performing Principal Axis Factor Analysis, and K-medoid clustering, the study discovers a list of 367 vulnerable occupations. The study further analyzes the last nine years of national employment data and finds that over the previous four years, the growth of vulnerable occupations is only half than that of non-vulnerable ones despite the long rally of economic expansion.


Introduction
Unemployment has been a crucial issue for governments as well as policymakers around the world for quite a long time. It has a consequence not only for the social, political and economic life of individuals but also for the overall economy and development of a country (Dickens et al. 1994). The social and political enterprise of people surrounding an unemployed person are also adversely affected. (McClelland et al. 2000). As unemployment has extensive ramifications in different policy areas ranging from welfare, education, tax and investment to health, crime, and other social issues as well as to political stability, both federal and state governments are keen to address this problem with all the tools within their arsenal, i.e., fiscal and monetary policies and other administrative strategies and schemes. However, without an effective and robust prediction mechanism concerning future employment scenarios, such policies would be pointless.
Therefore it is necessary to have a deep understanding of why unemployment occurs in the first place. There could be multifaceted reasons. According to Krugman (1994), the causes of unemployment can be separated into two broad categories: cyclical and structural. The former depends on fluctuations of aggregate demand and the latter on demographic shifts, changes in labor market institutions, or technological innovations.
Unlike cyclical unemployment, which is typically short term, easily predictable, and dependent on market forces, structural unemployment is longer lasting and caused by different externalities which may or may not be directly related to economic indicators (Mocan 1999). Such externalities could be policy shifts, globalization, or new product and process innovations. When there is an initiation of new business or production process, a severe skill mismatch can emerge; companies need competent workers, but the existing workers lack the necessary skills (Manacorda et al. 1999). As it takes substantial time to learn new skills, structural unemployment can last for a long time unless some radical initiatives in training and education are taken.
Lack of skill is not the only reason for structural unemployment. Sometimes a whole region can be affected because of a lack of farsightedness and vision from governments and policymakers. Fagerberg et al. (1997) investigate the causes of structural unemployment in different European countries and find that it prevails more in those countries with less investment in research and development. In these countries the diffusion of technology in production and manufacturing sectors is limited.
We can find a trace of structural unemployment in the US. As of the latest 'Job opening and labor turnout summary' published by BLS in November 2019, the number of job openings is 6.9 million, hires is 5.8 million, and separations (quits, layoffs, and discharges) is 5.3 million. The situation is there is a job for everyone, but still, there is a persistent unemployment rate of 3.8%. Most of these unemployed people are active job seekers but without any success. This scenario is a telltale sign of structural unemployment. For the first time since this Job Openings and Labor Turnover Survey (JOLTS) has been published, the ratio has fallen below 1, meaning more number of jobs are offered in the economy than the number of unemployed persons. The shaded area represents recession, as determined by the National Bureau of Economic Research.
There is another way to find out if an economy is facing structural unemployment, the Beveridge curve which shows an inverse relationship between labor demand and labor supply over time. The curve plots job opening rates on the vertical axis and unemployment rate on the horizontal axis. A shift of this curve to the right provides evidence of structural unemployment (Nickell et al. 2001). We can detect an apparent shift of this curve from early 2010 to the end of 2018 comparing to the 2000-2009 period. With further observation, we identify that in the last few months of 2018 the unemployment rate was lower than the job openings rate, confirming the existence of structural unemployment. There is thus a clear indication of skill mismatch between what employers want and what prospective workers can provide.

Causes of structural unemployment
There have been many efforts to find the cause of structural unemployment, but none of them has provided conclusions that hold true under every scenario or time frame. The most popular attempt is called the Skill-Biased Technological Change (SBTC) hypothesis, according to which the diffusion of technology increases the demand for high-skilled and educated workers and less-skilled workers become underemployed or sometimes completely out of employment. (Acemoglu 1995). SBTC involves a shift towards capital intensive production techniques that bolstered relative wages of skilled laborers by enhancing relative productivity. This phenomenon was conspicuous in the years after the 2nd World War when the widespread application of technology in production and manufacturing became commonplace (Violante, 2016).
A 'hollowing out of the middle' effect introduced doubts concerning the validity of SBTC theory. The overall demand for jobs is not perfectly linear with skill level and the demand for workers with mid-level skill or mid-ranked jobs has been declining in overall market share since the late 1970s relative to low-skilled and high-skilled workers, as shown by figure 3 and 4.
This phenomenon of job polarization could not be explained by the SBTC hypothesis. To overcome this drawback, Autor    The hypothesis postulates that routine-manual jobs are the most vulnerable and jobs that demand cognitive input and non-routine maneuvering are the least vulnerable to new and more advanced technology in the production and service sectors. Figure 5 supports of their claim. While Autor's hypothesis explain why the number of middleranked jobs is declining they have become irrelevant with the advent of Artificial Intelligence (AI), Machine Learning Algorithms, and high-end, agile robots. The tasks the new machines and algorithms perform cannot be defined by limited routine, non-routine or cognitive, manual task metrics. To predict job vulnerability due to this latest technology, it is necessary to take into consideration not only the skill level of workers or task constituents of a job, but all job requirement variables, i.e., abilities, capacities, experience, training and education, job context, etc. (Frey et al., 2017). A holistic approach has to be taken to grasp the true impact of the latest trends in technology on the overall job market.
There has been no recent attempt to pinpoint which jobs are susceptible and which are not, taking into account this very recent trends in innovation i.e., AI and Robotics. The previous literature has tried to detect vulnerability by sector (manufacturing, service, agriculture), or industries (health, education, finance, management, etc.) or types of jobs (routine or non-routine), or skill requirement of jobs (high or low) We know of no undertaking that investigates why the prospect of some employments are diminishing over the past several years despite the long rally of economic expansion. This study would like to shed some light on this issue in the line with 'task-attribute' approach, which is an amalgamation (broadly) of the classical hypotheses and present-day analysis and findings.

Datasets
For the study, two sources of datasets have been used, Bureau of Labor Statistics (BLS) and Occupational Information Network (O*NET). BLS provides full spectrum employment data from all the USA states and its territories. It also categorizes the data in terms of sectors and industries. This study has used employment data from 2010 to 2018. It is only concerned with the total number of national-level employment from each SOC (Standardized Occupational Code) and how its distribution changed each year. O*NET contains detailed occupational definitions of all the SOC-coded employments of BLS. O*NET explains all the jobs using more than 180 job features, under the broad categories of abilities, context, skills, knowledge, interests, work style, and work activities. The defining process is extremely elaborative and exhaustive. For each job and corresponding features, O*NET provides with an importance level of the feature for that particular job. In Figure  6, the study shows a standardized importance score of selected features for some occupations. The researchers interview people from different professions and ask them questions, both general and specific to their respective professions. Then they create a Likert scale ranging from 0 to 5. A score of 0 means that attribute is not at all important for that occupation and a score of 5 means very important. Based on these surveys they come up with an importance level for a particular attribute of a particular occupation. After that, the scores are standardized following a general formula. For example, we take the attribute 'speaking' for a lawyer and a paralegal. For both the professions, it is a very important requirement. But, their importance level is different. For lawyer, it is 70 and for a paralegal, it is 50. As these interviews are taken every year, any change in the importance level of an attribute of a particular occupation is reflected in the database within a year.
Cleaning and processing of datasets are imperative steps to make sense of these data. The way O*NET stores the information is difficult for comparison. The importance level of all the attributes (more than 180 variables for most of the occupations) is scattered over 1,110 files. PYTHON scripting facilitates downloading and merging these files. Because not all of the attributes are applicable to AI and robots, a subset of the whole dataset is created. As each attribute has an individual ID, it is used as the primary key to create a relational database using MYSQL.
For this analysis, all the occupations that are present in both O*NET and BLS databases and with relevant SOC codes are considered. After leaving the occupations with missing data, the study is left with 966 occupations. These occupations are the observations for this study.

Variables selection
Most of the relevant variables are initially chosen from the 180 odd variables listed in occupational characteristics based on the 'engineering bottleneck' concept. According to this concept even though AI and robots can perform different nonroutine cognitive and non-routine manual tasks, contemporary technology still falls short of expectation when it comes to such tasks as perception and manipulation, creative intelligence, social intelligence, etc. (Frey et al. 2013). The basic premise of the hypothesis is that there are some choke points that cannot be resolved by the current level of technological development when it comes to imitating human behavior, skill, and intuition that are critical for performing some occupational tasks, i.e., judgment, coordination, social perceptiveness, teamwork, etc. On top of these 'bottleneck' variables, we take into account contemporary research and articles on AI and robotics to decide on the occupational attributes that are going to be or already being emulated by AI or robots with higher precision and effectiveness, i.e., pattern recognition, repetitive motion, manual dexterity, etc. These attributes fall into routine work category, i.e., not much critical thinking, complex problem solving, or originality is required, rather following a well-defined set of rules is enough to perform the job properly. The database provides 41 attributes, which are either very difficult to emulate (engineering bottlenecks) or already being in the process of replacement (routine and repetitive) by AI and robots. There are some contextual attributes of some professions that could be detrimental for human lives, i.e., working in hazardous conditions, exposure to contamination and radiation, etc. As no one can take chances with human lives, these kinds of jobs are sure to be replaced by robots as soon as  Table 6, gives an example of some variables as per these three types of broad features. Finally, our database now constitutes a 966X45 matrix reduced from the initial 1110X180 matrix due to data availability and relevancy issue.

Data reduction using Principal Axis Factor Analysis
After merging and cleaning the dataset, 966 occupations remain, along with the 45 necessary and relevant variables scaled by their respective importance levels. As many of these variables were correlated (both Barlett sphericity and Kyser-Meyer-Olkin tests are performed), a principal axis factor analysis (PAFA) is undertaken to isolate the independent dimensions of variation. Principal axis factor analysis is a statistical technique that helps to reduce a large set of independent variables to a smaller and meaningful set of summary variables. It assists in exploring the latent construct of a phenomenon that may be present in several independent variables simultaneously.

Number of factors
Before performing the factor analysis, the study has performed 'Parallel Analysis' to find the optimum number of factors. In this case, the number could be from 6 to 8.
After performing the analysis, the best result is 7 factors, a solution that has the highest Tucker-Lewis value of 0.83. The root mean square of residuals (RMSR) is 0.02. This is acceptable as this value should be close to 0. The RMSEA index is 0.092, and BIC is 1477.06. Both these values are better than factor results with other factor numbers.  Figure 8 shows how the original variables loaded onto each factor in the 7-factor orthogonally rotated PAFA solution.

Interpretation of factors
We interpret the six basic skill-dimensions as follows: Problem-solving and repetitive work At one end, this factor involves spotting complex problems and assessing relevant information to develop and evaluate different options and implement the best solution. It also requires the performance of cost-benefit analysis of potential actions using logic and reasoning. This factor is important for identifying the strengths and weaknesses of alternative solutions, conclusions or approaches to problems. At the opposite end of the scale is repetitive work, doing a 'patterned' job that doesn't require out of the box thinking. Such jobs can be very easily codified, as there is not much dispersion from the linear and well-defined job description. As per the Occupational Safety and Health Administration (OSHA), "a highly repetitive job can be characterized by one of the following: A cycle time less than 30 seconds. Over 1,000 parts per shift, or more than 50% of the cycle time involving the same kind of fundamental cycle". Repetitive work can be harmful to a worker as it could be instrumental to musculoskeletal disorders. (PSHSA, 2010)

Negotiation and leadership capacity
This factor involves getting others onto the same page and trying to bridge the gaps in opinions and attitudes. It also requires adjusting one's actions in tandem to the counterpart's actions. Sometimes it could also demand to persuade others to change action or behavior as per the changing contexts. This factor is also important in handling and resolving complaints, discontents, and grievances.

Exposure to hazard
This factor requires doing a job that is exposed to contaminants (such as pollutants, gases, dust or odors), radiation (gamma, X-rays), risky machineries (moving parts, overhead cranes), very hot (above 90 F degrees) or very cold (below 32 F degrees) temperatures, loud noise (construction sites). Some jobs involve working in a high, uneven places (tree pruning, roofing). Few professions i.e., soldiers, deep-sea explorers, and firefighters are inherently exposed to various vulnerable scenarios. All such jobs that are exposed to hazardous conditions require a high level of depth perception (distinguishing between near and distant objects) and manual dexterity (agile body movement).

Empathy
This factor means being conscious of others' reactions and grasping why they react this way. It also requires being sensitive to others' needs and accommodating any legitimate demand. This factor involves being amiable to others on the job and demonstrating a cooperative, gracious attitude. It entails deep comprehension of human behavior and conduct and individual differences in ability, performance, personality, and interests.
Artistic ability This factor involves researching and developing new applications, processes or products. Sometimes it also requires the worker to design, create and implement unique, state-ofthe-art ideas and schemes.
Coordination/Leading capacity This factor entails engaging everyone to pursue a common goal. Coordination involves funneling and optimizing everyone's effort to achieve the target of the team as a whole. This factor helps in creating synergy among all the team players so that they can deliver what is expected from them. Knowledge over other team member's limitations and strengths is also a key component of this factor.

Susceptibility to AI and Robotics
If we observe closely we see that the factor 'Hazard' i.e., risk exposure (Ford 2015), and dexterity (Strickland 2016) is the most susceptible characteristic in terms of the scope and capacity of modern AI algorithms and robotics. Factor 'problem solving' displays an inverse relation between problem-solving and robotics-susceptible repetitive motion.
Occupations with low or negative scores on this factor are vulnerable. Other factors i.e., negotiation capacity, empathy, coordination and artistic are not vulnerable. (Russel et al. 2016). They are extremely difficult for AI and robots to emulate with the current level of research and development. This study then focuses on these susceptible factors to derive the list of occupations that are most likely to be affected by contemporary technological innovation. The list is created by taking the top 20 percent of occupations that scored high on the 'Hazard' factor and the bottom 20 percent of occupations that scored low on the 'Problem-solving' factor. Seven types of susceptibility are found based on these factors. (Figure 9)

Conditioned Vulnerability: K--Medoid Clustering
Not all occupations with similar vulnerabilities are equally at risk from technological change. Vulnerabilities need to be adjusted by accounting for other factors. For example, we can take the cases of barbers and surgeons. Both these professions are heavily dependent on dexterity with factor scores of 2.85 and 1.55 respectively. These occupations might be at risk due to cutting-edge development in robotic arms. But to perform the job of a surgeon unlike a barber, other factors are also critical. If we take into account other occupational attributes (critical thinking, coordination, judgment and decision, empathy), we can understand how distinct these two professions are in terms of replicability. We, therefore, need an adjusted classification of occupations that merges vulnerability with conditioning characteristics. For that we must take into consideration the 'bottleneck' attributes that are difficult to emulate.
To that end, we use k-medoid clustering (PAM, as Partitioning around Medoids), in the full 7-factor space that took into consideration all the relevant factors, vulnerable and 'anti' vulnerable.

Optimum Number of Clusters
To proceed with the k-medoid clustering procedure, we have to first decide on the number of optimum clusters. The fundamental characteristics of any clustering algorithm are that distance (similarity) of elements of the same cluster should be minimum and the intra-cluster distance (dissimilarity) should be maximum. (Rousseeuw P. J., 1987). The 'Silhouette Coefficient' technique is used to derive an initial number of clusters in the 6-space. Silhouette, s(i) is calculated using the following function: We can see silhouette value is highest for 7 number of clusters. If we take more than or less than 7 clusters silhouette value decreases. (Figure 11) Figure 11: Optimum number of clusters

K--Medoid Clustering
This algorithm (known as Partitioning around medoid) was first proposed in 1987 by two famous mathematicians, Kaufman and Rousseeuw. It is more robust to outliers and noises (Han et al. 2011) than the more popular clustering algorithm k-means clustering. While k-means uses a mean point as the center of the cluster (which may not be a real point in that cluster), k-medoid uses an actual point (medoid) or member in the cluster to categorize it. A medoid of a particular cluster is the most representative element of the cluster, i.e., a medoid's similarities with the other member of the same cluster is the minimum.

Procedure for K--Medoid Clustering
K-medoid initiates with selecting a random element for each number of cluster, i.e. k number of medoids for k number of clusters. All other elements are included in the nearby cluster in terms of the features or factor scores for the current scenario. After the initial point, a new medoid is chosen for every cluster and it is tested against the current medoid to see if the value cost function (dissimilarities between each data item and its corresponding medoid) is less or more. If cost is less than the previous medoid than the new point becomes the medoid for that cluster. After a sufficient number of iterations, the algorithm converges

K--Medoid Result
Of the 8 clusters of occupations, we can find only 3 of them have an immediate vulnerability because these clusters have high factor scores on susceptible factors or very low scores on 'bottle-neck' factors. (Figure 14). On the contrary, 5 clusters have high factor scores in more than 2 'bottle-neck' factors making them not immediately vulnerable. (Figure 15). There are altogether 408 occupations in these 3 clusters. A list of the occupations is provided in the appendix. One thing we must understand, here no attempt is made to make a ranking of vulnerability, i.e., high-mid-low vulnerability. Rather the study just looks at the vulnerable factors and segregates the occupations in terms of factor scores using k-medoid clustering algorithms.
A cursory look of the occupations gives us the idea that most of the jobs are low paid, highly repetitive, low skilled, and doesn't require more than a high school degree. Most of the occupations are from mining, agriculture, transportation, retail, and manufacturing industries. On the other hand, not surprisingly health care, hospitality, and education industries seem to be immune to these trends of prolific application of AI, and Robotics in workplace. Professional and business industries and state and federal government employees are also not so vulnerable.

Real Job Trends
The study further looks at the change of a number of jobs from all the occupations mentioned both in O*NET and BLS for the past 8 years. Initially, I measured the change in demand for jobs for all occupations. The study found that on average, the increase in jobs for the vulnerable occupations was only a little Figure 14: Factor scores of non-vulnerable clusters over 1%, whereas the increase in demand for the nonvulnerable jobs was almost 2.4% in these years. When I looked at this trend further, I noticed something interesting. I detected that the number of jobs that were more affected by AI and ML algorithms was decreasing at a higher rate (1.8% precisely). But, many of the vulnerable occupations that would mostly depend on dexterity or other 'Robotic' input were increasing, i.e., truck driver, firefighter etc. Again, not surprisingly number of warehouse helpers decreased two folds during the same period. My conclusion was that, even though current trends in robotics might put many jobs in a vulnerable condition, the threshold point of technology to replace human labor is still far away, and as of now, robots can replace them only in a 'controlled' environment.

Conclusion
We can see the demand for quite a handful of occupations is decreasing over the past several years. As, the US economy has not been experiencing an economic downturn, or no major trade agreement or disruptions has taken place during these years (trade war with China is rather a contemporary issue