{'fraction_non_alphanumeric': 0.02204498400150574, 'fraction_numerical': 0.00564652738565782, 'mean_word_length': 4.75870478254979, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
TrustSECO: An Interview Survey into Software Trust

The software ecosystem is a trust-rich part of the world. Collaboratively, software engineers trust major hubs in the ecosystem, such as package managers, repository services, and programming language ecosystems. This trust, however, is often broken by vulnerabilities, ransomware, and abuse from malignant actors. But what is trust? In this paper we explore, through twelve in-depth interviews with software engineers, how they perceive trust in their daily work. From the interviews we conclude three things. First, software engineers make a distinction between an adoption factor and a trust factor when selecting a package. Secondly, while in literature mostly technical factors are considered as the main trust factors, the software engineers in this study conclude that organizational factors are more important. Finally, we find that different kinds of software engineers require different views on trust, and that it is impossible to create one unified perception of trust. Keywords: software ecosystem trust, empirical software engineering, TrustSECO, external software adoption, cross-sectional exploratory interview analysis, trust perception.


Introduction
Software engineers use software packages for creating new solutions from different package managers. [Mojica et al., 2014] [Nguyen et al., 2020]. There is a significant amount of implicit trust in these packages. While the packages could easily be compromised, software engineers assume that as the package comes from a reliable source, it must be trustworthy. This is not always the case [Duan et al., 2020]. Moreover, there are several attack vectors that can compromise a package like registry exploitation of typo squatting [Hou F, 2020].
Before the factors that constitute trust in software packages and package repositories are looked upon, one must look at how software engineers choose software packages and how trust is gained. Moreover, what the impact factors are that influence this. The TrustSECO project aims to uncover all the factors that influence the trust that software engineers have in software packages. In order to uncover these impact factors, a survey will be developed that has as its main aim to uncover how software engineers perceive trust. [Vargas et al., 2020] has done similar research and found 26 factors that influence the selection process of software packages. Rather than looking at the whole selection process, this research will just focus on the trust aspect of that selection process. This will be done by analyzing existing literature and the results of cross-sectional interviews with experts. This information will then be the basis for a large scale survey. This research thus sets out to find what trust factors influence the decision to choose software packages.

Framework 2.1 Trust
One of the most important aspects of collaboration is trust [Bunduchi, 2013]. Moreover, trust creates the basis in decision making for the usage of long term product use. [Cho et al., 2015]. Using external software is just that, collaboration. In order to find out what factors induce trust in packages. One must take a look at the term trust. The term is widely used in computer science and has many different definitions across the spectrum. [Artz and Gil, 2007].
There has been a lot of research on this subject that lead to the following three most general and common definitions of trust [Artz and Gil, 2007].
• "Trust is a subjective expectation an agent has about another's future behavior based on the history of their encounters." [Mui et al., 2002] • "Trust is the firm belief in the competence of an entity to act dependably, securely, and reliably within a specified context." [Grandison and Sloman, 2000] • "Trust of a party A to a party B for a service X is the measurable belief of A in that B behaves dependably for a specified period within a specified context (in relation to service X)." [Olmedilla et al., 2006] The first definition is a reputation based one because it concerns the producer of the software rather than the software itself. While the producer is highly relevant in gaining trust in a software package. The producer of the package will not be more important than the product or service itself. This is because this research looks at what factors induce trust in software packages, not at the factors that induce trust in the entities that develop the software.
The second definition is a definition that suits this research better since it concerns the belief in the competence of the software. Therefore the characteristics for the software have to ensure that it acts dependably, securely and reliably. These characteristics will induce a list of factors that ensure that the system acts this way.
The third and final definition of trust concerns a service from a party to another party. This trust also comes from the belief in the product or service, rather than the party that produces this. Therefore this definition of trust will also suit this research.
The definition of trust that will be leading for this research is: "Trust of a party A to a party B for a service X is the measurable belief of A in that B behaves dependably for a specified period within a specified context (in relation to service X)." For this definition specifies the collaboration of two parties regarding a specific service or product. Thus the trust influence of the developer is not discarded while the focus is on the product, as is the scope for this research.

Bias
Bias is a form of error that can affect research. [Sica, 2006]. For this research, this may occur in the form of selection bias. Selection bias may occur on the participants in the interview if they do not represent the general ideas and thoughts of the average participant. [Hernán et al., 2004] Therefore it is important that the participants have different perspectives and roles on the same matter. This thus leads to participants in different organizations across different fields across different jobs and functions.
Another more intuitive form of bias present will be when several trust factors will be discussed in the interviews, personal bias. After all each participant has their own personal experiences with packages. Minimizing this bias will be next to impossible, since the participant might not even be aware of this bias. However this personal bias will be decreased by first letting the interviewee answer some more technical impersonal questions before diving into the personal part.

Research Methods
To find what factors influence trust in software packages, one must first find all the adoption factors that are considered when making such decision. After this, a selection can be made of the factors that actually contribute to trusting a package. In order to explore the current state of impact factors already present in literature, a literature study will be conducted. These results will be compared with the results from the cross-sectional interviews with software engineers to complement this current literature. This process is shown in figure 3.1.

Literature study
The literature study will be based on the SLR done by the TrustSECO te am. This SLR will to try uncover the already existing factors in literature. Various search queries will be entered in the following search engines:  The combination of these search terms will result in a list of articles. This list will be narrowed down through exclusion and inclusion rules. These are as follows: • Literature should be about open source software • Literature should list at least one impact factors on adopting open source software or one factor for gaining trust in a package • Literature should be public and accessible The last round of elimination will be done through abstract analysis of the articles. This will result in a final list of articles to be analyzed.These articles can be found in appendix 7.1.1 These articles will be scoured for adoption and trust factors and will be categorised as follows: The categorization is based on [Vargas et al., 2020] which holds an explanation as to why a certain factor is categorized as such. Technical factors are factors related to the release process, code quality attributes and the functionality. Organizational factors concern the individual perception, community around the project and other aspects of the organization where the package is developed. The economical aspects cover the financial aspect of package selection like licences, total cost of ownership and risks.

Interviews
The literature review will provide a solid foundation of knowledge to start creating interview questions. Interviews can be used to get detailed personal experiences and thought processes in the selection process [Hiller and DiLuzio, 2004]. These semi-structured interviews shall be conducted with software engineers, DevOps, architects and other experts (see table 3.1) The choice for semi-structured is made since this gives the freedom for the interviewee to really elaborate on personal experiences. In addition, this allows the interviewer to create a new line of questioning based on those experiences [Hove and Anda, 2005]. These interviews are held in English or Dutch. Since the literature provided English factors, a list of translated terms will be provided to ensure that the interviews in Dutch will not yield different results because of difference in understanding in certain terminology across these languages. Prior to the interview, an interview protocol is created based on [Jacob and Furgerson, 2012].
These semi-structured interviews are a form of exploratory research. The subject of exploratory research has been named by among others [Glaser and Strauss, 2017] with the discovery of grounded theory. The goal of exploratory research has been to form hypotheses rather than the testing of hypotheses [Kothari, 2004]. This is the case for this research since the objective is to define a hypothesis describing which factors lead to trusting a software package.
Each interview discusses roughly 9 questions regarding software trust. In order to ensure the participants have the required knowledge to answer the interview questions, certain standards have to be met, namely: • The participant needs to speak English or Dutch fluently • The participant has to have been involved with the selection process of software packages  The responses give insight as to why certain packages are chosen and the factors that contributed to this decision. Moreover to answer: How do software engineers develop trust in a package?
In order to answer this question, the interviews must first provide answer to the following questions: • What factors are important when selecting external software packages, what is the protocol?
• Which of the selecting factors contribute to gaining trust in a package?
• How do personal aspects influence trust in packages?
The first and second sub question may seem alike. However the literature review has shown that the factors that induce trust are a subset of the factors that influence the choice of a package. These sub questions will each be answered through a section in the interview. Each section contains the actual interview questions that the interviewee will be answering. Section one will contain Q1...Q4, section two will contain Q5 and Q6 and section three will contain Q7...Q9. The full interview structure can be seen in appendix 7.2.2.
These interviews are recorded and transcribed. After transcript approval from the participant a proper analysis will conclude the most important trust factors. This will create a list of factors that have quotes to back them up.

Literature study
The literature study will first take a look at what factors influence the adoption of certain software packages, these will be categorized based on the research of [Vargas et al., 2020]. What follows is an analysis of which of those factors actually lead to more trust in a software package. [Sánchez et al., 2018] was found during the SLR. This research describes a massive systematic literature review that was conducted to find the most important adoption factors when choosing open source software packages. This research will be setting the stage for the first part of the literature review. The factors will be discussed briefly to provide context and eventually be analyzed to see if they also induce trust.

Technical adoption factors
The technical factors are encountered in literature very frequently. These factors are concerning the technical aspects of software packages and are found in 49 of 54 pieces of literature. Impact factors such as compatibility with software, reliability usability and customization are the most common and thus have the highest importance. A brief description of the impact factors found according to [Wheeler, 2011] to get a good understanding how important certain factors can be: • Compatibility: This impact factor refers to the degree to which a piece of software integrates with existing software. Also whether or not additional programs are required to adopt this piece of software.
• Reliability: This factor measures if the software gives the wanted answers. This could be compared to availability. It is not a quantitative property and thus hard to measure.
• Usability: This describes how intuitive the program is for the user. This impacts the difficulty of the software to learn. When software is very usable it will be adopted faster.
• Customization: Customization is the degree to which a component can be changed to do something it could not do before. Configure changes to its initial configuration.
• Documentation: This refers to the available qualitative and quantitative documentation on a software package, this impact factor also falls under the 'support' impact factor, however it is heavily discussed in literature since it describes the technical capabilities of a package.
• Re-usability: Re-usability concerns the quantity of actual code that can be reused in different fashions than the particular library uses it for. The more general a piece of software is, the more goals it can serve.
• Triability: This factor describes the ease of implementation in a system. When this is the case, several other factors can be tested quite easily and thus benefit the package.
• Portability: This is the least mentioned factor and thus carries the least importance of the technical factors. It concerns the ease to deploy the package in multiple different systems.  Table 4.1: The 9 technical factors found in literature according to [Sánchez et al., 2018]

Human/organizational adoption factors
This second category of factors concerns the organizational aspects of the entity. This category of factors did also have a dominant presence in literature. Overall this was a bit less than the technical factors however, this category does contain the most important impact factor: "support". This was named 45 times out of the 54 pieces.
• Support: This impact factor also contains 'technicalities' like documentation and release frequency. It is a very broad impact factor since it contains a variety of factors that express the need to have a backup plan when the organization does not have the technical skills to solve certain problems. This ranges from documentation to the community.
• Training: A presence of sufficient training material ensures that technical staff can learn to fix problems themselves. It also ensures the users know the package is easy to learn and does not have to figure out the technical details themselves.
• Top management support: Some organizations' strategy can also impact the decision in which packages to adopt. The users select packages based on policies that the company has, e.g. privacy policy or information protection. [Vargas et al., 2020] • Attitude towards change: This factor describes how employees are looking towards the adoption of new packages.
• Case studies of FLOSS adoption: This factor describes the success of implementing new software packages. When a package is widely adopted it gains in reputation. Hence it can influence the decision process.
• Time adoption: This factor concerns the total time it takes to implement a package. The longer it takes to fully adapt to new software the less appealing it is.
• Centrality IT: This factor describes the dependency of the organization to the new software. Since the proper implementation of more important systems is way more urgent less important systems.
• Business process engineering: This describes "when an organization is changing its internal business processes due to any particular circumstance(e.g. quality improvement, organizational restructure)." This factor is only mentioned once throughout all literature and makes it the least named and thus least important.

Economic adoption factors
The last category of factors concerns the evaluation of economic factors. These factors are have the least dominance in literature however are still important for software practitioners to look upon.
• Total cost of ownership: Total cost of ownership contains all the costs related to the software package. This includes licensing, however also includes operational and support costs.  [Sánchez et al., 2018] • Licensing cost: Is a part of the total cost of ownership. This is the main cost that practitioners think about when discussing software costs, and concerns the cost of obtaining a particular license.

Organizational adoption factors Mentioned number of times
• Operational cost: This consists of three aspects: the cost to change from systems, the maintenance of the solution and the costs to implement the solution.
• Support cost: These are the costs related to external support as well as keeping the system updated. This is only referenced twice in literature and thus not a very influencing factor.

Economical adoption factors Mentioned number of times
Total cost of ownership 10 License cost 16 Operational cost 4 Support cost 2 Table 4.3: The 4 economical factors found in literature according to [Sánchez et al., 2018]

Trust factors
The factors that influence the adoption of open source software may differ from the factors that induce trust. Factors like functionality and reliability are constantly ranked as high for building trust, next to some other technical and organizational factors. [Del Bianco et al., 2011]. However it seems that the trustworthiness of a system can supersede other adoption factors. [Bernstein, 2005]. The research done by [Del Bianco et al., 2011] provides a list of factors that are believed to affect trustworthiness the most according to it's interviewees.
The two most important factors are "Reliability" and "Degree to which an OSS product satisfies/covers functional requirements the most". These factors are the two biggest technical impact factors in this research. The most important adoption factor for this research was support. However on the list of trustworthiness "Short-term support" is number fourteen on the list. This illustrates that even though for adoption it is the most important factor, for trusting certain software the technical factors are more important.
Another remarkable difference between adoption and trustworthiness factors is that the economic factors are mentioned 12(licence) and 13(total cost of ownership) times out of the 31 articles. Yet in the list of trustworthiness factors there is only one economical factor present out of the 37 named trust factors. That is also why there is no table for economical trust factors.
On the contrary, a factor that is in the second highest scoring group for trustworthiness is customer "Satisfaction". While it is trivial that this factor is important for most businesses, it is not explicitly named in literature about adopting open source software. This could be since it is often used to create a solution for customers rather than being the solution itself.

Interviews
The interview protocol is divided into 3 sections. These sections will be discussed separately and will answer the sub question set for that section.

Section 1, adoption factors and selection procedure
Section 1 of the interview consists of two separate parts containing four questions. The answers to these questions aim to answer the following sub question: What factors are important when selecting external software packages, what is the protocol?

Selection procedure
In order to get all the relevant information from the interviewees. The focus of the first section of the interview was to find out the procedure when selecting other packages, as well as finding out all the relevant factors. This procedure is highly relevant because it describes what role a developer has when selecting packages and the processes they have to go through. This sets the stage on their perspective on this matter. When asked about the procedure, all participants somehow mentioned that this was dependant of the situation. Important and sensitive project require delicate measures and a more sophisticated procedure.
Where less important projects would not require these. After that, the participants were asked to describe the typical procedure when selecting packages in their company. In some occasions there was a formal protocol when selecting packages (P1, P8, P12). These procedures stated the responsibilities of certain actors within the company and delegated roles. These procedures were set be-cause of the impact a vulnerability could lead to. P8: "So if we are want to use a package it first arrives in the 'demilitarized-zone' , then we conduct our analysis. Both by hand and with automated systems, and they tell us whether or not it meets all the set requirements." A more common practice amongst the interviewees is an informal procedure. (P2, P3, P5..P7, P9..P11). This was not a clear set of rules and responsibilities. However there is a common practice when it comes to selecting packages. The most important aspect across all informal procedures is that a decision to adopt a package is always discussed within the development team. P3: "We do this by reviewing each others work, and review of the library's code. Each code change is monitored by the team." In two occasions this sometimes has to be explicitly discussed with an architect or DevOps (P7, P11). This social security ensures that all code is reviewed and no code is accepted without being looked by at least two members of the team.
In addition to discussing these matters within the team, there is one other aspect that is heavily relied on when selecting packages with an informal selection process. This is the common sense and self-awareness of the developer. P:6 "... rules are not written in stone to be honest, and we just do what we think is right." While this is trivial, it is named very specifically by P3, P5, P6, P10. The participants that had a formal process did not mention this, since the risk a development team could have is safeguarded by the formal procedure. So for the informal procedure this implies that there is a lot of implicit trust amongst team members, this can bring a significant risk to the project.
When looking at the participants who either had a formal or informal protocol, one can see that P4 has none. This is because this participant has not operated in a development team. However when continuing the line of questioning, after a while he revealed he had some sort of self-made protocol in his mind for selecting packages. Through time a mind model was created on how to approach these matters.

Adoption factors
All the adoption factors that were found during the interviews are divided into the following categories: • Technical adoption factors

• Organizational adoption factors
• Economic adoption factors One may notice that the table holds more concrete examples of factors rather than the factors itself. Another noticeable aspect is that some of the named aspects may overlap in their meaning. E.g. the 'Number of users' and 'Number of contributors' are two separate aspects, where 'Community' also holds the number of users and contributors. This is because the majority of the participants described the actual actions and thought processes of why they would look at Table 4.6: Adoption factors overview a certain aspect. Therefore it would not be correct to categorize those specific aspects in a category, even though they do fall under that category. A brief description of the specific aspects will now follow, along with a list of quotes to back them up.

Technical factors
Appendix 4.1 holds the found technical aspects that influence the decision of which package to choose. After every aspect there is at least one quote from experts that support this. The following technical aspects were found in the interviews that influenced the interviewees as follows: • Compatibility -The package in question must align with the current framework or build. This is an aspect that has to hold before any other characteristics are considered.
• Documentation -Dependent on the size and the complexity, documentation can play a big role in the decision of a package. This can also illustrates a picture on how serious the project is.
• Complexity -A package must not be too complex for the problem that is trying to solve, and should make coding easier.
• Security -A package should be secure, however there is no clear ways to determine this were mentioned.
• Source code openness -If the source code is available, a global scan can give an indication of the quality of the code and what is does.
• Code quality -The code should do what is claims it does and nothing more. Tests can help to validate this.
• Customization -If a package does not fully cover your needed functionalities, it is very convenient is small adjustments can be made to get full coverage.
• Usability -If a package is easy to use, it will contribute to the selection of that package.

Organizational factors
Appendix 4.2 holds the found Organizational aspects that influence the decision of which package to choose. After every aspect there is at least one quote from experts that support this. The following organizational aspects were found in the interviews that influenced the interviewees as follows: • Active maintenance -If a package is actively maintained, and has been for a period of time. It illustrates a stability, which is an important factor in the selection process.
• Number of contributors -When a lot of contributors work on a project it increases the stability of the package as well because if one of the contributors quits, there are enough others to take over the workload.
• Contributor process -This describes the ease of becoming a contributor for a project. There is no general way of finding this out, however one can get a feel of how hard it is to make contributions to a project. If this is very easy, it also means that people with a bad agenda can do this and potentially create a vulnerability in an upcoming release.
• Git Issues -Dependant on the type of Git issues, this can influence the choice of a package a lot. Issues can be used for feature requests but also for bugs or possible conflicts. The fashion in which these are taken care of shows a good picture on how serious the project is.
• Number of users -The number of users influences the choice of packages in two ways. Firstly, if a lot of people use it, it is implicitly tested. So if there is a bug or vulnerability, it will eventually be found. The second reason is that if a vulnerability was to be found, a lot of people would experience this so it will be patched very fast.
• Backing company -A backing company can ensure the project has enough resources to continue and therefore contributes to the stability of the project. However if such backing company gains too much influence, it can alter certain aspects of the project that would endanger its integrity.
• Ease of integration -When a package is easy to integrate, it can be verified quickly and this is important when selecting packages.
• Community -A large and active community contributes to the selection of a package since the power of open source is because of the diversity of contributors, users and documentation writers from different backgrounds to contribute to one project.
• Known vulnerabilities -The Common Vulnerabilities and Exposures (CVE) holds a database that has all known public security vulnerabilities. This database can be accessed to see if a project has had vulnerabilities. The number of vulnerabilities and the resolve time can provide a good indication of how serious the project is.
• Reputation -The reputation can either be from the project itself, if it has made a name for itself. Or from the contributors or developing entity. One tends to believe that an entity with a good reputation creates good software, and that one with a bad reputation can create bad software. Yet an important aspect is that no reputation has no negative influence.
• Support -Support is not a necessary aspect, however does positively influence the choice of a package if present.
• Git stars -The Git stars can give somewhat of an indication on the popularity of the project.
• Knowledge of team -Rather than an aspect about the project, this is an aspect of the current team a developer is operating in. To look at the strengths and weaknesses of the team, and start selecting packages with that in mind rather than choosing a package solely based on characteristics of that package.

Economical factors
Appendix 7.3.3 holds the found economical aspects that influence the decision of which package to choose. After every aspect there is at least one quote from experts that support this. The following economical aspects were found in the interviews that influenced the interviewees as follows: • License -Licenses may restrict the usage of projects for certain purposes and may influence one's own project.
• Total cost of ownership -The total costs are considered when selecting a package. When using open source software this is not very relevant most of the times. However this can sometimes result in some tricky business.

Section 2, Trust factors and metrics
The second section of the interview consists of two questions regarding trust in software. Trust factors as well as trust metrics were discussed in order to answer the following subquestion: Which factors contribute to gaining trust in a package?

Trust factors
For the first three categories, the interviewees will think of all the factors that influence the decision of what packages will be selected. After this, they will create a subset of all those factors that actually cause them to trust a certain package. While these two aspects are very related, they are definitely not the same. P1: "For me the economical factors play a role in deciding what package, however not in trusting the package." As with the adoption factors, some participants named the actual factors where other named concrete examples of those factors. They are displayed separately to prevent information loss. The found trust influencing aspects can also be categorized as technical and organizational aspects. Appendix 7.4.1 holds the found technical trust influencing aspects. Each of these aspects is supported by at least one quote and appendix 7.4.2 holds the found organizational trust influencing aspects. Table 4.7 displays all aspects that were named during the interviews: There are some trust factors that are not mentioned at the adoption factors. These factors are: • Source code openness -When the source code is available it provides the possibility to go through the actual code and to let one decide for itself whether or not this piece of code is trustworthy. However as P1 pointed out: "There are so many sneaky ways to get a backdoor into software..." This does not guarantee that the code is safe however does contribute to the possible ways of finding out.
• Stackoverflow activity -This is a factor that one participant named specifically, and it is related to the community and the issues. Since stackoverflow is a question-and-answer platform that displays the activeness of the community and users.

Trust metrics
Trust factors and trust metrics seem very alike, however there certainly is a distinction between them, moreover in the context that they were asked. One could argue that the trust metrics contain the concrete examples of trust factors, Which is true. However, the distinction for this research comes from the context in which the two were asked. The question regarding the trust factors aims to uncover what aspects are relevant for an expert to gain trust in a software package. This question regards their personal perspective on what is important. Sometimes this would lead to concrete examples of trust factors: trust metrics. However, the question regarding trust metrics was asked after a detailed explanation about the TrustSECO project and what it aims to accomplish. A situation was described to the participants in which TrustSECO provided a trust score and they were asked what metrics they would value most in a trust score-breakdown. This contextual inconsistency makes all the difference here.
The remarkable aspect about this contextual difference is that some participants came up with metrics that represent a certain category. That were not named when asked about what factors are relevant when trusting a package, one question earlier.
As with the adoption and trust factors, some of the participants would name a category of metrics where others would name concrete examples. For now these will be named separately to prevent information loss. These can also be categorized in technical and organizational metrics. Table 4.8 shows what categories of metrics, or concrete examples of metrics were found: One may notice that P1 does not have any trust metrics, this is because the question was added after the first interview. Appendix 7.5.1 and 7.5.2 hold a list of all found categories of metrics and metrics themselves with quotes to back them up.

Section 3, Personal bias and experience
This section of the interview consists of three questions aimed to answer the question: How do personal aspects influence trust in a package?
This section was created since the literature mainly concerned characteristics of the packages and projects, rather than the actual people who have to make decisions based on these characteristics. The difficult part in this is that there is no one correct answer as to how personal aspects influence this trust. This is because each of the participants has another perspective on this due to different past experiences. However there is one main conclusion to be drawn from this. That is that each of the participants have changed the way they look at selecting software over the years. This is due to the experiences they, but also experiences that the people around them had. Some have experienced more severe situations than others, and this results in them being more careful and paying more attention to this problem.
Another important aspect that could be concluded from this last section of the interview, is that some participants came up with additional factors or metrics that were not mentioned before. This illustrates that this is not a subject that is consciously thought about a lot, since it takes some thought process to come up with all the relevant aspects related to this matter. Hence proving the importance of this research. This then also implicitly means that there could be more relevant factors that are not uncovered during the interviews and would be if there would be more or different questions. Some participants, who had taken prior interest in this matter did not experience this growth in the interview since they had given this subject some more in-depth thought at some point in their career.
The interviews lead to the thought that there is no one-size-fits-all list of factors why a certain individual trusts a certain package since it there was a lot of variance between participants. However that by looking at enough individuals, a well defined list of factors can be defined that determines if a package is trustworthy.

Discussion
One of the most prominent differences between the results from the literature and interviews, is the presence of technical factors for both the adoption and trust aspects. Technical factors are a lot more represented in literature than in the interviews. This is because only the factors or category of factors was noted if the participant named it before they were asked about it explicitly. Which revealed the following: a lot of the technical factors that were named in literature have to hold for interviewees to even consider looking at other factors. So this is definitely relevant when either adopting or trusting a package, and has to match with the current project in order for the project to be looked further into.
Another very important difference between some results are the results between the trust factors and trust metrics. First the participants were what factors could influence the trust on a package. After this they were asked what metrics they would like to seen in an overview for determining trust. The question was initially not for this research and was to get a better idea for the general TrustSECO project, however turned out to be quite the contrary. These two questions would sometimes give vastly different answers. This is due to a complication that this research has seen previously. Namely the difference in naming categories of factors vs. naming concrete examples of factors and the fact that there is a form of growth in the matter as the interview progresses. When discussing the trust metrics, the participants would sometimes name completely different aspects since the aspects had to be somewhat measurable and had not thought of certain aspects before.
This then also illustrates that there could be more trust factors than the participants have named collectively, since a lot of the steps and thought processes they go through are implicit.

TrustSECO
This research set out to create a list of adoption and trust factors for software packages. The purpose of this list is to serve as guidance for creating a survey that will sort the factors on importance. This will then be a major guidance for the TrustSECO project to weigh certain metrics connected to the prioritized factors, in order to create the final project. So from now on a survey needs to be created that ranks all the found factors, as well as all the metrics. The metrics are eventually going to be the most relevant for the TrustSECO project. A prototype of the project was recently created during Odyssey Momentum. This is a mass-online collaboration event in which the whole TrustSECO team invested a whole weekend to start creating the first version of the software. Appendix 7.6 holds more detail regarding my personal experience for this event.

Chapter 6 Conclusion
This research has shown that there are a lot of different factors that play a role when selecting or trusting software packages. It has also shown that these factors have a different impact on stakeholders depending on their role and the and situation.
For this research, there is a distinction between adoption factors and trust factors, both in literature and in the interviews. The adoption factors in literature share a great similarity with the ones found in the interviews. There are some differences. For example the interviews hold concrete examples of the better formulated categories in literature and there are few aspects that really differ from each other. Where the trust factors found in literature differ vastly from the majority of the found trust factors in the interviews. In literature the technical factors are the highest scoring factors where in the interviews the organizational seemed to be the most important.
Then there is the comparison between all found adoption and trust factors. Where the adoption factors display all the factors that are looked upon when selecting a package, the trust factors aim to illustrate what factors make people gain trust in a package. For the latter, this research has shown the difficulty to create a one size fits all trust factor list for individuals to gain trust in a package. This is because each individual has his or hers own past experiences with this matter, and thus has a different perspective on this. However, a good estimation of factors that make a package trustworthy can be generated by learning from enough individuals' experiences.

.3 Interview protocol
Utrecht University is researching the factors that influence software packages selection. Such an impact factor can be defined as a characteristic of a software package that results in trust in that package. The results of these interviews, together with a literature study will lead to a survey that, if done on a large scale, will reveal what these impact factors are. This interview will be about 45 minutes. This information will be available through the informed-consent, that needs to be signed before the interview starts.
State: name, age, nationality, education, function(profession), years of experience, organization, involvement in choosing packages, selection process used in the organization, type of organization, industrial sector of organization, usage of external products within organization 1. What is the protocol when choosing new external software products?
2. What factors related to technical aspects of the packages will you consider when making a decision on choosing a package?
3. What factors related to organizational aspects of the packages will you consider when making a decision on choosing a package?
4. What factors related to economical aspects of the packages will you consider when making a decision on choosing a package?
5. What of these factors, or any other unmentioned factors will make you gain trust in a package, when do you consider a package trusted?
6. What metrics would you like to see in an overview for determining trust?
7. How do you think your personal bias impacts your trust on packages?
8. When do you think a developer should be made aware that a package contains a vulnerability?
9. Do you have personal experience with vulnerabilities in packages? If so, can you comment on how you looked at them before, and after it happened?