{'fraction_non_alphanumeric': 0.03717639975918122, 'fraction_numerical': 0.011223875462286058, 'mean_word_length': 4.295343276784697, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Competitive Influence Maximization: Integrating Budget Allocation and Seed Selection

Today, many companies take advantage of viral marketing to promote their new products, and since there are several competing companies in many markets, Competitive Influence Maximization has attracted much attention. Two categories of studies exist in the literature. First, studies that focus on which nodes from the network to select considering the existence of the opponents. Second, studies that focus on the problem of budget allocation. In this work, we integrate these two lines of research and propose a new scenario where competition happens in two phases. First, parties identify the most influential nodes of the network. Then they compete over only these influential nodes by the amount of budget they allocate to each node. We provide a strong motivation for this integration. Also, in previous budget allocation studies, any fraction of the budget could be allocated to a node, which means that the action space was continuous. However, in our scenario, we consider that the action space is discrete. This consideration is far more realistic, and it significantly changes the problem. We model the scenario as a game and propose a novel framework for calculating the Nash equilibrium. Notably, building an efficient framework for this problem with such a huge action space and also handling the stochastic environment of influence maximization is very challenging. To tackle these difficulties, we devise a new payoff estimation method and a novel best response oracle to boost the efficiency of our framework. Also, we validate each aspect of our work through extensive experiments.


Introduction
Growing popularity of social networks like Instagram, Facebook, and their ability to propagate ideas and information so rapidly has made the Influence Maximization (IM) an eye-catching task. The IM problem in its classical form is to identify the top k influential nodes of a social network that can influence (directly or indirectly) the largest number of nodes in the network [1]. Viral marketing is a proper application of IM. A company selects a seed set of customers and activate them using free products, hoping that product adoption would propagate in the network through the word of mouth effect [2].
Kempe et al. [1] were the first to formulate the IM problem as a discrete stochastic optimization task and presented Independent Cascade (IC) and Linear Threshold (LT) models for influence diffusion. These stochastic diffusion models define how influence starts from the initial nodes and spreads through the network. They proved that under these two models, the IM problem is NP-hard and proposed a greedy seed selection algorithm that guarantees the (1 − 1/e − )-approximation of the optimal solution.
The observations from analyzing the seed selection studies and also the fact that the action space in the budget allocation scenario tends to be discrete, motivated us to integrate seed selection and budget allocation to propose a new scenario for the CIM problem. In our proposed scenario, the parties first identify the influential nodes of the network. Then, they compete over only these nodes, not the whole network, by the value of the budget they allocate to each influential node. Also, the considered action space is discrete. This modeling of the problem improves the discussed issues with previous studies. Figure 1 demonstrates a simple example of the proposed scenario. Both parties first recognize that nodes 1, 2, and 3 are the most influential nodes in the given social network. They have three units of budget and compete with each other over these influential nodes. Player 1 decides to allocate one unit of budget (a package that costs one unit of budget) to each of these influential nodes. However, player 2 decides to allocate two units to node 1 and one unit to node 2. In this case, node 1, for example, will prefer player 2 as he has suggested a package with a higher value.
We use game theory to analyze the scenario and calculate the Nash equilibrium by the Double Oracle algorithm [16]. This algorithm is specially designed for two-player zero-sum games with large action spaces like ours. For the diffusion dynamics, we have applied the competitive Independent Cascade (competitive-IC) model. Estimating the payoff in this diffusion model is very challenging. It is proven that calculating the influence spread under the IC model is #P-hard [17]. To tackle this difficulty, we have designed a novel method that assigns an influential value to nodes based on the concept of Reverse Reachable (RR) sets [18]. Also, our method is applicable in LT and Trigerring based diffusion models, since the concept of RR-sets is generalizable to these models [19]. To list our contributions: • We propose the two-phase budget allocation scenario where seed selection and budget allocation are integrated.
This scenario improves previous works on seed selection by changing the focus of the problem to convincing influential nodes to act as seed rather than which nodes from the network to select (we have a strong motivation for this change presented in Section 4). Also, the budget allocation scenario is enhanced in the following ways: 1. The focus of the competition is reduced to only influential nodes instead of the whole network. 2. The action space is considered to be discrete. 3. The scenario is extended to the IC, LT, and Triggering based diffusion models.
• We develop a novel competitive aware budget allocation framework for the CIM problem. We especially design a new payoff estimation method and a best response oracle that significantly improves the efficiency of our framework.
• We conduct exhaustive experiments on real-world networks to evaluate different aspects of our work. This is the first time that the budget allocation scenario is tested on real-world networks. The evaluations of [13,14,15] consist of only simple simulations.
This paper is organized as follows: Section 2 reviews the related work. In Section 3, some preliminaries are described. In Section 4, our discussion on seed selection studies is presented, and we discuss why these studies are unrealistic. The proposed scenario and the proposed framework will be described in Section 5 and Section 6. Experiments are reported in Section 7, and finally, we conclude our work in Section 8.

Related Work
Carnes et al. [4] and Bharathi et al. [3] are among the firsts that have addressed the Competitive Influence Maximization (CIM) problem. They first provide extensions to the Independent Cascade (IC) model for the competitive environment, and then analyzed the problem given the opponent's seed set, the follower's perspective. They similarly showed that under their diffusion models, the greedy algorithm provides the (1 − 1/e − )-approximation for the second player (the last player). Bharathi et al. further studied the best response strategy for the first player in very simple cases, like when the graph is directed lines.
Lin et al. [8] considered the scenario that seed selection happens in a multi-round manner and proposed a framework based on Q-learning to learn the optimal strategy for the parties. In their framework, actions were considered as single-party influence maximization algorithms. Also, diffusion happens according to the competitive Linear Threshold (competitive-LT) model, and the reward was calculated in the last round by difference of the number of activated nodes by parties. Their framework is applicable when the opponents' strategy is known or unknown. Li et al. [9] considered a more realistic scenario where parties select their seed sets simultaneously. They modeled the problem as a game with seed selection algorithms as actions and the expected influence as payoffs. Also, diffusion happens according to the competitive-IC model. These two works do not propose a new strategy for seed selection. Instead, a framework for selecting the best strategy from a set of seed selection algorithms available is proposed. Their solution is more practical since they do not consider any information about the seed set of the opponent.
In [5], a new model based on the competitive-LT model where each node can think about the incoming influence, and an influence maximization algorithm based on community detection were proposed. In [6], time limitation and time delay in the propagation process have been taken into account, and the competitive-IC model with meeting events was proposed. In [7], given the opponents' seed sets, they have tried to select the seed set that its spread will pass a certain threshold with a minimum cost. All these works have solved the problem with the assumption that the opponent's seed set is given (from the follower's perspective).
In [20], a two-player zero-sum extensive-form game is proposed that simulates the idea of the CIM problem. The players have an equal number of tokens, and in each stage of the game, they should choose one node from the graph to put a token on. When the number of tokens on a node reaches a threshold, the node will become activated and spreads its tokens to the neighbors. They have used Alpha-Beta pruning and Monte Carlo tree search algorithms to find the optimal strategy for the players.
In [10] and [11], the Q-learning based framework proposed in [8] is improved in the following way. In [10], transfer learning is applied to avoid retraining the Q-learning method when the social network changes. In [11], the factor of time is integrated into the competition, and a nested Q-learning algorithm is proposed.
Masucci and Silva [12] are the first to introduce the budget allocation scenario. They analyzed the case of two competitors and used game theory, especially Colonel Blotto games, to calculate the Nash equilibrium. They also used the voter model with discrete states for their diffusion dynamics. Later in [13], they extended the problem to the case of more than two players and presented the voter model with ranking scores. In both works, the considered action space was continuous, meaning that any fraction of the budget could be allocated to a node. Also, all players had an equal amount of budget.
Varma et al. [14] extended the voter model to the case where nodes have continuous states representing their opinion toward the players. Each player's objective is to maximize the overall opinion in the network. They mainly considered the case of two players, and again, the action space was continuous, and also the players' amount of budget could be different. They showed that under their proposed model, the game has a pure Nash equilibrium. In [15], they further extended their work to the case that marketing campaigns are repeated.

Competitive Independent Cascade Model
The competitive Independent Cascade (competitive-IC) model proposed in [3] is a widely accepted diffusion model for the competitive environment. A directed graph G = (V, E) is given and an activation probability P e ∈ [0, 1] is assigned to each edge. Nodes have two states, they are either inactive or activated i . In the latter case, i = 1, 2, . . . , n denotes the player, for whom the node is activated. Consider that S i denotes the seed set of the player i. Given the seed sets, influence propagates in the network through a progressive process as follows: • In the first step, nodes in each set S i become activated and their states are set to activated i . • In step t, in a random order, nodes activated in step t − 1 will try to activate their free neighbors. The activation succeeds with probability P e , and newly activated nodes take the state of the node who has activated them. • The process terminates when no new activation occurs.
According to the process, each newly activated node has only a single chance to activate its free neighbors, and also active nodes never change state. The influence spread of player i, I(S i ), denotes the number of nodes activated by S i when diffusion terminates.

Reverse Reachable Sets
The Reverse Reachable (RR) sets [18] is a proven method to approximate the influence spread and avoid the limitations of the greedy algorithm. In this section, we first describe this concept in the non-competitive environment, then it is described how this method is extended to the competitive problem when the opponent's seed set is given.

RR-sets in Non-competitive Problem
This is the definition of the random RR-sets under the classical IC model [18,19]: Let g be a graph in which diffusion is deterministic, meaning that, if there is a path from a node u to a node v, then u can influence v. An RR-set for a node v in the deterministic graph g is a set that includes all the nodes in g that has a path to v. v is called the root node. A random RR-set is an RR-set created for a root node selected randomly, also the deterministic graph g is sampled by removing each edge of G with probability 1 − P e (the root node and deterministic graph g are both sampled randomly).
Intuitively, the nodes in an RR-set have a chance to activate the root node of the RR-set, if they are selected as the seed set. Algorithm 1 describes how an RR set is created algorithmically. First, the RR-set and a queue are initialized with the root node selected uniformly at random (lines 1 and 2). Then, we start to move backward from the root node and add each visiting node to the RR-set according to the probability assigned to the corresponding edge. The process of moving backward is repeated for each node newly added (lines 3 to 6) [19]. for all w ∈ in_neighbors(u) do 6: add w to Q and RS according to the probability P w→u 7: return RS RR-set based methods like TIM/TIM+ [19] and IMM [21] build their seed selection algorithm upon this idea that when a sufficient number of random RR-sets are created, the expected spread of any seed set S can be estimated by the fraction of the RR-sets covered by the seed set (A seed set covers an RR set if it has a node in that): N is the number of nodes, θ is the number of RR-sets created, and C(S) is the number of RR-sets covered by the seed set S [19,21]. Algorithm 2 describes the outline of the RR-set based seed selection methods. First, a sufficient number of random RR-sets are created (line 1). Then, iteratively, the seed set is filled with the node that covers the largest number of the RR-sets (lines 3 to 6). Note that in each iteration, the RR-sets covered by the selected node are removed (line 6). v ← the node that covers the largest number of RR-sets in R 5: S ← S ∪ {v} 6: remove the RR-sets covered by v from R 7: return S

RR-sets in Competitive Problem
Lu et al. [22] extended the definition of RR-sets to the general case where two items that complement or compete with each other are diffused in the network. By this extension, they proposed the general-TIM algorithm for seed selection with the assumption that the other player's seed set is given. They proved that, in specific conditions, their algorithm provides the (1 − 1/e − ) approximation with a high probability. They further show that in the pure competition (our problem) their conditions completely hold.
According to their general definition of RR-sets, for creating a random RR-set for the competitive problem (under the competitive-IC model), the process is entirely the same as the non-competitive process, except that the backward phase stops when a node from the opponent's seed set is visited. We refer to the general-TIM algorithm with competitive RR-sets as the competitive-TIM algorithm.

Discussion on Seed Selection Scenarios
In this section, we conduct a step by step investigation regarding the scenarios where the focus of the competition is on seed selection, and discuss why these modelings are not realistic. For simplicity, our analysis is mainly around the case of two players, and the competitive-IC model is used for diffusion.
First, consider the follower's perspective scenario. In this scenario, the second player selects his seed set after the first player, and as a result, has complete knowledge of the opponent's seed set. For this player, the greedy algorithm provides a solution with the (1 − 1/e − )-approximation of the best response [3]. As described in Section 3.2.2, the competitive-TIM provides the same approximation with a high probability and is much more efficient than the greedy method [22]. Therefore, competitive-TIM is a perfect choice for the second player. Now consider the first player. As far as we know, no practical choice has been proposed for the first player, and we should refer to the non-competitive seed selection algorithms. There are generally two categories of seed selection strategies in the non-competitive literature. First, methods that do not guarantee any approximation. Second, algorithms with an approximation guarantee (1 − 1/e − ), like CELF [23], CELF++ [24], TIM+ [19] and IMM [21]. It makes much more sense to consider that the first player selects his seed set by one of the algorithms from the second category.
We conduct an experiment where the first player adopts the TIM+ algorithm (a representative from the second category), and the second player uses the competitive-TIM. Also, for both players, the seed set size is equal to k (|S 1 | = |S 2 | = k). Similarly, when this value is negative, the second player has won the game. Results show that the second player has lost the game in 3 out of 4 datasets, although he had complete knowledge of the opponent's seed set. Therefore, it seems that the first player has more chance to win, and the considered scenario is biased. Now, consider the scenario where the objective is to propose a framework for selecting the best seed selection strategy.
To analyze this case, we consider the scenario proposed in [9]. In this scenario, seed selection happens at the same time, and also seed sets can overlap. Overlapping nodes will be assigned to one of the players randomly. Considering the previous discussion, and also the fact that there is no competitive seed selection algorithm specific to this case, we would get again to the non-competitive seed selection methods with the greedy approximation ratio as the best choices. Note that the main difference among these methods is their running time, not their effectivity. Therefore, it is sensible that both players use IMM [21], which is the most efficient and scalable algorithm from this category, and so, their seed sets would be entirely the same. Even if the players adopt different methods from this category that has an approximation guarantee, again they will almost select the same nodes. To show this, we have calculated the number of nodes that overlap between the seed sets selected by CELF [23], CELF++ [24], TIM+ [19] and IMM [21] algorithms and report the result in Table 1 2 . The number of overlappings will even increase if we reduce the approximation error by increasing the number of Monte Carlo simulations in CELF and CELF++ and decrease in TIM+ and IMM. Therefore, the result of this scenario is the state that the players are targeting the same initial adopters. In conclusion, this observation indicates that the competition is beyond the choice of the seed selection method.

Proposed Scenario
Suppose that different companies are competing for more product sales in a social network. Each company has a marketing budget and several advertising packages with different values. The companies first identify the most influential nodes of the network, and then they decide which package should be allocated to which influential node. Nodes will prefer the package with the highest value.
Now we formally define the scenario. Consider that the set S = {s 1 , s 2 , ..., s n } with size n denotes the set of most influential nodes. Also, suppose that player i has a marketing budget k i and his available advertising packages are where d i denotes the value of the package, or equivalently the cost of the package for the company. By this formulation, the actions available to player i would be where a j indicates the value of the package allocated to the jth node in S, and also n j=1 a j ≤ k i . If a j = 0, it means that the player has ignored the node. Players simultaneously select their actions. Each node s i will be assigned to the seed set of the player that has offered the node a package with higher value compared to the other players. If two or more players have allocated the highest value, the node will be assigned to one of them randomly. When players play their actions (at the same time), their seed sets, S i , can be identified, and then diffusion happens according to the competitive-IC model described in Section 3.1.
As an example, consider Figure 1 part b. In this example, both players have an equal amount of budget, k 1 = k 2 = 3. Also, they have packages with similar values, D 1 = D 2 = {1, 2, 3}. The set of most influential nodes is S = {s 1 , s 2 , s 3 }. The first player has adopted action a 1 = {1, 1, 1}, and the second player has adopted action a 2 = {2, 1, 0}. Therefore, node s 1 will be assigned to the second player (who has offered a higher value) and node s 3 to the first player. Also, node s 2 would be assigned to one of the players randomly, as both players have allocated a package with equal value to this node. Suppose that the result is that this node is assigned to the second player. Finally, the seed sets would be S 1 = {s 3 } and S 2 = {s 1 , s 2 }. Then, diffusion happens, and the winner of the game would be identified, which is player 2 in this example.

Proposed Framework
We model the scenario as a normal form game and propose a novel framework for the case of two players. As the first step, n top influential nodes of the network (the nodes in set S, |S| = n) are identified using the TIM+ algorithm described in Section 3.2.1. This is the first phase of our framework. We define the utility (payoff) for the first player as E[I(S 1 ) − I(S 2 )], where I(S 1 ) and I(S 2 ) are the number of nodes activated by the first player's seed set (S 1 ) and the second player's seed set (S 2 ). Respectively, the second player's payoff would be E[I(S 2 ) − I(S 1 )]. In this way, we have formed a zero-sum game, payoff 1 + payoff 2 = 0. Zeros-sum games model the case of pure competition where any gain by one player results in the same amount of loss for the other player.
In a multi-agent environment, players can follow two kinds of strategies. Selecting one single action and playing it, pure strategy, or playing according to a probability distribution assigned to actions, mixed strategy. The pure strategy is a mixed strategy where probability one is assigned to a specific action. The most common and prominent solution concept in normal-form games is the Nash equilibrium. Nash equilibrium is a stable point where each player is playing the best response against its competitor. Therefore, no rational player would like to deviate from the strategy playing in Nash even if he gets to know the opponent's strategy. Every finite game has at least one Nash equilibrium [25].
Consider that the value of the game is defined as the first player's payoff in equilibrium (the state where both players are playing their respective Nash strategy). According to the Minimax theorem [25], in zero-sum games, the Nash strategy guarantees the player playing this strategy a minimum amount of payoff, which is equal to the value of the game. Note that when both players have an equal amount of budget, k 1 = k 2 , and also both have packages with the same values, D 1 = D 2 , the game would be symmetric. In symmetric games, the value of the game is zero [26]. Thus, in this case, the Nash strategy guarantees a payoff of at least zero, or in other words, guarantees not to lose any competition. This is a very promising guarantee.
To calculate the Nash equilibrium, we faced two challenging difficulties: 1. Our action space is considerably large. As an example, for the parameters k 1 = k 2 = 20, D 1 = D 2 = {1, 2, ..., 20} and n = 25, size of the action space is 1,761,039,350,070.
2. Considering the size of our action space, an efficient method for approximating the value of payoff, E[I(S 1 ) − I(S 2 )], is required. The widely adopted solution of directly diffusing the seed sets in the network [9,8,10,11] is not applicable here. Since, in this method, to reach an acceptable approximation, the diffusion must be repeated many times, which takes a long time.
To tackle the first difficulty, we have used the Double Oracle algorithm proposed in [16] for two-player zero-sum games.
The general idea of the method is that it considers a restricted game, where only a subset of the actions is available to each player. Then, the action space of the restricted game is iteratively enlarged until the algorithm converges. Algorithm 3 describes the outline of the Double Oracle algorithm. The available action sets are initialized with an arbitrary action from the action space (line 1). In each iteration, the Nash equilibrium of the restricted game is calculated (line 3). Using a best response oracle, the best response in pure strategies against the calculated equilibriums are identified and added to the action sets (lines 4-8). The algorithm converges when the calculated best responses are already in the action sets: both a 1 and a 2 are in A 1 and A 2 . In this case, nash 1 and nash 2 are the exact equilibrium of the game with the whole action sets. Another termination condition is to continue until v 1 − v 2 < , in this way, an approximation is calculated [16]. The algorithm guarantees to converge, and practically since only best responses are added in each iteration, it converges much faster than directly calculating the Nash equilibrium [16].
The Double Oracle algorithm works best in the problems that an efficient oracle for calculating the pure best response exists. To propose a very fast best response oracle and solve the problem of payoff estimation, we reuse the RR-sets created in the first phase and develop a heuristic method that assigns an influential value to the nodes in S. Then, we approximate the value of payoff using these calculated values. Suppose that the adopted actions by the first player and the second player are respectively a 1 = {a 1 1 , a 1 2 , ..., a 1 n }, and a 2 = {a 2 1 , a 2 2 , ..., a 2 n }, the payoff for the first player would be: where the function f (.) denotes the gain by node s i in the first player's payoff and is defined as: The above equation comes directly from the scenario and is common in other budget allocation studies [12,13]. The assignments in function f (.) is almost clear. When the player wins the node by assigning a higher value so his payoff would increase by value(s i ), and when he loses the node, his payoff will decrease. Also, when both players ignore the node (a 1 i = a 2 i = 0), the gain from this node for both players would be zero. In addition, when 0 < a 1 i = a 2 i again, the gain would be zero. Since, according to the scenario, in this case, the node will be assigned to one of the players randomly, and as our game is zero-sum, the expected gain is zero for both players. Calculation of payoff 2 is straight forward.
Note that the Nash strategy guaranteed a minimum amount of payoff, which is zero in the symmetric case. However, we are not calculating the exact value of payoff and the error of our heuristic influential value estimation method decreases the guaranteed value. Our experiments will show that this error is too small to affect the performance of the framework. Also, when the second termination condition is used in the double oracle algorithm, again, the guaranteed payoff would decrease by the value of the parameter.
In summary, our framework works in the following steps: 1. The set of most influential nodes, S, is identified by TIM+.
2. The influential values of the nodes in S are estimated.
3. The Nash equilibrium is calculated using the Double Oracle algorithm.
Next we will describe our heuritic influential value estimation method and the best response oracle.

Algorithm 3
The Double Oracle algorithm [16] 1: Initialize A 1 and A 2 with an arbitrary action from A 1 , A 2 2: repeat 3: nash 1 , nash 2 ← Calculate the nash equilibrium of the game where the first player can only play the actions in A 1 and the second player can only play the actions in A 2 4: Add a 1 to A 1 , and Add a 2 to A 2 9: until The termination condition is satisfied

Influential Value Estimation Method
Here we describe how we have reused the RR-sets created by the TIM+ algorithm in the first phase of our framework to assign an influential value to the nodes in S. In the TIM+ algorithm, θ random RR-sets are created according to the non-competitive definition of RR-sets, definition 1. Consider that R denotes this set of RR-sets. Recall from Section 3.2.1, that an RR-set is a set of nodes that have a chance to activate the root node of the RR-set. Also, the influence spread of any seed set S can be estimated by N θ C(S ), where N is the number of nodes in the network and C(S ) is the number of RR-sets that the seed set has at least a node in them (refer to Equation 1). Now, we describe our extension to the RR-set based methods. A node covers an RR-set if it is a member of the RR-set, and so has a chance to activate the root node of the set. In our problem, each node s i ∈ S can have three states. It is either in S 1 (belongs to the first player), or in S 2 (belongs to the second player), or it is ignored by the players and no budget is allocated to it (it is like the nodes out of S). Considering these cases, the effect of covering an RR-set that is also covered by some other nodes in S would not be equal to covering an RR-set that is just covered by a single node. This is the main difference between our problem and previous problems. To apply this difference, we assign a weight to each RR-set r ∈ R: w(r) = 1/|S ∩ r|, where |S ∩ r| indicates how many nodes from S cover r (or equivalently, number of the nodes in S that are also in r). Suppose that T (s ∈ r) returns 1 when s ∈ r is true and 0 otherwise. We estimate the influential value of a node s ∈ S by the sum of the weights of the RR-sets it covers: value(s) = N θ r∈R T (s ∈ r).w(r) Note that when the whole S belongs to one player, for example, when S 1 = S and S 2 = {}, our method's estimated value of the expected spread is exactly the same as the value estimated by non-competitive RR-set based methods (Section 3.2.1). Also, the competitive RR-set based method (described in Section 3.2.2) is not applicable here, since they estimate the spread given the other player's seed set. However, we do not have any information about the other player's seed set.

Best Response Oracle
The oracle calculates the best response in pure strategies against a given mixed strategy, q. The mixed strategy can be represented by the set of actions that probability greater than zero is assigned to them (support of the mixed strategy): q = {p 1 : a 1 , p 2 : a 2 , ..., p m : a m }, where p i > 0 is the probability that the player would play a i , and a 1 , a 2 , ..., a m are the actions in the support of q. Without loss of generality, we assume that the given mixed strategy is played by the second player, and we, as the first player, are searching for the best response. So, in this section, the notation of payoff (,) indicates the payoff for the first player. Also, we use k to denote the budget and D to denote the set of the advertising packages of the player for whom we are trying to find the best response.
Payoff of adopting a pure strategy B against the given mixed strategy q = {p 1 : a 1 , p 2 : a 2 , ..., p m : a m } would be: where payoff (B , a i ) is the payoff of B against the pure strategy (action) a i in the support of q. So, the oracle searches in pure strategies for the strategy B, that: Now we describe the algorithm to find the pure best response B. By our definition of actions, B = {b 1 , b 2 , ..., b n } and each a i in the support of q is a i = {a i 1 , a i 2 , ..., a i n }, where each b j , a i j is the value of the package allocated to the jth node in S under B and a i . According to Equation 5 and Equation 2, the payoff of pure best response B gainst the given mixed strategy q would be: and if we define the function F j (b j ) in the following way: Then, the payoff can be calculated based on the value of each b j : The function F j (b j ), calculates the total gain or loss of allocating a package with value b j ∈ D to the node s j . By this formulation, we can easily use dynamic programming to select the optimal value of each b j from the set D considering the constraint that n j=1 b j ≤ k. Furthermore, there is no need to explore the whole values of D for each b j , since, the value of function F j would not change for most of the values in D. In this way, we can largely reduce the search space. To clear the explanation, consider the following example.
Consider that we are trying to find the best response for the player that his budget is k = 3, and his set of available packages is D = {1, 2, 3}. Also, assume that n = 4 (n = |S|, number of the influential nodes identified in the first phase of our framework). The given mixed startegy is q = {0.8 : a 1 , 0.2 : a 2 } (with probability 0.8, a 1 would be played, and with probability 0.

Experiments
In this section, through exhaustive experiments, we separately evaluate the performance of the payoff estimation method and the proposed framework.
The framework and experiments are implemented in python language using networkx library 3 (version 1.11), and Nash equilibrium in the restricted game (line 3 of double oracle algorithm, algorithm 3) is calculated using gambit framework 4 (version 16). Experiments are conducted on a machine with Intel Core i5 3.4 GHz 4 cores 64-bit processor and 32 GB memory.
The source code of all the implementations are available in the project's repository 5 .

Experiment Setup
Our experiments are conducted on the standard datasets used in the literature of influence maximization. Datasets and their statistics are listed in Table 2. Ca-HepTh is the collaborative network from Arxiv. Ego-Facebook consists of  For the competitive-IC model, the propagation probability of each edge e is considered as P e = 1/indegree(v), where v is the node that e points to. This setting is widely adopted in previous studies [19,21,8]. Further parameters of each evaluation are described in their own subsection.

Evaluation of the Influential Value Estimation Method
The objective of the proposed heuristic method was to assign an influential value to the nodes in S and then calculate the payoff based on this value. There exists no similar method in the literature that calculates the payoff in this way (by assigning a value to the nodes), so, we could only validate our method against the followings: • Simple RR-set based method: This is the method that we have extended. In this method, the influential value of each node s i ∈ S is calculated by estimating the influence spread of the singleton set S = {s i } using the non-competitive RR-set based method (refer to Section 3.2.1): value(s i ) = E[I(S )] = N θ C(S ) = N θ C(s i ). • Centrality-based heuristics: Centrality-based measures like Degree, Betweenness, Closeness, etc. can also be used to assign an influential value to the nodes in S. These methods are used to benchmark the effectivity of the proposed method and are all available in the networkx library.
The evaluation is conducted in this way that, first, the set S with size n = 50 is identified. Then, 20 experiments are executed for each dataset. In each experiment, the nodes in S are randomly assigned to either S 1 , or S 2 , or none of the sets. This is to simulate different states that nodes in S might take in the competition. After these random assignments, the actual value of E[I(S 1 ) − I(S 2 )] is calculated by the average of I(S 1 ) − I(S 2 ) in 5000 rounds of repeated diffusion in the network according to the competitive-IC model. The absolute error would be the absolute difference between the actual value and the value estimated using the value-based methods. Finally, the mean absolute error of these 20 experiments for each method is reported in Table 3.
According to the results, our method has significantly performed better than the SimpleRRset method; this proves that our extension is valid. Also, the heuristic centrality-based measures have all failed. An abnormal pattern in the results is the significant failure of the simpleRRset method under the p2p dataset (mean absolute error of 626.42). This is because this method does not consider the fact that the created RR sets may be covered by more than one node from the set S, and this effect is more considerable under the p2p dataset compared to other datasets.

Evaluation of the Proposed Framework
In this section, we conduct several experiments to show the effectivity of our proposed framework. We mainly focus on the symmetric case where k 1 = k 2 = k and the advertising packages available to the players are D 1 = D 2 = {1, 2, ..., k}. This is because this case is more challenging. Winning the game in the other case, where one of the players has a larger budget, is not too difficult. Also, no similar framework exists in the literature to be compared with our method, and so, we have used the following strategies in our experiments: 1. 1each: One unit of the budget is allocated to the k nodes identified by TIM+.
2. 2each: Two units of the budget are allocated to the k/2 nodes identified by TIM+.
3. Random: Considering the k nodes, this strategy starts from the first node and allocates a random value greater than zero to each node with the considerataion to keep i a i = k.
4. Random(z): Like the above strategy, but it does not allocate a value greater than z to a node (the maximum amount of budget allocated to a node is z).
The 1each strategy resembles the studies that focused on the seed selection scenario, where the budget only identifies the seed set size. The Random strategy is an agent with no rationality, and Random(z) is an improved version of Random.
As the experimental parameters, we have tested the values of k = 10, 20, 30. For n, the size of the set S, the values of n = k, k + 5, k + 10 are tested to see which parameter would perform better. Note that it is not wise to set n < k since, in this way, the framework cannot investigate the case that the maximum possible number of nodes is selected. The double oracle algorithm is initialized (line 1 of Algorithm 3) with 1each strategy to make the results reproducible. Also, the complete convergence is used as the termination condition in k = 10, 20. However, in k = 30, due to long running time, for all datasets except p2p, we have used = 0.5 and for p2p we have used = 1 as termination condition. In the results, CBA(z) denotes our proposed framework, Competitive Budget Allocation (CBA) framework, where n = z.

General Experiments
As the first step, we benchmark the performance of our proposed framework (CBA), 1each, and 2each strategies by performing competitions between these strategies (as the first player) against the two Random, Random(3) strategies. Each competition is repeated 1000 times, and the win percentage of these methods is reported in Table 4. Note that, in the Influence Maximization game, a win is when I(S 1 ) − I(S 2 ) > 0. For each dataset and value of the budget, the method that has achieved the highest win percentage is highlighted. For example, in CA-HepTh dataset with k = 10, 1each strategy has defeated Random strategy in 86.4% of the competitions and has achieved the highest win percentage.
The Random(3) strategy is just the Random strategy improved by integrating a little bit of knowledge (to not allocate a value higher than 3). Note how the performance of the methods drops when tested against this strategy. In CA-HepTh dataset with k = 10, the performance of 1each drops from 86.4% to 54.4%. Also, mostly the 1each and 2each strategies have achieved the highest win percentage in the competitions against Random. However, when we are testing against Random(3), our proposed framework achieves the highest win percentage. Another important point is that the effect of a non-perfect strategy, like the random strategy, gets bold when the value of budget increases, and this is why the win percentages increase as the value of k increases.
In the next experiment, we conduct competitions between our method and 1each, 2each strategies. The average (avg.) and standard deviation (std.) of the I(S 1 ) − I(S 2 ) in 1000 repeated competitions alongside with the win percentage are reported in Table 5. For our framework, we have only reported CBA(k + 5) as the value of n = k + 5 has performed slightly better than other values of n. According to the results, in some cases, our method has won the game (win% 50%, and also avg. 0). In other cases, the results of the competition are a draw, win% 50%, and the value of avg. is small comparing to the value of std. However, our proposed framework has not lost any of these competitions with a large margin. Note that there is a correlation between win% and avg; the value of avg. increases when win% increases.
Another important point from Table 5 is the high value of the standard deviations. This is caused by two sources of randomness. First, the diffusion model is very stochastic. Second, the first player is playing the nash strategy, which is a mixed strategy, and so the actions are not deterministic and selected according to a probability distribution.
As the last experiment in this subsection, we conduct competitions between our method against itself with different values of n. This is mainly to identify which value of n performs better. Results are reported in Table 6. According to the results, none of the parameters could significantly defeat others. Therefore, it seems that there is no significant difference between different values of n tested; only n = k + 5 has slightly performed better.   The key results from the experiments conducted so far are: 1. The performance of our method was better than the 1each/2each strategies in benchmarking against Random(3). 2. We have not lost any competition against other strategies.
3. The hint that the result of n = k + 5 seems to be marginally better than other values of n. In the next subsection, we more accurately demonstrate the superiority of our framework compared to other strategies.

Why should one company adopt our framework?
From a game-theoretic point of view, in the symmetric case, which is the case of these experiments, the Nash equilibrium (proposed framework) guarantees not to be the absolute loser of any competition (refer to Section 6). However, for any non-nash strategy, there exists at least one strategy that can definitely defeat it. This is the main superiority of our framework.
To illustrate the point above, we formed competitions between 1each, 2each, Random(2), and Random (3) strategies against their pure best responses calculated by our best response oracle. Our method is also evaluated against these best responses and its own best response. Like before, each competition is repeated 1000 times, and results are reported in Table 7. The results of this experiment demonstrate how badly these strategies have lost the game to their best responses. In the Facebook dataset, for example, none of these methods could win any competition, win% of 0. However, our method's performance against these best responses and its own best response is completely following what was guaranteed.

Running Time and Number of iterations to converge
Finally, we report the running time and number of iterations it takes for the double oracle algorithm to converge in Table 8. Only the worst case of n = {k, k + 5, k + 10} for each value of k is reported. The cardinality of the action space is k+n−1 n−1 ; this is why we observe a significant increase in running time as we increase the value of k. Another important point is that the size of the graph does not affect the running time of the double oracle algorithm since we are solving the problem from a higher level of abstraction where the competition is formed around only the nodes in S. It is the distribution of the influential value of the nodes that affect the complexity of the game and the running time. For example, in k=30, the most significant running time was under the p2p dataset (with even a looser termination condition = 1), which is not our largest dataset.

Conclusion and Future Work
In this paper, we have presented the Competitive Influence Maximization problem from a higher level of abstraction, where parties first identify the influential nodes of the network. Then, they compete over these nodes by their advertising packages. Also, we have considered the action space to be discrete. This consideration makes a considerable difference to previous studies and is more realistic. We propose an efficient framework with a novel payoff estimation method that  calculates the Nash equilibrium. Our framework targets the case of two players and is entirely applicable when the players have different amounts of budget, and when their available advertising packages are different. Through several experiments, we evaluate each aspect of our work. Especially we show that any non-nash strategy would badly lose the game to its best response. However, this would not happen for our proposed framework, and it guarantees not to lose any competition in the symmetric case. This is the critical superiority of our method.
Our framework is mainly built upon the TIM+ algorithm. However, any other RR-set based method like IMM can easily be mapped to our framework. Also, since the concept of RR-sets applies to the Linear Threshold and Trigerring based models, our payoff method can be extended to these diffusion models.
As future works, we first intend to extend the framework to more than two players. This will not be easy since, for example, the Minimax theorem does not hold in that case. Also, so far, we could not provide a theoretical approximation guarantee for our payoff estimation method, and further work is required in that direction. Finally, analyzing other extensions that make the problem more realistic, like considering the tendency of influential customers, can be considered as future work.