I am working to build an deep reinforcement learning agent which can place orders (i.e. limit buy and limit sell orders). The actions are {"Buy": 0 , "Do Nothing": 1, "Sell": 2}.     Suppose that all the features are well suited for this task. I wanted to use just the standard "Profit &amp; Loss" as a reward, but I hardly thought to get something similar to the above image. The standard P&amp;L will simply place the pair (limit buy order, limit sell order) on every up movement. I don't want that because very often it won't cover the commission and it is not a good indicator to trade manually. I would be interested that the agent can maximize the profit and give me a minimum profit of $100 on every pair (limit buy order, limit sell order). On the picture, I would be interested in something similar to the above picture.  Is there a reward function that could allow me to get such a result? If so, what is it?  UPDATE  Is the following utility function can work with the purpose of that question?  $$ U(x) = \max(\$100, x) $$  That seems correct, but I don't know how the agent will be penalized if it covers a wrong transaction, i.e. the pair (limit buy order, limit sell order) creates a loss of money.
I am working to build a reinforcement agent with DQN. The agent would be able to place buy and sell orders for a day trading purpose. I am facing a little problem with that project. The question is "how to tell the agent to maximize the profit and avoid the transaction where the profit is less than 100$".  I want to maximize the profit inside a trading day and avoid to place the pair (limit buy order, limit sell order) if the profit on that transaction is less than 100$. The idea here is to avoid the little noisy movements. Instead, I prefer long beautiful profitable movements. Be aware that I thought using the "Profit &amp; Loss" as the reward.  "I want the minimal profit per transaction to be 100$" ==> It seems this is not something that is enforceable. I can train the agent to maximize profit per transaction, but how that profit is cannot be ensured.  At the beginning, I wanted to tell the agent, if the profit of a transaction is 50 dollars, I will remove 100 dollars, then It becomes a penalty of 50 dollars for the agent. I thought it was a great way to tell the agent to not place a limit buy order if you are not sure it will give us a minimal profit of 100$. It seems that all I would be doing there is simply shifting the value of the reward. The agent only cares about maximizing the sum of rewards and not taking care of individual transactions.   How to tell the agent to maximize the profit and avoid the transaction where the profit is less than 100$? With that strategy, what guarantee that the agent will never make a buy/sell decision that results in less than 100 dollars profit? Does the sum of reward - # transaction * 100 can be a solution?
iMessage doesn't say "delivered" or "not delivered" I sent an iMessage from my iPod touch to my friends iphone on Monday. It didn't say delivered or not delivered. It was just nothing. His phone is on (I know because my other friends are iMessaging him) and my friend said her messages don't say "delivered" at the bottom but they still talk. Mine always say delivered at the bottom. So is this an error, like it didn't go through, or is he just ignoring me? What are good out of the box alternative geocoding solutions to Esri's services? Esri's  will retire at the end of the year, what are good GIS services that can replace the North American and US Streets geocoders?  I have totally excluded Esri's replacement, the  as it is a subscription service. Add year to question and answer dates I'm having some difficulty understanding how to interpret variable importance output from the Random Forest package. Mean decrease in accuracy is usually described as "the decrease in model accuracy from permuting the values in each feature".   Is this a statement about the feature as a whole or about specific values within the feature? In either case, is the Mean Decrease in Accuracy the number or proportion of observations that are incorrectly classified by removing the feature (or values from the feature) in question from the model?   Say we have the following model:  require(randomForest) data(iris) set.seed(1) dat &lt;- iris dat$Species &lt;- factor(ifelse(dat$Species=='virginica','virginica','other')) model.rf &lt;- randomForest(Species~., dat, ntree=25, importance=TRUE, nodesize=5) model.rf varImpPlot(model.rf)     Call:  randomForest(formula = Species ~ ., data = dat, ntree = 25,  proximity = TRUE, importance = TRUE, nodesize = 5)  Type of random forest: classification Number of trees: 25 No. of variables tried at each split: 2          OOB estimate of  error rate: 3.33% Confusion matrix:           other virginica class.error other        97         3        0.03 virginica     2        48        0.04     In this model, the OOB rate is rather low (around 5%). Yet, the Mean Decrease in Accuracy for the predictor (Petal.Length) with the highest value in this measure is only around 8.   Does this mean that removing Petal.Length from the model would only result in an additional misclassification of 8 or so observations on average?   How could the Mean Decrease in Accuracy for Petal.Length be so low, given that it's the highest in this measure, and thus the other variables have even lower values on this measure? What does one second after big bang mean? Consider the following statement:     Hadron Epoch, from $10^{-6}$ seconds to $1$ second: The temperature of the   universe cools to about a trillion degrees, cool enough to allow   quarks to combine to form hadrons (like protons and neutrons).   What does it mean to say "from $10^{-6}$ seconds to $1$ second"?   How is time being measured?  One particle might feel just $10^{-20}\ \mathrm s$ having passed and another could feel $10^{-10}\ \mathrm s$ having passed.   Is saying "1 second after the big bang" a meaningful statement? I have a class with two class methods (using the classmethod() function) for getting and setting what is essentially a static variable.  I tried to use the property() function with these, but it results in an error.  I was able to reproduce the error with the following in the interpreter: class Foo(object):     _var = 5     @classmethod     def getvar(cls):         return cls._var     @classmethod     def setvar(cls, value):         cls._var = value     var = property(getvar, setvar)  I can demonstrate the class methods, but they don't work as properties: &gt;&gt;&gt; f = Foo() &gt;&gt;&gt; f.getvar() 5 &gt;&gt;&gt; f.setvar(4) &gt;&gt;&gt; f.getvar() 4 &gt;&gt;&gt; f.var Traceback (most recent call last):   File &quot;&lt;stdin&gt;&quot;, line 1, in ? TypeError: 'classmethod' object is not callable &gt;&gt;&gt; f.var=5 Traceback (most recent call last):   File &quot;&lt;stdin&gt;&quot;, line 1, in ? TypeError: 'classmethod' object is not callable  Is it possible to use the property() function with @classmethod decorated functions? How do I use an NBT editor to modify item properties? Text spirals with TikZ I just ran across this, and I'm curious about whether it can be replicated in TeX...    Ideally one would have the spiral getting tighter and tighter as one got to the centre, as in    I know TikZ has the ability to fit text along a path, but I'm not sure how one might make text get smaller and smaller.  The closest I can find is , but that uses manually inserted font commands.  Edit: I got what I wanted using mwibrow's answer: Recursively delete empty directories in Windows How to determine $\int x^2 e^{-x^2}\mathrm dx$? I was wondering how to determine $$\int x^2 e^{-x^2}\mathrm dx?$$ I tried to move the exponential into the integrator, and apply integration by parts, but that will make the integrand more complicated. Thank you! How do I install wine? Can someone please explain how I can install  in the following ways?   By using the package manager? By downloading and installing a pre-built package? Prove that f is one-to-one on D How to close (kill) ssh ControlMaster connections manually The easiest thing to do is [to] call for a taxi No group of order 400 is simple I just bought "Reputation to the max". I bought this first, because I really want to help the cleanup of the vote-to-close queue, which, despite the efforts I've been hearing about, is still quite large. According to SO, my reputation is now 15,750, but that queue still says I need at least 3k rep to review! That is clearly a bug! In C, how do I restrict the scope of a global variable to the file in which it's declared? Debian : Verify disk encryption password for already mounted system I am working on our Debian server on which we have a Raid drive encrypted with 2 devices. I did this around 1.5 years back and have never rebooted since. Now, we want to reboot for some reasons, but I have to make sure that the password I have is the correct one, or we will be stuck.   How can we verify the password for an encrypted-raid? Thank you.  Raid details :    mdadm --detail /dev/md0    /dev/md0:         Version : 1.2   Creation Time : Thu Feb 11 14:43:40 2016      Raid Level : raid1      Array Size : 1953382336 (1862.89 GiB 2000.26 GB)   Used Dev Size : 1953382336 (1862.89 GiB 2000.26 GB)    Raid Devices : 2   Total Devices : 2     Persistence : Superblock is persistent      Update Time : Fri Mar 10 10:37:42 2017           State : clean   Active Devices : 2 Working Devices : 2  Failed Devices : 0   Spare Devices : 0             Name : HOSTNAME:0  (local to host HOSTNAME)            UUID : 5c450558:44a2b1e9:83cb4361:9c74df8e          Events : 49      Number   Major   Minor   RaidDevice State        0       8        1        0      active sync   /dev/sda1        1       8       17        1      active sync   /dev/sdb1   Thank you.   Update  dmsetup table --showkeys CryptVol: 0 Large_Number crypt aes-cbc-essiv:sha256 RANDOM_TEXT  0 253:0 4096 RaidVG-LVMVol: 0 Large_Number linear 9:0 2048 Basic Electronics Book Necessary and sufficient condition for the matrix $A = I - 2 x x^t$ to be orthogonal