title: What is the best way to train a price prediction machine learning model for car data with text and integer values in the CSV file?
body: <p>Thank you for coming to this post, it's a bit long but I hope it's worth the read. I have been trying, very unsuccessfully, I might add, to train a machine learning model to predict the price of a car based on entries in a CSV file. I have about 200,000 rows of data in a CSV file and each row has various different entries. What I want to do is give a batch of say 50 to 500 (perhaps more or less, if it will generate better results) rows of data to tensorflow/keras at a time and ask that it predicts the corresponding prices for each row (where each row represents a unique car). Then I want to enable the model to see the correct price for each of the rows in the batch and subsequently try to predict the price of a further 50 to 500 (again perhaps more or less than this) rows of data. I want this process of guessing prices and then being told the correct price to continue for the entirety of the training data. I imagine this process of training will be quite slow and for the first few thousand (or perhaps even 10 thousand +) rows of data the model will be wildly inaccurate at predicting the correct price. What I hope is that after all the training data has been seen by the model I will be able to validate its accuracy with test data and I hope that the model will be accurate enough for my needs. I hope I have explained myself clearly with what I am trying to achieve.</p>
  
  <p>To give further information to this post my CSV file is set out as follows:</p>
  
  <p>(The cars shown below are just some entries taken from the CSV)</p>
  
  <p>Note how in the Alfa Romeo Giulia there is no information in the "ATTENTION GRABBER" column, as shown by the double comma after "168 miles". This is quite common for some entries not to have any data in them, and is not limited to this column. </p>
  
  <pre><code>CAR NAME, YEAR, MILEAGE, ATTENTION GRABBER, KEY SPECS, DESCRIPTION, URL, PRICE
  Abarth 1.4 595 3d 144 BHP, 2018, 8000 miles, ASCARI 180 LIMITED EDITION, |2018 (18 reg)|Hatchback|8000 miles|1.4L|143bhp|Manual|Petrol|1 owner|Full service history|, VELOCES ARE VERY PLEASED TO PRESENT FOR SALE THIS AWESOME 595 ASCARI ABARTH …, https://www.autotrader.co.uk/classified/advert/201910173426267?model=595&amp;sort=distance&amp;make=Abarth&amp;radius=1501&amp;onesearchad=Used&amp;onesearchad=Nearly%20New&amp;onesearchad=New&amp;advertising-location=at_cars&amp;postcode=KY103AA&amp;page=20, 11990
  Alfa Romeo 159 Sportwagon 1.9 JTDM 16v TI Estate 5dr Diesel …, 2009, 100000 miles, RARE AUTO • TAN LEATHER • MINT, |2009 (09 reg)|Estate|100000 miles|1.9L|150bhp|Automatic|Diesel|, JUST ARRIVED IN STOCK A VERY RARE &amp; SOUGHT AFTER ALFA ROMEO 159 TI 1.9 DIESEL …, https://www.autotrader.co.uk/classified/advert/201912095183372?model=159%20Sportwagon&amp;sort=distance&amp;make=Alfa%20Romeo&amp;radius=1501&amp;onesearchad=Used&amp;onesearchad=Nearly%20New&amp;onesearchad=New&amp;advertising-location=at_cars&amp;postcode=KY103AA&amp;page=2, 5975
  Alfa Romeo Giulia 2.0 TB 280 Veloce 4dr Auto, 2019, 168 miles,, |2019 (69 reg)|Saloon|168 miles|2.0L|276bhp|Automatic|Petrol|1 owner|, Alfa Romeo Giulia 2.0 TB 280 Veloce 4dr Auto White 1 owner £31000 Apple car …, https://www.autotrader.co.uk/classified/advert/202002087091280?model=Giulia&amp;sort=distance&amp;make=Alfa%20Romeo&amp;radius=1501&amp;onesearchad=Used&amp;onesearchad=Nearly%20New&amp;onesearchad=New&amp;advertising-location=at_cars&amp;postcode=KY103AA&amp;page=12, 31000
  Alfa Romeo Giulietta 1.4 MULTIAIR LUSSO TB 5d 170 BHP DNA …, 2012, 53610 miles, GREAT HISTORY, |2012 (62 reg)|Hatchback|53610 miles|1.4L|170bhp|Manual|Petrol|, BUY NOW PAY APRIL! FINANCE AVAILABLE  This vehicle is presented in excellent …, https://www.autotrader.co.uk/classified/advert/202001276642755?model=Giulietta&amp;sort=distance&amp;make=Alfa%20Romeo&amp;radius=1501&amp;onesearchad=Used&amp;onesearchad=Nearly%20New&amp;onesearchad=New&amp;advertising-location=at_cars&amp;postcode=KY103AA&amp;page=25, 5250
  </code></pre>
  
  <p>Also note how the information in the colums YEAR and MILEAGE is also in the column KEY SPECS which potentially makes these columns altogether irrelevant since the data is already there without those columns. I decided to include them however since i'm not sure how machine learning works exactly and I'm not sure how well the model would be able to see those information points within the column KEY SPECS, so I thought it may be useful to extract them out and make new columns for them. If a ML model is fully capable of seeing this information from within the key specs column then I could simply delete the columns YEAR and MILEAGE. Further thing to note is that each car annoyingly has a different amount of KEY SPECS, what I mean by this is that, if we take the first example we have:</p>
  
  <pre><code>Abarth 1.4 595 3d 144 BHP, 2018, 8000 miles, ASCARI 180 LIMITED EDITION, |2018 (18 reg)|Hatchback|8000 miles|1.4L|143bhp|Manual|Petrol|1 owner|Full service history|, VELOCES ARE VERY PLEASED TO PRESENT FOR SALE THIS AWESOME 595 ASCARI ABARTH …, https://www.autotrader.co.uk/classified/advert/201910173426267?model=595&amp;sort=distance&amp;make=Abarth&amp;radius=1501&amp;onesearchad=Used&amp;onesearchad=Nearly%20New&amp;onesearchad=New&amp;advertising-location=at_cars&amp;postcode=KY103AA&amp;page=20, 11990
  </code></pre>
  
  <p>We can see that the KEY SPECS are:</p>
  
  <pre><code>|2018 (18 reg)|Hatchback|8000 miles|1.4L|143bhp|Manual|Petrol|1 owner|Full service history|
  </code></pre>
  
  <p>On another car however the KEY SPECS could be:</p>
  
  <pre><code>|2013 (13 reg)|120000 miles|8 owners|
  </code></pre>
  
  <p>Which I presume is less than ideal since there isn't a consistent number of key spec variables, in the first example there were 9 variables and in the second, example there were only 3 variables and the variables don't match up like for like between the examples either. So the second variable for the first example was "Hatchback" and the second variable for the second example was "120000 miles", I assume this is a problem, since 120000 miles is not comparable to Hatchback (again I don't know how ML models work so i'm not sure if this is actually a problem at all).</p>
  
  <p>Now that I feel I have explained my problem as best as I can, i'll tell you a few things that I have tried already. First I thought that I could use one hot encoding on each of my columns separately. I started by turning the first column "CAR NAME" into a gigantic list of lists, where each list within the overall list contained a string that denoted the name of that car. For instance if I were to do this to the cars in my example I would generate a list that looks like this:</p>
  
  <pre><code>[['Abarth 1.4 595 3d 144 BHP'], ['Alfa Romeo 159 Sportwagon 1.9 JTDM 16v TI Estate 5dr Diesel …'], ['Alfa Romeo Giulia 2.0 TB 280 Veloce 4dr Auto'], ['Alfa Romeo Giulietta 1.4 MULTIAIR LUSSO TB 5d 170 BHP DNA …']]
  </code></pre>
  
  <p>The next step I took was generating a new list that, instead of containing many lists, just contained each word. So I would generate a list that looked like this:</p>
  
  <pre><code>['Abarth', '1.4', '595', '3d', '144', 'BHP', 'Alfa', 'Romeo', '159', 'Sportwagon', '1.9', 'JTDM', '16v', 'TI', 'Estate', '5dr', 'Diesel', '…', 'Alfa', 'Romeo', 'Giulia', '2.0', 'TB', '280', 'Veloce', '4dr', 'Auto', 'Alfa', 'Romeo', 'Giulietta' '1.4', 'MULTIAIR', 'LUSSO', 'TB', '5d', '170', 'BHP', 'DNA', '…']
  </code></pre>
  
  <p>Then I took only the unique words from that list to make another new list, and after this I used word2int to assign an integer value to each of the words from my final list that did not contain any duplicate words. What I then tried to do was use one hot vectors to train the model but my computer locked up and froze from the work load. So I could not even train the first column "CARE NAMES" I assume because of the sheer size of the vocabulary and due to how inefficient one hot vectors are. The code that I used is as follows:</p>
  
  <p>FEEL FREE TO SKIP PAST THIS BIT OF CODE AS ITS PROBABLY NOT THAT RELEVANT ANYWAY</p>
  
  <pre><code># function to convert numbers to one hot vectors
  def to_one_hot(data_point_index, vocab_size):
      temp = np.zeros(vocab_size)
      temp[data_point_index] = 1
      return temp
  x_train = [] # input word
  y_train = [] # output word
  for data_word in data:
      x_train.append(to_one_hot(word2int[ data_word[0] ], vocab_size))
      y_train.append(to_one_hot(word2int[ data_word[1] ], vocab_size))
  # convert them to numpy arrays, Now we have our x_train and our y_train! yay!
  x_train = np.asarray(x_train)
  y_train = np.asarray(y_train)
  
  print(x_train.shape, y_train.shape)   # (x_train.shape, y_train.shape) = (392978, 1698) (392978, 1698)
                                        # This means 39,2978 training points, where each point has 1,698 dimensions
  
  
  x = tf.placeholder(tf.float32, shape=(None, vocab_size))
  y_label = tf.placeholder(tf.float32, shape=(None, vocab_size))
  EMBEDDING_DIM = 5 # you can choose your own number
  W1 = tf.Variable(tf.random_normal([vocab_size, EMBEDDING_DIM]))
  b1 = tf.Variable(tf.random_normal([EMBEDDING_DIM])) #bias
  hidden_representation = tf.add(tf.matmul(x,W1), b1)
  #Next, we take what we have in the embedded dimension and make a prediction about the neighbour. To make the prediction we use softmax.
  W2 = tf.Variable(tf.random_normal([EMBEDDING_DIM, vocab_size]))
  b2 = tf.Variable(tf.random_normal([vocab_size]))
  prediction = tf.nn.softmax(tf.add( tf.matmul(hidden_representation, W2), b2))
  # Now all that's left to do is train it
  sess = tf.Session()
  init = tf.global_variables_initializer()
  sess.run(init) #make sure you do this!
  # define the loss function:
  cross_entropy_loss = tf.reduce_mean(-tf.reduce_sum(y_label * tf.log(prediction), reduction_indices=[1]))
  # define the training step:
  train_step = tf.train.GradientDescentOptimizer(0.1).minimize(cross_entropy_loss)
  n_iters = 10000    #Suggested 10,000 iterations, takes forever!!!
  # train for n_iter iterations
  for _ in range(n_iters):
      sess.run(train_step, feed_dict={x: x_train, y_label: y_train})
      print('loss is : ', sess.run(cross_entropy_loss, feed_dict={x: x_train, y_label: y_train}))
  print('----------')
  print(sess.run(W1))
  print('----------')
  print(sess.run(b1))
  print('----------')
  vectors = sess.run(W1 + b1)
  print(vectors[ word2int['sportback'] ])
  </code></pre>
  
  <p>So after trying this and getting precisely nowhere I tried to use dense word embeddings instead of one hot vectors. I'm not really sure this was the right call either. I'd love someone to shed some light on what I'm trying to achieve and guide me in the right direction. Sorry for the long post, but I think it was necessary to fully explain myself.</p>
  
  <p>Finally just to reiterate what I'm actually asking for, is help to train a model to predict the price of a car, by using the 200,000 (or so) rows of data that I have, taking in mind that the data which I have is jumbled up with some columns having text, some columns having numbers and some having both text and numbers.</p>
  
  <p>Thank you so much if you made it to the bottom of this post, any help that you can give me is going to go a long way. I hope that other users of Stack Exchange can benefit from this question as well.</p>