title: NLTK - Remove Tags From Parsed Chunks
body: <pre><code>#!/usr/bin/env python
  # -*- coding: utf-8 -*-
  import os
  import nltk
  import re
  
  from nltk.tree import *
  from nltk.chunk.util import tagstr2tree
  from nltk import word_tokenize, pos_tag
  
  text = "Yarın, Mehmet ile birlikte Ankara'da ki Nüfus Müdürlüğü'ne, Aziz 
  Yıldırım ile birlikte, Şükrü Saraçoğlu Stadı'na gideceğiz.".decode("utf-8")
  
  tagged_text = pos_tag(word_tokenize(text))
  tagged_text2 = word_tokenize(text)
  
  grammar = "NP:{&lt;NNP&gt;+}"
  
  cp = nltk.RegexpParser(grammar)
  result = cp.parse(tagged_text)
  
  for tree in result:
      print(tree)
  
  wrapped = "(ROOT "+ str(result) + " )"  # Add a "root" node at the top
  trees = nltk.Tree.fromstring(wrapped, read_leaf=lambda x: x.split("/")[0])
  
  for tree in trees:
      print(tree.leaves())
  
  for tree2 in result:
      print(nltk.Tree.fromstring(str(tree2), read_leaf=lambda x: x.split("/")[0]))
  </code></pre>
  
  <p>The Output:</p>
  
  <pre><code>(NP Yar\u0131n/NNP)
  (u',', ',')
  (NP Mehmet/NNP)
  (u'ile', 'NN')
  (u'birlikte', 'NN')
  (NP Ankara'da/NNP ki/NNP Nufus/NNP Mudurlugu'ne/NNP)
  (u',', ',')
  (NP Aziz/NNP Y\u0131ld\u0131r\u0131m/NNP)
  (u'ile', 'NN')
  (u'birlikte', 'NN')
  (u',', ',')
  (NP Sukru/NNP Saracoglu/NNP Stad\u0131'na/NNP)
  (u'gidece\u011fiz', 'NN')
  (u'.', '.')
  
  
  ['Yar\\u0131n', ',', 'Mehmet', 'ile', 'birlikte', "Ankara'da", 'ki', 'Nufus', "Mudurlugu'ne", ',', 'Aziz', 'Y\\u0131ld\\u0131r\\u0131m', 'ile', 'birlikte', ',', 'Sukru', 'Saracoglu', "Stad\\u0131'na", 'gidecegiz', '.']
  
  
  (NP Yar\u0131n)
  (u',', ',')
  (NP Mehmet)
  (u'ile', 'NN')
  (u'birlikte', 'NN')
  (NP Ankara'da ki Nufus Mudurlugu'ne)
  (u',', ',')
  (NP Aziz Y\u0131ld\u0131r\u0131m)
  (u'ile', 'NN')
  (u'birlikte', 'NN')
  (u',', ',')
  (NP Sukru Saracoglu Stad\u0131'na)
  (u'gidece\u011fiz', 'NN')
  (u'.', '.')
  </code></pre>
  
  <p>I referenced from :<a href="https://stackoverflow.com/questions/33705555/how-can-i-remove-pos-tags-before-slashes-in-nltk">How can I remove POS tags before slashes in nltk?</a></p>
  
  <p>I want to grouping proper names and remove the tags but when i used the solution it effects the whole text and after that my chunk parse is gone. I really tried the understand the tree structure but how can i apply the the removing function in for statement. I want my Output like:</p>
  
  <p><strong>My desired output:</strong></p>
  
  <pre><code>[Yar\u0131n]
  [,]
  [Mehmet]
  [ile]
  [birlikte]
  [Ankara'da ki Nufus Mudurlugu'ne]
  ...
  ...
  </code></pre>
  
  <p>Also i can't deal with utf-8 as you see my output is full of non-ascii characters. How can i deal with it ?</p>
  
  <p><strong>EDIT:</strong></p>
  
  <pre><code>for i in range(len(tree)):
      arr.append(nltk.Tree.fromstring(str(tree[i]), read_leaf=lambda x: x.split("/")[0]).leaves())
      print(arr[i])
  </code></pre>
  
  <p>I found what shoul i write in the code but now i have the following error. I think i can't append punctuations on my array. </p>
  
  <pre><code>['Yar\\u0131n']
  Traceback (most recent call last):
    File "./chunk2.py", line 61, in &lt;module&gt;
      arr.append(nltk.Tree.fromstring(str(tree[i]), read_leaf=lambda x: x.split("/")[0]).leaves())
    File "/usr/local/lib/python2.7/dist-packages/nltk/tree.py", line 630, in fromstring
      cls._parse_error(s, match, open_b)
    File "/usr/local/lib/python2.7/dist-packages/nltk/tree.py", line 675, in _parse_error
      raise ValueError(msg)
  ValueError: Tree.read(): expected u'(' but got ','
              at index 0.
                  ","
                   ^
  </code></pre>