title: Cannot pip install package in virtualenv on EC2
body: <p>I'm seeing this weird issue on ec2. I'm trying to install <code>lsm-db</code> package inside my virtualenv, it says it's successfully installed but when trying to import the package or do <code>pip list</code> it's not there.</p>
  
  <p>I created this virtualenv by</p>
  
  <pre class="lang-sh prettyprint-override"><code>virtualenv -p python3.6 venv
  source venv/bin/activate
  </code></pre>
  
  <p>then running <code>which python</code> and <code>which pip</code> shows the correct ones inside <code>venv</code> as <code>&lt;path-to-my-project&gt;/venv/bin/pip</code>. Then I run <code>pip install -r requirements.txt</code> where <code>sqlalchemy</code> and <code>lsm-db</code> are included. A warning is shown</p>
  
  <pre><code>WARNING: Building wheel for lsm-db failed: [Errno 13] Permission denied: '/home/ec2-user/.cache/pip'
    WARNING: Building wheel for sqlalchemy failed: [Errno 13] Permission denied: '/home/ec2-user/.cache/pip'
  Failed to build lsm-db sqlalchemy
  Installing collected packages: sqlalchemy, flask-sqlalchemy, lsm-db, MarkupSafe, click
    Running setup.py install for sqlalchemy ... done
    Running setup.py install for lsm-db ... done
  Successfully installed MarkupSafe-1.1.1 click-7.0 flask-sqlalchemy-2.4.0 lsm-db-0.6.4 sqlalchemy-1.3.5
  </code></pre>
  
  <p>The last line shows they are correctly installed despite the warnings. However, they are not. Running pip install lsm-db gives</p>
  
  <pre class="lang-sh prettyprint-override"><code>▶ pip install lsm-db
  WARNING: The directory '/home/ec2-user/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
  WARNING: The directory '/home/ec2-user/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
  Collecting lsm-db
    Downloading https://files.pythonhosted.org/packages/17/f2/dc68ecdece424eb206c42b8db924c5a7f6dd97d26df42a1c9fb41a6fe555/lsm-db-0.6.4.tar.gz (316kB)
       |████████████████████████████████| 317kB 14.7MB/s
  Building wheels for collected packages: lsm-db
    WARNING: Building wheel for lsm-db failed: [Errno 13] Permission denied: '/home/ec2-user/.cache/pip'
  Failed to build lsm-db
  Installing collected packages: lsm-db
    Running setup.py install for lsm-db ... done
  Successfully installed lsm-db-0.6.4
  </code></pre>
  
  <p>Again it shows it's installed, but it's not. Not in pip list, can't import in Python. What is going on here? Why do I see permission issue inside <code>virtualenv</code>? I didn't create the <code>virtualenv</code> with sudo if that's a potential problem. Is it related to the way I installed <code>virtualenv</code>?</p>
  
  <hr>
  
  <p>EDIT</p>
  
  <p>I changed the permission and reinstall pip with <code>sudo yum install python36-pip</code>. Now I'm seeing this... sorry it seems like a new problem</p>
  
  <pre><code>Running setup.py install for lsm-db ... error
      ERROR: Complete output from command /home/ec2-user/&lt;project&gt;/venv/bin/python3.6 -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-bhhmd3j8/lsm-db/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-__gotpo6/install-record.txt --single-version-externally-managed --compile --install-headers /home/ec2-user/&lt;project&gt;/venv/include/site/python3.6/lsm-db:
      ERROR: /tmp/pip-install-bhhmd3j8/lsm-db/setup.py:11: UserWarning: Cython not installed, using pre-generated C source file.
        warnings.warn('Cython not installed, using pre-generated C source file.')
      running install
      running build
      running build_ext
      building 'lsm' extension
      creating build
      creating build/temp.linux-x86_64-3.6
      creating build/temp.linux-x86_64-3.6/src
      gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.6m -c lsm.c -o build/temp.linux-x86_64-3.6/lsm.o
      lsm.c:32:20: fatal error: Python.h: No such file or directory
       #include "Python.h"
                          ^
      compilation terminated.
      error: command 'gcc' failed with exit status 1
      ----------------------------------------
  ERROR: Command "/home/ec2-user/&lt;project&gt;/venv/bin/python3.6 -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-bhhmd3j8/lsm-db/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-__gotpo6/install-record.txt --single-version-externally-managed --compile --install-headers /home/ec2-user/&lt;project&gt;/venv/include/site/python3.6/lsm-db" failed with error code 1 in /tmp/pip-install-bhhmd3j8/lsm-db/
  </code></pre>
  
  <hr>
  
  <p>EDIT</p>
  
  <p>Thanks for the pointers, now I installed the python development package on ec2 and tried to install lsm-db again</p>
  
  <pre><code>&gt; sudo yum install python36-devel
  ...
  &gt; pip install lsm-db
  Collecting lsm-db
    Using cached https://files.pythonhosted.org/packages/17/f2/dc68ecdece424eb206c42b8db924c5a7f6dd97d26df42a1c9fb41a6fe555/lsm-db-0.6.4.tar.gz
  Building wheels for collected packages: lsm-db
    Building wheel for lsm-db (setup.py) ... done
    Stored in directory: /home/ec2-user/.cache/pip/wheels/ed/32/8a/b0f3aa15b68f1e6b8f2dcc6285de4796b6b58af71c5f280f48
  Successfully built lsm-db
  Installing collected packages: lsm-db
  Successfully installed lsm-db-0.6.4
  </code></pre>
  
  <p>again, it shows it's successful, and <code>which python</code> gives me the right python in my venv. However, I have the exact same result as the beginning, <code>pip list</code> does not have <code>lsm-db</code> and I cannot import it in python.</p>
  
  <p>This is entirely an issue on ec2, I can successfully use <code>lsm-db</code> in my virtualenv on my mac. There must be something off with <code>virtualenv</code> on ec2 here, why the right pip in venv cannot install a package in venv?? I'm completely lost. If anyone has experience please chime in!</p>
  
  <p>(To work around this problem, I'm switching from <code>lsm-db</code> to <code>sqlitedict</code> in my code. <code>sqlitedict</code> can be installed no problem, so it narrows the suspect down to just <code>lsm-db</code> specific issues.)</p>
  
  <p>This question from 2016 is more similar to mine but it has no conclusion why or any solution.
  <a href="https://stackoverflow.com/questions/35050524/pip-says-that-pymongo-has-been-installed-but-doesnt-show-it-as-installed">Pip says that pymongo has been installed, but doesn&#39;t show it as installed</a></p>
  
  <p>Another potentially related question:
  <a href="https://stackoverflow.com/questions/50704882/amazon-ec2-virtualenv-pip-says-it-installed-numpy-but-python-cant-find-it">Amazon EC2 virtualenv: pip says it installed numpy but python can&#39;t find it</a></p>