title: Running 32-bit code in 64-bit process on Linux - memory access
body: <p>I'm experimenting with running 32-bit code inside a 64-bit Linux process. The 32-bit code is completely self-contained, it makes direct IA32 system calls on its own. If I were to load this code in a 32-bit process, it would run just fine.</p>
  
  <p>Initially, I thought I could just allocate a stack for the 32-bit code, switch to it and everything would work fine, but that didn't go so well. Mainly because stack-related instructions (POP/PUSH/...) were doing 8-byte shifts instead of 4 bytes.</p>
  
  <p>By googling, I learned that I can transition to 32-bit mode by switching to segment selector 0x23. Unfortunately, segments are something I know very little about.</p>
  
  <p>I am able to transition to 32-bit mode with something like this (inline AT&amp;T assembly):</p>
  
  <pre><code>movl $0x23, 4(%%rsp) // segment selector 0x23
  movq %0, %%rax
  movl %%eax, (%%rsp) // target 32-bit address to jump to
  lret
  </code></pre>
  
  <p>Where %0 contains a 32-bit address of where the code is mapped. The code starts running, I can see that PUSH/POP now works the way it should, but it crashes even earlier than when I ran the code in 64-bit mode on a seemingly innocuous instruction:</p>
  
  <pre><code>0x8fe48201      mov    0xa483c(%rbx),%ecx
  </code></pre>
  
  <p>Where <code>%rbx</code> (or more like <code>%ebx</code> since this code is already 32-bit, GDB just doesn't know that) contains <code>0x8fe48200</code>. The address it's trying to read from (<code>0x8feeca3c</code>) is valid and readable (according to <code>/proc/XXX/maps</code>) and when I read it from within GDB, it contains the expect value.</p>
  
  <p>Yet, Linux sends a <code>SIGSEGV</code> to the process on this instruction and the faulting address is <code>0</code> (as reported by <code>strace</code> or <code>p $_siginfo._sifields._sigfault.si_addr</code> inside <code>gdb</code>). Somehow it seems <code>0x8feeca3c</code> is not a valid address in the 32-bit realm.</p>
  
  <p>Any ideas how to proceed?</p>
  
  <p>UPDATE: I have written a minimal example that segfaults reading address 0, although address 0 is not really being referenced. It seems that reading any addresses in memory fails (even reading the address of the instruction that has just been executed!), although stack operations work OK.</p>
  
  <pre class="lang-c prettyprint-override"><code>#include &lt;sys/mman.h&gt;
  #include &lt;string.h&gt;
  #include &lt;stdio.h&gt;
  #include &lt;stdint.h&gt;
  
  // 32-bit code we're executing
  const unsigned char instructions[] = {
          0x6a, 0, // push 0
          0x58, // popl %eax
          0xe8, 0, 0, 0, 0, // call the next line to get our location in memory
          0x5b, // pop %ebx
          // THE FOLLOWING mov SEGFAULTS, but it is well within the mapped area (which has size 0x3000)
          // A simpler "mov (%ebx), %eax" (0x8b, 0x03) would fail as well
          0x8b, 0x83, 0, 0x20, 0, 0, // mov 0x2000(%ebx), %eax
          0xf4 // hlt, not reached
  };
  
  int main()
  {
          void* area;
          void* stack;
  
          area = mmap(NULL, 3*4096, PROT_WRITE|PROT_READ|PROT_EXEC, MAP_ANONYMOUS | MAP_PRIVATE | MAP_32BIT, -1, 0);
          memcpy(area, instructions, sizeof(instructions));
  
          stack = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE | MAP_32BIT, -1, 0);
          stack = (void*) (((uint64_t) stack) + 4096 - 4);
  
          memset(((char*)area) + 2*4096, 0xab, 100); // Place 0xAB in the area we mov from in 32-bit instructions
  
          // Switch to 32-bit mode and jump into the code
          __asm__ volatile("movq %1, %%rsp;" \
                           "subq $8, %%rsp;" \
                           "movl $0x23, 4(%%rsp);" \
                           "movq %0, %%rax;" \
                           "movl %%eax, (%%rsp);" \
                           "lret" :: "m"(area), "r"(stack) :);
  }
  </code></pre>