title: Execute arbitrary python code remotely - can it be done?
body: <p>I'm working on a grid system which has a number of very powerful computers. These can be used to execute python functions very quickly. My users have a number of python functions which take a long time to calculate on workstations, ideally they would like to be able to call some functions on a remote powerful server, but have it appear to be running locally.</p>
  
  <p>Python has an old function called "apply" - it's mostly useless these days now that python supports the extended-call syntax (e.g. **arguments), however I need to implement something that works a bit like this:</p>
  
  <pre><code>rapply = Rapply( server_hostname ) # Set up a connection
  result = rapply( fn, args, kwargs ) # Remotely call the function
  assert result == fn( *args, **kwargs ) #Just as a test, verify that it has the expected value. 
  </code></pre>
  
  <p>Rapply should be a class which can be used to remotely execute some arbitrary code (<code>fn</code> could be literally anything) on a remote server. It will send back the result which the <code>rapply</code> function will return. The "result" should have the same value as if I had called the function locally.</p>
  
  <p>Now let's suppose that <code>fn</code> is a user-provided function I need some way of sending it over the wire to the execution server. If I could guarantee that fn was always something simple it could could just be a string containing python source code... but what if it were not so simple? </p>
  
  <p>What if <code>fn</code> might have local dependencies: It could be a simple function which uses a class defined in a different module, is there a way of encapsulating <code>fn</code> and everything that <code>fn</code> requires which is not standard-library? An ideal solution would not require the users of this system to have much knowledge about python development. They simply want to write their function and call it.</p>
  
  <p>Just to clarify, I'm not interested in discussing what kind of network protocol might be used to implement the communication between the client &amp; server. My problem is how to encapsulate a function and its dependencies as a single object which can be serialized and remotely executed. </p>
  
  <p>I'm also not interested in the security implications of running arbitrary code on remote servers - let's just say that this system is intended purely for research and it is within a heavily firewalled environment. </p>