{"action":"created","issue":{"url":"https://api.github.com/repos/pybind/pybind11/issues/610","repository_url":"https://api.github.com/repos/pybind/pybind11","labels_url":"https://api.github.com/repos/pybind/pybind11/issues/610/labels{/name}","comments_url":"https://api.github.com/repos/pybind/pybind11/issues/610/comments","events_url":"https://api.github.com/repos/pybind/pybind11/issues/610/events","html_url":"https://github.com/pybind/pybind11/pull/610","id":202274689,"number":610,"title":"Eigen/numpy referencing","user":{"login":"jagerman","id":4459524,"avatar_url":"https://avatars.githubusercontent.com/u/4459524?v=3","gravatar_id":"","url":"https://api.github.com/users/jagerman","html_url":"https://github.com/jagerman","followers_url":"https://api.github.com/users/jagerman/followers","following_url":"https://api.github.com/users/jagerman/following{/other_user}","gists_url":"https://api.github.com/users/jagerman/gists{/gist_id}","starred_url":"https://api.github.com/users/jagerman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jagerman/subscriptions","organizations_url":"https://api.github.com/users/jagerman/orgs","repos_url":"https://api.github.com/users/jagerman/repos","events_url":"https://api.github.com/users/jagerman/events{/privacy}","received_events_url":"https://api.github.com/users/jagerman/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":34,"created_at":"2017-01-21T00:27:47Z","updated_at":"2017-02-24T22:15:10Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/pybind/pybind11/pulls/610","html_url":"https://github.com/pybind/pybind11/pull/610","diff_url":"https://github.com/pybind/pybind11/pull/610.diff","patch_url":"https://github.com/pybind/pybind11/pull/610.patch"},"body":"pybind's current Eigen support relies on copying whenever transferring data back and forth between Eigen C++ types and numpy arrays.  In many cases we can avoid such copies by either having Eigen reference a numpy array data, or having numpy reference eigen's data.  This PR does that.\r\n\r\n(I'd like to leave it here for a while for people to comment/criticize/etc. before merging!)\r\n\r\nFull details are in the documentation commit, but the basic highlights are:\r\n\r\n- when returning an rvalue Eigen concrete type (e.g. `MatrixXd`), we now return a numpy `array` *without copying* which references the `MatrixXd` data.  The MatrixXd itself gets stashed into a `capsule` which is tied to the lifetime of the returned `array`.\r\n- when returning a reference (e.g. `const MatrixXd &`) we copy by default, but you can get a numpy reference instead via `return_value_policy::reference` or a reference tied to the parent via `::reference_internal`.\r\n- when passing a numpy array to a function taking an `Eigen::Ref<MatrixXd>` or `Eigen::Ref<const MatrixXd>`, we avoid copying *if possible*.  Unfortunately it is often *not* possible: `Eigen::Ref` by default assumes a contiguous inner stride, which means contiguous columns for a MatrixXd.  Two possible solutions:\r\n    - Thus you either need to pay attention to the order of passed numpy arrays (i.e. creating with `order='F'`), or change Eigen types from `MatrixXd` to `Matrix<double, Dynamic, Dynamic, RowMajor>`.  But even this isn't always enough: numpy's `a.transpose()`, for example, returns a view into the original numpy array, but just flips the column/row majors.\r\n    - A nicer solution is to take arguments as `Eigen::Ref<Eigen::MatrixXd, 0, Eigen::Stride<Eigen::Dynamic, Eigen::Dynamic>>`.  In addition to numpy arrays of either row/column contiguity, it also allows non-contiguous inputs such as array slices.  Since this is really cumbersome to write, I added `py::EigenDRef<MatrixType>`/`py::EigenDMap<MatrixType>`/`py::EigenDStride` as useful shortcuts.\r\n- mutable Eigen arguments (e.g. `Eigen::Ref<MatrixXd>`) are supported.  Unlike non-mutable references (e.g. `Eigen::Ref<const MatrixXd>`) you won't be allowed to call such a function if a conversion of copy is required.\r\n- you can return `Eigen::Block`s, `Eigen::Map`s, and `Eigen::Ref`s when return a numpy array that references the block/map/matrix data without copying it.  You'll very often want to use a `py::return_value_policy::reference_internal` here (if a method returning internal data) or some other keep-alive mechanism, for obvious reasons.  You can explicitly ask for data to be copied into a new numpy array by using `return_value_policy::copy`.\r\n- we honour `flags.writeable`: you can't pass a `flags.writeable = False` numpy array as an `Eigen::Map<Matrix>` argument.  We also honour the other way: if you return a const Matrix value, reference, or Eigen::Ref, you end up with an array with `flags.writeable = False`.\r\n- you can even do some crazy nested numpy-eigen-numpy-eigen-... referencing, if you feel so inclined (one of the added tests does this just to see that it works).\r\n\r\nThis PR also contains a few other mostly-related odds and ends that I ran into when writing all of this up."},"comment":{"url":"https://api.github.com/repos/pybind/pybind11/issues/comments/282420315","html_url":"https://github.com/pybind/pybind11/pull/610#issuecomment-282420315","issue_url":"https://api.github.com/repos/pybind/pybind11/issues/610","id":282420315,"user":{"login":"wjakob","id":1203629,"avatar_url":"https://avatars.githubusercontent.com/u/1203629?v=3","gravatar_id":"","url":"https://api.github.com/users/wjakob","html_url":"https://github.com/wjakob","followers_url":"https://api.github.com/users/wjakob/followers","following_url":"https://api.github.com/users/wjakob/following{/other_user}","gists_url":"https://api.github.com/users/wjakob/gists{/gist_id}","starred_url":"https://api.github.com/users/wjakob/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wjakob/subscriptions","organizations_url":"https://api.github.com/users/wjakob/orgs","repos_url":"https://api.github.com/users/wjakob/repos","events_url":"https://api.github.com/users/wjakob/events{/privacy}","received_events_url":"https://api.github.com/users/wjakob/received_events","type":"User","site_admin":false},"created_at":"2017-02-24T22:15:10Z","updated_at":"2017-02-24T22:15:10Z","body":"(and thanks for benchmarking, that's a relief)"}}
{"actor":{"display_login":"wjakob"}}