title: How to crop too large to fit in memory shapefiles with geoarrow?
body: <p>I have a lot of shapefiles I would like to crop, and I decided to give a try to the new <code>geoarrow</code> R package.</p>
  <pre class="lang-r prettyprint-override"><code>library(sf)
  #&gt; Linking to GEOS 3.10.2, GDAL 3.4.3, PROJ 8.2.0; sf_use_s2() is TRUE
  library(arrow)
  #&gt; 
  #&gt; Attaching package: 'arrow'
  #&gt; The following object is masked from 'package:utils':
  #&gt; 
  #&gt;     timestamp
  library(geoarrow)
  </code></pre>
  <p>Lets create some data</p>
  <pre class="lang-r prettyprint-override"><code>nc &lt;- sf::read_sf(system.file(&quot;shape/nc.shp&quot;, package = &quot;sf&quot;))
  
  nc1 &lt;- nc
  nc1$id &lt;- 1
  
  nc2 &lt;- nc
  nc2$id &lt;- 2
  
  tmpdir &lt;- tempdir(check = TRUE)
  </code></pre>
  <p>Write the data as a dataset</p>
  <pre class="lang-r prettyprint-override"><code>lapply(list(nc1, nc2), \(x) {
    write_dataset(x, tmpdir, partitioning = &quot;id&quot;)
  })
  #&gt; [[1]]
  #&gt; NULL
  #&gt; 
  #&gt; [[2]]
  #&gt; NULL
  </code></pre>
  <p>I can not crop the shapefiles like this:</p>
  <pre class="lang-r prettyprint-override"><code>open_dataset(tmpdir) |&gt; 
    st_crop(c(xmin = -80, xmax = -78, ymin = 34.5, ymax = 36))
  #&gt; Error in UseMethod(&quot;st_crop&quot;): no applicable method for 'st_crop' applied to an object of class &quot;c('FileSystemDataset', 'Dataset', 'ArrowObject', 'R6')&quot;
  </code></pre>
  <p>However, it works if I pull the data in memory before the operation:</p>
  <pre class="lang-r prettyprint-override"><code>open_dataset(tmpdir) |&gt; 
    geoarrow_collect_sf() |&gt; 
    st_crop(c(xmin = -80, xmax = -78, ymin = 34.5, ymax = 36))
  #&gt; Warning: attribute variables are assumed to be spatially constant throughout all
  #&gt; geometries
  #&gt; Simple feature collection with 52 features and 15 fields
  #&gt; Geometry type: POLYGON
  #&gt; Dimension:     XY
  #&gt; Bounding box:  xmin: -80 ymin: 34.5 xmax: -78 ymax: 36.00415
  #&gt; Geodetic CRS:  NAD27
  #&gt; # A tibble: 52 × 16
  #&gt;     AREA PERIMETER CNTY_ CNTY_ID NAME  FIPS  FIPSNO CRESS_ID BIR74 SID74 NWBIR74
  #&gt;  * &lt;dbl&gt;     &lt;dbl&gt; &lt;dbl&gt;   &lt;dbl&gt; &lt;chr&gt; &lt;chr&gt;  &lt;dbl&gt;    &lt;int&gt; &lt;dbl&gt; &lt;dbl&gt;   &lt;dbl&gt;
  #&gt;  1 0.128      1.55  1897    1897 Fran… 37069  37069       35  1399     2     736
  #&gt;  2 0.17       1.68  1903    1903 Guil… 37081  37081       41 16184    23    5483
  #&gt;  3 0.111      1.39  1904    1904 Alam… 37001  37001        1  4672    13    1243
  #&gt;  4 0.104      1.29  1907    1907 Oran… 37135  37135       68  3164     4     776
  #&gt;  5 0.077      1.27  1908    1908 Durh… 37063  37063       32  7970    16    3732
  #&gt;  6 0.142      1.64  1913    1913 Nash  37127  37127       64  4021     8    1851
  #&gt;  7 0.219      2.13  1938    1938 Wake  37183  37183       92 14484    16    4397
  #&gt;  8 0.201      1.80  1968    1968 Rand… 37151  37151       76  4456     7     384
  #&gt;  9 0.18       2.14  1973    1973 Chat… 37037  37037       19  1646     2     591
  #&gt; 10 0.094      1.31  1979    1979 Wils… 37195  37195       98  3702    11    1827
  #&gt; # … with 42 more rows, and 5 more variables: BIR79 &lt;dbl&gt;, SID79 &lt;dbl&gt;,
  #&gt; #   NWBIR79 &lt;dbl&gt;, id &lt;int&gt;, geometry &lt;POLYGON [°]&gt;
  </code></pre>
  <p>The problem is that I have a lot of shapefiles which can not be pulled into memory. Any possibility to crop all these shapefiles within <code>geoarrow</code>? If not, I will just loop all shapefile individually.</p>
  <p><sup>Created on 2022-06-28 by the <a href="https://reprex.tidyverse.org" rel="nofollow noreferrer">reprex package</a> (v2.0.1)</sup></p>