{'fraction_non_alphanumeric': 0.02713439506260419, 'fraction_numerical': 0.009576845316213244, 'mean_word_length': 4.662582847961438, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
More than a Million Ways to Be Pushed: A High-Fidelity Experimental Data Set of Planar Pushing

Pushing is a motion primitive useful to handle objects that are either too large or too heavy to grasp. It is also at the core of much of robotic manipulation, in particular when motion or reconfiguration is involved. It seems reasonable then to wish for robots to have reliable models to predict and understand how pushed objects move. However, these models comprise many parameters, such as those involving friction, that are hard to measure precisely. It is in the essence of frictional interaction and therefore of sliding that when you repeat two experiments with as close as possible to identical setups, two slightly different outcomes happen. In robotics, we often appeal to assumptions and approximations which render models that are computable, but restricted and inaccurate. Just how close are those models? How reasonable are the assumptions they are based on? To help in answering these questions, and to get a better experimental understanding of pushing, we present a comprehensive and high-fidelity data set of planar pushing experiments. The data set contains timestamped poses of a pusher and a pushed object, as well as forces at the interaction. We vary the push interaction in 6 dimensions: surface material, shape of the pushed object, contact position, pushing direction, pushing speed, and pushing acceleration. We automate the data capturing process with an industrial robot along precisely controlled position-velocity-acceleration pushing trajectories, which give us dense samples of positions and forces of uniform quality. We finish the paper by characterizing frictional properties at the interaction, as well as describing and evaluating the most common assumptions and simplifications made by models of frictional pushing in robotics.


I. INTRODUCTION
Pushing is a widely used motion primitive for robotic manipulation. It can aid in the positioning and reorientation of parts [1,2,3,4], facilitate grasping under pose uncertainty [5], grasping in clutter [6], or help in the transportation of large or heavy objects [7,8]. The mechanics of pushing have also been used to aid perception, for example to track the pose of a pushed object [9,10,11], to estimate its shape [11], and to identify inertial parameters such as mass, moment of inertia or coefficient of friction [12,13]. All these applications rely on a good understanding of the mechanics of pushing, which let us predict how an object moves under a certain push.
At an analytical level, pushing is a well understood problem. For decades, the mechanics and robotics communities have developed models to explain the interaction at the interface between a pushed object and a support surface [1,14,10,15,8]. These are usually based on Coulomb's friction law, often rewritten and generalized as maximumpower inequality [1,16]. In Section II we detail these, and see how they have led to compact and deterministic models that, under sufficient assumptions, can be used to explain and control the motion of a pushed object.
The reality, however, is bitter. Predicting the real motion of a pushed object is not a trivial task. In practice, the degree of variability and sensitivity is significant, and there is no computationally efficient and accurate prediction algorithm to explain it.
More recently, data-driven models [17,7,18,19] are an alternative approach to analysis. Though different techniques have been developed, they are still primitive and either do not offer generality, or do not address the observed variability. The lack of common data sets or benchmarks could provide an explanation as to why learning has not had yet the same effect that it has had in other disciplines such as computer vision. Data sets would enable research in model development, and would allow evaluation and comparison of solutions. Although robots excel at accuracy and repetition, capturing large amounts of real data requires setting and resetting experiments between trials, which is difficult without human intervention. This is in stark contrast, for example to collecting digitized daily images for computer vision research [20].
In this study, we have automated the setting and execution of controlled pushing experiments, and captured a large high-fidelity data set of pushing interactions. The data set, as detailed in Section III, includes a wide variety of controlled pushes recorded at a high sample rate with both a force/torque sensor, and a Vicon tracking system. One particularly relevant portion of the data set is that it contains a significant amount of dynamic pushing where the inertial components have an appreciable effect over frictional forces, for which there is little work in the literature.
In summary, the main contributions of this paper are: • A data set of planar pushing interactions containing time series of high-fidelity poses of the pusher and the pushed object, and forces experienced by the pusher. This yields more than a million time stamped data points of poses and forces for each combination of shape and material. • An evaluation of common assumptions that analytical models of pushing make. The data set covers variations in contact location, pusher velocity magnitude and direction, pusher acceleration, object shape, and surface material. To our knowledge, this is the first data set of this caliber. We hope that this data will provide a tool for an experimental study of pushing.

II. RELATED WORK
We review here several works that provide or use analytical motion models of pushing. Many make a quasi-static assumption. Others incorporate dynamics while relying on other assumptions. In the context of pushing, quasi-static interaction means that inertia components are negligible, and only contact forces, frictional forces, and gravity are considered. Due to the fact that the speed and scale of objects in most robotic manipulations fit inside this regime, the quasi-static assumption makes the problem more tractable, and thus leads to simpler models [21].
Mason [1] starts the line of work by proposing a voting theorem to determine the rotation direction of a pushed object. Goyal et al. [16] proposes a limit surface representation for motion-force mapping of a sliding object. This serves as the foundation of many following work on push prediction.  [8] Lee and Cutkosky [22] were the first to approximate the limit surface as an ellipsoid to improve computation time for motion-force mapping. Lynch et al. [23] applies the ellipsoidal approximation to derive a closed-form analytical solution to the push prediction problem for quasi-static pushing. It accommodates both sticking and sliding pushing. Howe and Cutkosky [24] explore more approximation methods of limit surfaces, and provide a guide for choosing them based on the contact distribution, computation cost and accuracy. Lynch and Mason [3] provide a practical solution to the unpredictability of motion of the pushed object by stable pushing with a fence-shaped finger. These models have been used for example for planning robust push-grasps for prismatic objects by Dogar and Srinivasa [25] or for planning in-hand manipulation with patch contacts by Chavan Dafle and Rodriguez [26]. Peshkin and Sanderson [14] address the uncertainty of pushing by sampling all possible pressure distributions and predict a range of possible object motions. Jia and Erdmann [10] investigate a dynamic analysis of pushing but assume that the pusher and the object interact through a frictionless contact. Behrens [8] also gives a dynamic analysis but assumes sticking pushing. We summarize the assumptions and approximations made in the above references in Table I, and we validate the conditions that are marked with asterisk in later sections.

III. DATA SET
This data set records the poses of a pusher and the pushed object together with the forces applied to the pusher from various push experiments. The variations cover 6 dimensions: object's shape, surface material, pusher's speed and acceleration, and initial contact position and push direction. The detailed properties of the shape and the surface material are specified in the data set website [27]. Here we describe the 6 dimensions in detail (see also Table II).
· Shape. Testing different shapes can provide insights into phenomena such as how the frictional properties are affected by changes in the support pressure distribution. To this end, we used 3 rectangles with different aspect ratios (rect1-3), 3 right triangles with different skews (tri1-3), 3 ellipses with different eccentricities (ellip1-3), 1 hexagon (hex), and 1 butterfly shape (butter) ( Table III).
· Surface material. The support surface is of great importance as it dictates the frictional properties between it and the object. To investigate its effect, we choose four surfaces: i) ABS, ii) Delrin, iii) plywood, iv) polyurethane. We will refer to the surface instances we conduct experiments on as abs, delrin, plywood, and pu respectively throughout the paper. In section V, we characterize relevant frictional properties of these surfaces.
· Contact position. As the location of the initial contact changes along the perimeter of the object, we expect a different behavior of the object. Depending on its shape, each object is pushed at 33-44 nearly evenlyspaced locations.
· Push direction. For each contact point we vary the direction of the pusher from −80 • to 80 • around the contact normal with increments of 20 • , for a total of 9 directions.
Note that the contact position and orientation are only valid for the initial contact. After that, the contact configuration may change. That is, we do not control the contact position and push direction in terms of the object frame for a whole trajectory. This is because we want the pusher-object interaction to happen not only at those discrete values, but over its nearby region. The relative poses between pusher and object at every time instance are recorded throughout the whole push. Therefore, the record is valid along the whole trajectory. Also, we are interested in capturing the transition between sticking and sliding pushing, so we would like the push interaction to evolve over time.
On the other hand, we decide not to control force during pushing for three reasons. First, although we do not control the force, we record the force and pose of the object at a high frame rate. The data is still useful for studying how the force information relates to the object pose and the pusher motion. Second, controlling the force acting on the object is challenging because the interaction is through friction, so the force that can be exerted on the object is limited by the friction cone between the pusher and the object. Since the friction coefficient has large variability, the cone we compute may be wrong. If the targeting force is more than the friction can bear, the controller can increase the robot acceleration to extremely high. Third, the position, speed and acceleration of the pusher can be controlled very accurately from an industrial robot, whereas force sensors typically have much lower signal to noise ratio. Therefore we choose position control instead of force control.

IV. DATA COLLECTION SPECIFICATIONS
In this section we will provide a detailed overview of the data collection system and the automated process designed for recording the pushes. The collected data set is obtained from the setup depicted in Figure 1 where a 6 DOF industrial robotic manipulator uses a cylindrical rod to push the objects.

A. Hardware Specification
Robot. The system uses a ABB IRB120 industrial robotic arm with 6 DOF and very high accuracy, that allows us to control precisely the position, velocity and acceleration of its tool center point (TCP). The robot has a horizontal reach of 580 mm and a payload of 3 Kg which is sufficient for the forces that it needs to exert, since the pushed objects are in the order of 1 Kg. The robot is equipped with a stiff cylindrical steel pusher which is attached to the robot in line with the 6th link and has a length of 156 mm. This minimizes occlusions to the motion tracking system. The diameter of the pusher is 9.5 mm.
Sensor instrumentation. We use an ATI Gamma forcetorque sensor rigidly attached to the 6th link of the robot to measure the interaction in the plane of the planar pushing. The cylindrical pusher is mounted directly to the measurement plate of the sensor. The sensor is of high sensitivity with force resolution of 1/160 N in x and y direction, and torque resolution of 1/2000 Nm in z direction.
We track the pose of the object with a Vicon motion tracking system, composed of 4 Bonita cameras with a wide field of view that experimentally gives sufficient views in the range of motions of the pushed object. Each object is fitted with 4 non-symmetrically placed markers to provide accurate tracking to improve visibility. Although 3 are in theory sufficient, 4 asymmetric markers give more stable readings. facing up.
The noise in the recording system is quite small. The average error in the force is 0.04 N, and the noise in the object and pusher position is of 0.07 mm for its center pose and 0.04 • for its orientation.
Objects. We use a total of 11 objects, all water-jet cut in stainless steel for durability. They are bead blasted to remove burrs, retaining a more realistic "rough" surface. Objects weight range between 0.75 and 1.4 Kg depending on the shape. All objects are 13 mm thick. The frictional coefficient between the pusher and the object is approximately 0.25, which was determined using a traditional variable slope experiment.

B. Software description
To facilitate integration of various components of the system such as the robot control, the force-torque sensor, and the

C. Data Collection Process
The automated experiments used to collect the push data can be summarized as: 1. The tracker locates the current pose of the object. Note that the reset mechanism, key for capturing a very large quantity of experiments, is simply enabled by gluing a thick washer to the top of the object, so that the pusher can slide inside and bring the object to the center of the plate.

D. Comprehensive Straight Line Pushing
To collect a comprehensive data set, we enumerate all kinds of values defined in the 6 dimensions and let the robot execute each experiment by following the above procedure. The pusher starts in contact with the object and follows a straight line of 5 cm. The figure under the paper title shows 6 examples of straight-line pushes. Note that the first one is resulted from high-velocity pushing, therefore the object butter slides away from the pusher in the end.
The data collection results in a total of 6,000 pushes per object and surface. Each push has an average of 200 timestamped interactions. In total we ends up with more than a million triples of pusher motion, object motion and force received by pusher.

V. CHARACTERIZING DYNAMIC FRICTION ON THE SURFACES
Pushing involves dynamic frictional interaction between the object and the surface as well as between the object and the pusher. In this section, we conduct experiments to understand how the dynamic coefficient of friction (DCoF) between the object and the 4 kinds of surface materials changes due to some factors. Among many of them, we focus on the following five factors because they are normally present in most basic robotic pushing scenarios: 1) location on surface; 2) number of times the object slides on surface; 3) speed of the sliding motion; 4) direction of the sliding motion without rotation; 5) direction of the sliding motion with rotation.
To study these effects, we design a cage to push the rectangle rect1 as shown in Figure 2. The cage finger does not clamp the object but traps it, with only a small gap (< 1 mm) between the finger and the object boundary. Because of the geometric interference, the robot has full control over the object motion. Dynamic friction force is measured as the reaction force in the horizontal plane on the force/torque sensor at the robot wrist. The theoretical DCoF is computed by dividing the measured reaction force f f by the supporting normal force f n , DCoF = f f fn . For collecting data for the first 3 experiments in the above list, the robot performs a line scanning motion. In each line pass of the scan, the robot pushes the block from left to right and back to the starting point at left. It then moves down to transit to the next line scan. Neighboring scanlines are separated by 5 mm for the first factor for obtaining high spatial resolution, and 100 mm for the second and third. One complete scan of a surface consists of scanning consecutive all lines from top to bottom. We trim the data captured during the transition to study data only from sliding at constant speed. For collecting data for the first two factors, the sliding speed is 20 mm/s; for the third factor, we conduct 10 scans of the 10 speeds mentioned in Section III.
For the fourth factor, in order to slide through a point on the surface from different directions, we start each scan from a point on a circle centered at that point on the surface and stop at the opposite side of the circle. The starting points are separated by 5 degrees on the circumference. The sliding speed is 20 mm/s. For the fifth factor, the procedure is similar to the fourth but we need to generate motion of different ratios of rotation and translation. For each scan, the starting angle is θ and ending angle is −θ with θ spreading over −88 • to 88 • with 4 degree increment. The radius of the circle runs through 50, 25, 12.5, and 0 mm.
Location on surface. Due to imperfection of surfaces, DCoF is generally not spatially uniform. Figure 3 shows the spatial distribution we recovered for all for surfaces along with their ranges. The areas mapped are 20 cm by 43 cm wide. It is interesting to note that even seemingly smooth and uniform materials such as delrin have distinguishable differences on the surface. Figure 4 shows the histogram of the measured DCoF for each surface material. Sorting standard deviation from low to high, we have delrin: 0.016, abs: 0.017, pu: 0.024, and much larger pu: 0.064. The distributions are similar to Gaussian distribution, and can be served as a basic friction model of sliding on these surfaces.
Time dependence. In general, a surface becomes smoother after being rubbed, which is called a polishing process. Similarly, objects sliding on the surface polish the surface and therefore change its DCoF. Here we quantify the polishing effect. Figure 5 shows the DCoF against the number of scans performed on a surface. We can observe a decreasing trend for all materials. To compare the surfaces, after 100 scans, the DCoFs of them change as follows:  • plywood: 0.28 to 0.24 (-11.3%); • pu: 0.29 to 0.28 (-2.3%). Figure 5 shows that for example delrin and abs have a break-in period after which DoF converges to an almost constant value. For some materials like plywood, the breakin period is much longer. For other materials, like pu, the break-in period is almost non-existent.
Speed of sliding motion. Coulomb friction model states that the magnitude of the friction force does not depend on object sliding speed. We validate this statement on all the surface materials we use. In Figure 6, we show the DCoF as a function of the object sliding speed. All the surface materials except for pu are pretty much consistent with the statement, with only a slight increase for larger velocities. The DCoF of pu increases to 1.0 when the speed reaches 500 mm/s. The phenomenon of increased friction due to speed is observed in [28] on rubbers, and [29] states that pu possess this characteristic. Therefore, if the velocity of a sliding object on pu spans a wide range, the change in DCoF should be taken into account.
Direction of sliding motion without rotation. Principle of Maximum-power inequality. If a friction force is opposite to sliding motion, and its magnitude is a constant independent of sliding direction, we say the sliding has isotropic friction; otherwise, anisotropic. We plot the friction force collected only at the center point with 1 mm radius in Figure 7. This plot is called limit curve (LC), if we assume the static friction and the DCoF are the same. That is, if the object is stationary, the friction will be inside the curve; if the object is sliding then the friction will lie on LC boundary. A LC will be close to a circle if the material exhibits isotropic friction. The figure shows that among the four materials, abs and delrin are close to isotropic but not perfect, plywood slightly less, and pu the least. For pu, the ratio between the largest friction and the smallest is around 3/2, which is a significant difference. This explains partly the wide deviation of the DCoF as observed in Figure 4.
From the previous directional data, we can verify if the velocity and the friction force satisfy the principle of maximum-power inequality. The principle states that where LC is the limit curve, f and v are a corresponding pair of friction force and sliding velocity, and f * is any other friction force in the LC. The principle states that the combination of velocity at contact and force will be such that the dissipated power is maximum, or in most cases, friction will oppose velocity.
In a general pushing setting, this is a principle difficult to impose, since it is a complicated constraint, but since our experiment is forcing a desired velocity on the object, it is trivial to verify.
We check whether where f i is an array of forces collected when the object is being pushed in the v i direction. We only use data when the object is passing the same point to avoid spatial difference of DCoF. Figure 8 shows ∆P as a function of pushing direction. For all the material except pu, ∆P is very close to 0, which means maximum power inequality applies to them. For pu, there are 2 regions of velocities in which ∆P is significantly less than 0. They correspond to the abrupt transitions at the top and bottom of the limit curve of pu in Figure 7.
Direction of sliding motion with rotation. Limit Surface.
To consider motion of a 2D rigid body, direction of motion and friction force will include not only translational but rotational component. This extends the concept of limit curve into limit surface (LS), which resides in a force-moment space and is visualized in the top of Figure 9. A LS of a body-surface pair works as follows. If the body is sliding, the generalized friction force lies on LS; otherwise, inside LS. The motion corresponding to that force is in surface normal direction [16].
Here we verify the ellipsoid approximation of the limit surface by comparing the approximated ellipsoid to a limit surface constructed from real measurements. In the bottom of Figure 9, cross section plots of limit surfaces collected from real experiment are shown. Using the most basic approach, the ellipsoid is assumed to be centered at the origin, and is fitted using measured moment magnitude from pure rotational motion and force magnitude from pure translational motion. The shade region shows the 2σ region. We have the following observations. First, due to different kinds of uncertainty in friction, the real limit surface is not a thin surface but a thick surface with uncertainty. Secondly, although measurement are noisy, we can see that the underlying curve of the data is close to the ellipses but not exactly the same. Third, the data points from abs and plywood are biased toward the left side because they are no perfectly isotropic. Fourth, delrin has the most symmetric LS, and pu has the least.  From the stochastic property of friction observed in Section V, we can imagine that pushing on these surfaces is a stochastic process. Here, we want to get a taste of how the uncertainty looks like and motivate some future work for in-depth studies. We provide a case study of an experiment from which we can gain some insights about the following questions.
• Does the object displacement distribution resemble a Gaussian distribution? • How uncertain is the distribution? • Are the predictions from a physical model close to data? • Are the object mean displacements the same on all surfaces? In this experiment, we let the robot conduct the same kind of straight-line pushes repeatedly on an object, and build a distribution of object ending poses. We focus on one • surface: all 4 materials. Additionally, the pusher pushes for 15 cm instead of 5 cm as in the comprehensive data set, which will help the pushing go through sticking and sliding phases. We denote the object trajectory as starting from (x, y, θ) o = (0, 0, 0), and ending at (∆x, ∆y, ∆θ). Figure 10 shows the result, and we observe the following. First, from the histogram and the scatter plot the distribution of the final pose seems to have at least three modes, and its shape does not resemble a Gaussian distribution. Second, the standard deviation (std) of ending poses depends on the surface type. We normalize the error by the mean displacement to get an error rate. Also, we compute the translational and rotational stds separately (Table IV). We can observe qualitatively that the deviation is positively related with the friction property characterized in Section V. It will be interesting to investigate quantitatively what is the exact relationship.
Third, we compare the mean trajectory with the output of a physics model from [23]. Apparently, there is a gap between them, and this will be an interesting future direction to evaluate the difference in a more comprehensive way, e.g., in what condition the prediction is good and when it is bad. In addition, we not only interested in predicting the mean displacement but also a distribution of displacement by a certain push. Fourth, in theory under quasi-static assumption and uniform friction, the displacement should be the same even DCoFs are different. However, as the result shows, the mean displacements are actually quite different from each other. This shows that uncertainty in friction really needs to be taken into account.

VII. CONCLUSION
In this paper we present two main contributions: • First, a large and high-fidelity experimental data set of planar pushing interactions. The data spans six different dimensions of the pushing problem: the shape of the pushed object, the material of the surface where it slides, the location of contact between pusher and slider, and the direction, velocity, and acceleration along which the pusher moves. Overall, these generate more than   [23]. (e) Histogram of ∆x, ∆y, and ∆θ respectively. a million timestamped samples of positions of pusher and slider, as well as interaction forces. • Second, we describe and evaluate the most common assumptions and approximations used in models of planar pushing. The results say that while some assumptions such as the maximum power inequality are generally good representations of the relationship between the directions of friction and motion, other assumptions are not equally respected. In particular the relationship between the magnitudes of normal force and friction force at contact (i.e. the coefficient of friction) is not necessarily linear, and the best linear approximation changes in space, with orientation, with velocity, and even with time. Some materials, mainly those which are more rigid yield slightly better approximations. Our current and future work include: leveraging this data set to develop more accurate semi-parametric models of frictional pushing; investigating their use in the context of simulation, planning, and control; as well as continued efforts in collecting experimental data for prehensile and nonprehensile contact interactions. Of particular interest are outof-plane motions that involve different manipulation actions such as rolling or toppling.
Our long term goal is to steer away from a manipulation paradigm that relies heavily on open loop executions of motions that are planned with simple deterministic models of frictional interaction.