{'fraction_non_alphanumeric': 0.03526056520611875, 'fraction_numerical': 0.006913836314925244, 'mean_word_length': 4.609306834706738, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Curiosity-driven Intuitive Physics Learning

Biological infants are naturally curious and try to comprehend their physical surroundings by interacting, in myriad multisensory ways, with different objects - primarily macroscopic solid objects - around them. Through their various interactions, they build hypotheses and predictions, and eventually learn, infer and understand the nature of the physical characteristics and behavior of these objects. Inspired thus, we propose a model for curiosity-driven learning and inference for real-world AI agents. This model is based on the arousal of curiosity, deriving from observations along discontinuities in the fundamental macroscopic solid-body physics parameters, i.e., shape constancy, spatial-temporal continuity, and object permanence. We use the term body-budget to represent the perceived fundamental properties of solid objects. The model aims to support the emulation of learning from scratch followed by substantiation through experience, irrespective of domain, in real-world AI agents.


I. INTRODUCTION
How do we learn? How do we choose what to learn? How do we continually store and update knowledge? When do we understand that we have understood?... It's been centuries that philosophers and scientists have been mulling over such questions, and these have-and continue to-influence research across disciplines; the fields of Artificial Intelligence and Machine Learning [1] being at the pinnacle of such endeavours. Despite present-day machine learning advancements, it is difficult for a machine to execute a task that a human naturally performs. There is yet a huge gap between a machine's ability to observe, encode experiences and formulate reasons. Here we present an approach that can potentially be used to design machines that can 'understand' the physical world around them. Our model draws inspiration from the way infants learn [2]. It begins with parsing the surroundings by sensing all that is in the physical space around them, interacting with the objects to understand the dynamics, experimenting with them, and deriving hypotheses and predictions, and eventually reasoning on the body budget parameters (shape constancy, spatial-temporal continuity, object permanence). Sensing refers to touch, smell, hearing, smell, and taste. Physical space is the materialistic world around infant.These basic senses help humans identify components in their physical space and their behavior, followed by updating knowledge gathered on the objects through reasoning. Whenever an unexpected event happens, curiosity gets triggered, which contributes to the urge to 'know or learn more' about such an event, further supported with valid reasons (with as much confidence as is possible). The way reinforcement learning works is broadly aligned with the biological infants learning about the physical space. They perceive, they act, develop the knowledge about the physical space, build the parameters about it, interlink the knowledge with the parameters to have a virtual simulation of the physical space around them. The infants learn a lot by observing the actions, states, results and then try to imitate similar actions. In our work, we have focused on the visual sense to demonstrate the proposed method for curiosity-driven learning. We have considered three fundamental solid body physics properties, essential for the comprehension of object-dynamics, i.e., Spatial-temporal continuity, Object Permanence, and Shape constancy, as discussed in the paper [3].

II. RELATED WORK
By focusing on learning as a function of inferences on fundamental properties of solid objects, the designed mechanism is envisioned to serve as a substrate for generalized learning and transfer learning. Some design inspirations are: • Nguyen et al. 2020 [3] discuss the state-of-the-art method to learn intuitive physics with the help of measuring the element of surprise and explaining it. Surprises are explained by a function of motion-based surprise and appearance-based surprise parameters. The evaluation metrics used were Absolute Error Rate and Relative Error Rate for parameters of Object Permanence, Shape Constancy, and Spatial-temporal Continuity.  [4] used curiosity to explain the need to explore the environment and discover novel states. Curiosity is considered an intrinsic reward that enables the agent to explore the environment more. The method empowers an agent to learn generalizable skills even in the absence of an explicit goal. • Zhou et al. (2020) [5] present 'meta-imitation learning' as a mechanism of learning from few demonstrations and trying to imitate them through trial-n-error.  [9] proposed an approach for learning to learn while learning. Obtained results have proven fastening initial learning and faster convergence on subsequent tasks.

III. METHODOLOGY AND SYSTEM DESIGN
We propose a method for curiosity-driven learning of intuitive physics through interactions with everyday objects. We have considered object's body-budget parameters (shape constancy, spatial-temporal continuity, and object permanence) to parse and understand the physical world around. As stated in the article [3], these parameters come handy while parsing natural physical objects-object interactions. All these parameters are represented in the form of a linear plot w.r.t. time / frame-sequence. These plots demonstrate the behavior of the object in a given event (from the data-set) as a line plot. These plots are then examined to identify discontinuity in the object responses if any, and further trigger the curiosity component if discontinuity is observed.

A. Dataset
Dataset used for learning and training of our agent is the Intuitive Physics Dataset [12]. This dataset constitutes a set of possible and impossible events presented through animations. Every event in the dataset has some objects, which can be any or all of a cube, cone, and a sphere. These events may or may not contain an occluder which is a wall in our case. These events can be further classified as static or dynamic based on the object's movement in the event. The sample event of possible and impossible with occluder as a wall is shown in Fig.1

B. System Architecture
The proposed solution is shown in Fig 2. The video input is given to YOLO-v4 model for object detection and localization [7]. The localized object information is used to obtain linear plots for the body budget parameters. These linear plots are then used to analyze discontinuities in the responses as shown in  If occluder is detected for the same time when discontinuities are observed, then the agent accepts this as the reason for the object behavior, and matches the decision with the ground truth which is already provided with the dataset. If the responses are matched, the agent proceeds further for a new event, else it is labeled as an exceptional case. This exception case is still stored in the knowledge base of the agent, which further seeks similar responses to gain confidence about that event. If the agent cannot find reasoning for the discontinuities, it is tagged as an impossible event and further matched with the ground truth. If ground truth matches with the labeling by the agent, then it proceeds further for a new event else added as an exception. The exception added is further generalized if the number of events gets the same responses for the same state and is also labeled as a reasoning label, the same way we humans accept some facts which we might not aware of the reasoning yet but accepts the existence of it.

C. Knowledge Update and Inference
• A generated score for each body budget parameter represents unique characteristics about the object in the given environment. • We consider a hypothetical space that we term latent space. The latent space is similar to plotting data points in a 2D plane and classifying these data points using clustering. Latent space is divided into three spaces that represents classes. • We plot the scores obtained for an unknown object in this latent space. The closer the calculated data point, the more confident the agent will be to classify the   (1), and 'B' represents confidence which is calculated as shown in equation (4) and inference is shown in equation (5) D. Curiosity Driven Learning • With a discontinuity in the linear plot of the body budget, the curiosity block gets activated. • On activation, the agent revisits the event and tries to find out the reasoning for the discontinuity as shown in Fig.  2 • If no reasoning found, it will add exceptional value to the reasoning and keep the information as per ground truth. • With further past experience, this exception will be removed if multiple similar experiences are observed.

E. Equations
As discussed in Section III.B, Z-numbers constitutes of < X, A, B >. For an unknown object, 'X' will be null initially. 'A' being the score of the object w.r.t. the body budget parameters is shown in equation (1). Score generated for object permanence is given in equation (2).
S op is Score for Object permanence, S sc is Score for Shape Constancy, S stc is Score for Spatial Temporal Continuity.S sc is obtained from YOLO-v4 object prediction scores. Fig. 3. Example Event. This is the actual dataset sample event whose body budget concept's Linear plot is shown in Fig. 4. This event has 2 different objects, a sphere and a cone moving from one end to another. Sphere follows the normal path and does not have any surprise factor whereas cone suddenly appears in one time frame, follows a continuous path and then disappears which give rise to curiosity Riochet et al. [12] Fig is the prediction score of the detected object. Impact Value is the unique value specific to a class. For our consideration, we have assigned constant value of 10 for sphere, value of 100 for cone and a value of 1000 for cube. As we are doing summation, we will be having these value in terms of 100's etc. giving unique score for a given class. 'N' represents total number of frames. The range for S op will be different for different class having unique range for each class. S sc is normalised euclidean distance between object's images.
Here, Count O(x t ) is the count of frames in which the object was detected.
Here, A avgc is average 'A' score for a particular class, A u is 'A' value of unknown class. Based on this, 'Z' will be calculated as in equation (5) Z = C, f or C with min (B c ) IV. EXPERIMENT For the experiment described in Table 1, we considered the ground truth as 'sphere' and obtained the following values: N = 90, P yolo (O(x(t)) = 0.6 (for sphere) and 0.17(for cone). As this does not involve any occluder, α = β = γ = 0.33, Count O(x t ) for sphere = 48, for cone = 0.

V. CONCLUSION AND DISCUSSIONS
To the best of our knowledge, the proposed idea of curiositydriven learning mechanism is a novel technique. It shows promise in applications / areas that require learning from scratch and learning from learning. Some directions for future work are as follows, • Incorporation of a knowledge graph towards never-ending learning and Z-number based in-policy reinforcement and transfer learning. • Knowledge-clustering and generation of concept-clouds and 'subjective' experiences. • Robots learning from scratch, and eventually from experiences, in any domain. O(x t ) ← Objects located in x t frame 5: G(x t ) ← Ground truth of objects 6: EoF ← End of frame 7: O(x t ), G(x t ) ∈ {C 1 , C 2 , C 3 , C 4 } 8: C 1 , C 2 , C 3 , C 4 are classes 9: while EoF != True do 10: if O(x t ) ∈ {C 1 , C 2 , C 3 } then