I have post already the question few months ago about my project that I'm starting to work on. This post can be see here:    Now, I know this is based around multivariate time series analysis and tasks are to classify and cluster the data. I have gathered some materials (e-books, tutorials etc.) on this but still can't see a more detailed picture of how even I should start. Here's the tutorial that looks like it might be helpful but the thing is my data looks differently and I'm not really sure if this can be applied to my work.    So basically, my questions are:  How I can start on some very basic analysis? How to read data so it any meaning for me. Any tips and advises will be much appreciated! Note: I'm just the beginner in data science.
Background I'm working on a time series data set of energy meter readings. The length of the series varies by meter - for some I have several years, others only a few months, etc. Many display significant seasonality, and often multiple layers - within the day, week, or year. One of the things I've been working on is clustering of these time series. My work is academic for the moment, and while I'm doing other analysis of the data as well, I have a specific goal to carry out some clustering. I did some initial work where I calculated various features (percentage used on weekends vs. weekday, percentage used in different time blocks, etc.). I then moved on to looking at using Dynamic Time Warping (DTW) to obtain the distance between different series, and clustering based on the difference values, and I've found several papers related to this. Question Will the seasonality in a specific series changing cause my clustering to be incorrect? And if so, how do I deal with it? My concern is that the distances obtained by DTW could be misleading in the cases where the pattern in a time series has changed. This could lead to incorrect clustering. In case the above is unclear, consider these examples: Example 1 A meter has low readings from midnight until 8AM, the readings then increase sharply for the next hour and stay high from 9AM until 5PM, then decrease sharply over the next hour and then stay low from 6PM until midnight. The meter continues this pattern consistently every day for several months, but then changes to a pattern where readings simply stay at a consistent level throughout the day. Example 2 A meter shows approximately the same amount of energy being consumed each month. After several years, it changes to a pattern where energy usage is higher during the summer months before returning to the usual amount. Possible Directions  I've wondered whether I can continue to compare whole time series, but split them and consider them as a separate series if the pattern changes considerably. However, to do this I'd need to be able to detect such changes. Also, I just don't know if this is a suitable way or working with the data. I've also considered splitting the data and considering it as many separate time series. For instance, I could consider every day/meter combination as a separate series. However, I'd then need to do similarly if I wanted to consider the weekly/monthly/yearly patterns. I think this would work, but it's potentially quite onerous and I'd hate to go down this path if there's a better way that I'm missing.  Further Notes These are things that have come up in comments, or things I've thought of due to comments, which might be relevant. I'm putting them here so people don't have to read through everything to get relevant information.  I'm working in Python, but have rpy for those places where R is more suitable. I'm not necessarily looking for a Python answer though - if someone has a practical answer of what should be done I'm happy to figure out implementation details myself. I have a lot of working &quot;rough draft&quot; code - I've done some DTW runs, I've done a couple of different types of clustering, etc. I think I largely understand the direction I'm taking, and what I'm really looking for is related to how I process my data before finding distances, running clustering, etc. Given this, I suspect the answer would be the same whether the distances between series are calculated via DTW or a simpler Euclidean Distance (ED). I have found these papers especially informative on time series and DTW and they may be helpful if some background is needed to the topic area:
Why is the Finder app always open? First time Mac user. I am confused by the fact that the finder application is always open. For example, when switching apps with Cmd+Tab, it is always there, even when there are no windows opened for the app. Cmd-tabbing to Finder with no windows/tabs opened for it has no effect (it seems to me it would more intuitive to open a new tab when this action is chosen).   Furthermore, there is no 'Close app' option when I right click the Finder icon in the dock. Am I misunderstanding something about the Finder app? Is it required for proper functioning of Mac OS? From what I see, Finder is just a file browser, which by no means should be forced to be open 100% of the time. My multimeter is rated to 600V.  How can I readily (and safely) use it to measure higher voltages? I've been programming in Java for a while and just got thrown onto a project that's written entirely in C#.  I'm trying to come up to speed in C#, and noticed enums used in several places in my new project, but at first glance, C#'s enums seem to be more simplistic than the Java 1.5+ implementation.  Can anyone enumerate the differences between C# and Java enums, and how to overcome the differences?  (I don't want to start a language flame war, I just want to know how to do some things in C# that I used to do in Java).  For example, could someone post a C# counterpart to Sun's famous Planet enum example?  public enum Planet {   MERCURY (3.303e+23, 2.4397e6),   VENUS   (4.869e+24, 6.0518e6),   EARTH   (5.976e+24, 6.37814e6),   MARS    (6.421e+23, 3.3972e6),   JUPITER (1.9e+27,   7.1492e7),   SATURN  (5.688e+26, 6.0268e7),   URANUS  (8.686e+25, 2.5559e7),   NEPTUNE (1.024e+26, 2.4746e7),   PLUTO   (1.27e+22,  1.137e6);    private final double mass;   // in kilograms   private final double radius; // in meters   Planet(double mass, double radius) {       this.mass = mass;       this.radius = radius;   }   public double mass()   { return mass; }   public double radius() { return radius; }    // universal gravitational constant  (m3 kg-1 s-2)   public static final double G = 6.67300E-11;    public double surfaceGravity() {       return G * mass / (radius * radius);   }   public double surfaceWeight(double otherMass) {       return otherMass * surfaceGravity();   } }  // Example usage (slight modification of Sun's example): public static void main(String[] args) {     Planet pEarth = Planet.EARTH;     double earthRadius = pEarth.radius(); // Just threw it in to show usage      // Argument passed in is earth Weight.  Calculate weight on each planet:     double earthWeight = Double.parseDouble(args[0]);     double mass = earthWeight/pEarth.surfaceGravity();     for (Planet p : Planet.values())        System.out.printf("Your weight on %s is %f%n",                          p, p.surfaceWeight(mass)); }  // Example output: $ java Planet 175 Your weight on MERCURY is 66.107583 Your weight on VENUS is 158.374842 [etc ...] How to find correct size inner tube What is the use for non-inverting input resistance in a negative feedback opamp? Here in the Figure R5=50K. This resistor value is not in the equations to obtain output voltage, and in many tutorials non-inverting input is directly grounded without a resistor. What might be the reason to use it here? How to extract the last inserted row in SQL Server? When an insert statement is executed one or more rows is inserted into the table, is there any way to extract the last inserted row in SQL Server? Short story (50s? 60s?) with a man trapped inside a sentient cave Evaluating this integral : $ \int \frac {1-7\cos^2x} {\sin^7x \cos^2x} dx $ The question :  $$ \int \frac {1-7\cos^2x} {\sin^7x \cos^2x} dx $$  I tried dividing by $\cos^2 x$ and splitting the fraction.  That turned out to be complicated(Atleast for me!)  How do I proceed now? What can I do when getting "We are no longer accepting questions/answers from this account"? Multiple Installations and Understanding $PATH I have an older version of Git installed at:  /usr/bin/   I recently downloaded a newer version to:  /usr/local/bin   When I type:  which git   I get the location of the old version. I believe that this is just because /usr/bin/ appears before /usr/local/bin in my $PATH variable and so the older version of git is 'found' first.   To test this, I renamed the older version of git to "git_old". Now when I type:  which git   I get the location of the newer version, as expected. But when I type:  git --version   I get the following error:  -bash: /usr/bin/git: No such file or directory   I'm just wondering why my computer is going back to looking in the old location for Git? How to read ND filter description? When I search on Amazon for eg. neutral density 67mm, I get long list of different filters. Some of them are marked like: ND2, ND4, ND8 etc. I'm guessing this means 2-, 4- or 8-stops filter, am I right?  But what about filters, that say 0.6 or 0.9? What does this mean?   Is there any other thing (apart of stops and diameter) that I should also pay attention to when choosing a filter? Remotely changing hard drive boot order, windows 7 Does electricity flow on the surface of a wire or in the interior? How is Starkiller Base's beam visible from Takodana in The Force Awakens? On Takodana,      the main characters and various bar patrons can see the planet Hosnian Prime being destroyed by Starkiller Base with their (respective) naked eyes. How did they see this? Is the Hosnian system close enough to see? If it's not very close and they could just see the explosion, how did the light from it reach them so quickly? Comparing currency values as floats does not work bordermatrix with blocks I need to write a complicated matrix, that has a structure as in the attached picture. I have come across numerous questions where I am given the picture such as the above one been asked "how many rectangles are there?". I have even come across some slightly different images that instead of rectangles you are supposed to find the number of triangles. Well, I was thinking whether there is any formula or strategy that is used to solve these problems without having to manually count every shape.  Help would be appreciated.  Thank you :) How and why do accelerating charges radiate electromagnetic radiation? Let's consider it case by case:  Case 1: Charged particle is at rest. It has an electric field around it. No problem. That is its property.  Case 2: Charged particle started moving (it's accelerating). We were told that it starts radiating EM radiation. Why? What happened to it? What made it do this?  Follow up question: Suppose a charged particle is placed in uniform electric field. It accelerates because of electric force it experiences. Then work done by the electric field should not be equal to change in its kinetic energy, right? It should be equal to change in K.E + energy it has radiated in the form of EM Waves. But then, why don't we take energy radiated into consideration when solving problems? (I'm tutoring grade 12 students. I never encountered a problem in which energy radiated is considered.) When are "if" and "whether" equivalent? Are if and whether equivalent in sentences like the ones below?     How to determine if my saddle is too high?   How to determine whether my saddle is too high?         We should check if everything is okay now.   We should check whether everything is okay now. Find the number of solutions of $x^2=x\mod m$ for any integer $m$.  I was thinking about bringing the $x$ over to the other side and somehow apply chinese remainder thm.