text:
|Publication number
|Publication type
|Application number
|Publication date
|16 Mar 2006
|Filing date
|5 Jan 2005
|Priority date
|6 Aug 2004
|Publication number
|Christina LaComb, Bethany Hoogs, Jason Miele, Deniz Doganaksoy, Radu Neagu, Corey Bufi, Abha Moitra, Andrew Deitsch, Richard Arthur
|Original Assignee
|Lacomb Christina A, Hoogs Bethany K, Miele Jason P, Doganaksoy Deniz S, Neagu Radu E, Bufi Corey N, Abha Moitra, Deitsch Andrew I, Arthur Richard B
|Export Citation
|BiBTeX, EndNote, RefMan
|Patent Citations (13), Referenced by (25), Classifications (6), Legal Events (1)
|External Links: USPTO, USPTO Assignment, Espacenet
This application claims priority under 35 U.S.C. �119(e) from Provisional Application No. 60/599,511 filed on 6 Aug. 2004.
The systems and techniques described herein relate generally to displaying data for analysis. More specifically, these systems and techniques relate to a data presentation to draw attention to anomalies within a set of related performance data.
The role of an analyst is generally to examine data associated with a particular entity, and to use that data to more completely understand the entity. This understanding can then be used to evaluate the entity, or to make plans for how to improve the performance of the entity, or compare entities to one another. Examples of such analysis include: the examination of medical records to determine to what degree a treatment on a patient (or group of patients) has been effective; examination of athletic performance by a coach to determine how best to improve the performance of the players; examination of historical failure data to determine optimal insurance rates for particular insured equipment; and examination of historical financial data to evaluate the financial health of a target company.
Taking this last example, understanding the financial health of a company is an important factor in evaluating a potential business interaction with that company. An understanding of a company's financial health can be used to help evaluate the risks involved in doing business with that company, and can form a basis for predicting the expected benefits from the potential business relationship or transaction. However, fraudulent financial filings by the company can provide a misleading picture of the financial health of a company. Companies that engage in such fraudulent financial behavior can collapse in ways not reflected by the apparent financial health reflected by their financial information.
As a result of recent collapses of companies that were hiding their financial difficulties behind fraudulent financial data, investors and creditors are seeking ways to identify false or misleading financials before the time where the company's dire financial straits become apparent due to earnings shortfalls, scandals or bankruptcy. Even when financial data is available that can be used to evaluate whether or not there are warning signs that could be found within the financial data, it can be a difficult process to identify where and to what degree the financial data presented is anomalous or merits further investigation or consideration.
Therefore, there is a continued need for improvement in the presentation of data to facilitate evaluation of performance associated with the data of a target entity and to facilitate comparison between entities.
In one embodiment of the systems and methods provided herein, a method for displaying anomaly measures associated with a target entity is presented. Each anomaly measure is associated with a performance metric and a time period, and for each metric, the method involves: determining a value for an anomaly score associated with the metric; determining a set of ranges of anomaly scores; associating a displayable characteristic with each of the set of ranges; assigning a displayable characteristic to each anomaly score based on the range into which the anomaly score falls; associating the displayable characteristic for that anomaly score with a cell on a grid, the cell being associated with the particular anomaly measure and time period of the corresponding performance metric; and presenting the data on a display medium. The set of ranges are selected such that the total range of possible anomaly scores is divided into a set such that each range in the set is separated from an adjacent range by a breakpoint.
In another embodiment of the systems and methods described, the target entity is a company, and the performance metrics are financial metrics representing the financial results of the target company.
In another embodiment of the systems and methods described, anomaly scores are determined by identifying a target value; collecting context data on the target entity; and calculating an anomaly score using the target value and context data using an exceptional statistical measurement. The target value is the value of the performance metric associated with the target entity for a specific time period.
In a further embodiment of the systems and methods described, calculating an anomaly score includes the steps of generating a measure of central tendency for the target value and context data using an exceptional technique; generating a measure of variation for the target value and context data using an exceptional technique; and generating an anomaly score based on the measure of central tendency, the measure of variation, and the target value.
In another embodiment of the systems and methods described herein, the anomaly score is generated using an equation of the form
where A is the anomaly score, Xt is the target value, CT is the measure of central tendency, and V is the measure of the variation.
In another embodiment of the systems and methods described herein, the breakpoints between the ranges of anomaly score values can be adjusted, collectively, in order to alter the center of the ranges and the size of the ranges.
In another embodiment of the systems and techniques described, a visualization of a set of anomaly measures associated with a target entity is created. Each of the set of anomaly measures is associated with a performance metric and a time period, and the visualization comprises a grid of cells; a set of ranges of anomaly scores; and a set of displayable characteristics. The grid of cells is disposed on a display medium and arranged into rows and columns such that each cell belongs to one row and one column. Each cell in a particular row or column corresponds to either the same performance metric or the same time period. An anomaly score associated with the target entity, time period and performance metric corresponding to the row and column of the cell is associated with the cell. The displayable characteristic associated with the range into which the anomaly score falls is associated with the cell and displayed on the display medium at the location of the cell.
The above mentioned and other features will now be described with reference to the drawings of embodiments of the visualizations. The drawings are intended to illustrate, but not to limit, the embodiments described. The drawings contain the following figures:
As noted above, analysts examine data and improve the understanding of the entities represented by the data in order to more effectively manage tasks related to that entity. In general, analysis can comprise examining historical data associated with the performance of various aspects of a particular entity. Such historical data can come from many sources and in many forms. One common form is numerical data that represents measurements of various aspects of the performance of the entity. These measurements are referred to as 'performance metrics'. In general, any historical result that can be represented as a number associated with that result can be considered a performance metric.
Generally, a performance metric is associated with a specific period of time. For instance, a common type of measurement that is carried out on automobiles involves determining how many kilometers that the car can travel for each liter of fuel that is burned. This particular performance metric, kilometers-per-liter, can be calculated on the basis of a particular trip, a particular time period, or a particular tank of fuel. The particular value that is calculated may vary depending on the time period associated with the metric. For example, over a lifetime of operation, a car may get 10 kilometers-per-liter, while when measured over a specific time period, for example, June 2003 to August 2003, the kilometers-per-liter may be 12.
A technique will be described for representing performance metrics associated with a particular entity. One goal of these techniques is to allow someone who is performing the analysis to more quickly focus on the most significant portions of the data, and to more effectively evaluate the entity being described by the performance metric data.
In particular, a series of examples will be discussed giving details of the operation of various systems and techniques for providing visualizations to aid in analysis of data. The particular examples discussed below are related to financial analysis. As a result, the entities being examined are target companies operating in various industries, and they are measured using a variety of financial performance metrics, as will be discussed in further detail below. However, it will be understood that the systems and techniques discussed are applicable to entities in many other fields of analysis, each of which may use its own particular performance metrics. These may include, but are not limited to: long term health-care analysis of health metrics (analyzing measurements of blood pressure, weight, white blood cell count, and so on); medical research (analyzing measurements of organ function, blood chemistry or other treatment specific measurements); mechanical equipment performance (measurement of time between failures, downtime, availability, cost to repair failures, and so on); television ratings performance (number of households viewing, advertising revenue per half-hour, intent-to-view tracking, cost of production per half-hour, and so on); and athletic performance and statistics (defensive yards given up per game, batting average, win-loss ratio, and so on).
Financial analysts, such as managers of investment portfolios, analysts working for companies extending credit, and loan officers, make decisions everyday based on perceptions of a company's financial health. Their basis for this perception is generally in large part taken from information on the company's financial statement. Taken at its simplest, such financial analysts look for any financial data that doesn't seem to fit in, either because it represents an unusual financial circumstance for the company (which may indicate poor financial health), or because it doesn't conform to the analyst's existing knowledge of the company's financial circumstances (which may indicate improper or fraudulent financial reporting).
Such 'out of the ordinary' financial data is referred to generally as an 'anomaly'. A financial analyst would like to be able to recognize any financial anomalies, and to determine the significance of these anomalies as quickly and effectively as possible. Properly recognized and understood, financial anomalies can act as early warning signs of financial decline or fraud, which can allow an analyst to undertake the appropriate detailed investigation and consideration necessary to the proper evaluation of business transactions with companies exhibiting such anomalous finances. Such evaluation may help avoid transactions that are undesirable by recognizing developing problems before they happen, or by shaping the terms of agreements to take into account the nature of the anomalies detected.
In the discussion of the described systems and techniques below, the particular company of interest to a financial analyst or other investigator is referred to as the 'target' company. The target company is evaluated by determining values for one or more financial metrics and comparing these values to financial metric values for either: (a) the same company at earlier times; or (b) peer companies to the target company. These comparisons are used to generate a visual representation of the variations between the metrics for the target company and its historical performance and peers.
As discussed herein, a 'financial metric' may be any piece of financial data that is associated with the performance or operation of a company over a particular time period. For instance, a classic financial metric is net income. Other financial metrics include, but are not limited to: total revenue; inventory on hand; capital expenses; interest payments; debt; and earnings before interest, taxes, depreciation and amortization (EBITDA). While these and many other financial metrics are known in the art, their usage to identify financial anomalies has become progressively more difficult over time. As financial accounting has become increasingly complex, it has become more difficult to systematically identify financial statement fraud or financial decline.
Even when a broad scope of well-considered financial metrics is used to analyze the financial health of a target company, it can still be difficult to rapidly determine whether a particular value of a metric indicates a cause for further investigation or reconsideration of a potential business transaction or not. For example, knowing that a company generated 1.4 million dollars worth of sales last year is of little use without other indicia for comparison. Therefore, rather than simply learning the value of the metric, the analyst would like to determine whether the financial metric's value is out of the ordinary for the company, i.e. whether the metric is anomalous. The definition of an anomaly may change from one financial metric to the next. Limitations on anomalous values may also vary based on factors such as target company size, the industry in which the target company operates, and with the passage of time. In particular, changes over time can reflect both changes in the operation of the company, as well as changes in the overall economic environment.
In order to account for these variations and determine whether or not a given value for a financial metric for a target company is outside an expected range (i.e., anomalous), context information is used to form a basis for the analysis of the target company's financial metric data. As noted above, this context information can be taken from two primary sources: the target company's past performance, and the performance of the target company's peers. By using such context information to quantify the typical amount of variation present within the industry or within the company's own performance, it is possible to systematically and rigorously compare current financial metric data to context data and accurately assess the level of anomalous financial data in the target company's financial statements. In particular, the techniques described herein are well suited to identifying anomalous values in small sets of data. This can be significant because the amount of context data that is appropriate and relevant is often limited.
As noted above, context information is used to properly evaluate the degree to which a given financial metric is anomalous. In order to have an effective evaluation, the context data is selected to be appropriately relevant to the target financial metric for the target company. When selecting the appropriate context data over the time domain, it is generally desirable to look at the closest data available to the time period of interest. Since the time period of interest is usually the most recent data available, the appropriate scope of time to consider is a sequence of the most recent financial data available for the company—for example, the scope might correspond to the last 3 years of data.
Proper context data that accounts for the financial behavior of the industry and overall economic environment is found by using an appropriate group of 'peer' companies to the target company. A group of companies from the same industry and of similar size is selected to act as the appropriate peer group for the target company. "Similar size" may be determined by comparing one or more of a variety of indices of size. In one particular embodiment, "similar size" is determined by total sales. It will be understood that a variety of measurements, including the financial metrics themselves, can be used as the index of size. It is generally desirable to choose the peer group such that the target company lies in the middle of the group as measured by the selected index of size. This provides equal representation in the peer group of companies that are larger and smaller than the target company.
In a further particular embodiment, the peer companies may be selected from the group of companies that are classified within the same Standard Industrial Code (SIC) as the target company. If a database of companies with appropriate financial data is available, such as the database of information made available by Mergent, Inc., the peer group can be selected to be the companies in the database in the same SIC as the target company, and exhibiting the next four highest and next four lowest values for the index of size, e.g., total sales. It will be understood that other sized groups of similarly sized companies can be chosen, but that as noted above, it may be desirable to maintain a group of peer companies to both sides of the target company's size when possible.
In the event that there are not four companies that exhibit indices of size greater than the target company, it can often be effective to compare metrics that have been normalized by the appropriate size metric. For instance, if an analyst were using a target metric of outstanding debt to evaluate a target company, each peer company's debt could be normalized by being divided by that company's total assets, for example. Other financial metrics could also be used for normalization, including but not limited to total revenue or market capitalization.
By establishing the appropriate context, both in time and across the industry to the peers of the target company, the need for a subjective assessment as to whether a given financial metric is anomalously high or low can be avoided, and objective and automatic calculation can be made to detect and quantify financial anomalies.
Note that a financial metric's value can be either anomalously high, or anomalously low. While there generally is a particular direction that is recognized as being the preferable trend in a value (e.g., it is generally better to have high revenues than low revenues), it should be noted that this technique is designed to identify and quantify anomalies regardless of their polarity. This allows for the evaluation of data that appears to be too good to be trusted and may in fact represent a misleading or suspicious value for a financial metric. It also can be significant for detection of anomalies identified by simultaneous behavior of more than one financial metric. However, as will be discussed in greater detail below, the display of anomalous data will differ depending on whether the anomaly is in a positive or a negative direction.
In order to quantify the degree and direction of anomalousness associated with a particular value for a particular target company, statistical analysis of the value of the metric for that company can be carried out in comparison to: (a) a body of data representing the past behavior of that metric for the company; or (b) the behavior of that metric compared to the corresponding metric for peer companies to the target company. This comparison can be used to associate a score representing the degree of anomalousness associated with a particular value in comparison to the population to which it is being compared. This score can be calculated in various ways, some of which are discussed further below.
Such an 'anomaly score' for each financial metric for the target company can be calculated. For a given target company, each financial metric can be analyzed to determine the degree to which the value for that metric is different from the appropriate context data for that company and that metric. Depending on the nature of the context used (i.e., over time as opposed to across an industry), there are two different types of anomaly scores that can be calculated: the 'anomaly-within' score, and the 'anomaly-between' score.
'Anomaly-within' scores are scores calculated based upon the set of data representing a particular financial metric for a target company taken over different time periods. For instance, this data may represent financial metrics from successive fiscal quarters. The target value is generally the most recent value of the metric. In this way, anomaly-within scores measure a given company's financial data against its own past performance.
'Anomaly-between' scores are scores based upon the set of data for a given financial metric taken for a target company and a group of peer companies, all for the same time period. This data may represent the performance of a group of similarly situated companies all considered in a particular fiscal quarter. The anomaly-between scores measure a given company's financial data against the performance of its peer group.
The financial metrics that can be used to calculate these anomaly scores can include any of the financial metrics discussed herein, and may be taken from any source that provides appropriate data for comparison. These sources can include, but are not limited to: balance sheets, income statements, and cash flow statements, as well as metrics that are output by other financial analysis techniques.
Such information can be manually located and collated, or can be identified automatically. In addition to techniques known in the art for reading and analyzing sources of financial data, additional techniques that may be of use are described in copending U.S. patent application Ser. No. 10/401,310 entitled "Mathematical Decomposition of Table-Structured Electronic Documents" (Attorney Docket 126304) filed on 27 Mar. 2003, copending U.S. patent application Ser. No. 10/400,982 entitled "Automated Understanding and Decomposition of Table-Structured Electronic Documents" (Attorney Docket 126305) filed on 27 Mar. 2003, and copending U.S. patent application Ser. No. 10/401,259 entitled "Automated Understanding, Extraction and Structured Reformatting of Information in Electronic Files" (Attorney Docket 126311) filed on 27 Mar. 2003, the entirety of all of which is hereby incorporated by reference herein.
As noted above, one statistical technique to evaluate the degree to which a particular value in a group is an outlier, i.e. is anomalous, is to calculate what is known as a 'z-score' for the value in the group. Typical z-scores are based upon a calculation of the mean and the standard deviation of the group, and the technique for calculating z-scores is well known in the art. While such a statistical technique can be effective in evaluating the degree to which a single entry is anomalous in a well-populated group, z-scores can be shown to lose their effectiveness as an indication of anomalousness when used on sets of data that have only a few values. This lack of discriminating power with small data sets limits the utility of a z-score in many types of analysis discussed herein.
As a result, while it is possible to use z-scores as anomaly scores, it is often not desirable to do so. Therefore, while anomaly scores need not be z-scores, they may still use certain elements that are similar to those used in calculating the z-score. For instance, standard z-scores are based on a measurement of the central tendency of the group, and the variation within the group. In calculating a z-score, the central tendency is represented by the mean of the group, while the variation is represented by the standard deviation of the group. Such generalized elements such as central tendency and variation for the group can be useful in defining more effective anomaly scores.
The anomaly score calculation may be generally of the form
where A is the anomaly score, Xt represents the target value, CT represents a measure of central tendency of the set, and V represents a measure of the variation in the set.
As will be understood by those of skill in the art, the measure of central tendency may include any number of different calculations that describe the central tendency of the set of values, including but not limited to: mean, geometric mean, median, and mode. Similarly, those of skill in the art will appreciate that any of a variety of measures of variation may be used as well, including but not limited to: range, variance, standard deviation, coefficient of variance, and standard error. It will also be understood that the measurements of central tendency and variation may be based on more than one of the types of calculations, for instance, the measure of central tendency could be a weighted average of the mean and the mode, based upon the number of occurrences of the mode value among the data.
A set of techniques based upon the use of 'exceptional' statistical calculations are described herein that enable financial analysts to perform the desired forms of analysis on small sets of data, while retaining the ability to identify anomalous values within that small set. In general, an 'exceptional' technique, also referred to herein as an exceptional statistical technique, an exceptional measurement or an exceptional calculation, may be defined as a technique for calculating a statistical value associated with a set of data and a target value, such that the target value is excluded from the calculation of the exceptional measurement. Examples will be discussed in greater detail below. By using an exceptional technique, the particular target value within a group is prevented from skewing the measurements used to characterize that group.
As noted above, 'exceptional' measurements for central tendency and variation are used in one embodiment of an effective anomaly score calculation. In particular embodiments, techniques making use of the 'exceptional mean' and the 'exceptional deviation' are used.
In accordance with the definition provided above, the 'exceptional mean' is the mean of a set of data, excluding the target value. For example, consider the set of five values comprising 4, 5, 12, 13, and 16. The ordinary mean of this set of data is 10 (the sum of the values is 50, which when divided by the number of values, yields 10). However, the exceptional mean for this set of data, when the third value, 12, is the target value, is 9.5 (the sum of the four values in the set excluding the target value is 38, which when divided by the number of values excluding the target value, yields 9.5).
In a similar manner, the 'exceptional deviation' is the standard deviation of the set of data when the target value is excluded from the set of data. For example, consider the set of five values discussed above: 4, 5, 12, 13, and 16. The ordinary standard deviation is calculated by taking the square root of the variance about the mean of the group (in this case, the standard deviation is approximately 5.2). However, if the target value we are using in our anomaly score is the third value (i.e., 12), then the exceptional deviation is the standard deviation of the set comprising: 4, 5, 13 and 16 (approximately 5.9).
Note that the exceptional mean and exceptional deviation for a group may change depending upon which value in the group is the target value. Also note that in our example above, the exceptional mean is smaller than the ordinary mean, while the exceptional deviation is larger than the standard deviation.
As mentioned above, the use of context allows for the central tendency and variation used in the calculation of an anomaly score to vary based upon changes in the context data. By creating such sensitivity to changes over time within the target company, as well as sensitivity to variations in the particular industry of the target company, the anomaly score may be more effective at reflecting the degree that a given metric is a true outlier, and not merely indicative of a larger trend that includes the target company. More details and a further discussion of anomaly scores and exceptional statistical measures can be found in co-pending patent application "METHODS AND SYSTEMS FOR ANOMALY DETECTION IN SMALL DATASETS", application Ser. No. ______ (Attorney Docket 137267), the entirety of which is hereby incorporated by reference herein.
The detection of anomalies, however, is less effective if they are not relayed to account managers or other investment professionals in a way that motivates them to take appropriate investigative action. One mode for communicating the scope and nature of the anomalies present within the financial metrics of a target company is to prepare a visual display, or 'visualization', of the anomalies associated with the target company.
Such a visualization can be used to direct the attention of the account manager or other decision maker to those aspects of a particular target company that are most in need of more detailed investigation and evaluation to determine the potential impact and underlying cause of the identified financial anomalies. The technical effect of these presentation techniques is to allow the identification of the severity, frequency, and even the underlying causes of anomalous conditions, and to illustrate the relationship between identified anomalies. All of this information can be used to help draw conclusions about the risk of fraud, default or financial instability associated with the target company. Furthermore, appropriate visualizations may be used to compare the results from different target companies in order to assess relative strengths and risks of those companies.
One particular embodiment of a visualization is illustrated in
In this embodiment, each column generally represents one time period associated with the financial metrics being analyzed. As discussed above, this period may vary based on the availability of the financial metric data, and need not correspond to a specific length of time for every anomaly map. In the illustrated anomaly map 100, each column corresponds to a single fiscal quarter. For instance, column 110 represents the financial metrics associated with the first fiscal quarter of 2002. While ten columns representing periodic data are included in
In the embodiment of
With each column representing a time period, and each row representing a financial metric, the cells within the body of the anomaly map each represent the financial metric of the cell's row when evaluated for the target company in the time period associated with the cell's column. For instance, cell 150 represents the financial metric of "Long Term Debt" associated with row 160, for the time period of the fourth fiscal quarter of 2003, which is associated with column 170.
As noted above, anomaly scores that can be of especial use in directing the attention of a decision maker to the most relevant portions of an anomaly map can generally be divided into the categories of 'anomaly-within' and 'anomaly-between' scores. It is generally desirable to include both anomaly-within and anomaly-between scores on an anomaly map, and various arrangements can be used to do so.
In the illustrated example of
Another approach to the presentation of these scores is to place the anomaly-within and anomaly-between scores for a single metric on adjacent rows of the anomaly map in order to facilitate a rapid comparison between a target company's performance versus its own past compared to its performance versus the economy in its industry. An alternate approach is to have one anomaly map for the anomaly-within scores (performance versus past) and a separate anomaly map for the anomaly-between scores (performance versus industry).
Other techniques that may be used include the use of split cells. For instance, in each cell within the body of the anomaly map, the cell may be physically divided such that, for example, the left half of the cell represents the anomaly-within score and the right half the anomaly-between score, with a separate characteristic displayed in each portion. Such a technique can also be used with the cell split into separate portions vertically (top and bottom), or on an angle.
Another technique that can be used is to use different types of displayable characteristics for the anomaly-within and anomaly-between scores, and to display them in the same cell. For instance, a characteristic such as color could be associated with an anomaly-within score for a given metric and time period, and a separate characteristic, such as the size of the indicator, or the intensity of the color could be used for the anomaly-between score of the same metric and time period.
Each cell within the body of the anomaly map 100 is associated with a displayable characteristic that is associated with the value of the anomaly score corresponding to the time period and metric for that cell. Various displayable characteristics may be employed depending on the nature of the medium of presentation of the anomaly map. Examples include but are not limited to: color choice, color intensity, cross-hatching patterns, patterns of colors, scrolling or moving patterns, blinking, and such other display characteristics as would be known to those of skill in the art.
It will be understood that not every displayable characteristic is suitable for every display medium. For example, the use of blinking or moving patterns within a cell will be limited to displays, such as computer or television monitors, that are capable of displaying time-varying patterns. By contrast, more simple characteristics such as color choice and intensity are suitable for any color-capable display medium. Some patterns, such as cross hatching, are usable even in media where color is unavailable.
As can be seen in
For example, in one embodiment, the characteristic identified with block 210 is used to represent the most highly negative range of anomaly scores and the characteristic shown in block 220 is used for scores that are less negative. The characteristic in block 230 is used for slightly negative anomaly scores. Block 240 illustrates the characteristic used for neutral scores that deviate neither significantly negatively or positively from the group norm. The characteristic shown in block 250 is used for anomaly scores that are slightly positive. Block 260 illustrates the characteristic for somewhat positive deviations, and block 270 corresponds to the characteristic used to identify highly positive deviations.
The demarcations or breakpoints between the various categories corresponding to the blocks 210, 220, 230, 240, 250, 260, 270 of the legend 200 can be varied to adjust the sensitivity of the presentation made by the anomaly map. In one embodiment, the highly negative category, illustrated by the characteristic in block 210, is used for anomaly scores that are less than −50. This is indicated by the breakpoint 280, having a value of −50, illustrated between the display characteristics for the most negative score range (block 210) and the second most negative score range (block 220).
Similarly, it can be observed that breakpoint 282 with a value of −6 separates the somewhat negative category (block 220) from the slightly negative category (block 230). Breakpoints 284, 286, 288 and 290 separate the remaining categories from one another as shown in legend 200. These six breakpoints divide the spectrum of possible anomaly scores into seven ranges that each have an associated display characteristic. Some techniques for varying these breakpoints are discussed in greater detail below.
Within the body of the heat map, the characteristic displayed within each cell represents the range into which the value of the anomaly score corresponding to that cell falls. In
By associating the display in each cell with a visual characteristic, an account manager or other decision maker can quickly identify what portions of the map represent positive deviations, and which represent negative deviations. Furthermore, because of the organization of the anomaly map, it is easy for the account manager to identify particular time periods (vertical groupings) that tended toward particular types of deviations, as well as particular aspects of business finance (horizontal groupings) that tended toward positive or negative deviations.
For example, in the anomaly map illustrated in
Similarly, it can be seen that the horizontal grouping shown in row 130 indicates ongoing underperformance on the metric of Total Revenue for the target company over time when compared to its industry peers. A financial analyst may decide that such an indication gives a reason to look further into the revenues of this company, and the way in which these revenues are reported.
Of course, various other arrangements of characteristics associated with anomaly score ranges could be used. In general, display characteristics that allow for users of the visualization tool to rapidly grasp the direction (positive or negative) and magnitude of the anomaly score will be effective.
One alternate embodiment might substitute different characteristics for the various ranges. A variety of different mappings are possible. Several exemplary embodiments are shown in Table 1. For example, the embodiment discussed above with reference to
TABLE 1 Anomaly Score Range 1 2 3 4 5 Highly Dark red Red Solidly Tight Flashing negative filled vertical red cross hatching Somewhat Light red Orange Dotted Loose Red negative vertical cross hatching Slightly Pale red Yellow Single dot Single Light negative vertical red line Neutral White Green Blank Blank White SlightlyX Pale Blue Slash Single Light positive green horizontal green line Somewhat Light Indigo Cross Loose Green positive green hatching horizontal cross hatching Highly positive Dark Violet Double Tight Flashing green cross horizontal green hatching cross hatching
As shown in the table, possible display characteristics can include characteristics that are most effective on displays such as computer screens, television monitors, or other dynamic display media. These include such characteristics as: flashing indicators, variations in the speed at which an indicator flashes, moving patterns (such as a scrolling marquee style display within a cell), alternating patterns (such as a cell that flashes between red and yellow), and compound indicators that make use of more than one type of characteristic in a single cell (for instance a number or cross-hatch pattern indicated in a colored cell, or a cell that alternates between two different colors).
While such characteristics can be effective at placing more information within a single cell of the anomaly map, it is desirable that the anomaly map present the information in a way that allows it to be easily understood and accessed by the decision maker. As a result, it may be the case that the use of compound display characteristics will be better suited to items that deserve special attention, rather than areas where subtle discrimination is called for.
Another type of visualization that can be used to direct the attention of an account manager or other decision maker is illustrated in
The warnings being shown in the map in
A red flag represents an aggregation of anomaly scores for multiple related financial metrics. For example, a red flag might be triggered in the event of anomalously high revenue combined with anomalously high inventory value. By combining the individual metrics, the decision to signal a red flag will be based on bringing together information from several sources, which will increase the likelihood of catching an actual event deserving of attention.
Red flags differ from anomaly scores in that a red flag either occurs during a particular time period or does not, but there is no quantity associated with the flag. Because the basis for red flags may include one-time events and other non-numeric data, statistical analysis like that used in generating anomaly scores is less effective. As a result, there is no need for a legend mapping various levels of red flag values to different display characteristics. There is a simply a single symbol or display characteristic that is used in a cell to indicate that the particular red flag is raised for the indicated time period. In
A red flag may also be useful for presenting direct indicators for single-time events. For example, when examining the occurrence of infrequent events such as FTC or SEC investigations, management changes, mergers or acquisitions, or other one-time or rare events, it is often simply effective to indicate whether or not such an event occurred within a particular time period.
This can be done by placing a recognizable display characteristic in the body of a red flag map 300, as shown in
This can be seen on
In the illustrated embodiment, a solidly filled cell indicates the presence of a red flag, and an empty cell represents the absence of a flag. Another characteristic that can be used to indicate a red flag is a red cell, when a color display medium is used. As discussed above with respect to the various possible display characteristics for anomaly score ranges, it is possible to use a wide variety of display characteristics as are known in the art in order to indicate the presence or absence of a red flag, including all of the various characteristics noted above for use with anomaly map visualizations.
As in the anomaly map visualization, vertical groupings of flags can be rapidly identified as showing a time period in which there were multiple red flags to be noted (see columns 320 and 330 for example), while horizontal groupings illustrate persistent occurrence of a single red flag (see rows 340 and 350 for example).
In addition to horizontal groupings in a single row showing persistent flags for the same type of event, the arrangement of the red flags into rows can be used to draw the appropriate attention of an analyst. By arranging the rows such that the red flags for related aspects of financial analysis are located closer together than the red flags for unrelated aspects of the target company's finances, it is possible to more rapidly identify those general areas in which red flags are occurring and deserve further investigation.
For example, as seen in
For example, as can be seen in row 420, the red flag associated with a sharp increase in inventory is indicated in various periods—illustrated in the body of the map of red flags by the "filled in" display characteristic. In row 430, the map indicates that the red flag for unusually high debt given tangible assets is also illustrated sporadically over time, and not always in the same time periods as the red flags in row 420. Both of these red flags are related to misleading financial warning signs, and both are associated with the assets of the company, either tangible assets or inventory. By locating these indicators in rows that are disposed within the map close to each other, an overall pattern of potentially misleading financial data associated with assets can be seen. If these red flags were located in widely separated rows, the overall pattern would seem much less coherent and related.
In an alternative embodiment, rather than grouping related red flags together, the red flags are ranked with respect to the frequency with which each one occurs. In this embodiment, those red flags occurring most frequently are placed at the top of the map and those red flags occurring infrequently or not at all are placed at the bottom of the map. Although useful in assisting the analyst to quickly assess the most chronic problems, this placement becomes difficult to quickly assimilate since the order of the red flags is no longer fixed.
As a result, effective placement of rows for red flags that are related or associated with one another helps to create groupings of related red flags that more easily draw the attention of the decision maker when the red flag map is viewed. In essence, the same amount of overall indicated variation is more significant if it is arranged in a way that suggests a coherent pattern, rather than simple random occurrences. By arranging the rows in such an ordered manner, such patterns are more easily detected visually and can be investigated more effectively.
The techniques and systems described herein can be used to provide an advantage to an account manager or other user who is tasked with evaluating the risks associated with a business transaction with a target company based upon its financial history. At their simplest, the visualizations presented herein provide a way for rapidly assessing the overall degree of anomalousness associated with a given target company's financial performance. However, the appropriate use and understanding of the techniques discussed herein allows a greater appreciation of the nature and significance of any anomalous financial performance by the target company.
One example of such information is that the visualizations discussed herein provide a way to rapidly identify the frequency or persistence of anomalies, particularly if a single anomaly shows up consistently in the absence of other anomalous behavior. Such an observed pattern in the visualization focuses the attention of the decision maker on the particular anomaly for that company, rather than industry or the particular time period.
Another example of a rapidly identifiable pattern would be if a number of anomalous results were all observed in a single time period in an otherwise non-anomalous visualization. Such indications direct the attention of the decision maker to the occurrences of that particular time period.
Such visualizations that indicate the direction, severity, and chronology of various anomalous behavior allow for the decision maker to more rapidly assimilate and comprehend the nature of the financial behavior of the target company. Patterns of anomalies may also be used as indicators of larger financial behaviors, e.g. anomalous results in a particular pattern may indicate a general trend of aggressive revenue recognition within a particular company, or indicate that there is a tendency to overstate the quality of the company's earnings, or to overvalue intangible assets in order to protect the bottom line. Such patterns of "anomaly signatures" can be recognized visually based on the way that anomalies cluster over time and in particular rows.
Another advantage is that the visualizations can be used to provide a way for the decision maker to rapidly access further supporting information that may be useful in properly understanding the underlying causes and significance of anomalous financial results. For instance, each block on an anomaly map can be associated with a link to supporting information that corroborates the result identified in that block.
For example, in an anomaly map that was displayed using an interactive medium, such as a computer, a hot link using HTML or another markup language can be used to access supporting data or further details. If the anomaly map of
A similar technique can be used on a red flag map by linking blocks associated with each red flag to appropriate supporting data. This could include material such as press releases indicating one-time events (such as management changes), and links to the underlying financial metrics for red flags that are associated with collections of individual anomaly score results. With such links in place, these visualizations can be used as an anomaly exploration tool with which to browse a company's financial status and history as characterized by its financial anomalies, financial trends, and company behavior patterns.
In general, such visualizations allow decision makers to analyze the financial behavior of a target company for a potential business transaction. For instance, two companies may have the same amount of their overall financial results within the last three years that would be considered anomalous, but the way that those anomalous metrics are distributed can be different in ways that dramatically impact the overall financial risk associated with the company.
For instance, a company with a moderate amount of slightly negative and slightly positive anomalies scattered throughout its anomaly map might be presumed to be experiencing normal variations and drift due to market forces or other vagaries of the economy. On the other hand, the same number of anomalies concentrated within one or two fiscal periods indicates something else entirely. It is much more likely that there were particular events or strategies that resulted in anomalous behavior localized at those times.
Even in the case where most of the anomalies are found in one or two time periods, the impression of the company may be different if those anomalous results are recent as opposed to if they are more remote in time. A company that showed anomalous behavior at a particular time and then shows no anomalous financial results since that time appears to have taken corrective action. On the other hand, anomalous results that continue to the current time may indicate behavior that still requires correction.
Similarly, if the same amount of anomalous behavior is found consistently over time, but only in some metrics, then there may well be a reason to investigate the performance associated with these metrics in more depth. It could be the case that such a set of anomalies is consistently observed because that anomaly is endemic to that particular target company, or it may be the case that there is consistent poor financial management with respect to those issues at that company.
In addition to the embodiments described above, other embodiments may include additional or alternate aspects, as described below. For example, in one other embodiment of the systems and techniques provided herein, a facility to vary the breakpoints between the various levels of associated display characteristics can be provided. Such a feature can be used to allow the user to select the level of anomaly score at which a particular cell will switch from one display characteristic to another. For example, rather than having the display characteristic illustrated in block 210 of
Such a change would result in a greater number of cells falling into the range of scores corresponding to block 220, and a smaller number falling into the range corresponding to block 210. By altering the breakpoints in this way, there will be less of the most extremely negative display characteristic displayed.
Note that this change does not change the underlying anomaly score calculations, but only the display characteristic that is associated with any particular resulting anomaly score. In general, by altering these breakpoints (280, 282, 284, 286, 288, 290), it is possible to increase the ability of the user to distinguish between particular anomaly score ranges of interest.
In addition to presenting the user with the ability to alter each of the breakpoints between the display characteristics in the legend 200, a more general and broad control over the center point and sensitivity of the display may be provided through the use of a pair of user controls.
One such example is shown in
In the example discussed above with respect to
Altering the center of the display range can be useful to get a better view of a particular company's anomaly map that has more values to one side of the current center point than the other. For example, if an account manager wished to get a more clear view of the financial metrics of a company where the majority of the anomaly scores were negative, skewing the center of the display range downward would make use of more of the effective range available to differentiate between the varying degrees of negative scores.
The sensitivity adjustment operates in a similar way to the center point adjustment in that it is used to adjust the breakpoints between the ranges associated with each display characteristic. However, instead of shifting all the breakpoints the same way, the sensitivity adjustment allows a user to change the size of each range associated with a particular display characteristic. For example, in
For example, to allow more discrimination between similar anomaly scores, the sensitivity is increased, making the ranges smaller, and allowing for finer visual discrimination between anomaly scores. The trade off is that more scores will be pushed into the more extreme categories, resulting in less significance to the display characteristics associated with the most extreme anomaly scores.
By varying these breakpoint parameters, the visualization tool can be tuned to view particular anomaly maps in a manner that provides for the most useful decision making analysis for the account manager or other financial analyst.
It will also be understood by those of skill in the art that in addition to the buttons 650, 660 shown in
In another embodiment of the visualization tools presented herein, it is possible to use a different number of display characteristics than are discussed above and illustrated in
If display characteristics that can be varied continuously (or nearly continuously) are available (such as the brightness of a color, or the speed of a moving pattern), then an effectively infinite number of levels can be made available. In such circumstances, the display characteristic is related in some way directly to the particular anomaly score, rather than being grouped into a single range, all of which is displayed in the same manner. It will be understood to those of skill in the art that such continuously variable display characteristics will be more effective on certain types of display media than others. For instance, anomaly visualizations that are transmitted by facsimile or presented in monochrome are less well suited to continuous variations in color or shading than visualizations that will be viewed directly on color displays.
Another embodiment makes use of a separate anomaly map for the anomaly-within and the anomaly-between visualizations, and allows a user to toggle between the two displays. If such a display is used and the time periods and financial metrics are located in the same places on both the anomaly-within and anomaly-between maps, this technique can draw an analyst's attention to those areas where the performance of the target company differs when compared to its past versus when compared to its peers. Such deviations will show up as cells that change their displayed characteristic dramatically when the views are toggled. For instance, if using a color mapping scheme such as the scheme in column 1 of Table 1, cells that change from a red hue to a green hue when the visualizations are toggled are cells with that indicate good performance in one comparison and poor performance in the other.
In a further embodiment of the systems and techniques for anomaly visualization, it is also possible to include a display characteristic that is associated with cells in a map for which no evaluation was made. This can be useful in circumstances where, for example, data for some time periods is unavailable for a particular target company or its peers, red flag calculations that require unavailable data, or such other circumstances when a calculation cannot be made.
Examples of display characteristics that can be used for such "no test" cells can include a single slash or an "X" through a cell, or the use of a neutral color, such as gray. The choice of display characteristic is not limited in any way, but may be most effective when chosen to contrast with all of the other display characteristics in use in the legend of that particular visualization.
The use of such "no test" characteristics allows a decision maker to distinguish between financial records that were examined and produced no red flags or anomalous results, and financial results that are simply unknown. This prevents inadvertent conclusions that everything is non-anomalous in a particular time period when the truth is that there is no data to support a conclusion either way.
In addition to the embodiments described above with reference to financial metrics and financial analysis, it will be appreciated that the general analysis and visualization systems and techniques described herein may also be used in the context of other types of analysis of performance metrics. Such contexts could include, without limitation: medical studies, television ratings, real-estate pricing, insurance estimating, athletic performance monitoring, equipment reliability improvement, and health care monitoring.
For example, the use of anomaly maps may be effective in analyzing a patient's health and identifying anomalous areas in which a doctor might focus his examination of a patient. For instance, if blood pressure were to be monitored and recorded periodically for a large group of patients, it would be possible to identify, even in the absence of pre-defined normal values, anomalous results for a patient when compared to his own history, as well as to the results of his peers. In one instance, it might be observed that a patient's blood pressure was consistently rising over a period of time. However, such a gradual rise might not be anomalous when compared to the blood pressure of the individual's peers, all of whom might be experiencing increasing blood pressure of a non-anomalous nature, simply due to aging. Such a result could be quickly identified through the use of an anomaly map as described above where blood pressure was a performance metric associated with the target entity of the patient, and where other patients with similar demographic characteristics (for example, age and gender) formed the peer group. By comparing the anomaly-within and anomaly-between maps, a doctor or other medical practitioner could more rapidly identify those aspects of the patient's health metrics that deserved further attention and those that simply represented ordinary variation.
The various embodiments of anomaly visualizations and the techniques for creating and using them described above thus provide a way for analysts such as account managers and financial analysts to evaluate the target entity. These techniques and systems also provide a way to compare companies and to evaluate the risk associated with business transactions with target companies.
Of course, it is to be understood that not necessarily all such objects or advantages described above may be achieved in accordance with any particular embodiment. Thus, for example, those skilled in the art will recognize that the systems and techniques described herein may be embodied or carried out in a manner that achieves or optimizes one advantage or group of advantages as taught herein without necessarily achieving other objects or advantages as may be taught or suggested herein.
Furthermore, the skilled artisan will recognize the interchangeability of various features from different embodiments. For example, the use of links in cells to provide access to underlying data described with respect to one embodiment can be adapted for use with the sensitivity and centering adjustments described with respect to another. Similarly, the various features described, as well as other known equivalents for each feature, can be mixed and matched by one of ordinary skill in this art to construct visualization techniques and systems in accordance with principles of this disclosure.
Although the systems herein have been disclosed in the context of certain embodiments and examples, it will be understood by those skilled in the art that the invention extends beyond the specifically disclosed embodiments to other alternative embodiments and/or uses of the systems and techniques herein and obvious modifications and equivalents thereof. Thus, it is intended that the scope of the invention disclosed should not be limited by the particular disclosed embodiments described above, but should be determined only by a fair reading of the claims that follow.
|Cited Patent
|Filing date
|Publication date
|17 Oct 1997
|16 Nov 1999
|Martz; David R.
|Method for relational ordering and displaying multidimensional data
|25 Oct 2002
|9 Aug 2005
|Michaud Partners, Llp
|Portfolio rebalancing by means of resampled efficient frontiers
|22 Oct 2001
|12 Dec 2006
|Starmine Corporation
|Security analyst estimates performance viewing system and method
|14 Dec 2000
|2 May 2002
|Johnson Christopher Donald
|Methods and apparatus for automated underwriting of segmentable portfolio assets
|31 Jul 2002
|5 Feb 2004
|Feldman Stanley J.
|Method for enterprise valuation
|21 Aug 2003
|8 Jul 2004
|Dirk Noetzold
|Method and system for valuation of complex systems, in particular for corporate rating and valuation
|17 Jun 2003
|23 Dec 2004
|Oracle International Corporation
|Audit management workbench
|19 Apr 2004
|6 Jan 2005
|Portfolio Search, Inc.
|Methods for evaluating the financial strength of a holding in comparison to other holdings
|29 Oct 2002
|2 Jun 2005
|Eder Jeff S.
|Method of and system for analyzing, modeling and valuing elements of a business enterprise
|29 Nov 2004
|9 Jun 2005
|Joel Jameson
|Methods and systems for accurately representing corporate financial results in light of equity-based compensation and contingent transactions
|6 Apr 2004
|6 Oct 2005
|Pricewaterhousecoopers Llp
|Systems and methods for investigation of financial reporting information
|2 Aug 2004
|8 Dec 2005
|Accenture Global Services Gmbh
|Shareholder value tool
|30 Oct 2003
|27 Jul 2006
|Ran Zilberman
|Electronic interpretation of financials
|Citing Patent
|Filing date
|Publication date
|16 Oct 2007
|1 Dec 2009
|General Electric Company
|Method and system for predicting turbomachinery failure events employing genetic algorithm
|2 Feb 2007
|14 Sep 2010
|Microsoft Corporation
|Detecting and displaying exceptions in tabular data
|2 Feb 2007
|14 Sep 2010
|Microsoft Corporation
|Dynamically detecting exceptions based on data changes
|2 Aug 2006
|30 Nov 2010
|Pensiondcisions Limited
|Method and system for measuring investment performance
|9 Dec 2004
|7 Dec 2010
|International Business Machines Corporation
|Method and system for displaying prioritization of metric values
|10 Dec 2008
|31 Jan 2012
|At&T Intellectual Property I, Lp
|System and method for content validation
|26 Aug 2011
|28 May 2013
|Bank Of America Corporation
|Financial statement analyzer
|21 Dec 2011
|19 Aug 2014
|At&T Intellectual Property Ii, L.P.
|System and method for content validation
|30 Jan 2012
|9 Sep 2014
|Mastercard International Incorporated
|Methods and systems for analyzing weirdness of variables
|18 May 2010
|13 Jan 2015
|Amazon Technologies, Inc.
|Systems and methods for selecting components for inclusion in portions of a displayable file
|7 Jul 2011
|17 Feb 2015
|International Business Machines Corporation
|Interactive data visualization for trend analysis
|23 May 2014
|31 Mar 2015
|Visual BI Solutions Inc.
|Multi-selection widget for selecting multiple time periods at multiple levels within a time dimension hierarchy
|7 Feb 2008
|12 May 2015
|Future Route Limited
|Methods and apparatus for analysing and/or pre-processing financial accounting data
|11 Jan 2013
|26 May 2015
|Surfwax, Inc.
|User interfaces for displaying relationships between cells in a grid
|9 Dec 2004
|4 May 2006
|International Business Machines Corporation
|Method and system for displaying prioritization of metric values
|2 Aug 2006
|1 Mar 2007
|Mannion Graham J
|Method and system for measuring investment performance
|7 Feb 2008
|12 Aug 2010
|Future Route Limited
|Methods and Apparatus for Analysing and/or Pre-Processing Financial Accounting Data
|6 Apr 2010
|8 Sep 2011
|Jeremy Elser
|System and methods for comparing real properties for purchase and for generating heat maps to aid in identifying price anomalies of such real properties
|8 Sep 2011
|Jeremy Elser
|System and Methods for Management of Real Property
|1 Dec 2011
|Quantum Intelligence, Inc.
|Multiple Domain Anomaly Detection System and Method Using Fusion Rule and Visualization
|2 Feb 2012
|Nokia Corporation
|Displaying information
|10 Jan 2013
|International Business Machines Corporation
|Interactive data visualization for trend analysis
|26 Aug 2011
|28 Feb 2013
|Bank Of America
|Financial statement analyzer
|3 Mar 2011
|9 Sep 2011
|Jeremy Elser
|System and methods for management of real property and for comparing real properties for purchase
|30 Mar 2012
|11 Oct 2012
|Chicago Mercantile Exchange Inc.
|Index based on temporally staggered value samples
|U.S. Classification
|International Classification
|Cooperative Classification
|European Classification
|11 Mar 2005
Owner name: GENERAL ELECTRIC COMPANY, NEW YORK
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LACOMB, CHRISTINA ANN;HOOGS, BETHANY KNIFFIN;MIELE, JASON PAUL;AND OTHERS;REEL/FRAME:016361/0371;SIGNING DATES FROM 20050216 TO 20050228


text:
Automation blooms, training falters
The regulatory-laden federal procurement process is a tough beast to turn in any new direction. But some experts see a glimmer of hope for a bit of change in the coming five years.
The process will become more automated, some experts say. Malleable systems for writing contracts and new technologies will streamline workflow and decision-making processes. Those systems will also help capture individual departments' preferences for instituting policies and regulations, said Ray Bjorklund, senior vice president and chief knowledge officer at FedSources.
Some of the drive toward technology-based approaches comes from the younger professionals who are taking over procurement. They use technology throughout their daily lives and are likely to want to use tablet computers and smart phones to do their work, said Peter Tuttle, vice president of distributed solutions and a former Army contracting officer.
A day is coming when procurement software will no longer be the conventional end-to-end, all-encompassing solution, Tuttle said. Instead, it will be separated into modules that are designed to run on mobile devices and tailored to perform a specific step of the procurement process.
For example, a procurement officer could buy a module for one part of the acquisition process from one company, and buy software from a separate company for another part of the process. Then the officer could pay for the right to use both modules—for a single use—by using a purchase card, Tuttle said. However, it may not happen within five years.
"As long as the federal government provides common interface and standardization guidance, vendors can use their creativity to construct software solutions," he said.
The knowledge that comes from personal interest could become more valuable because training is likely to falter, some experts say. Along with the general strain on the federal workforce, that will make recruiting and retaining top-caliber people even more of a challenge, Bjorklund and several other experts said.
"I'd like to think the acquisition workforce will be better trained and that the role of acquisition professionals will evolve to that of a business adviser, rather than a buyer," said Larry Allen, president of Allen Federal Business Partners. "We've been saying that, though, for at least a dozen years now."
Unfortunately, training is always one of the first areas to take a hit when budgets have to be cut, and some agencies have already frozen the hiring of new employees, said Jaime Gracia, president and CEO of Seville Government Consulting.
"Combined with institutional knowledge leaving at alarming rates, which will worsen, I see a bleak horizon," Gracia said.
Allen said officials face two issues: They must learn how to attract midcareer people with experience, and then they must give them high-level responsibilities without upsetting the people who have worked in the organization for a long time but don't have the skills to be in senior positions.
Another aspect of procurement, insourcing vs. outsourcing, is also coming to a head, Allen said. Insourcing is likely to disappear as a significant trend in the next five years because the government will realize it can't afford to build and maintain the infrastructure to support a larger workforce, he added.
"That's not to say that we won't see tactical forays into insourcing as political winds shift, but there will be no — dare I say it living in Virginia? — 'tectonic' movement on this front," Allen said.
Read more of Federal IT's next act or return to the 2011 Federal List.
About the Author
Matthew Weigelt is a freelance journalist who writes about acquisition and procurement.
E-Mail this page
Printable Format
Advanced Search
Meet 100 women and men who are doing great things in federal IT.
Click here for the full list
In the newest Federal Employee Viewpoint Survey, feds show they're feeling better than they have over the past few years – but there's still room for improvement, especially in mission-critical IT.
The General Services Administration's Enterprise Infrastructure Solutions contract will anchor the agency's NS 2020 telecommunications strategy.
House and Senate both approve a measure to maintain funding at fiscal 2015 levels through Dec. 11.
The aviation regulator missed a Congressional deadline to develop rules of the sky-ways for drones.
Agency heads, acquisition officers and even CIOs don't need to be hard-core techies. FCW's four-part look at the expertise that's really required to make federal IT run.
FCW recognizes 14 individuals having an outsized impact early in their careers.
As Steve Kelman returns from a summer of aggressive medical treatment, he considers the all-too-common aversion to short-term sacrifices.
Massive open online courses can offer specialized IT instruction on a flexible schedule and on the cheap. That may not always mesh with government's preference for structure and certification, however.
The federal government craves more cybersecurity professionals. These six schools are helping meet that demand.
My fear with relying too much on automation for procurement is that the CO won't read the whole contract from end to end and word for word. A contract is a legal document which, if a loss is involved, the writer of the contract takes the loss. In this case Uncle Sam. This same type of situation occurs from too much reliance on Spell Checkers. The amount of typos and missing or incorrect wording seems to have increased.
8609 Westwood Center Drive, Suite 500Vienna, VA 22182-2215 703-876-5100
© 1996-2013 1105 Media, Inc. All Rights Reserved. This copy is for your personal, non-commercial use only.To order presentation-ready copies for distribution to colleagues, clients or customers, visit: www.1105Reprints.com

subset: commoncrawl


text:
Listen or Download Podcast Here
The Westover COW & PLOW CLUBS have, with great sadness, was discontinued effective August 2014.
Westover is now open the 2nd weekend of each month
>> Wines & Sparklers
>> Ports & Dessert Wines
>> $7 tasting fee deducted from your purchase
We are open 12-5 daily by appointment 510-537-3932 ext 1
We are open 12-5 on the folloing dates with no appointment (drop in) 9/27, 10/3, 10/12, 10/13, 11/14, 11/15, 12/12, 12/13
Most Holidays (please check our website or call)
Beautiful Setting for Picnics
Large Selection of Wines, Ports and Sparkling Wines
Westover's quiet and relaxing environment provides the ideal place to hold a special tasting event.
Enjoy the grounds
Beautiful backdrop
Westover produces the widest variety of ports in the United States.
We offer more varietals of wine than any other winery in the Livermore Valley.
Enjoy the beautiful grounds
Our 20+ varieties of port range from delicate, lightly sweet white ports to rich 12-year barrel aged tawnies, Portuguese style ports and unique varietal-spefic offerings such as petite sirah or cabernet sauvignon port. Westover also produces vineyard and varietal specific red and white wines, as well as a variety of champagnes and sparkling wines such as our acclaimed Cuvee La Peche.


text:
Junior Member
Last Activity: 5th October 2015 07:17 PM
About Me
- About warci
- Service provider
- Germany - Telekom Deutschland
- City
- Lower Saxony
- Home country
- Germany
- Devices
Showing Friends 1 to 1 of 1
- Krocko
- Senior Member
Most Thanked
Post Summary
jureca, djluisbento HIMA_UHL_TWRP_Nandroid_1.32.401.15 // Created with TWRP 184.108.40.206 (read only) More files for SKU 401, 206, 161 and 111 are collected here: http://www.handy-faq.de/forum/htc_one_m9_custom_roms/317884-htc_one_m9_firmware_ruu...
It worked, thanks :D If anyone should use another editor like Hex Workshop the position can be displayed as 004083FE (see attached files).
exocetdj For the collection: 18.104.22.168 TWRP-Backup incl. boot image, system partition (read-only) and Stock-Recovery
Yeah right I can see now the 0PJAIMG.zip would be the better option. Just need to figure which is the correct file to flash, the R1 or the R2.... Found this site which has some interesting info http://www.handy-faq.de/forum/htc_one_m9_custom_ro...
Hey Sneakyghost, i think the following command can be useful for your flashing tutorials or at least for your Task Scripts. It's the universal command (works fine on the m7, m8 and m9) to reboot directly into the RUUmode: adb reboot oem-78...


text:
Search Results: ''
Fluency in Play: Computer Game Design for Less Commonly Taught Language Pedagogy
Paperback: $10.01
Ships in 3-5 business days
AZ 100 Indie Films: A Centennial Celebration of the State of Arizona
Paperback: $15.00
Ships in 3-5 business days
A catalog of the 100 films selected by the Arizona Media Arts Center to celebrate the 2012 Centennial of Arizona statehood. Includes copious film and filmmaker information, a guide to film themes for... More > teachers and scholars, and a still from each film. Black & White with Color cover. 200 pages.< Less
Top 10see more >
Why Is This Sex Book...
Paperback: $15.50
Scaling Agile: A Lean...
Paperback: $20.00
7 Steps to Health and...
Paperback: $49.92
Speaking Magic
Paperback: $12.95
Getting Value out of...
Paperback: $14.99
Mickeyisms: 30 Tips for...
Hardcover: $23.65
The Other Side of the...
Paperback: $300.00
How to Follow Jesus
Paperback: $9.99
Breaking The Power Of...
Paperback: $10.00
Age Verification
The page you are attempting to access contains content that is not intended for underage readers.
Please verify your birth date to continue.


text:
Often when the word "hero" comes to mind, we think of caped crusaders, soldiers on the battlefield, and occasionally a toasted Quiznos...More →
Look. We all know this world is an unstable place. Regime change is all around, and its only a matter of time before the new...More →
A few weeks back, a a 160-foot giant squid was reported to have washed up on the Californian coast. Though it proved...More →
Millions of species have had their chance to make it big on planet Earth, but only one group has been the undisputed...More →
Welcome once again to Science Fact or Science Fiction, a little game we like to play here at Mindhut that will put...More →
Cash is an interesting concept. A group of people can pick any item to make currency by agreeing upon its value. This...More →
Everybody knows that the government knows exactly what they're doing and would never spend public funds carrying out weird or pointless experiments,...More →
If you Google Image search "god of the Internet," this resplendent image of a cat is the number one result. Indeed, kittehs...More →
The private spaceflight industry has really been kicking into high gear recently, so it may not be long before you can buy...More →
Super Earth. Just the name sounds pretty awesome. But Super Earths are actually a thing and have been in the news lately....More →


text:
Easy Valentine's Day Craft: Warm Fuzzies!
by Cindy Hopper
Do you have someone you can share a warm fuzzy with on Valentine's Day? Help your child share the love with these pom pom creatures, an easy-to-make gift for Valentine's Day.
The Lice Whisperer
by Amalah
My daughter keeps getting head lice from her best friend! How do I put a stop to it without embarrassing everyone?
How My Family Has Good Mornings
by Rachel Meeks
My family places importance on good mornings, but we are not good at mornings.
Gentle Care for Winter Scalps
by Amalah
A dry, itchy, flakey scalp is even less fun when it's attached to your preschooler.
Favorite Valentine Ideas (in addition to our own)
We love Valentine's Day over here. For years we have been creating and gathering ideas for easy Valentine's Day cards for kids, homemade Valentine's Day gifts, and fun Valentine's Day projects.
But I also love what I see out there on the web and want to share some of my favorite Valentine's Day gift ideas for kids (which I have been obsessively collecting on my Pinterest board).
Good Boys vs. Bad Guys
by Amalah
My little boys just want to be the "bad guys" all the time. How do I keep their play appropriate and safe?


text:
Thanks for taking the time to report this user.
Your participation helps keep Airbnb a safe and trusted community, and it doesn't go unnoticed!
Nicholas has completed online and offline ID verification.
Learn more »
It is always a pleasure to have such wonderful guests. They were a joy to host!


text:
After serving out a year of hard labor in the salt mines of Endovier for her crimes, 18-year-old assassin Celaena Sardothien is dragged before the Crown Prince. Prince Dorian offers her her freedom on one condition: she must act as his champion in a competition to find a new royal assassin.
Her opponents are men-thieves and assassins and warriors from across the empire, each sponsored by a member of the king's council. If she beats her opponents in a series of eliminations, she'll serve the kingdom for three years and then be granted her freedom. Celaena finds her training sessions with the captain of the guard, Westfall, challenging and exhilirating. But she's bored stiff by court life. Things get a little more interesting when the prince starts to show interest in her... but it's the gruff Captain Westfall who seems to understand her best.
Then one of the other contestants turns up dead... quickly followed by another. Can Celaena figure out who the killer is before she becomes a victim? As the young assassin investigates, her search leads her to discover a greater destiny than she could possibly have imagined.
©2012 Sarah J. Maas (P)2013 Audible, Inc.
"Readers seeking the political intrigue of Kristen Cashore's Graceling and its sequels or the deadly competition at the heart of The Hunger Games will find both in Maas's strong debut novel. . . . The verve and freshness of the narration make for a thrilling read." (Publishers Weekly, starred review)
"A teenage assassin, a rebel princess, menacing gargoyles, supernatural portals and a glass castle prove to be as thrilling as they sound. . . . Celaena is still just a teenager trying to forge her way, giving the story timelessness. This commingling of comedy, brutality and fantasy evokes a rich alternate universe with a spitfire young woman as its brightest star." (Kirkus Reviews)
"Throne of Glass by Sarah J. Maas is a tale full of surprises and shadows, where deadly seeds are sown with the promise of more danger to come. A fascinating glimpse into the dark side of Cinderella." (Colleen Houck, NYT Bestselling author of the Tiger's Curse series)
"The World's Most Powerful Assassin is a Sap."
So you've been put into a death camp prison mine at age 17. No one survives this death camp prison. You were there for 2 years. And you're the world's most powerful assassin; the queen of the underworld. And you've seen your people bled to death and you're full of rage and the milk of revenge.
So of course, when you're freed, the first thing you're going to do is become a total airhead and totally like fall in love with two guys because OMG one is like a total Prince, but the other one is like soooo loyal and stalwart. And of course, you will totally eat lots of amazing chocolates that will be listed out individually, because chocolate is like sooo delicious, right?
Don't get me wrong. I've been known to enjoy some full on romance fantasy adventures. I've read and enjoyed Sharon Shinn's Twelve Houses series, and one of those books, the one with Justin and the Lirren girl with the starry hair? That shit was as cheesy and romantic as it gets.
I've even read the first of 4521 trilogies in Jacqueline Carey's Kushiel Cycle. So don't think that I'm all EWWW, because of the elements of romance that this novel trots out. I'm not all EWWW, I'm all PLEEEEEEZE, because it just doesn't make any sense How are you going to be the world's most powerful assassin and the queen of the underworld and be a total napkin as well?
Anyway. That's what you get.
"Save the credit!"
Maybe an angst-ridden pre-teen would find all the drama and whining more acceptable.
Anything else.
Although all the characters constantly sounded arrogant and bratty, I don't think the narrator can be held solely responsible; the author's writing is just as much to blame.
All of them! The "world's best assassin" "wrings her hands in worry" and lets people sneak up on her on a regular basis. She spent her career carefully hiding her identity, yet now constantly whines that no one respects her or takes her seriously because they don't know who she is. The "spoiled prince" is an arrogant womanizing jerk, who refuses to marry because every woman he's ever met is "mentally inferior". Listening to them is like listening to a bunch of immature 12-year-old spoiled rotten brats.
Save the credit!!
I have 3 kids that occupy all the TV's in the house...enough said...addicted to audiobook ....shhhh don't tell my husband ;-)
"fun YA series... I could get hooked on"
The fiercest assassin in all the land turns out to be an 18 year old Girl. A kings tournament of thieves & assassins temporarily wins Celaena Sardothien her freedom from the salt mines to compete. The tournament champion wins their freedom and becomes the king's personal assassin for 4 years with a handsome salary, but there is dark magic effecting all of the land and an uprising is eminent. Celaena seems to be the key to fighting the dark magic and possible over throw the King. The only reason I didn't give all 5 stars is because there were slow spots in the book, but the second book seems to have more action in it so I'm going to stick with the series.
"It's no Game of Thrones..."
I have never felt compelled to write a review before, but I want to save you from a mistake. This book has a splash on the cover that says "Fans of Game of Thrones and Hunger Games will love this!" I think George R.R. Martin is a genius and his books have cost me numerous nights of sleep. And who doesn't love Katniss? So if this book had a strong sassy no bs female lead in a world similar to GOT this could be worth a read? No.The lead character can wield a weapon, though the fight specifics are often lacking. "She knocked him out." "She hit the target dead center." Her emotional development is young. She's vain and annoying. Maybe she will grow up as the series plays out, but I won't be along for the ride to find out.Though Katniss and Celaena would be fighting on the same side I think Katniss would be doing a lot of eye rolling in having to deal with Celaena. The male characters are a bit more interesting. If you are a hard core GOT fan you will be bored silly. This book has nowhere the depth, plot nor characters to hold your attention. Hunger Games is a more fair comparison and while some of the story is similar (strong female lead out to thwart an unfair society) the comparisons end there.I probably would have liked this book when I was in high school. If you are a grown up look elsewhere...
PS I would not have bothered to write this review had the publisher not tried to hoodwink GOT and HG fans...
"Intresting Book"
It has a lot of characteristics of a teen fantasy novel but is much better written then most.
It was a good book, I would defiantly read the rest of the series.
Very good book. Never had a dull moment. I would recommend this book to anyone who enjoys a solid story, good narration and a little bit of adventure and romantic drama. This book would be good for adults and teens.
"Loved it!"
Absolutely! I read this and it's sequel within a couple of days, it had me absolutely hooked - can't wait for the third one!!
Definitely Chaol - I won't give away any spoilers by explaining why
If you like this one and it's follow up I would suggest listening to Samantha Shannon's 'The Bone Season'.
Say something about yourself!
I found that the characters did not really live up to their background or expectations.
Everyone seem to be a 12-year-old adolescent.
Whether it was the performer, writer or director the entire story was mostly annoying.
I went through half of it and at the end all I could say was: wow what an annoying leading character this was.
I simply would not recommend this storyline or series.
Amber and Izzy
"The Audible version is Great! Worth the credit!"
This book was highly recommended by a friend, when it was first published. I prefer listening to books, so I was disappointed when I discovered that it wasn't available at Audible, so I ended up reading it. I was so excited to see it available on Audible. The performance by Elizabeth Evans brought a lot more excitement and fun to the book! I enjoyed it very much and am starting the second book!!
I am not a fan of romance, and in this book, it is secondary to the plot. I love the "kick-ass" Caleana charater! She is witty and smart too!
This book is a good YA fantasy, and works for adults in the same way as The Hunger Games. I am a big fan of The Dresden Files, Jane Yellowrock, Mercy Thompson, Kate Daniels, and I enjoyed this book!!
"Seriously good"
The narrator seemed to be really into the story, which of course adds to the experience. Then, the plot, characters, drama, etc. all were fantastic. The heroine is amazing: strong, humorous, badass, and relatable (maybe not the mad skills but her humanity).
Maas does a great job creating a whole world. It's a well thought-out fantasy universe and the characters seem real (real to that world, obviously it's not set in our time/world, etc.). The main three characters are the best, and although there may be the start to a love triangle, that's not the main focus to the storyline. It's more about a girl who was raised to be a killer, who was imprisoned for said killing, then gets the chance for freedom--and what she'll have to do to gain that freedom. Along the way, the reader gets to experience her thoughts, emotions, experiences, and more, and it's extremely well done.
I don't want to give away spoilers, but I enjoyed the action sequences. They were well written in the way that you could really imagine what was going on, like watching it through your mind's eye. Also, I liked anything where Calaena and the Crown Prince were together.
I listen to my audiobooks on my commutes, and sometimes I would take the long way home, desperately wanting to continue the story.
"Enjoyable :)"
I tried reading the book several time and just didnt seem to get into it, but listening to the book was for me so much better, the narrator did a great job.
I recommend this for an enjoyable listen :)
"Amazing book! Hard to put down!"
Great listen! Very action filled and quick paced, at the same time descriptive and emotional. Has everything you would ever need inside a book - crime, action, romance, drama, fantasy, historical feeling - all included!
Reminds me in some way of Game of Thrones (I loved the series, haven't read the book).
Interesting storytelling - from a perspective of a character ("I" form). It is interesting because, though the main character remains Celaena, author gives you insight from the other characters point of view, too, in that way quenching your thirst for information about their feelings.
Great narrator! I think she fits very well.
A captivating listen, if you enjoy game of thrones, name of the wind you will thoroughly enjoy this
"Fabulous characters"
This is a great story to fall in love with. The characters are beautifully constructed and the plot is engaging and twists enough to prevent prediction.
I defy you not to love Celaena Sardothien!
"I loved reading the book and enjoyed the audiobook"
Awesome, exciting, interesting.
Final fight scene between Celeana and Cane.
A badass heroine with a will of steel.
I thought the narrator's was a bit annoying.
This is an excellent book! I couldn't stop listening to this, the story, the plot, honestly made me feel part of it.
"Celaena Sardothien - even her name is feisty"
Loved the heroine, defiant but cool, a young woman who is able to find a way to be herself, even when everyone believes she is broken and controlled - excellent.
I am currently reading book three and enjoying it immensely; her characters are believable and attractive - you care about what happens to them, how much they hurt and all of their triumphs.
Elizabeth Evans's voice was not something that impacted strongly on me; I had read the first two books so wondered how much hearing it read might detract from the story, however, quite soon her reading allowed the protagonists to speak for themselves, without her voice intruding.
Doubt it, films rarely do justice to a good book
"I love this series"
I loved this book and I didn't have time to re-read it in physical form so I bought the audiobook. It more than met my expectations and was enjoyable to listen to. When looking back at it, I wish there had been more content as a whole, as in it could have been longer because I enjoyed everything about it. But, as a first novel it was excellent. I love the main character, because she isn't a stereotype and you should definitely at least listen to this one.
Report Inappropriate Content
If you find this review inappropriate and think it should be removed from our site, let us know. This report will be reviewed by Audible and we will take appropriate action.


text:
No Content Found
Street life: 20 years of international face-to-face fundraising
It is 20 years since face-to-face fundraising burst onto our streets and revolutionised individual giving. Despite its ripe old age, there is still life in the old legs yet, finds Stephen Cotterill.
Corporate Partnerships Survey 2012
from £35.00
Fundraising (with optional website)
from £89.00
Featured Jobs
Charity Finance Summit 2015
8 Oct 2015
Charity Technology Conference 2015
19 Nov 2015
Fundraising Live 2016
11 Feb 2016


text:
Temporary Price Drop
$49.99 - $109.98 -
List Price: $59.95 - $169.99
Was: - (Save )
Ends: Today at ET!
You save 35%
Clear Selection
{{{true:'No image available for this option.', false:'Hover an option to enlarge.'}[largeSwatch.src === 0]}}
This option is not available with your selected options.
HN-ESC312 The Step
The FREEStyle StepItem # HN-ESC312
Roll over to zoom
Click for full screen
- Non-slip platform with support blocks
- Adjusts from 4 to 6 to 8 inches high
- Supports up to 275/300 lbs.
- Choose from Circuit size or Health Club size
- Manufacturer's warranty included (see product guarantee)
Full Description
Delivery Estimate
More From This Brand
Guarantee & Returns
Recommended Accessories
Similar Steppers
Sunny Health & Fitness Mini Stepper with Resistance Bands
Sunny Health & Fitness Twister Stepper with Handle Bar
Marcy Deluxe Mini Stepper
Customer Reviews
Earn up to $3 in My Reward points
for reviewing this product when you join My Hayneedle Rewards
Write a review
200 points for each review = $2
300 points for reviews including an image or video = $3 Learn More
for reviewing this product when you join My Hayneedle Rewards
Write a review
200 points for each review = $2
300 points for reviews including an image or video = $3 Learn More
Customer Q & A
Be the first to ask a question about this product.
Enter your question, and one of our Customer Care experts will respond via email and also post the answer here.
Ask a Question
Email Address
Please Enter
Your Valid Email
Your Valid Email
Your Question
Please Enter
Your Question
Your Question
Sponsored Links
The Step
The FREEStyle Step
Finding a way to work out doesn't have to be a chore anymore thanks to The Freestyle Step. This step system includes a teal non-slip platform with purple support blocks that adjust the height from 4 to 6 to 8 inches depending on the intensity of the workout. The Circuit Size platform's stepping surface measures 25W x 11D inches and supports up to 275 pounds. Assembled dimensions: 28.75L x 14W x 8H inches. The Health Club Size platform's stepping surface measures 40W x 14D and supports up to 300


text:
bank of america corp (BAC:New York)
Last $15.69 USD
Change Today 0.00 / 0.00%
Volume 44.0M
As of 1:12 PM 10/6/15 All times are local (Market data is delayed by at least 15 minutes).
BOARD MEMBERS AFFILIATED WITH Graham Burton Goldsmith B Bus (ACCTG), FCPA, FAICD, F Fin.*
Chief Executive Officer and Co-Chief Investment Officer
*Data is at least as current as the most recent Definitive Proxy.
|Recently Viewed
MOST CONNECTED PEOPLE TO Graham Burton Goldsmith B Bus (ACCTG), FCPA, FAICD, F Fin.
|Robert John Edgar
|49 Relationships
|Graham John Kraehe
|37 Relationships
|Alice Joans M. Williams
|33 Relationships
|Neil Gregory Chatfield
|32 Relationships
|Kathryn J. Fagg
|28 Relationships
Sponsored Financial Commentaries
Sponsored Links
Report Data Issue
To contact BANK OF AMERICA CORP, please visit . Company data is provided by Capital IQ. Please use this form to report any data issues.
Please enter your information in the following field(s):
Your requested update has been submitted
Our data partners will research the update request and update the information on this page if necessary. Research and follow-up could take several weeks. If you have questions, you can contact them at firstname.lastname@example.com.


text:
Team Plays Well As Miller Hits ODAC's Top Ten October 2, 2015 A total group effort pushed the EMU volleyball women to a 3-1 win over North Carolina Wesleyan in Harrisonburg Friday night. As the Royals... [full story]
Royals Can't Blemish Roanoke's Hot Start September 30, 2015 EMU's volleyball women came up short in their upset bid of undefeated Roanoke College Wednesday, falling 3-0 in Harrisonburg. The two teams... [full story]
Volleyball Women Split To Maintain .500 Record September 26, 2015 For the second weekend in a row, the volleyball women split a tri-match. EMU's women romped to a sweep over West Virginia Tech Saturday in... [full story]
Huge Run Caps EMU's ODAC Sweep Of Randolph September 22, 2015 The Royals exploded on a match-ending 20-3 run to put the exclamation point on a sweep of visiting Randolph College Tuesday night. It was the... [full story]
Defense Leads EMU To Sweep To Wrap Tri-Match September 19, 2015 On a Jekyll-and-Hyde day, EMU's volleyball women ended on a high note, handily sweeping Ferrum to earn a split at a tri-match on their home... [full story]
ODAC Power Randolph-Macon Pushes Past EMU The Royals ran into a powerhouse Randolph-Macon team Friday night at home, and came out on the short end of a 3-0 decision. It will be a quick turnaround for Eastern Mennonite, as they step back onto the floor at 11:00am Saturday, playing Virginia Wesleyan as part of a tri-match in Yoder Arena. (Sep 18, 2015) [more]
Royals Athlete of the Week: Katie Miller Libero Katie Miller has been a dependable force and her teammates trust her on the floor. She is also the new record holder for career digs at EMU. (Sep 17, 2015) [more]
Women Erase Deficits In Sweep Of Mount Aloysius The Royals got their home stand started on the right foot, smacking a season high attack percentage in a hard-fought sweep of Mount Aloysius. Eastern Mennonite overcame sizeable deficits in each of the final two sets to improve to 4-4. (Sep 15, 2015) [more]


text:
Warm French Lentil Salad
A classic French dish, warm lentil salad is at once healthful, hearty, and comforting.
Total Time:
Level: Moderate
Serves: 4
Advertisement - Continue Reading Below
- 7 oz. French green (de Puy) lentils
- 3 sprig fresh oregano
- ¼ tsp. oregano leaves
- 1 medium onion
- 3 large stalks celery
- 2 large peppers (red, orange, or yellow)
- 2 tbsp. extra virgin olive oil
- salt
- pepper
- 3 tbsp. red wine vinegar
- 1 tbsp. Fresh dill
- 6 oz. baby spinach
- 2 oz. feta cheese
- In 4-quart saucepan, combine lentils, oregano sprigs, and 3 cups water. Cover; heat to boiling. Reduce heat to maintain simmer; cook 25 to 30 minutes or until tender.
- Meanwhile, chop onion, celery, and peppers. In 12-inch skillet, heat 1 tablespoon oil on medium. Add onion and celery; cook 7 to 8 minutes or until tender, stirring. Add peppers, 2 tablespoons water, and 1/4 teaspoon each salt and freshly ground black pepper. Cook 5 minutes or until peppers are crisp-tender, stirring.
- Drain lentils (discard sprigs); transfer to bowl. Stir in vinegar, dill, oregano leaves, onion mixture, 1/4 teaspoon each salt and freshly ground black pepper, and remaining oil.
- Divide spinach among 4 serving plates; spoon lentil mixture over. Top with feta, and garnish with dill.
More recipes like this
What do you think?
14 Genius Tricks to Keep Fresh Food From Spoiling Fast
Because you didn't pay $4.50 for a pint of moldy strawberries.
Prince Harry and William Ate These Mouse Cupcakes
And now you can, too!
How to Brew the Perfect Cup of Coffee
Everything you need to know — from the beans to the temperature.
McDonald's Will Soon Start Serving Organic Burgers
But there's only one place to get them.
More From Food & Recipes