The `Frame` module provides an F#-friendly API for working with data frames. The module follows the usual desing for collection-processing in F#, so the functions work well with the pipelining operator (`|>`). For example, given a frame with two columns representing prices, we can use `Frame.diff` and numerical operators to calculate daily returns like this: let df = frame [ "MSFT" => prices1; "AAPL" => prices2 ] let past = df |> Frame.diff 1 let rets = past / df * 100.0 rets |> Stats.mean Note that the `Stats.mean` operation is overloaded and works both on series (returning a number) and on frames (returning a series). The functions in this module are designed to be used from F#. For a C#-friendly API, see the `FrameExtensions` type. For working with individual series, see the `Series` module. The functions in the `Frame` module are grouped in a number of categories and documented below. Accessing frame data and lookup ------------------------------- Functions in this category provide access to the values in the fame. You can also add and remove columns from a frame (which both return a new value). - `addCol`, `replaceCol` and `dropCol` can be used to create a new data frame with a new column, by replacing an existing column with a new one, or by dropping an existing column - `cols` and `rows` return the columns or rows of a frame as a series containing objects; `getCols` and `getRows` return a generic series and cast the values to the type inferred from the context (columns or rows of incompatible types are skipped); `getNumericCols` returns columns of a type convertible to `float` for convenience. - You can get a specific row or column using `get[Col|Row]` or `lookup[Col|Row]` functions. The `lookup` variant lets you specify lookup behavior for key matching (e.g. find the nearest smaller key than the specified value). There are also `[try]get` and `[try]Lookup` functions that return optional values and functions returning entire observations (key together with the series). - `sliceCols` and `sliceRows` return a sub-frame containing only the specified columns or rows. Finally, `toArray2D` returns the frame data as a 2D array. Grouping, windowing and chunking -------------------------------- The basic grouping functions in this category can be used to group the rows of a data frame by a specified projection or column to create a frame with hierarchical index such as `Frame<'K1 * 'K2, 'C>`. The functions always aggregate rows, so if you want to group columns, you need to use `Frame.transpose` first. The function `groupRowsBy` groups rows by the value of a specified column. Use `groupRowsBy[Int|Float|String...]` if you want to specify the type of the column in an easier way than using type inference; `groupRowsUsing` groups rows using the specified _projection function_ and `groupRowsByIndex` projects the grouping key just from the row index. More advanced functions include: `aggregateRowsBy` which groups the rows by a specified sequence of columns and aggregates each group into a single value; `pivotTable` implements the pivoting operation [as documented in the tutorials](../frame.html#pivot). The `stack` and `unstack` functions turn the data frame into a single data frame containing columns `Row`, `Column` and `Value` containing the data of the original frame; `unstack` can be used to turn this representation back into an original frame. A simple windowing functions that are exposed for an entire frame operations are `window` and `windowInto`. For more complex windowing operations, you currently have to use `mapRows` or `mapCols` and apply windowing on individual series. Sorting and index manipulation ------------------------------ A frame is indexed by row keys and column keys. Both of these indices can be sorted (by the keys). A frame that is sorted allows a number of additional operations (such as lookup using the `Lookp.ExactOrSmaller` lookup behavior). The functions in this category provide ways for manipulating the indices. It is expected that most operations are done on rows and so more functions are available in a row-wise way. A frame can alwyas be transposed using `Frame.transpose`. ### Index operations The existing row/column keys can be replaced by a sequence of new keys using the `indexColsWith` and `indexRowsWith` functions. Row keys can also be replaced by ordinal numbers using `indexRowsOrdinally`. The function `indexRows` uses the specified column of the original frame as the index. It removes the column from the resulting frame (to avoid this, use overloaded `IndexRows` method). This function infers the type of row keys from the context, so it is usually more convenient to use `indexRows[Date|String|Int|...]` functions. Finally, if you want to calculate the index value based on multiple columns of the row, you can use `indexRowsUsing`. ### Sorting frame rows Frame rows can be sorted according to the value of a specified column using the `sortRows` function; `sortRowsBy` takes a projection function which lets you transform the value of a column (e.g. to project a part of the value). The functions `sortRowsByKey` and `sortColsByKey` sort the rows or columns using the default ordering on the key values. The result is a frame with ordered index. ### Expanding columns When the frame contains a series with complex .NET objects such as F# records or C# classes, it can be useful to "expand" the column. This operation looks at the type of the objects, gets all properties of the objects (recursively) and generates multiple series representing the properties as columns. The function `expandCols` expands the specified columns while `expandAllCols` applies the expansion to all columns of the data frame. Frame transformations --------------------- Functions in this category perform standard transformations on data frames including projections, filtering, taking some sub-frame of the frame, aggregating values using scanning and so on. Projection and filtering functions such as `[map|filter][Cols|Rows]` call the specified function with the column or row key and an `ObjectSeries<'K>` representing the column or row. You can use functions ending with `Values` (such as `mapRowValues`) when you do not require the row key, but only the row series; `mapRowKeys` and `mapColKeys` can be used to transform the keys. You can use `reduceValues` to apply a custom reduction to values of columns. Other aggregations are available in the `Stats` module. You can also get a row with the greaterst or smallest value of a given column using `[min|max]RowBy`. The functions `take[Last]` and `skip[Last]` can be used to take a sub-frame of the original source frame by skipping a specified number of rows. Note that this does not require an ordered frame and it ignores the index - for index-based lookup use slicing, such as `df.Rows.[lo .. hi]`, instead. Finally the `shift` function can be used to obtain a frame with values shifted by the specified offset. This can be used e.g. to get previous value for each key using `Frame.shift 1 df`. The `diff` function calculates difference from previous value using `df - (Frame.shift offs df)`. Processing frames with exceptions --------------------------------- The functions in this group can be used to write computations over frames that may fail. They use the type `tryval<'T>` which is defined as a discriminated union: type tryval<'T> = | Success of 'T | Error of exn Using `tryval<'T>` as a value in a data frame is not generally recommended, because the type of values cannot be tracked in the type. For this reason, it is better to use `tryval<'T>` with individual series. However, `tryValues` and `fillErrorsWith` functions can be used to get values, or fill failed values inside an entire data frame. The `tryMapRows` function is more useful. It can be used to write a transformation that applies a computation (which may fail) to each row of a data frame. The resulting series is of type `Series<'R, tryval<'T>>` and can be processed using the `Series` module functions. Missing values -------------- This group of functions provides a way of working with missing values in a data frame. The category provides the following functions that can be used to fill missing values: * `fillMissingWith` fills missing values with a specified constant * `fillMissingUsing` calls a specified function for every missing value * `fillMissing` and variants propagates values from previous/later keys We use the terms _sparse_ and _dense_ to denote series that contain some missing values or do not contain any missing values, respectively. The functions `denseCols` and `denseRows` return a series that contains only dense columns or rows and all sparse rows or columns are replaced with a missing value. The `dropSparseCols` and `dropSparseRows` functions drop these missing values and return a frame with no missing values. Joining, merging and zipping ---------------------------- The simplest way to join two frames is to use the `join` operation which can be used to perform left, right, outer or inner join of two frames. When the row keys of the frames do not match exactly, you can use `joinAlign` which takes an additional parameter that specifies how to find matching key in left/right join (e.g. by taking the nearest smaller available key). Frames that do not contian overlapping values can be combined using `merge` (when combining just two frames) or using `mergeAll` (for larger number of frames). Tha latter is optimized to work well for a large number of data frames. Finally, frames with overlapping values can be combined using `zip`. It takes a function that is used to combine the overlapping values. A `zipAlign` function provides a variant with more flexible row key matching (as in `joinAlign`) Hierarchical index operations ----------------------------- A data frame has a hierarchical row index if the row index is formed by a tuple, such as `Frame<'R1 * 'R2, 'C>`. Frames of this kind are returned, for example, by the grouping functions such as `Frame.groupRowsBy`. The functions in this category provide ways for working with data frames that have hierarchical row keys. The functions `applyLevel` and `reduceLevel` can be used to reduce values according to one of the levels. The `applyLevel` function takes a reduction of type `Series<'K, 'T> -> 'T` while `reduceLevel` reduces individual values using a function of type `'T -> 'T -> 'T`. The functions `nest` and `unnest` can be used to convert between frames with hierarchical indices (`Frame<'K1 * 'K2, 'C>`) and series of frames that represent individual groups (`Series<'K1, Frame<'K2, 'C>>`). The `nestBy` function can be used to perform group by operation and return the result as a series of frems. [category:Frame and series operations]

public static class FrameModule