Skip to content

Dataframe schemas

These are the unselected Arrow/Polars schemas from src/marketdata/storage/schema.py. Expressions supplied to GetDataframe may change the returned columns. Runtime quote output is separate from the full instrument catalogue.

Column Polars type Meaning
isin String Instrument identifier
lastPrice Int64 Last trade price
lastPricePct Float32 Last-price percentage change supplied by the source
closePrice Int64 Closing price
closePricePct Float32 Closing-price percentage change supplied by the source
limits Array(Struct, configured depth) Fixed-depth book rows

Prices use source/native units. Missing data remains nullable. A missing quote or count is not evidence of zero liquidity.

The option frame contains every base column plus:

Column Polars type Meaning
buyIv Float64 Calculated bid-side implied volatility
sellIv Float64 Calculated ask-side implied volatility
blackScholesPrice Float64 Calculated theoretical option price

Calculated values depend on instrument metadata, available history and current quotes. Missing or invalid inputs can produce null model prices or NaN implied volatilities; check for both null and non-finite values. The configured risk_free_rate defaults to 0.3.

limits has fixed depth, controlled by market_data.dataframe_max_limits_rows (default five). Rows are stored in best-to-deeper order. Each struct contains:

Field Polars type Meaning
buyPrice, sellPrice Int64 Bid/ask prices
buyVolume, sellVolume Int64 Bid/ask volumes
buyCount, sellCount Int64 Order counts when supplied by the source
timestamp Datetime(us) Book-row time; no timezone annotation in the Arrow schema
buyValue, sellValue Float64 Price multiplied by volume where available

Book timestamps and snapshot capture timestamps are distinct. Repeated snapshots do not establish that each stored row was refreshed. Preserve the timezone semantics of the producing source when interpreting a timestamp without a timezone annotation; do not silently assign the consumer host’s timezone.

Persistence schemas add the following fields to the base/option schema:

Column Polars type Meaning
snapshot_id String Snapshot identifier
captured_at Datetime(us, UTC) Capture time
market_type UInt8 Market discriminator

These metadata columns belong to recorded snapshots, rather than the default GetDataframe Arrow frames. The gRPC response carries capture time separately in GetDataframeResponse.timestamp.

See the Market Data protobuf reference for the separate scalar wire types used by GetBestLimits.