I run pandas OLS on a data set. If I run it with less than 20 time series in the x-value, everything works fine
Is there a maximum of dependants pandas.ols can handle?
This is what I’m doing, except that I have the data on file instead of fetching it with the DataReader:
from pandas import Series, DataFrame, ols
from pandas.io.data import DataReader
from DataContainer import DataContainer
import random
window = 21
basic = DataReader("BHI", "yahoo")
print len(basic)
dependance = 15
sp100 = [
"AAPL", "ABT", "ACN", "AEP", "ALL", "AMGN", "AMZN", "APC",
"AXP", "BA", "BAC", "BAX", "BK", "BMY", "BRK.B", "CAT", "C", "CL",
"CMCSA", "COF", "COP", "COST", "CPB", "CSCO", "CVS", "CVX", "DD", "DELL",
"DIS", "DOW", "DVN", "EBAY", "EMC", "EXC", "F", "FCX", "FDX", "GD", "GE",
"GILD", "GOOG", "GS", "HAL", "HD", "HNZ", "HON", "HPQ", "IBM", "INTC",
"JNJ", "JPM_1", "KFT", "KO", "LLY", "LMT", "LOW", "MA", "MCD", "MDT", "MET",
"MMM", "MO", "MON", "MRK", "MS", "MSFT", "NKE", "NOV", "NSC", "NWSA",
"NYX", "ORCL", "OXY", "PEP", "PFE", "PG", "PM", "QCOM", "RF", "RTN",
"SBUX", "SLB", "SLE", "SO", "SPG", "T", "TGT", "TWX", "TXN", "UNH", "UPS",
"USB", "UTX", "VZ", "WAG", "WFC", "WMB", "WMT", "XOM"
]
keys = random.sample(sp100, dependance)
data = {key: DataReader(key, "yahoo") for key in keys}
vals = {key: DataFrame(data=Series(data[key], name=key), index=basic.index) for key in data}
model = ols(y=basic, x=vals, window=window)
The error occurs as soon as dependance >= 20, but never for dependance <20.
The vals dict is just there cause my local datastructure gives the same name to every DataFrame, which ols doesn’t like, and I didn’t find a better way to rename the DataFrames.
Ok, this seemed to be a bug/feature of pandas 0.9.0 and/or statsmodels 0.4.2. Upgraded to pandas 0.10.0 and statsmodels 0.5.0 and everything works fine now.