[MSNoise] (no subject)

Wed May 14 07:27:09 UTC 2014

Hi Oscar,

(cc to the mailing list, it can be very useful to others too)

Le 13/05/2014 17:11, Oscar Alberto Castro Artola a écrit :
> Hello Thomas,
>
> I have checked for CCF for 2006-01-01 and is not empty. Everything 
> seems to be alright! Although I realized that when my filters are too 
> narrow I get this error:
>
> *oscar at bayta:~/sweet_noise$ py s05compute_mwcs.py*
> 2014-05-13 10:03:03,361 [INFO] We will recompute all MWCS based on the 
> new REF for TO.BUCU:TO.XALI
> 2014-05-13 10:03:03,798 [INFO] We will recompute all MWCS based on the 
> new REF for TO.PALM:TO.PLLI
> 2014-05-13 10:03:04,213 [INFO] We will recompute all MWCS based on the 
> new REF for TO.PALM:TO.XALI
> 2014-05-13 10:03:04,609 [INFO] We will recompute all MWCS based on the 
> new REF for TO.PLLI:TO.XALI
> 2014-05-13 10:03:05,989 [INFO] There are MWCS jobs for some days to 
> recompute for TO.BUCU:TO.XALI
> 2014-05-13 10:03:06,009 [DEBUG] Processing MWCS for: 
> TO_BUCU_TO_XALI.ZZ.04 - 2006-01-06 - 01 days
> /home/oscar/anaconda/lib/python2.7/site-packages/numpy/core/_methods.py:55: 
> RuntimeWarning: Mean of empty slice.
>   warnings.warn("Mean of empty slice.", RuntimeWarning)
> /home/oscar/anaconda/lib/python2.7/site-packages/numpy/core/_methods.py:77: 
> RuntimeWarning: Degrees of freedom <= 0 for slice
>   warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning)
> Traceback (most recent call last):
>   File "s05compute_mwcs.py", line 122, in <module>
>     cur, ref, f.mwcs_low, f.mwcs_high, goal_sampling_rate, -maxlag, 
> f.mwcs_wlen, f.mwcs_step)
>   File "/home/oscar/sweet_noise/MWCS.py", line 145, in mwcs
>     res = sm.regression.linear_model.WLS(phi, v, w**2).fit()
>   File 
> "/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/regression/linear_model.py", 
> line 381, in __init__
>     weights=weights, hasconst=hasconst)
>   File 
> "/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/regression/linear_model.py", 
> line 79, in __init__
>     super(RegressionModel, self).__init__(endog, exog, **kwargs)
>   File 
> "/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/base/model.py", 
> line 137, in __init__
>     self.initialize()
>   File 
> "/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/regression/linear_model.py", 
> line 88, in initialize
>     self.rank = rank(self.exog)
>   File 
> "/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/tools/tools.py", 
> line 381, in rank
>     D = svdvals(X)
>   File 
> "/home/oscar/anaconda/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", 
> line 146, in svdvals
>     check_finite=check_finite)
>   File 
> "/home/oscar/anaconda/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", 
> line 100, in svd
>     full_matrices=full_matrices, overwrite_a=overwrite_a)
> ValueError: failed to create intent(cache|hide)|optional array-- must 
> have defined dimensions but got (0,)
>
> That filter is between 0.04 - 0.046 Hz.

1/ the reason for the first error (filters too narrow) is that you don't 
have enough data points for doing weighted linear regression. If you use 
a 10 seconds MWCS_wlen at 20Hz, you have 200 points of data to compute. 
The FFT is done on the next power of 2 samples, so 256 here. Meaning, 
you have 256 FFT datapoints between 0 and Nyquist=10Hz, or a frequency 
step of 0.039 Hz. You filter is 0.006 Hz wide, so, at best, you'll have 
1 data point (+ [0,0]) to compute the phase shift, this is not enough. 
If you want to use such a narrow band, you should either CC at 100 Hz, 
or edit the s05 file to padd the input data with zeros, in order to have 
more resolution on the FFT (go up to the second or third next power of 2).

> When my filters are broader there is no problem with 
> /s05compute_mwcs.py/, but when I try to run /s06compute_dtt.py/ I get 
> this:
>
> *oscar at bayta:~/sweet_noise$ py s06compute_dtt.py *
> */home/oscar/anaconda/lib/python2.7/site-packages/numpy/oldnumeric/__init__.py:11: 
> ModuleDeprecationWarning: The oldnumeric module will be dropped in 
> Numpy 1.9*
> *  warnings.warn(_msg, ModuleDeprecationWarning)*
> *2014-05-13 10:07:17,935 [INFO] *** Starting: Compute DT/T ****
> */home/oscar/anaconda/lib/python2.7/site-packages/setuptools-2.2-py2.7.egg/pkg_resources.py:991: 
> UserWarning: /home/oscar/.python-eggs is writable by group/others and 
> vulnerable to attack when used with get_resource_filename. Consider a 
> more secure location (set with .set_extraction_path or the 
> PYTHON_EGG_CACHE environment variable).*
> *2014-05-13 10:07:18,139 [DEBUG] Found 731 updated days*
> *2014-05-13 10:07:18,203 [INFO] Loading mov=1 days for filter=12*
> *2014-05-13 10:07:18,204 [DEBUG] Processing 2005-01-01*
> *2014-05-13 10:07:18,216 [DEBUG] Processing 2005-01-02*
> *2014-05-13 10:07:18,222 [DEBUG] Processing 2005-01-03*
> *2014-05-13 10:07:18,228 [DEBUG] Processing 2005-01-04*
> *2014-05-13 10:07:18,235 [DEBUG] Processing 2005-01-05*
> *2014-05-13 10:07:18,243 [DEBUG] Processing 2005-01-06*
> *2014-05-13 10:07:18,250 [DEBUG] Processing 2005-01-07*
> *2014-05-13 10:07:18,257 [DEBUG] Processing 2005-01-08*
> *2014-05-13 10:07:18,263 [DEBUG] Processing 2005-01-09*
> *2014-05-13 10:07:18,271 [DEBUG] Processing 2005-01-10*
> *2014-05-13 10:07:18,278 [DEBUG] Processing 2005-01-11*
> *2014-05-13 10:07:18,284 [DEBUG] Processing 2005-01-12*
> *Traceback (most recent call last):*
> *  File "s06compute_dtt.py", line 304, in <module>*
> *    VecXfilt, prepend=False)*
> *  File 
> "/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/tools/tools.py", 
> line 289, in add_constant*
> *    var0 = data.var(0) == 0*
> *  File 
> "/home/oscar/anaconda/lib/python2.7/site-packages/numpy/core/_methods.py", 
> line 111, in _var*
> *    ret = ret.dtype.type(ret / rcount)*
> *AttributeError: 'float' object has no attribute 'dtype'*
>
2/ the second error is a bug in numpy 1.8.0, you should try to update 
numpy to 1.8.1 ! This is not MSNoise :-)

Best regards,

Thomas