[MSNoise] msnoise compute_cc problem

Thomas Lecocq thomas.lecocq at oma.be
Tue Nov 10 08:31:06 UTC 2015 

Laura,

It's most probably because you are using a 32bits system, allowing a max 
of 1024 MB of RAM being used... Could you work on a 64 bits machine ?

Thomas

Le 04/11/2015 10:18, Carretero Medina Laura a écrit :
> Hi Thomas,
>
> I've plotted the data_availability for the whole period and it seems to be correct (attached image). After updating ObsPy to 0.10.2 I've tried to compute the cross-correlation functions and it reports this error:
>
>
> 2015-11-03 15:27:38 [INFO] *** Starting: Compute CC ***
> /home/juan/anaconda/lib/python2.7/site-packages/setuptools-5.8-py2.7.egg/pkg_resources.py:1032: UserWarning: /home/juan/.python-eggs is writable by group/others and vulnerable to attack when used with get_resource_filename. Consider a more secure location (set with .set_extraction_path or the PYTHON_EGG_CACHE environment variable).
> 2015-11-03 15:27:38 [INFO] Will compute ZZ
> 2015-11-03 15:27:38 [INFO] New CC Job: 2014-01-24 (6 pairs with 4 stations)
> Traceback (most recent call last):
>    File "/home/juan/anaconda/bin/msnoise", line 9, in <module>
>      load_entry_point('msnoise==1.3.1', 'console_scripts', 'msnoise')()
>    File "/home/juan/anaconda/lib/python2.7/site-packages/msnoise-1.3.1-py2.7.egg/msnoise/scripts/msnoise.py", line 393, in run
>      cli(obj={})
>    File "/home/juan/anaconda/lib/python2.7/site-packages/click/core.py", line 700, in __call__
>      return self.main(*args, **kwargs)
>    File "/home/juan/anaconda/lib/python2.7/site-packages/click/core.py", line 680, in main
>      rv = self.invoke(ctx)
>    File "/home/juan/anaconda/lib/python2.7/site-packages/click/core.py", line 1027, in invoke
>      return _process_result(sub_ctx.command.invoke(sub_ctx))
>    File "/home/juan/anaconda/lib/python2.7/site-packages/click/core.py", line 873, in invoke
>      return ctx.invoke(self.callback, **ctx.params)
>    File "/home/juan/anaconda/lib/python2.7/site-packages/click/core.py", line 508, in invoke
>      return callback(*args, **kwargs)
>    File "/home/juan/anaconda/lib/python2.7/site-packages/msnoise-1.3.1-py2.7.egg/msnoise/scripts/msnoise.py", line 174, in compute_cc
>      main()
>    File "/home/juan/anaconda/lib/python2.7/site-packages/msnoise-1.3.1-py2.7.egg/msnoise/s03compute_cc.py", line 271, in main
>      basetime, tramef_Z = preprocess(db, stations, comps, goal_day, params, tramef_Z)
>    File "/home/juan/anaconda/lib/python2.7/site-packages/msnoise-1.3.1-py2.7.egg/msnoise/s03compute_cc.py", line 118, in preprocess
>      stream[i] = check_and_phase_shift(trace)
>    File "/home/juan/anaconda/lib/python2.7/site-packages/msnoise-1.3.1-py2.7.egg/msnoise/api.py", line 1221, in check_and_phase_shift
>      FFTdata = FFTdata * np.exp(1j * 2. * np.pi * fftfreq * dt)
> MemoryError
>
>
> There don't seem to be problems with the files either... I don't know what to do to fix it.
>
> Thank you very much for your time.
>
> Laura
> ________________________________________
> De: msnoise-bounces at mailman-as.oma.be [msnoise-bounces at mailman-as.oma.be] en nombre de Thomas Lecocq [thomas.lecocq at seismology.be]
> Enviado: martes, 03 de noviembre de 2015 11:01
> Para: msnoise at mailman-as.oma.be
> Asunto: Re: [MSNoise] msnoise compute_cc problem
>
> Hi Laura,
>
> Could you try to update ObsPy to 0.10.2 first ? If you restart the code,
> does it crash again ? How does the data_availability look for this day ?
> No problems with the files ? Like a crazy sampling_rate ?
>
> Cheers,
>
> Thomas
>
> Le 3/11/2015 10:24, Carretero Medina Laura a écrit :
>> Hi,
>> I've installed MSNoise 1.3.1 on a Ubuntu 32bit computer, but when I try to compute the cross- correlation functions it does it well for the first 3 years of data (from 1 January 2011 to 31 December 2013), but after that it reports this error:
>>
>>
>> 2015-11-02 18:45:57 [INFO] *** Starting: Compute CC ***
>>
>> /home/juan/anaconda/lib/python2.7/site-packages/setuptools-5.8-py2.7.egg/pkg_resources.py:1032: UserWarning: /home/juan/.python-eggs is writable by group/others and vulnerable to attack when used with get_resource_filename. Consider a more secure location (set with .set_extraction_path or the PYTHON_EGG_CACHE environment variable).
>>
>> 2015-11-02 18:45:58 [INFO] Will compute ZZ ZT
>>
>> 2015-11-02 18:45:58 [INFO] New CC Job: 2014-01-15 (6 pairs with 4 stations)
>>
>> Traceback (most recent call last):
>>
>>     File "/home/juan/anaconda/bin/msnoise", line 9, in <module>
>>
>>       load_entry_point('msnoise==1.3.1', 'console_scripts', 'msnoise')()
>>
>>     File "/home/juan/anaconda/lib/python2.7/site-packages/msnoise-1.3.1-py2.7.egg/msnoise/scripts/msnoise.py", line 393, in run
>>
>>       cli(obj={})
>>
>>     File "/home/juan/anaconda/lib/python2.7/site-packages/click/core.py", line 700, in __call__
>>
>>       return self.main(*args, **kwargs)
>>
>>     File "/home/juan/anaconda/lib/python2.7/site-packages/click/core.py", line 680, in main
>>
>>       rv = self.invoke(ctx)
>>
>>     File "/home/juan/anaconda/lib/python2.7/site-packages/click/core.py", line 1027, in invoke
>>
>>       return _process_result(sub_ctx.command.invoke(sub_ctx))
>>
>>     File "/home/juan/anaconda/lib/python2.7/site-packages/click/core.py", line 873, in invoke
>>
>>       return ctx.invoke(self.callback, **ctx.params)
>>
>>     File "/home/juan/anaconda/lib/python2.7/site-packages/click/core.py", line 508, in invoke
>>
>>       return callback(*args, **kwargs)
>>
>>     File "/home/juan/anaconda/lib/python2.7/site-packages/msnoise-1.3.1-py2.7.egg/msnoise/scripts/msnoise.py", line 174, in compute_cc
>>
>>       main()
>>
>>     File "/home/juan/anaconda/lib/python2.7/site-packages/msnoise-1.3.1-py2.7.egg/msnoise/s03compute_cc.py", line 266, in main
>>
>>       basetime, tramef_Z, tramef_E, tramef_N = preprocess(db, stations, comps, goal_day, params, tramef_Z, tramef_E, tramef_N)
>>
>>     File "/home/juan/anaconda/lib/python2.7/site-packages/msnoise-1.3.1-py2.7.egg/msnoise/s03compute_cc.py", line 144, in preprocess
>>
>>       trace.detrend(type="linear")
>>
>>     File "/home/juan/anaconda/lib/python2.7/site-packages/obspy-0.9.2-py2.7-linux-i686.egg/obspy/core/util/decorator.py", line 249, in new_func
>>
>>       return func(*args, **kwargs)
>>
>>     File "/home/juan/anaconda/lib/python2.7/site-packages/obspy-0.9.2-py2.7-linux-i686.egg/obspy/core/util/decorator.py", line 232, in new_func
>>
>>       return func(*args, **kwargs)
>>
>>     File "/home/juan/anaconda/lib/python2.7/site-packages/obspy-0.9.2-py2.7-linux-i686.egg/obspy/core/trace.py", line 1732, in detrend
>>
>>       self.data = func(self.data, **options)
>>
>>     File "/home/juan/anaconda/lib/python2.7/site-packages/scipy/signal/signaltools.py", line 1560, in detrend
>>
>>       coef, resids, rank, s = linalg.lstsq(A, newdata[sl])
>>
>>     File "/home/juan/anaconda/lib/python2.7/site-packages/scipy/linalg/basic.py", line 550, in lstsq
>>
>>       resids = np.sum(np.abs(x[n:])**2, axis=0)
>>
>> MemoryError
>>
>> I've made more attempts but now it takes 4000 seconds per day and sometimes it is interrupted by reporting the same error above.
>>
>> Some of the parameters used are: cc_sampling_rate: 20 Hz, ZZ correlation only, resampling method: resample, filter parameters (Low: 0.01, Mwcs low: 0.12, Mwcs high: 0.98, high: 1, Rms threshold: 0.0, Mwcs wlen:10 and Mwcs step: 5)
>>
>> What can I do? Thank you very much in advance.
>> Laura.
>>
>>
>> _______________________________________________
>> MSNoise mailing list
>> MSNoise at mailman-as.oma.be
>> http://mailman-as.oma.be/mailman/listinfo/msnoise
> _______________________________________________
> MSNoise mailing list
> MSNoise at mailman-as.oma.be
> http://mailman-as.oma.be/mailman/listinfo/msnoise
>
>
> _______________________________________________
> MSNoise mailing list
> MSNoise at mailman-as.oma.be
> http://mailman-as.oma.be/mailman/listinfo/msnoise

More information about the MSNoise mailing list