[MSNoise] failed to run 01.scan_archive_threaded.py
Thomas Lecocq
thomas.lecocq at oma.be
Wed Oct 23 09:55:21 UTC 2013
with only 1 thread, to test?
thom
Le 23/10/2013 11:54, Wang, Xiao a écrit :
> Please see the following output after I add up "print source":
>
> msnoise at seisapp:~/MSNoise-Temp/MSNoise-1.2$ python test.s01scan_archive.py init 4
> 2013-10-23 10:55:34,388 [INFO] *** Starting: Scan Archive ***
> mtime: -1
> init man!
> will work on 4 threads
> /home/msnoise/SDS/2010/3A/L002/HHE.D/3A.L002..HHE.D.2010.089
> /home/msnoise/SDS/2010/3A/L002/HHE.D/3A.L002..HHE.D.2010.090
> /home/msnoise/SDS/2010/3A/L002/HHE.D/3A.L002..HHE.D.2010.091
> /home/msnoise/SDS/2010/3A/L002/HHE.D/3A.L002..HHE.D.2010.092
> Traceback (most recent call last):
> File "test.s01scan_archive.py", line 204, in <module>
> t.start()
> File "/home/msnoise/anaconda/lib/python2.7/threading.py", line 743, in start
> _start_new_thread(self.__bootstrap, ())
> thread.error: can't start new thread
> Error: msr_unpack_data((null)): Cannot allocate diff buffer
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot allocate diff buffer
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: msr_unpack(): Cannot allocate memory
> Error: msr_init(): Cannot allocate memory
> Error: msr_unpack_data((null)): Cannot (re)allocate memory
> Error: seg_init(): Cannot allocate memory
> Error: msr_init(): Cannot allocate memory
> Segmentation fault
>
>
>
> ________________________________________
> From: msnoise-bounces at mailman-as.oma.be [msnoise-bounces at mailman-as.oma.be] on behalf of Thomas Lecocq [thomas.lecocq at oma.be]
> Sent: 23 October 2013 10:48
> To: msnoise at mailman-as.oma.be
> Subject: Re: [MSNoise] failed to run 01.scan_archive_threaded.py
>
> Xiao,
>
> edit s01scan_archive.py and add this line
>
> print source
>
> before
>
> data = read(source, quality=False)
>
> that should be at Line72.
>
> and send the output,
>
> Thom
>
>
> Le 23/10/2013 11:42, Wang, Xiao a écrit :
>> In the SDS folder, there are only mseed files, no other format files.
>>
>> Xiao
>>
>>
>> -----Original Message-----
>> From: msnoise-bounces at mailman-as.oma.be [mailto:msnoise-bounces at mailman-as.oma.be] On Behalf Of Thomas Lecocq
>> Sent: 23 October 2013 10:36
>> To: msnoise at mailman-as.oma.be
>> Subject: Re: [MSNoise] failed to run 01.scan_archive_threaded.py
>>
>> Xiao,
>>
>>
>> Le 23/10/2013 11:29, Wang, Xiao a écrit :
>>> I put my SDS data files in /home/msnoise/SDS directory. In table config, I set value of data_folder is /home/msnoise/SDS, value of data_structure is SDS, and value of export_format is MSEED.
>> This looks OK. Do you have anything else than mseed files in the folders ?
>>
>> Thomas
>>> Xiao
>>>
>>> ________________________________________
>>> From: msnoise-bounces at mailman-as.oma.be
>>> [msnoise-bounces at mailman-as.oma.be] on behalf of Thomas Lecocq
>>> [thomas.lecocq at oma.be]
>>> Sent: 23 October 2013 10:24
>>> To: msnoise at mailman-as.oma.be
>>> Subject: Re: [MSNoise] failed to run 01.scan_archive_threaded.py
>>>
>>> Xiao,
>>>
>>> what is your data archive format and path ? (the configuration bits)
>>>
>>> Thomas
>>>
>>> Le 23/10/2013 11:23, Wang, Xiao a écrit :
>>>> Hi Thomas,
>>>>
>>>> I try script python s01scan_archive.py init 4, but I still get same exception. I check database and tables of jobs, filters and data_availability are empty.
>>>>
>>>> Regards,
>>>>
>>>> Xiao
>>>>
>>>>
>>>> Screenshot:
>>>>
>>>> msnoise at seisapp:~/MSNoise-Temp/MSNoise-1.2$ python s01scan_archive.py
>>>> init 4
>>>> 2013-10-23 10:21:32,796 [INFO] *** Starting: Scan Archive ***
>>>> mtime: -1
>>>> init man!
>>>> will work on 4 threads
>>>> 2013-10-23 10:21:33,090 [DEBUG] 3A.L002..HHE.D.2010.089: Before
>>>> Start-Date! (0.25) Traceback (most recent call last):
>>>> File "s01scan_archive.py", line 203, in <module>
>>>> t.start()
>>>> File "/home/msnoise/anaconda/lib/python2.7/threading.py", line 743, in start
>>>> _start_new_thread(self.__bootstrap, ())
>>>> thread.error: can't start new thread
>>>> 2013-10-23 10:21:33,173 [DEBUG] 3A.L002..HHE.D.2010.090: Before
>>>> Start-Date! (0.33) Traceback (most recent call last):
>>>> File "_ctypes/callbacks.c", line 314, in 'calling callback function'
>>>> File "/home/msnoise/anaconda/lib/python2.7/site-packages/obspy-0.8.4-py2.7-linux-i686.egg/obspy/mseed/core.py", line 261, in allocate_data
>>>> data = np.empty(samplecount, dtype=DATATYPES[sampletype])
>>>> MemoryError Segmentation fault
>>>>
>>>>
>>>> ________________________________________
>>>> From: msnoise-bounces at mailman-as.oma.be
>>>> [msnoise-bounces at mailman-as.oma.be] on behalf of Thomas Lecocq
>>>> [thomas.lecocq at oma.be]
>>>> Sent: 23 October 2013 08:59
>>>> To: msnoise at mailman-as.oma.be
>>>> Subject: Re: [MSNoise] failed to run 01.scan_archive_threaded.py
>>>>
>>>> Hello Xiao,
>>>>
>>>> That is weird, could you test using more than 1 thread:
>>>>
>>>> python s01scan_archive.py init 4
>>>>
>>>> where 4 means 4 threads
>>>>
>>>>
>>>> Thomas
>>>>
>>>> Le 22/10/2013 16:20, Wang, Xiao a écrit :
>>>>> Hi Thomas,
>>>>>
>>>>> I downloaded MSNoise 1.2 and passed s001configurator.py and s002populate_station_table.py. But I encountered error when I call script python s01scan_archive.py init.
>>>>>
>>>>> Any idea?
>>>>>
>>>>> Xiao
>>>>>
>>>>>
>>>>> The following is the screenshot:
>>>>>
>>>>> msnoise at seisapp:~/MSNoise-Temp/MSNoise-1.2$ python
>>>>> s001configurator.py Updating Config Table Updating Station Table
>>>>> Updating Filter Table Done !
>>>>>
>>>>> msnoise at seisapp:~/MSNoise-Temp/MSNoise-1.2$ python
>>>>> s002populate_station_table.py
>>>>>
>>>>>>> Populating the Station table
>>>>> Adding: 3A L002
>>>>> Adding: 3A L003
>>>>> Adding: 3A L004
>>>>> Adding: 3A L005
>>>>> Adding: 3A L006
>>>>> Adding: 3A L007
>>>>> Adding: 3A L008
>>>>> Adding: 3A L009
>>>>> Adding: 3A L101
>>>>> Adding: 3A L102
>>>>> Adding: 3A L103
>>>>> Adding: 3A L104
>>>>> Adding: 3A L105
>>>>> Adding: 3A L106
>>>>> Adding: 3A L107
>>>>> Adding: 3A L201
>>>>> Adding: 3A QC08
>>>>> Adding: 3A QC11
>>>>> Adding: 3A U16B
>>>>> Adding: 3A U40B
>>>>> Adding: 3A U69B
>>>>> Adding: 3A UO2B
>>>>>
>>>>> msnoise at seisapp:~/MSNoise-Temp/MSNoise-1.2$ python
>>>>> s01scan_archive.py init
>>>>> 2013-10-22 15:14:14,516 [INFO] *** Starting: Scan Archive ***
>>>>> mtime: -1
>>>>> init man!
>>>>> will work on 1 threads
>>>>> Traceback (most recent call last):
>>>>> File "s01scan_archive.py", line 203, in <module>
>>>>> t.start()
>>>>> File "/home/msnoise/anaconda/lib/python2.7/threading.py", line 743, in start
>>>>> _start_new_thread(self.__bootstrap, ())
>>>>> thread.error: can't start new thread Traceback (most recent call
>>>>> last):
>>>>> File "_ctypes/callbacks.c", line 314, in 'calling callback function'
>>>>> File "/home/msnoise/anaconda/lib/python2.7/site-packages/obspy-0.8.4-py2.7-linux-i686.egg/obspy/mseed/core.py", line 261, in allocate_data
>>>>> data = np.empty(samplecount, dtype=DATATYPES[sampletype])
>>>>> MemoryError Segmentation fault
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Please donwload the latest MSNoise release, there are quite a
>>>>>> number of bugfix and improvements on the way memory and threads are handled !
>>>>>>
>>>>>> Let me know how it goes !
>>>>>>
>>>>>> Thomas
>>>>>>
>>>>>> ps : please subscribe to the list in order to have your messages
>>>>>> delivered automatically (this one went to "moderation")
>>>>> _______________________________________________
>>>>> MSNoise mailing list
>>>>> MSNoise at mailman-as.oma.be
>>>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>>> --
>>>> Dr. Thomas Lecocq
>>>> Geologist
>>>>
>>>> Seismology - Gravimetry
>>>> Royal Observatory of Belgium
>>>>
>>>> *
>>>> * * * * *
>>>> * * * *
>>>> ---------
>>>> http://www.seismology.be
>>>> http://twitter.com/#!/Seismologie_be
>>>>
>>>> _______________________________________________
>>>> MSNoise mailing list
>>>> MSNoise at mailman-as.oma.be
>>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>>> _______________________________________________
>>>> MSNoise mailing list
>>>> MSNoise at mailman-as.oma.be
>>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>> --
>>> Dr. Thomas Lecocq
>>> Geologist
>>>
>>> Seismology - Gravimetry
>>> Royal Observatory of Belgium
>>>
>>> *
>>> * * * * *
>>> * * * *
>>> ---------
>>> http://www.seismology.be
>>> http://twitter.com/#!/Seismologie_be
>>>
>>> _______________________________________________
>>> MSNoise mailing list
>>> MSNoise at mailman-as.oma.be
>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>> _______________________________________________
>>> MSNoise mailing list
>>> MSNoise at mailman-as.oma.be
>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>> --
>> Dr. Thomas Lecocq
>> Geologist
>>
>> Seismology - Gravimetry
>> Royal Observatory of Belgium
>>
>> *
>> * * * * *
>> * * * *
>> ---------
>> http://www.seismology.be
>> http://twitter.com/#!/Seismologie_be
>>
>> _______________________________________________
>> MSNoise mailing list
>> MSNoise at mailman-as.oma.be
>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>> _______________________________________________
>> MSNoise mailing list
>> MSNoise at mailman-as.oma.be
>> http://mailman-as.oma.be/mailman/listinfo/msnoise
> --
> Dr. Thomas Lecocq
> Geologist
>
> Seismology - Gravimetry
> Royal Observatory of Belgium
>
> *
> * * * * *
> * * * *
> ---------
> http://www.seismology.be
> http://twitter.com/#!/Seismologie_be
>
> _______________________________________________
> MSNoise mailing list
> MSNoise at mailman-as.oma.be
> http://mailman-as.oma.be/mailman/listinfo/msnoise
> _______________________________________________
> MSNoise mailing list
> MSNoise at mailman-as.oma.be
> http://mailman-as.oma.be/mailman/listinfo/msnoise
--
Dr. Thomas Lecocq
Geologist
Seismology - Gravimetry
Royal Observatory of Belgium
*
* * * * *
* * * *
---------
http://www.seismology.be
http://twitter.com/#!/Seismologie_be
More information about the MSNoise
mailing list