[MSNoise] s02new_jobs
Thomas Lecocq
thomas.lecocq at oma.be
Wed Nov 20 06:44:19 UTC 2013
Kasper,
Le 20/11/2013 05:36, Kasper van Wijk a écrit :
> Thomas, we reinstalled with your latest versions, and now have the same
> problem as Esteban, I am afraid:
>
> [kasper at kvan637-277769 MSNoise-master]$ python s02new_jobs.py
> 2013-11-20 17:30:32,540 [INFO] *** Starting: New Jobs ***
> Traceback (most recent call last):
> File "s02new_jobs.py", line 89, in <module>
> if day != old_day and day_pairs != []:
> NameError: name 'day' is not defined
You get this error because 1) you have no new files detected in the
archive and 2) I did not code this possibility properly. The problem
indeed comes from the archive scanning process, it seems:
>
> My data structure is BUD (and appears correct in the configurator (snapshot
> attached), and there is more than one day's worth of data in the structure
> to correlate. File names are like this:
>
> ABAZ.NZ..EHE.2013.322 ABAZ.NZ..EHN.2013.322 ABAZ.NZ..EHZ.2013.322
> ABAZ.NZ..EHE.2013.323 ABAZ.NZ..EHN.2013.323 ABAZ.NZ..EHZ.2013.323
>
> I did notice among a large amount of output to screen that there was an
> error in
>
>> python s01scan_archive.py init
> ......
> 2013-11-20 17:27:11,154 [DEBUG] Already Exists: "ABAZ.NZ..EHZ.2013.322"
> (read:0.19 (0.22) seconds | save:0.0275 seconds)
> Traceback (most recent call last):
> File "s01scan_archive.py", line 203, in <module>
> t.start()
> File "/usr/lib64/python2.7/threading.py", line 746, in start
> _start_new_thread(self.__bootstrap, ())
> thread.error: can't start new thread
> 2013-11-20 17:27:11,324 [DEBUG] Already Exists: "ABAZ.NZ..EHZ.2013.323"
> (read:0.16 (0.16) seconds | save:0.0038 seconds)
> .....
>
> but it did finish the scanning.
There seems definitively to be a bug in the threading interface using
only 1 thread, so maybe try with 2 or 4 : "s01scan_archive.py init 2". I
should rewrite the way args are passed to this script, it's not obvious.
When files already exist in the DB, which seems to be the case here,
they are marked "M"odified ONLY if the data they contain is different.
If not, they remain "A"rchive, and are not found by the new_jobs
process. If you really want to re-process those files, you should
manually change the flag to "M"odified or "N"ew of all the lines in the
DB. I can make a small script to do that if you need.
Hope this helps,
Cheers,
Thomas
>
> I am running this under linux, FC19.
>
> Cheers,
> Kasper
>
>
> On 19 November 2013 19:09, Thomas Lecocq <thomas.lecocq at oma.be> wrote:
>
>> Hi Esteban,
>>
>> Le 19/11/2013 00:59, Esteban Chaves Sibaja a écrit :
>>> Hi,
>>>
>>> I'm trying to run my data as example: just 2 stations and ech record has
>> 24 hours (SAC format):
>>> /DATA/YZ/MIRM/MIRM.YZ.HHZ.2013.199
>>> /DATA/YZ/GRAS/GRAS.YZ.CR.HHZ.2013.199
>>>
>>>
>>> But I got the next error after type >> python s02new_jobs.py:
>>>
>>> File "s02new_jobs.py, line 89, in <module>
>>> if day != old_day and day_pairs != []:
>>> NameError: name 'day' is not defined
>>>
>>> So. What should be the correct format of the input data? SAC or mseed?
>>> and how can I solve this time problem?
>> Currently, only the archive formats defined as :
>>
>> data_structure['SDS'] =
>> "YEAR/NET/STA/CHAN.TYPE/NET.STA.LOC.CHAN.TYPE.YEAR.DAY"
>> data_structure['BUD'] = "NET/STA/STA.NET.LOC.CHAN.YEAR.DAY"
>> data_structure['IDDS'] =
>> "YEAR/NET/STA/CHAN.TYPE/DAY/NET.STA.LOC.CHAN.TYPE.YEAR.DAY.HOUR"
>> data_structure['PDF'] =
>> "YEAR/STA/CHAN.TYPE/NET.STA.LOC.CHAN.TYPE.YEAR.DAY"
>>
>>
>> are known by MSNoise. You have to define your own format in the
>> data_structures.py file
>>
>> data_structure['NAME'] = "NET/STA/STA.NET.LOC.CHAN.YEAR.DAY"
>>
>> Once done, the "s002populate_station_table.py" script should find your
>> stations/network and the rest of the workflow should work fine.
>>
>> Let me know how it goes,
>>
>> Thomas
>>
>>> Sorry about the question, but I'll appreciate the help to solve it and
>> continue using my data.
>>> Esteban
>>>
>>>
>>> On Nov 18, 2013, at 4:00 AM, msnoise-request at mailman-as.oma.be wrote:
>>>
>>>> Send MSNoise mailing list submissions to
>>>> msnoise at mailman-as.oma.be
>>>>
>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>>> or, via email, send a message with subject or body 'help' to
>>>> msnoise-request at mailman-as.oma.be
>>>>
>>>> You can reach the person managing the list at
>>>> msnoise-owner at mailman-as.oma.be
>>>>
>>>> When replying, please edit your Subject line so it is more specific
>>>> than "Re: Contents of MSNoise digest..."
>>>>
>>>>
>>>> Today's Topics:
>>>>
>>>> 1. Re: MSNoise Digest, Vol 2, Issue 4 (Thomas Lecocq)
>>>>
>>>>
>>>> ----------------------------------------------------------------------
>>>>
>>>> Message: 1
>>>> Date: Mon, 18 Nov 2013 10:20:00 +0100
>>>> From: Thomas Lecocq <thomas.lecocq at oma.be>
>>>> To: msnoise at mailman-as.oma.be
>>>> Subject: Re: [MSNoise] MSNoise Digest, Vol 2, Issue 4
>>>> Message-ID: <5289DBC0.7070502 at oma.be>
>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>>
>>>> Hi all,
>>>>
>>>> It seems to be a OSX-specific problem, I'll report it to the Anaconda
>>>> guys to try to debug...
>>>>
>>>> Sorry about that,
>>>>
>>>> Thomas
>>>>
>>>> Le 15/11/2013 20:37, Esteban Chaves a ?crit :
>>>>> Hi All,
>>>>>
>>>>> I got the same problem. I did install my MSNoise and dependencies in
>> MAC
>>>>> OSx 10.9.
>>>>> and when I typed:
>>>>>
>>>>>>> python s001configurator.py I got this:
>>>>> Qt internal error: qt_menu.nib could not be loaded. The .nib file
>> should be
>>>>> placed in QtGui.framework/Versions/Current/Resources/ or in the
>> resources
>>>>> directory of your application bundle.
>>>>>
>>>>> Abort trap: 6
>>>>>
>>>>>
>>>>> I'm using Anaconda, and after reading trough the web my solution was
>>>>> using pythonw instead of python, but when the GUI from the Configurator
>>>>> appears, I
>>>>> can't
>>>>> see all the corresponding spaces. For example I can't change the path
>> for
>>>>> my data, because I can't see the space there. Then after any change I
>> made
>>>>> I got the same message:
>>>>>
>>>>> Updating Config Table
>>>>>
>>>>> Updating Station Table
>>>>>
>>>>> Updating Filter Table
>>>>>
>>>>> Done !
>>>>>
>>>>> //anaconda/bin/pythonw: line 2: 6112 Segmentation fault: 11
>>>>> //anaconda/python.app/Contents/MacOS/python $@
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Esteban.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Esteban J. Chaves
>>>>> PhD Student in Seismology
>>>>> Earth and Planetary Sciences
>>>>> Earth and Marine Science Building
>>>>> office C317B
>>>>> University of California, Santa Cruz
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Nov 14, 2013 at 4:00 AM, <msnoise-request at mailman-as.oma.be>
>> wrote:
>>>>>> Send MSNoise mailing list submissions to
>>>>>> msnoise at mailman-as.oma.be
>>>>>>
>>>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>>>>> or, via email, send a message with subject or body 'help' to
>>>>>> msnoise-request at mailman-as.oma.be
>>>>>>
>>>>>> You can reach the person managing the list at
>>>>>> msnoise-owner at mailman-as.oma.be
>>>>>>
>>>>>> When replying, please edit your Subject line so it is more specific
>>>>>> than "Re: Contents of MSNoise digest..."
>>>>>>
>>>>>>
>>>>>> Today's Topics:
>>>>>>
>>>>>> 1. Segmentation fault when running s001configurator.py (Wang,
>> Xiao)
>>>>>>
>>>>>> ----------------------------------------------------------------------
>>>>>>
>>>>>> Message: 1
>>>>>> Date: Wed, 13 Nov 2013 14:57:11 +0000
>>>>>> From: "Wang, Xiao" <Xiao.Wang at liverpool.ac.uk>
>>>>>> To: "msnoise at mailman-as.oma.be" <msnoise at mailman-as.oma.be>
>>>>>> Subject: [MSNoise] Segmentation fault when running s001configurator.py
>>>>>> Message-ID:
>>>>>> <
>> AFD7E1C355F302448EFE8C6EAE6FB563AEBB7335 at CHEXMBX1.livad.liv.ac.uk
>>>>>> Content-Type: text/plain; charset="iso-8859-1"
>>>>>>
>>>>>> Not every time, but sometime I encountered "Segmentation fault" when
>>>>>> running s001configurator.py. The database works fine and can be
>> updated
>>>>>> properly when the problem happens. I am using msnoise 1.2.
>>>>>>
>>>>>> Following is the output of the problem:
>>>>>>
>>>>>> student at seisapp:~/msnoise$ python s001configurator.py
>>>>>> Updating Config Table
>>>>>> Updating Station Table
>>>>>> Updating Filter Table
>>>>>> Done !
>>>>>> Segmentation fault
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Xiao
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>>
>>>>>> _______________________________________________
>>>>>> MSNoise mailing list
>>>>>> MSNoise at mailman-as.oma.be
>>>>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>>>>>
>>>>>>
>>>>>> End of MSNoise Digest, Vol 2, Issue 4
>>>>>> *************************************
>>>>>>
>>>>> _______________________________________________
>>>>> MSNoise mailing list
>>>>> MSNoise at mailman-as.oma.be
>>>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>>>
>>>> ------------------------------
>>>>
>>>> _______________________________________________
>>>> MSNoise mailing list
>>>> MSNoise at mailman-as.oma.be
>>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>>>
>>>>
>>>> End of MSNoise Digest, Vol 2, Issue 6
>>>> *************************************
>>> _______________________________________________
>>> MSNoise mailing list
>>> MSNoise at mailman-as.oma.be
>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>> _______________________________________________
>> MSNoise mailing list
>> MSNoise at mailman-as.oma.be
>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>
>>
>>
>> _______________________________________________
>> MSNoise mailing list
>> MSNoise at mailman-as.oma.be
>> http://mailman-as.oma.be/mailman/listinfo/msnoise
More information about the MSNoise
mailing list