[MSNoise] Structure & filenaming of archive

[Thomas Lecocq]

Hi Ilias,

This is most probably because your modified files (names) don't actually 
include the network code inside the data (read: in the header of the 
data chunks in miniseed, when read by obspy).

if scan_archive doesn't NET, STA and COMP, it's not worth going further 
down the workflow.

Best regards,

Tom


On 16/10/2017 21:23, Ilias Papadopoulos wrote:
> Hello community,
>
> It is my first attempt to archive properly the data I have in hand. It 
> is a messy business, resulting from the mess of the original archive.
>
> The data are originally in 20 minutes chunks Seisan format. They have 
> been converted to mseed. I am trying to bring them to a structure & 
> filename that can be recognized as a default format. I will give you 
> an example:
>
> original file name: 2017-02-02-0000.GCMP.HHE.mseed
>
> modified file name:TR.GCMP.00.HHE.D.2017.033-0000.mseed
>
> The structure of my folder(s) is: 
> data_archive/msnoise_archive/YEAR/NET/STA/CHAN.TYPE/*.mseed
>
> My workflow so far:
>
> msnoise populate -> identifies NET STA
>
> msnoise scan_archive --init -> Reads the mseed chunks, identifies 
> correctly 72files/day/station, assigns flags=A
>
> Checking the Dashboard for Data availability, it has correctly 
> read/placed the STA/Comp/Path/Time variables/Samplerate, gives flag N. 
> BUT it doesn't identify the NET.
>
> msnoise new_jobs --init --> Updating: NET STA to flag=A
>
> Checking again the Dashboard, flag of data still remains N, and 
> obviously no new jobs are added. What am I missing?
>
>
> Any help is appreciated.
>
> Ilias
>
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------- 
>
>
> Here is the msnoise info outcome:
>
> [smn at smn test3]$ msnoise info
>
> General:
> MSNoise is installed in: /home/smn/obspy/lib/python3.5/site-packages
>  - db.ini is present
>
>
> Configuration:
>  - /home/smn/Desktop/kickemjenny/SDS_archive exists
>  - CROSS_CORRELATIONS does not exists (and that is normal because 
> keep_all=False)
>
> Raw config bits: "D"efault or "M"odified (green)
>  M data_folder: /home/smn/Desktop/kickemjenny/SDS_archive
>  D output_folder: CROSS_CORRELATIONS
>  D data_structure: SDS
>  D network: *
>  D channels: *
>  M startdate: 2016-12-30
>  D enddate: 2018-01-01
>  D analysis_duration: 86400
>  D cc_sampling_rate: 20.0
>  D resampling_method: Lanczos
>  D preprocess_lowpass: 8.0
>  D preprocess_highpass: 0.01
>  M remove_response: Y
>  D response_format: dataless
>  M response_path: /home/smn/Desktop/kickemjenny
>  D response_prefilt: (0.005, 0.006, 30.0, 35.0)
>  D maxlag: 120.
>  D corr_duration: 1800.
>  D overlap: 0.0
>  D windsorizing: 3
>  D whitening: A
>  D stack_method: linear
>  D pws_timegate: 10.0
>  D pws_power: 2.0
>  D crondays: -1
>  D components_to_compute: ZZ
>  D autocorr: N
>  D keep_all: N
>  D keep_days: Y
>  D ref_begin: 1970-01-01
>  D ref_end: 2018-01-01
>  D mov_stack: 5
>  D export_format: MSEED
>  D sac_format: doublets
>  D dtt_lag: static
>  D dtt_v: 1.0
>  D dtt_minlag: 5.0
>  D dtt_width: 30.0
>  D dtt_sides: both
>  D dtt_mincoh: 0.65
>  D dtt_maxerr: 0.1
>  D dtt_maxdt: 0.1
>  D plugins:
>
> Filters:
> ID: [low:high]  [mwcs_low:mwcs_high]    mwcs_wlen    mwcs_step used
> 01: [0.100:1.000] [0.110:0.990] 012 004 Y
>
> Stations:
> TR.GCMP -61.4260 12.4860 0.0 DEG Y
> TR.MCLT -60.9412 13.7096 0.0 DEG Y
>
> CC Jobs:
>
> DTT Jobs:
>
>
>
>