[MSNoise] MSNoise CPU-GPU-RAM Processing

[Thomas Lecocq]

Dear Prima,

First, (the easy answer), the fact that the GPU stays at 0% is perfectly 
normal, as MSNoise doesn't use GPU calculation (or Cuda, or anything 
related to GPUs).

The increase between the -t 1 and -t 2 seems logical in terms of timing. 
But from 2 to 4, you don't seem to gain any timing increase.

So:
- How do you calculate/get the RAM/CPU values ?
- What OS do you work on ?
- Which Python version/architecture do you use ? (use bugreport.py -a 
and send it to the list)
- Are you sure your machine is a quadcore ? or could it be a dual core, 
capable of hyperthreading (2x2) ?
- Are you using a MySQL server ? On your local machine or on a remote one ?

Scanning the archive actually doesn't consume much power, but requires 
quite a large number of readings, so your disk performance could be an 
issue too. I haven't yet played a lot with Numba, it's a thing to check 
in the future. If you succeed in doing that, PR (pull requests) are 
welcome ! This said, there are few chances the first steps (scan + jobs) 
can be accelerated dramatically, as the first mainly relies on Obspy and 
the second on the power of your database machine.

Best,

Thomas

Le 17/05/2014 13:25, Wardhani, Prima Wira Kusuma a écrit :
> Dear Thomas,
> Dear All,
>
> I want to ask about Scan Archive, Creating New Jobs and CC using MSNoise
> using CPU-GPU-RAM computing.
> I processed 49 seismic station, with 2 months data available. For scanning
> the archive *s01scan_archive.py*, using --init option, and thread option :
>
> --init -t 1 : consume 400MB RAM, 17% CPU, 0% GPU : ~24 hours
> --init -t 2 : consume 1.6GB RAM, 17% CPU, 0% GPU : ~13 hours
> --init -t 4 : consume 1.6GB RAM, 17% CPU, 0% GPU : ~13 hours
>
> the next threads option RAM comsume stuck in 1.6GB, no increase processing
> using multicore CPU.
> After finish I tried create new jobs using *s02new_jobs.py* for all
> station, and it
> takes 38 minutes for [DEBUG] all station pair for 1 day. I checked the
> process only consume 1.6GB RAM, 17% CPU, no more than this value.
> How to optimalize the Processing using RAM-CPU-and GPU? Because it takes
> along time to process all my Data,, at least it can be optimize using
> RAM-CPU it will be better. My Computer using 16GB RAM and 4Core at 3.2GHz, but
> it only used 1.6GB only.. I tried to insert NumbaPro, but for some reason,
> it failed,.
>
> Can us suggest how to use parallel computing using GPU-CPU-RAM for optimize
> MSNoise code?
>
> Thank you,
>
>
> WARDHANI, Prima Wira Kusuma
> Master of Earthquake Engineering and Engineering Seismology
> Universite Joseph Fourier - ISTerre Room 015
> prima.dhani06 at gmail.com
> +62-81548815815
> +33-663964113
>
> がんばります！
> _______________________________________________
> MSNoise mailing list
> MSNoise at mailman-as.oma.be
> http://mailman-as.oma.be/mailman/listinfo/msnoise