[MSNoise] Station list
VICIC BLAZ [PHD0900016]
BLAZ.VICIC at phd.units.it
Tue Sep 20 16:29:38 UTC 2016
The problem was actually in PyTables... I reinstalled it with apt-get and it did not work...then i checked with anaconda. it was not installed there, so i installed it, and now it works.
Next problem is, as i saw the same, as Emily had last month... was it resolved?
i try to stack and i get:
Lets STACK !
2016-09-20 18:21:23 [DEBUG] Starting the ref stack
[]
2016-09-20 18:21:24 [DEBUG] Processing SL_CEY:SL_JAVS-ZZ-2
Traceback (most recent call last):
File "/home/blaz/miniconda2/bin/msnoise", line 11, in <module>
sys.exit(run())
File "/home/blaz/miniconda2/lib/python2.7/site-packages/msnoise/scripts/msnoise.py", line 614, in run
cli(obj={})
File "/home/blaz/miniconda2/lib/python2.7/site-packages/click/core.py", line 716, in __call__
return self.main(*args, **kwargs)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/click/core.py", line 696, in main
rv = self.invoke(ctx)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/blaz/miniconda2/lib/python2.7/site-packages/click/core.py", line 889, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/click/core.py", line 534, in invoke
return callback(*args, **kwargs)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/msnoise/scripts/msnoise.py", line 301, in stack
main('ref', interval)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/msnoise/s04stack.py", line 203, in main
updated_days = updated_days_for_dates(db, start, end, pair.replace('_', '.'), jobtype='CC', interval=datetime.timedelta(days=interval),returndays=True)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/msnoise/api.py", line 1189, in updated_days_for_dates
days = session.query(Job).filter(Job.pair == pair).filter(Job.day >= date1).filter(Job.day <= date2).filter(Job.jobtype == jobtype).filter(Job.lastmod >= lastmod).group_by(Job.day).order_by(Job.day).all()
File "/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2613, in all
return list(self)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2761, in __iter__
return self._execute_and_instances(context)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2776, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
return meth(self, multiparams, params)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
compiled_sql, distilled_params
File "/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
context)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
exc_info
File "/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
context)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
cursor.execute(statement, parameters)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/pymysql/cursors.py", line 166, in execute
result = self._query(query)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/pymysql/cursors.py", line 322, in _query
conn.query(q)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/pymysql/connections.py", line 835, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/pymysql/connections.py", line 1019, in _read_query_result
result.read()
File "/home/blaz/miniconda2/lib/python2.7/site-packages/pymysql/connections.py", line 1302, in read
first_packet = self.connection._read_packet()
File "/home/blaz/miniconda2/lib/python2.7/site-packages/pymysql/connections.py", line 981, in _read_packet
packet.check_error()
File "/home/blaz/miniconda2/lib/python2.7/site-packages/pymysql/connections.py", line 393, in check_error
err.raise_mysql_exception(self._data)
File "/home/blaz/miniconda2/lib/python2.7/site-packages/pymysql/err.py", line 107, in raise_mysql_exception
raise errorclass(errno, errval)
sqlalchemy.exc.InternalError: (pymysql.err.InternalError) (1055, u"Expression #1 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'msnoise.jobs.ref' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by") [SQL: u'SELECT jobs.ref AS jobs_ref, jobs.day AS jobs_day, jobs.pair AS jobs_pair, jobs.jobtype AS jobs_jobtype, jobs.flag AS jobs_flag, jobs.lastmod AS jobs_lastmod \nFROM jobs \nWHERE jobs.pair = %(pair_1)s AND jobs.day >= %(day_1)s AND jobs.day <= %(day_2)s AND jobs.jobtype = %(jobtype_1)s AND jobs.lastmod >= %(lastmod_1)s GROUP BY jobs.day ORDER BY jobs.day'] [parameters: {u'day_2': datetime.date(2016, 9, 20), u'lastmod_1': datetime.datetime(2016, 9, 19, 18, 21, 24, 191369), u'day_1': datetime.date(1970, 1, 1), u'jobtype_1': 'CC', u'pair_1': 'SL.CEY:SL.JAVS'}]
So, as you mentioned, i added sql_mode=TRADITIONAL line in my.cnf, and the same as for Emily,
sudo service mysql restart did not work. i added [mysql] in front of the line that i added, and now restart works, but I get the same error as before when calling
msnoise stack -r -m
any idea? or should i install older version of mysql?
thanks
________________________________
From: msnoise-bounces at mailman-as.oma.be <msnoise-bounces at mailman-as.oma.be> on behalf of Thomas Lecocq <Thomas.Lecocq at seismology.be>
Sent: Tuesday, September 20, 2016 5:10:46 PM
To: msnoise at mailman-as.oma.be
Subject: Re: [MSNoise] Station list
Hi,
you probably checked "keep all" parameter. it's really not properly
tested. just avoid it for the moment.
Thomas
On 20/09/2016 15:50, VICIC BLAZ [PHD0900016] wrote:
> Sorry, but there is another problem...
>
> i ran:
>
> msnoise -t 2 scan_archive --init
>
> msnoise new_jobs
>
> without problems.
>
>
> then i tried with
>
> msnoise -t 4 compute_cc
>
>
> it starts to work, but after some time it produces this error:
>
> ...
>
> 2016-09-20 16:42:18 [DEBUG] Azimuth=258.9
> Process Process-1:
> Traceback (most recent call last):
> File "/home/blaz/miniconda2/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
> self.run()
> File "/home/blaz/miniconda2/lib/python2.7/multiprocessing/process.py", line 114, in run
> self._target(*self._args, **self._kwargs)
> File "/home/blaz/miniconda2/lib/python2.7/site-packages/msnoise/s03compute_cc.py", line 683, in main
> export_allcorr(db, ccfid, allcorr[ccfid])
> File "/home/blaz/miniconda2/lib/python2.7/site-packages/msnoise/api.py", line 825, in export_allcorr
> df.to_hdf(os.path.join(path, date+'.h5'), 'data')
> File "/home/blaz/miniconda2/lib/python2.7/site-packages/pandas/core/generic.py", line 1101, in to_hdf
> return pytables.to_hdf(path_or_buf, key, self, **kwargs)
> File "/home/blaz/miniconda2/lib/python2.7/site-packages/pandas/io/pytables.py", line 259, in to_hdf
> complib=complib) as store:
> File "/home/blaz/miniconda2/lib/python2.7/site-packages/pandas/io/pytables.py", line 389, in __init__
> 'importing'.format(ex=str(ex)))
> ImportError: HDFStore requires PyTables, "No module named tables" problem importing
>
> I tried reinstalling PyTables without luck.
>
>
> Thanks!
>
>
> ________________________________
> From: msnoise-bounces at mailman-as.oma.be <msnoise-bounces at mailman-as.oma.be> on behalf of VICIC BLAZ [PHD0900016] <BLAZ.VICIC at phd.units.it>
> Sent: Tuesday, September 20, 2016 3:43:22 PM
> To: msnoise at mailman-as.oma.be
> Subject: Re: [MSNoise] Station list
>
> Thanks, it works now.
>
>
> Blaž
>
> ________________________________
> From: msnoise-bounces at mailman-as.oma.be <msnoise-bounces at mailman-as.oma.be> on behalf of Thomas Lecocq <Thomas.Lecocq at seismology.be>
> Sent: Tuesday, September 20, 2016 3:01:07 PM
> To: msnoise at mailman-as.oma.be
> Subject: Re: [MSNoise] Station list
>
> Hi,
>
> You have to import data_structure too :
>
> from msnoise.data_structures import data_structure
>
>
> otherwise it'll fail (your custom.py file is not valid, fails at import).
>
> Thomas
>
>
> On 20/09/2016 12:59, VICIC BLAZ [PHD0900016] wrote:
>> Thank you.
>>
>>
>> Which brings me to the 2nd question...
>>
>> I tried to populate with this command, but of course it is not working...
>>
>>
>> What I tried:
>>
>>
>> in the folder
>>
>> msnoise/data I have put a miniseed file of one station of one day.
>>
>> Name of the miniseed is in the format:
>>
>> STA.NET.CHAN.YEAR.DAY
>>
>>
>> as data_folder in settings I have set this folder.
>>
>>
>> in the data_structures.py I added this line:
>>
>> data_structure['SEI'] = "STA.NET.CHAN.YEAR.DAY"
>>
>>
>> i created a custom.py script:
>>
>>
>> from obspy import read
>>
>> data_structure['SEI'] = "STA.NET.CHAN.YEAR.DAY"
>>
>> def populate():
>> st = read("*")
>> st.merge()
>>
>> stations = []
>>
>> for tr in st:
>> sta = str(tr.stats.station)
>> net = str(tr.stats.network)
>> stations.append(str("%s_%s") % (net, sta))
>>
>> return stations
>>
>> So,now i have main folder with sub folder data, where I have miniseed file. in the main folder I put custom.py and db.ini and, if I understand right, run msnoise populate from here. In the browser I set data_folder to where my miniseed is and data_structure as SEI.
>>
>> When i run the populate command, i get error:
>>
>>
>> ()
>>>> Populating the Station table
>> ()
>> Can't parse the archive for format SEI !
>> trying to import local parser (should return a station list)
>> ()
>> No file named custom.py in the /home/blaz/Documents/msnoise folder
>>
>> (custom.py IS in /home/blaz/Documents/msnoise)
>> I hope I was clear enough.
>>
>> Thanks!
>>
>>
>>
>>
>>
>> ________________________________
>> From: msnoise-bounces at mailman-as.oma.be <msnoise-bounces at mailman-as.oma.be> on behalf of Thomas Lecocq <Thomas.Lecocq at seismology.be>
>> Sent: Tuesday, September 20, 2016 12:56:56 PM
>> To: msnoise at mailman-as.oma.be
>> Subject: Re: [MSNoise] Station list
>>
>> Hi,
>>
>> Yes, this bug was recently reported by another user and I haven't had
>> time to fix it.
>>
>> The best way to add the station currently is to use the "msnoise
>> populate" command.
>>
>> Cheers,
>>
>> Thomas
>>
>>
>> On 20/09/2016 11:51, VICIC BLAZ [PHD0900016] wrote:
>>> Dear all
>>>
>>>
>>> I would like to try to use msnoise. I installed it on Ubuntu 16.04, and as far as I know, everything was installed as planed.
>>>
>>> Now, I open msnoise admin and try to manually add stations in the browser. I set NET, STA, X, Y, ALT, COORD and click save. When I do that, I get a lot of errors:
>>>
>>>
>>> *
>>> File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask/app.py", line 2000, in __call__
>>>
>>> return self.wsgi_app(environ, start_response)
>>>
>>> *
>>> File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask/app.py", line 1991, in wsgi_app
>>>
>>> response = self.make_response(self.handle_exception(e))
>>>
>>> *
>>> File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask/app.py", line 1567, in handle_exception
>>>
>>> reraise(exc_type, exc_value, tb)
>>>
>>> *
>>> File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask/app.py", line 1988, in wsgi_app
>>>
>>> response = self.full_dispatch_request()
>>>
>>> *
>>> File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask/app.py", line 1641, in full_dispatch_request
>>>
>>> rv = self.handle_user_exception(e)
>>>
>>> *
>>> File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask/app.py", line 1544, in handle_user_exception
>>>
>>> reraise(exc_type, exc_value, tb)
>>>
>>> *
>>> File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask/app.py", line 1639, in full_dispatch_request
>>>
>>> rv = self.dispatch_request()
>>>
>>> *
>>> File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask/app.py", line 1625, in dispatch_request
>>>
>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>
>>> *
>>> File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask_admin/base.py", line 69, in inner
>>>
>>> return self._run_view(f, *args, **kwargs)
>>>
>>> *
>>> File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask_admin/base.py", line 368, in _run_view
>>>
>>> return fn(self, *args, **kwargs)
>>>
>>> *
>>> File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask_admin/model/base.py", line 1920, in create_view
>>>
>>> model = self.create_model(form)
>>>
>>> *
>>> File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask_admin/contrib/sqla/view.py", line 1028, in create_model
>>>
>>> if not self.handle_view_exception(ex):
>>>
>>> *
>>> File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask_admin/contrib/sqla/view.py", line 1011, in handle_view_exception
>>>
>>> return super(ModelView, self).handle_view_exception(exc)
>>>
>>> *
>>> File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask_admin/contrib/sqla/view.py", line 1022, in create_model
>>>
>>> model = self.model()
>>>
>>> TypeError: __init__() takes exactly 9 arguments (1 given)
>>>
>>>
>>>
>>> What am I doing wrong?
>>>
>>>
>>> Thanks,
>>>
>>> Blaz
>>> _______________________________________________
>>> MSNoise mailing list
>>> MSNoise at mailman-as.oma.be
>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>> _______________________________________________
>> MSNoise mailing list
>> MSNoise at mailman-as.oma.be
>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>> _______________________________________________
>> MSNoise mailing list
>> MSNoise at mailman-as.oma.be
>> http://mailman-as.oma.be/mailman/listinfo/msnoise
> _______________________________________________
> MSNoise mailing list
> MSNoise at mailman-as.oma.be
> http://mailman-as.oma.be/mailman/listinfo/msnoise
> _______________________________________________
> MSNoise mailing list
> MSNoise at mailman-as.oma.be
> http://mailman-as.oma.be/mailman/listinfo/msnoise
> _______________________________________________
> MSNoise mailing list
> MSNoise at mailman-as.oma.be
> http://mailman-as.oma.be/mailman/listinfo/msnoise
_______________________________________________
MSNoise mailing list
MSNoise at mailman-as.oma.be
http://mailman-as.oma.be/mailman/listinfo/msnoise
More information about the MSNoise
mailing list