gdat issueshttps://gitlab.epfl.ch/spc/gdat/-/issues2024-02-19T13:51:28Zhttps://gitlab.epfl.ch/spc/gdat/-/issues/64Job Failed #8328542024-02-19T13:51:28ZOlivier SauterJob Failed #832854Job [#832854](https://gitlab.epfl.ch/spc/gdat/-/jobs/832854) failed for 467e9cbfff6cfe59f1d0c38a2b802a18f9ef6712:
@amerle something changed for the deployment?Job [#832854](https://gitlab.epfl.ch/spc/gdat/-/jobs/832854) failed for 467e9cbfff6cfe59f1d0c38a2b802a18f9ef6712:
@amerle something changed for the deployment?Antoine MerleAntoine Merlehttps://gitlab.epfl.ch/spc/gdat/-/issues/63Modify code following renaming of genlib's writestruct function2023-11-15T08:00:05ZAntoine MerleModify code following renaming of genlib's writestruct functionSee spc/tcv/tbx/genlib#29.See spc/tcv/tbx/genlib#29.https://gitlab.epfl.ch/spc/gdat/-/issues/62Make routines for IDS connection matrix interpretation available to MEQ2023-10-23T10:18:59ZFrancesco CarpaneseMake routines for IDS connection matrix interpretation available to MEQ**To**: @osauter @amerle
**For visibility**: @cheiss @ffelici
**Motivation**: Several users/developers are working on IDS-> MEQ parsing routines.
**Topic**: These 2 routines [here](https://gitlab.epfl.ch/spc/gdat/-/blob/master/mat...**To**: @osauter @amerle
**For visibility**: @cheiss @ffelici
**Motivation**: Several users/developers are working on IDS-> MEQ parsing routines.
**Topic**: These 2 routines [here](https://gitlab.epfl.ch/spc/gdat/-/blob/master/matlab/TCV_IMAS/extract_info_connection_matrix.m) [here](https://gitlab.epfl.ch/spc/gdat/-/blob/master/matlab/TCV_IMAS/plot_connection_matrices.m) are useful to interpret the connection matrix stored in the IDS to obtain `Twa`. I was using them to create the windings structure [here](https://gitlab.epfl.ch/spc/tcv/tbx/meq/-/blob/iter_dev/liugids.m#L68) and I would like to leverage them again for https://gitlab.epfl.ch/spc/tcv/tbx/meq/-/merge_requests/632
**Proposed solution**:
1) Bring them to MEQ.
2) Create a dedicate folder IDS with IDS utils, available as submodule for `gdat` and `MEQ`.
3) ??https://gitlab.epfl.ch/spc/gdat/-/issues/61Error when loading 'powers' for some recent shots (>78000)2023-10-09T12:34:06ZCassandre Ekta ContréError when loading 'powers' for some recent shots (>78000)For shots
```
[78360 78359 78358]
```
`gdat(shot,'powers')` throw this error:
```
> Error using interp1>reshapeAndSortXandV (line 409)
> X must be a vector.
>
> Error in interp1 (line 91)
> [X,V,orig_size_v] = reshapeAndSortXand...For shots
```
[78360 78359 78358]
```
`gdat(shot,'powers')` throw this error:
```
> Error using interp1>reshapeAndSortXandV (line 409)
> X must be a vector.
>
> Error in interp1 (line 91)
> [X,V,orig_size_v] = reshapeAndSortXandV(varargin{1},varargin{2});
>
> Error in gdat_tcv (line 2477)
> ip_t = interp1(ip.t,ip.data,gdat_data.ohm.t);
>
> Error in gdat (line 190)
> [gdat_data,gdat_params,error_status,varargout] = feval(['gdat_' lower(machine_eff)],args{:});
```
Then, `78415` (which has evaluation error on all signals) throws an error instead of indicating that the data is empty.
```
> One or more output arguments not assigned during call to "interpos".
> Error in gdat_tcv (line 2476)
> vloop_smooth=interpos(-63,vloop.t,vloop.data,gdat_data.ohm.t,tension);
>
> Error in gdat (line 190)
> [gdat_data,gdat_params,error_status,varargout] = feval(['gdat_' lower(machine_eff)],args{:});
```Cassandre Ekta ContréCassandre Ekta Contréhttps://gitlab.epfl.ch/spc/gdat/-/issues/60Field provenance in IDS nbi when writing into IMAS 3.382023-05-15T16:09:25ZMatteo Vallarmatteo.vallar@epfl.chField provenance in IDS nbi when writing into IMAS 3.38@mmarin @osauter
1. I read the NBI IDS from TCV
2. I try to write the NBI IDS in ITER (both on IMAS 3.34 and IMAS 3.38) and the following error occurs
```
>> [ids_put_status] = ids2database(73116,0,0,ids);
these subfields are not ids na...@mmarin @osauter
1. I read the NBI IDS from TCV
2. I try to write the NBI IDS in ITER (both on IMAS 3.34 and IMAS 3.38) and the following error occurs
```
>> [ids_put_status] = ids2database(73116,0,0,ids);
these subfields are not ids names, so not used: nbi_description params_tcv2ids wall_description
Warning : while putting IDS nbi, the written IDS has non-empty obsolescent node ids_properties/source. Please consider updating the code to avoid using obsolescent nodes.
problems in putting data in database:
Error using ids_put
Unable to get field provenance from input structure
... in structure ids_properties/provenance
... in IDS nbi
Error in ids2database (line 122)
ids_put(idx,ids_to_get,ids2put.(ids_to_get));
```
@mmarin did the full `tcv2ids2database` on `spcimas` and it went throughhttps://gitlab.epfl.ch/spc/gdat/-/issues/59Dimunits names for time & rho2023-05-12T14:20:04ZCassandre Ekta ContréDimunits names for time & rhoCould we standardize the `dimunits` labels for the time, rhopol_norm and rhotor_norm?
### Time
Mainly specified as `s`, I sometimes noticed it was called:
- TCV time (sec)
- time [s]
### Rhopolnorm
Has many nicknames. As a note in gda...Could we standardize the `dimunits` labels for the time, rhopol_norm and rhotor_norm?
### Time
Mainly specified as `s`, I sometimes noticed it was called:
- TCV time (sec)
- time [s]
### Rhopolnorm
Has many nicknames. As a note in gdat_tcv:
- [line 2189](matlab/TCV/gdat_tcv.m#L 2189) : ‘sqrt(psi_norm)'
- [line 2706](matlab/TCV/gdat_tcv.m#L 2706),[2747](matlab/TCV/gdat_tcv.m#L 2747),[3491](matlab/TCV/gdat_tcv.m#L 3491) : rho_pol~sqrt(\psi_norm)
- [line 3043](matlab/TCV/gdat_tcv.m#L 3043) : rhopol\_norm
- [line 3491](matlab/TCV/gdat_tcv.m#L 3491) : sqrt(psi\_norm)
- [line 1082](matlab/TCV/gdat_tcv.m#L 1082) (CXRS) : ‘’ (empty)
- and sometimes as 'psi_TCV' (not found in gdat_tcv.m, maybe directly from the node?)Olivier SauterOlivier Sauterhttps://gitlab.epfl.ch/spc/gdat/-/issues/58IDF, IDR: rho grid label is not given in dimunits2023-05-12T14:13:23ZCassandre Ekta ContréIDF, IDR: rho grid label is not given in dimunitsIn `gdat_aug`, IDF/IDR data are processed with the "signal" method, but on [line 349](matlab/AUG/gdat_aug.m#L349) `aatmp` gives an empty `dimunits`. Could we add a switch case there to specify that it is `rhotornorm` for IDR and `rhopoln...In `gdat_aug`, IDF/IDR data are processed with the "signal" method, but on [line 349](matlab/AUG/gdat_aug.m#L349) `aatmp` gives an empty `dimunits`. Could we add a switch case there to specify that it is `rhotornorm` for IDR and `rhopolnorm` for IDF?Olivier SauterOlivier Sauterhttps://gitlab.epfl.ch/spc/gdat/-/issues/57Job Failed #4092412022-06-29T09:33:28ZOlivier SauterJob Failed #409241Job [#409241](https://gitlab.epfl.ch/spc/gdat/-/jobs/409241) failed for 853e79e6920bb2345ed0014513b1fbf2beb86468:
@amerle gdat tests fail from time to time only with matlab960 when constructing eqdsk, it seems it does not save properly ...Job [#409241](https://gitlab.epfl.ch/spc/gdat/-/jobs/409241) failed for 853e79e6920bb2345ed0014513b1fbf2beb86468:
@amerle gdat tests fail from time to time only with matlab960 when constructing eqdsk, it seems it does not save properly the files during the call ala:
[fnames_readresults]=read_results_for_chease(65565,1,1,3,[],[],[],0,0,1,[129,129]);
and so does not find the file /tmp/$USER/eqdsksigns.65565t1.0000 which is then read by read_eqdsk
Any idea/suggestions? I could never reproduce the error on lacshttps://gitlab.epfl.ch/spc/gdat/-/issues/56Problems loading the core_profiles ids2022-04-08T08:26:10ZMichele MarinProblems loading the core_profiles idsI found a problem loading the core_profiles ids. The commands are
```
[ids_from_tcv]=tcv2ids(64958,'ids_names',{'core_profiles'});
[ids_from_tcv]=tcv2ids(73461,'ids_names',{'core_profiles'});
```
The first works on spcimas but does not...I found a problem loading the core_profiles ids. The commands are
```
[ids_from_tcv]=tcv2ids(64958,'ids_names',{'core_profiles'});
[ids_from_tcv]=tcv2ids(73461,'ids_names',{'core_profiles'});
```
The first works on spcimas but does not work on lac9, the second works on lac9 but does not work on spcimas. The error message is
```
cd /home/matlab/crpptbx-9.6.0/gdat/TCV_IMAS; git describe --tags --first-parent --abbrev=11 --long --dirty --always: Signal 1
Warning: !!!!!!!!!!!!!!!!!!!!!!!!!\results::thomson:times is empty? Check
> In gdat_tcv>get_thomson_raw_data (line 3571)
In gdat_tcv (line 2089)
In gdat (line 190)
In tcv_get_ids_core_profiles (line 39)
In gdat_tcv (line 1487)
In gdat (line 190)
In tcv2ids (line 117)
!!!!!!!!!!!!!!!!!!!!!!!!! cannot continue with ne_rho
Warning: problems with: psi = psitbxtcv(gdat_data.shot,t_eq,'*0',psitbx_str);
> In gdat_tcv>psi_scatvol_zshift (line 3769)
In gdat_tcv (line 2103)
In gdat (line 190)
In tcv_get_ids_core_profiles (line 39)
In gdat_tcv (line 1487)
In gdat (line 190)
In tcv2ids (line 117)
gdat_data.shot = 73461
Name Size Bytes Class Attributes
t_eq 0x0 0 double
Warning: problems calling gdat_tcv
> In gdat (line 196)
In tcv_get_ids_core_profiles (line 39)
In gdat_tcv (line 1487)
In gdat (line 190)
In tcv2ids (line 117)
there is a problem with: tcv_get_ids_core_profiles , may be check if it exists in your path or test it by itself
Warning: problems calling gdat_tcv
> In gdat (line 196)
In tcv2ids (line 117)
Index exceeds matrix dimensions.
Error in gdat_tcv>psi_scatvol_zshift (line 3772)
disp(['t_eq(1) = ' num2str(t_eq(1))])
Error in gdat_tcv (line 2103)
[psiscatvol,psi_max] = psi_scatvol_zshift(gdat_data, zshift, psitbx_str, gdat_params);
Error in gdat (line 190)
[gdat_data,gdat_params,error_status,varargout] = feval(['gdat_' lower(machine_eff)],args{:});
Error in tcv_get_ids_core_profiles (line 39)
temp_1d.ne_rho = gdat(params_cores_profiles.shot,params_eff);
Error in gdat_tcv (line 1487)
[ids_top,ids_top_description] = feval(['tcv_get_ids_' ids_top_name],shot,ids_empty,gdat_data.gdat_params);
Error in gdat (line 190)
[gdat_data,gdat_params,error_status,varargout] = feval(['gdat_' lower(machine_eff)],args{:});
Error in tcv2ids (line 117)
tmp = gdat(shot,gdat_params);
```
It seems that gdat cannot find the thomson data, but I do not know why... The error is the same in both servers (but on different shots).https://gitlab.epfl.ch/spc/gdat/-/issues/55call to gdat 'powers' not working if no ohmic power2023-05-12T14:15:04ZMatteo Vallarmatteo.vallar@epfl.chcall to gdat 'powers' not working if no ohmic powerThe following call
```powers = gdat(69866,'powers')```
gives the following error
```Warning: problems calling gdat_tcv
> In gdat (line 196)
One or more output arguments not assigned during call to "interpos".
Error in gdat_tcv (line ...The following call
```powers = gdat(69866,'powers')```
gives the following error
```Warning: problems calling gdat_tcv
> In gdat (line 196)
One or more output arguments not assigned during call to "interpos".
Error in gdat_tcv (line 2441)
gdat_data.data(:,1) = interpos(-21,gdat_data.ohm.t(ij),gdat_data.ohm.data(ij),gdat_data.t);
Error in gdat (line 190)
[gdat_data,gdat_params,error_status,varargout] = feval(['gdat_' lower(machine_eff)],args{:});
```
It is given by the fact that
`ptot_ohm = tdi('\results::conf:ptot_ohm'); `
is filled with NaNs
@osauter should we add a `try/catch` on `gdat` or should we solve the problem at a higher level (i.e. fill whichever node for the ohmic power `gdat` is using)?https://gitlab.epfl.ch/spc/gdat/-/issues/54Build distribution.markers IDS2022-03-30T09:27:07ZMatteo Vallarmatteo.vallar@epfl.chBuild distribution.markers IDSRelated to https://gitlab.epfl.ch/spc/tcv/analysis/transp/-/issues/22
I'll build a routine to build the distribution function represented as markers on IMAS. <br>
@osauter this is not TCV specific, but it will use TRANSP output. Where d...Related to https://gitlab.epfl.ch/spc/tcv/analysis/transp/-/issues/22
I'll build a routine to build the distribution function represented as markers on IMAS. <br>
@osauter this is not TCV specific, but it will use TRANSP output. Where do you suggest to put it? And which routine should I use as a backbone?https://gitlab.epfl.ch/spc/gdat/-/issues/53Move standard tests to spc-ci2022-03-03T16:16:33ZAntoine MerleMove standard tests to spc-ciAntoine MerleAntoine Merlehttps://gitlab.epfl.ch/spc/gdat/-/issues/52Clarify how to handle invalid profile data2022-02-03T10:11:40ZFederico FeliciClarify how to handle invalid profile dataFollowing discussion in RAPTOR: https://gitlab.epfl.ch/spc/raptor/-/commit/33811c45281388434dab1f33da4794d47b6efe76#note_111832
To be clarified how to handle invalid points being passed by `gdat` in such a way that users don't need to ...Following discussion in RAPTOR: https://gitlab.epfl.ch/spc/raptor/-/commit/33811c45281388434dab1f33da4794d47b6efe76#note_111832
To be clarified how to handle invalid points being passed by `gdat` in such a way that users don't need to second-guess.https://gitlab.epfl.ch/spc/gdat/-/issues/51Ensure test coverage for TCV to IDS scripts2022-01-31T14:42:10ZFederico FeliciEnsure test coverage for TCV to IDS scriptsOlivier SauterOlivier Sauterhttps://gitlab.epfl.ch/spc/gdat/-/issues/50add "`[L,LX] = liuqe()`" where appropriate in TCV2ids2022-01-31T14:37:05ZOlivier Sauteradd "`[L,LX] = liuqe()`" where appropriate in TCV2idsThe following discussion from !116 should be addressed:
- [ ] @ffelici started a [discussion](https://gitlab.epfl.ch/spc/gdat/-/merge_requests/116#note_110825):
> I recommend using `liuqe` functions such as `[L,LX] = liuqe()`...The following discussion from !116 should be addressed:
- [ ] @ffelici started a [discussion](https://gitlab.epfl.ch/spc/gdat/-/merge_requests/116#note_110825):
> I recommend using `liuqe` functions such as `[L,LX] = liuqe()`...Antoine MerleAntoine Merlehttps://gitlab.epfl.ch/spc/gdat/-/issues/49mds-ssh for JET connection should become from public instead of merle's folde...2022-01-31T14:34:38ZOlivier Sautermds-ssh for JET connection should become from public instead of merle's folder at some pointThe following discussion from !116 should be addressed:
- [ ] @ffelici started a [discussion](https://gitlab.epfl.ch/spc/gdat/-/merge_requests/116#note_110822): (+2 comments)
> if this is temporary, let's add an issue to keep trac...The following discussion from !116 should be addressed:
- [ ] @ffelici started a [discussion](https://gitlab.epfl.ch/spc/gdat/-/merge_requests/116#note_110822): (+2 comments)
> if this is temporary, let's add an issue to keep track of the fact this needs to be removed when this mds interface goes public?Olivier SauterOlivier Sauterhttps://gitlab.epfl.ch/spc/gdat/-/issues/48Is field launcher.frequency.data filled?2021-12-21T15:26:17ZMatteo Vallarmatteo.vallar@epfl.chIs field launcher.frequency.data filled?@osauter
It looks like the fields
```
launcher.frequency.data
launcher.frequency.time
```
are not filled when producing the IDS, and when reading from the data in IMAS format.
MWE: `[ids_from_tcv,varargout] = tcv2ids(64770,'ids_names'...@osauter
It looks like the fields
```
launcher.frequency.data
launcher.frequency.time
```
are not filled when producing the IDS, and when reading from the data in IMAS format.
MWE: `[ids_from_tcv,varargout] = tcv2ids(64770,'ids_names', {'ec_launchers'})`
- [x] Check the matlab structure has the field in it
- [ ] Check the data written to IMAS format have the fieldMatteo Vallarmatteo.vallar@epfl.chMatteo Vallarmatteo.vallar@epfl.chhttps://gitlab.epfl.ch/spc/gdat/-/issues/47Error when calling gdat in a loop for different shots2021-12-09T17:11:01ZMatteo Vallarmatteo.vallar@epfl.chError when calling gdat in a loop for different shots@osauter
I don't get why this code is not working.
It looks like the second time gdat runs, it fails to fetch `ne_rho` data.
```
%% ne, te profiles
shots=[72338, 72101];%, 68669];
trial_indx=1;
% shots=[68773, 68779];%, 68669];
delta_s...@osauter
I don't get why this code is not working.
It looks like the second time gdat runs, it fails to fetch `ne_rho` data.
```
%% ne, te profiles
shots=[72338, 72101];%, 68669];
trial_indx=1;
% shots=[68773, 68779];%, 68669];
delta_shots=[1, 1];
time_analysis = [0.7, 1.5];%, 1.554];
col=['k', 'r', 'b'];
leg=[];
time_out = [[0,2.]; [0,2]];%; [0,2]];
linewidth=2.5;
figure('Position', [10, 10, 1000, 600]);
[ha, ~] = tight_subplot(2, 2, 0.1, 0.1, [0.1, 0.01]);
% xxticks=0.2:0.2:1.2;
ax1=ha(1); ax2=ha(2); ax3=ha(3); ax4=ha(4);
for aa=[ax1,ax2,ax3,ax4], hold(aa, 'on'); end
% ax1=subplot(4,1,1);
for ss=1:numel(shots)
ne=gdat_tcv(shots(ss), 'ne_rho', 'time_out', time_analysis(ss), 'trial_indx', trial_indx);
plot(ax1, ne.fit.x, ne.fit.data*1e-19, col(ss), 'LineWidth', linewidth);
te=gdat_tcv(shots(ss), 'te_rho', 'time_out', time_analysis(ss), 'trial_indx', trial_indx);
plot(ax2, te.fit.x, te.fit.data*1e-3, col(ss), 'LineWidth', linewidth);
cxrs=gdat_tcv(shots(ss), 'cxrs', 'time_out', time_analysis(ss), 'trial_indx', trial_indx);
plot(ax3, cxrs.ti.fit.rho, cxrs.ti.fit.data*1e-3, col(ss), 'LineWidth', linewidth);
plot(ax4, cxrs.vtor.fit.rho, cxrs.vtor.fit.data, col(ss), 'LineWidth', linewidth);
label=sprintf('#%i (\\delta<0); t=%.2f s', shots(ss), time_analysis(ss));
if delta_shots(ss)>0
label=sprintf('#%i (\\delta>0); t=%.2f s', shots(ss), time_analysis(ss));
end
leg = [leg; label];
clear label ne te cxrs;
end
% ylim(ax1, [200, 300]);
ylabel(ax1, 'n_e [10^{19} m^{-3}]');
% xticklabels(ax1, []); xticks(ax1, xxticks);
% ax2=subplot(4,1,2);
ylabel(ax2,'T_e [keV]');
ylabel(ax3,'T_i [keV]');
ylabel(ax4,'V_{\phi} [km/s]');
```
I get the following error
```
>> Warning: !!!!!!!!!!!!!!!!!!!!!!!!!\results::thomson:times is empty? Check
> In gdat_tcv>get_thomson_raw_data (line 3535)
In gdat_tcv (line 2083)
!!!!!!!!!!!!!!!!!!!!!!!!! cannot continue with ne_rho
Undefined function or variable 'is'.
Error in iround_os (line 17)
sigi=sig(is);
Error in gdat_tcv>psi_scatvol_zshift (line 3726)
t_eq = t_psi(iround_os(t_psi,t_th));
Error in gdat_tcv (line 2097)
[psiscatvol,psi_max] = psi_scatvol_zshift(gdat_data, zshift, psitbx_str, gdat_params);
```https://gitlab.epfl.ch/spc/gdat/-/issues/46Discrepancy in equilibrium ids from liuqe and from eqdsk2021-11-01T14:06:42ZMatteo Vallarmatteo.vallar@epfl.chDiscrepancy in equilibrium ids from liuqe and from eqdsk@osauter @amerle @mmarin
I was generated the equilibrium ids from liuqe data (`tcv2ids` function, standard call, only asking for `equilibrium` and `core_profiles` ids).
In such case, reading the ids yelds a negative q profile. ~~This fe...@osauter @amerle @mmarin
I was generated the equilibrium ids from liuqe data (`tcv2ids` function, standard call, only asking for `equilibrium` and `core_profiles` ids).
In such case, reading the ids yelds a negative q profile. ~~This feels weird... should IMAS convention be COCOS=11 (thus q>0)?~~ In the matlab structure out of `tcv2ids` q is positive. In the database, it is negative.
If I try to generate the equilibrium using an `eqdsk` in the ids I get a positive q profile.
I am confident about the cocos of the `eqdsk`: I generate it with `cocos_out=2`, and `find_cocos` tells me it is either `1` or `2`. I anyway tried converting from the `eqdsk` telling it was `cocos=1` but I get the same result (looking only at q).
Have you ever encountered similar issues?https://gitlab.epfl.ch/spc/gdat/-/issues/45Proper error handling of mds-related errors2021-10-23T16:30:31ZFederico FeliciProper error handling of mds-related errorsIt is proposed to use `mds` status flag passed through `tdi` to check for errors, instead of tests of the returned data directly.
For example replacing
```
d = tdi('expression')
if ~ischar(d.data) && ~isempty(d.data)
% use data
else
w...It is proposed to use `mds` status flag passed through `tdi` to check for errors, instead of tests of the returned data directly.
For example replacing
```
d = tdi('expression')
if ~ischar(d.data) && ~isempty(d.data)
% use data
else
warning('problem')
end
```
with
```
[d] = tdi('expression')
if ~rem(d.status,2) % unambigious way to check for problems
warning('problem')
end
```
### Previous discussion:
- [ ] @vallar started a [discussion](https://gitlab.epfl.ch/spc/gdat/-/merge_requests/114#note_104015): (+10 comments)