gdat issueshttps://gitlab.epfl.ch/spc/gdat/-/issues2023-11-15T08:00:05Zhttps://gitlab.epfl.ch/spc/gdat/-/issues/63Modify code following renaming of genlib's writestruct function2023-11-15T08:00:05ZAntoine MerleModify code following renaming of genlib's writestruct functionSee spc/tcv/tbx/genlib#29.See spc/tcv/tbx/genlib#29.https://gitlab.epfl.ch/spc/gdat/-/issues/62Make routines for IDS connection matrix interpretation available to MEQ2023-10-23T10:18:59ZFrancesco CarpaneseMake routines for IDS connection matrix interpretation available to MEQ**To**: @osauter @amerle
**For visibility**: @cheiss @ffelici
**Motivation**: Several users/developers are working on IDS-> MEQ parsing routines.
**Topic**: These 2 routines [here](https://gitlab.epfl.ch/spc/gdat/-/blob/master/mat...**To**: @osauter @amerle
**For visibility**: @cheiss @ffelici
**Motivation**: Several users/developers are working on IDS-> MEQ parsing routines.
**Topic**: These 2 routines [here](https://gitlab.epfl.ch/spc/gdat/-/blob/master/matlab/TCV_IMAS/extract_info_connection_matrix.m) [here](https://gitlab.epfl.ch/spc/gdat/-/blob/master/matlab/TCV_IMAS/plot_connection_matrices.m) are useful to interpret the connection matrix stored in the IDS to obtain `Twa`. I was using them to create the windings structure [here](https://gitlab.epfl.ch/spc/tcv/tbx/meq/-/blob/iter_dev/liugids.m#L68) and I would like to leverage them again for https://gitlab.epfl.ch/spc/tcv/tbx/meq/-/merge_requests/632
**Proposed solution**:
1) Bring them to MEQ.
2) Create a dedicate folder IDS with IDS utils, available as submodule for `gdat` and `MEQ`.
3) ??https://gitlab.epfl.ch/spc/gdat/-/issues/61Error when loading 'powers' for some recent shots (>78000)2023-10-09T12:34:06ZCassandre Ekta ContréError when loading 'powers' for some recent shots (>78000)For shots
```
[78360 78359 78358]
```
`gdat(shot,'powers')` throw this error:
```
> Error using interp1>reshapeAndSortXandV (line 409)
> X must be a vector.
>
> Error in interp1 (line 91)
> [X,V,orig_size_v] = reshapeAndSortXand...For shots
```
[78360 78359 78358]
```
`gdat(shot,'powers')` throw this error:
```
> Error using interp1>reshapeAndSortXandV (line 409)
> X must be a vector.
>
> Error in interp1 (line 91)
> [X,V,orig_size_v] = reshapeAndSortXandV(varargin{1},varargin{2});
>
> Error in gdat_tcv (line 2477)
> ip_t = interp1(ip.t,ip.data,gdat_data.ohm.t);
>
> Error in gdat (line 190)
> [gdat_data,gdat_params,error_status,varargout] = feval(['gdat_' lower(machine_eff)],args{:});
```
Then, `78415` (which has evaluation error on all signals) throws an error instead of indicating that the data is empty.
```
> One or more output arguments not assigned during call to "interpos".
> Error in gdat_tcv (line 2476)
> vloop_smooth=interpos(-63,vloop.t,vloop.data,gdat_data.ohm.t,tension);
>
> Error in gdat (line 190)
> [gdat_data,gdat_params,error_status,varargout] = feval(['gdat_' lower(machine_eff)],args{:});
```Cassandre Ekta ContréCassandre Ekta Contréhttps://gitlab.epfl.ch/spc/gdat/-/issues/60Field provenance in IDS nbi when writing into IMAS 3.382023-05-15T16:09:25ZMatteo Vallarmatteo.vallar@epfl.chField provenance in IDS nbi when writing into IMAS 3.38@mmarin @osauter
1. I read the NBI IDS from TCV
2. I try to write the NBI IDS in ITER (both on IMAS 3.34 and IMAS 3.38) and the following error occurs
```
>> [ids_put_status] = ids2database(73116,0,0,ids);
these subfields are not ids na...@mmarin @osauter
1. I read the NBI IDS from TCV
2. I try to write the NBI IDS in ITER (both on IMAS 3.34 and IMAS 3.38) and the following error occurs
```
>> [ids_put_status] = ids2database(73116,0,0,ids);
these subfields are not ids names, so not used: nbi_description params_tcv2ids wall_description
Warning : while putting IDS nbi, the written IDS has non-empty obsolescent node ids_properties/source. Please consider updating the code to avoid using obsolescent nodes.
problems in putting data in database:
Error using ids_put
Unable to get field provenance from input structure
... in structure ids_properties/provenance
... in IDS nbi
Error in ids2database (line 122)
ids_put(idx,ids_to_get,ids2put.(ids_to_get));
```
@mmarin did the full `tcv2ids2database` on `spcimas` and it went throughhttps://gitlab.epfl.ch/spc/gdat/-/issues/59Dimunits names for time & rho2023-05-12T14:20:04ZCassandre Ekta ContréDimunits names for time & rhoCould we standardize the `dimunits` labels for the time, rhopol_norm and rhotor_norm?
### Time
Mainly specified as `s`, I sometimes noticed it was called:
- TCV time (sec)
- time [s]
### Rhopolnorm
Has many nicknames. As a note in gda...Could we standardize the `dimunits` labels for the time, rhopol_norm and rhotor_norm?
### Time
Mainly specified as `s`, I sometimes noticed it was called:
- TCV time (sec)
- time [s]
### Rhopolnorm
Has many nicknames. As a note in gdat_tcv:
- [line 2189](matlab/TCV/gdat_tcv.m#L 2189) : ‘sqrt(psi_norm)'
- [line 2706](matlab/TCV/gdat_tcv.m#L 2706),[2747](matlab/TCV/gdat_tcv.m#L 2747),[3491](matlab/TCV/gdat_tcv.m#L 3491) : rho_pol~sqrt(\psi_norm)
- [line 3043](matlab/TCV/gdat_tcv.m#L 3043) : rhopol\_norm
- [line 3491](matlab/TCV/gdat_tcv.m#L 3491) : sqrt(psi\_norm)
- [line 1082](matlab/TCV/gdat_tcv.m#L 1082) (CXRS) : ‘’ (empty)
- and sometimes as 'psi_TCV' (not found in gdat_tcv.m, maybe directly from the node?)Olivier SauterOlivier Sauterhttps://gitlab.epfl.ch/spc/gdat/-/issues/58IDF, IDR: rho grid label is not given in dimunits2023-05-12T14:13:23ZCassandre Ekta ContréIDF, IDR: rho grid label is not given in dimunitsIn `gdat_aug`, IDF/IDR data are processed with the "signal" method, but on [line 349](matlab/AUG/gdat_aug.m#L349) `aatmp` gives an empty `dimunits`. Could we add a switch case there to specify that it is `rhotornorm` for IDR and `rhopoln...In `gdat_aug`, IDF/IDR data are processed with the "signal" method, but on [line 349](matlab/AUG/gdat_aug.m#L349) `aatmp` gives an empty `dimunits`. Could we add a switch case there to specify that it is `rhotornorm` for IDR and `rhopolnorm` for IDF?Olivier SauterOlivier Sauterhttps://gitlab.epfl.ch/spc/gdat/-/issues/55call to gdat 'powers' not working if no ohmic power2023-05-12T14:15:04ZMatteo Vallarmatteo.vallar@epfl.chcall to gdat 'powers' not working if no ohmic powerThe following call
```powers = gdat(69866,'powers')```
gives the following error
```Warning: problems calling gdat_tcv
> In gdat (line 196)
One or more output arguments not assigned during call to "interpos".
Error in gdat_tcv (line ...The following call
```powers = gdat(69866,'powers')```
gives the following error
```Warning: problems calling gdat_tcv
> In gdat (line 196)
One or more output arguments not assigned during call to "interpos".
Error in gdat_tcv (line 2441)
gdat_data.data(:,1) = interpos(-21,gdat_data.ohm.t(ij),gdat_data.ohm.data(ij),gdat_data.t);
Error in gdat (line 190)
[gdat_data,gdat_params,error_status,varargout] = feval(['gdat_' lower(machine_eff)],args{:});
```
It is given by the fact that
`ptot_ohm = tdi('\results::conf:ptot_ohm'); `
is filled with NaNs
@osauter should we add a `try/catch` on `gdat` or should we solve the problem at a higher level (i.e. fill whichever node for the ohmic power `gdat` is using)?https://gitlab.epfl.ch/spc/gdat/-/issues/54Build distribution.markers IDS2022-03-30T09:27:07ZMatteo Vallarmatteo.vallar@epfl.chBuild distribution.markers IDSRelated to https://gitlab.epfl.ch/spc/tcv/analysis/transp/-/issues/22
I'll build a routine to build the distribution function represented as markers on IMAS. <br>
@osauter this is not TCV specific, but it will use TRANSP output. Where d...Related to https://gitlab.epfl.ch/spc/tcv/analysis/transp/-/issues/22
I'll build a routine to build the distribution function represented as markers on IMAS. <br>
@osauter this is not TCV specific, but it will use TRANSP output. Where do you suggest to put it? And which routine should I use as a backbone?https://gitlab.epfl.ch/spc/gdat/-/issues/51Ensure test coverage for TCV to IDS scripts2022-01-31T14:42:10ZFederico FeliciEnsure test coverage for TCV to IDS scriptsOlivier SauterOlivier Sauterhttps://gitlab.epfl.ch/spc/gdat/-/issues/50add "`[L,LX] = liuqe()`" where appropriate in TCV2ids2022-01-31T14:37:05ZOlivier Sauteradd "`[L,LX] = liuqe()`" where appropriate in TCV2idsThe following discussion from !116 should be addressed:
- [ ] @ffelici started a [discussion](https://gitlab.epfl.ch/spc/gdat/-/merge_requests/116#note_110825):
> I recommend using `liuqe` functions such as `[L,LX] = liuqe()`...The following discussion from !116 should be addressed:
- [ ] @ffelici started a [discussion](https://gitlab.epfl.ch/spc/gdat/-/merge_requests/116#note_110825):
> I recommend using `liuqe` functions such as `[L,LX] = liuqe()`...Antoine MerleAntoine Merlehttps://gitlab.epfl.ch/spc/gdat/-/issues/49mds-ssh for JET connection should become from public instead of merle's folde...2022-01-31T14:34:38ZOlivier Sautermds-ssh for JET connection should become from public instead of merle's folder at some pointThe following discussion from !116 should be addressed:
- [ ] @ffelici started a [discussion](https://gitlab.epfl.ch/spc/gdat/-/merge_requests/116#note_110822): (+2 comments)
> if this is temporary, let's add an issue to keep trac...The following discussion from !116 should be addressed:
- [ ] @ffelici started a [discussion](https://gitlab.epfl.ch/spc/gdat/-/merge_requests/116#note_110822): (+2 comments)
> if this is temporary, let's add an issue to keep track of the fact this needs to be removed when this mds interface goes public?Olivier SauterOlivier Sauterhttps://gitlab.epfl.ch/spc/gdat/-/issues/48Is field launcher.frequency.data filled?2021-12-21T15:26:17ZMatteo Vallarmatteo.vallar@epfl.chIs field launcher.frequency.data filled?@osauter
It looks like the fields
```
launcher.frequency.data
launcher.frequency.time
```
are not filled when producing the IDS, and when reading from the data in IMAS format.
MWE: `[ids_from_tcv,varargout] = tcv2ids(64770,'ids_names'...@osauter
It looks like the fields
```
launcher.frequency.data
launcher.frequency.time
```
are not filled when producing the IDS, and when reading from the data in IMAS format.
MWE: `[ids_from_tcv,varargout] = tcv2ids(64770,'ids_names', {'ec_launchers'})`
- [x] Check the matlab structure has the field in it
- [ ] Check the data written to IMAS format have the fieldMatteo Vallarmatteo.vallar@epfl.chMatteo Vallarmatteo.vallar@epfl.chhttps://gitlab.epfl.ch/spc/gdat/-/issues/47Error when calling gdat in a loop for different shots2021-12-09T17:11:01ZMatteo Vallarmatteo.vallar@epfl.chError when calling gdat in a loop for different shots@osauter
I don't get why this code is not working.
It looks like the second time gdat runs, it fails to fetch `ne_rho` data.
```
%% ne, te profiles
shots=[72338, 72101];%, 68669];
trial_indx=1;
% shots=[68773, 68779];%, 68669];
delta_s...@osauter
I don't get why this code is not working.
It looks like the second time gdat runs, it fails to fetch `ne_rho` data.
```
%% ne, te profiles
shots=[72338, 72101];%, 68669];
trial_indx=1;
% shots=[68773, 68779];%, 68669];
delta_shots=[1, 1];
time_analysis = [0.7, 1.5];%, 1.554];
col=['k', 'r', 'b'];
leg=[];
time_out = [[0,2.]; [0,2]];%; [0,2]];
linewidth=2.5;
figure('Position', [10, 10, 1000, 600]);
[ha, ~] = tight_subplot(2, 2, 0.1, 0.1, [0.1, 0.01]);
% xxticks=0.2:0.2:1.2;
ax1=ha(1); ax2=ha(2); ax3=ha(3); ax4=ha(4);
for aa=[ax1,ax2,ax3,ax4], hold(aa, 'on'); end
% ax1=subplot(4,1,1);
for ss=1:numel(shots)
ne=gdat_tcv(shots(ss), 'ne_rho', 'time_out', time_analysis(ss), 'trial_indx', trial_indx);
plot(ax1, ne.fit.x, ne.fit.data*1e-19, col(ss), 'LineWidth', linewidth);
te=gdat_tcv(shots(ss), 'te_rho', 'time_out', time_analysis(ss), 'trial_indx', trial_indx);
plot(ax2, te.fit.x, te.fit.data*1e-3, col(ss), 'LineWidth', linewidth);
cxrs=gdat_tcv(shots(ss), 'cxrs', 'time_out', time_analysis(ss), 'trial_indx', trial_indx);
plot(ax3, cxrs.ti.fit.rho, cxrs.ti.fit.data*1e-3, col(ss), 'LineWidth', linewidth);
plot(ax4, cxrs.vtor.fit.rho, cxrs.vtor.fit.data, col(ss), 'LineWidth', linewidth);
label=sprintf('#%i (\\delta<0); t=%.2f s', shots(ss), time_analysis(ss));
if delta_shots(ss)>0
label=sprintf('#%i (\\delta>0); t=%.2f s', shots(ss), time_analysis(ss));
end
leg = [leg; label];
clear label ne te cxrs;
end
% ylim(ax1, [200, 300]);
ylabel(ax1, 'n_e [10^{19} m^{-3}]');
% xticklabels(ax1, []); xticks(ax1, xxticks);
% ax2=subplot(4,1,2);
ylabel(ax2,'T_e [keV]');
ylabel(ax3,'T_i [keV]');
ylabel(ax4,'V_{\phi} [km/s]');
```
I get the following error
```
>> Warning: !!!!!!!!!!!!!!!!!!!!!!!!!\results::thomson:times is empty? Check
> In gdat_tcv>get_thomson_raw_data (line 3535)
In gdat_tcv (line 2083)
!!!!!!!!!!!!!!!!!!!!!!!!! cannot continue with ne_rho
Undefined function or variable 'is'.
Error in iround_os (line 17)
sigi=sig(is);
Error in gdat_tcv>psi_scatvol_zshift (line 3726)
t_eq = t_psi(iround_os(t_psi,t_th));
Error in gdat_tcv (line 2097)
[psiscatvol,psi_max] = psi_scatvol_zshift(gdat_data, zshift, psitbx_str, gdat_params);
```https://gitlab.epfl.ch/spc/gdat/-/issues/45Proper error handling of mds-related errors2021-10-23T16:30:31ZFederico FeliciProper error handling of mds-related errorsIt is proposed to use `mds` status flag passed through `tdi` to check for errors, instead of tests of the returned data directly.
For example replacing
```
d = tdi('expression')
if ~ischar(d.data) && ~isempty(d.data)
% use data
else
w...It is proposed to use `mds` status flag passed through `tdi` to check for errors, instead of tests of the returned data directly.
For example replacing
```
d = tdi('expression')
if ~ischar(d.data) && ~isempty(d.data)
% use data
else
warning('problem')
end
```
with
```
[d] = tdi('expression')
if ~rem(d.status,2) % unambigious way to check for problems
warning('problem')
end
```
### Previous discussion:
- [ ] @vallar started a [discussion](https://gitlab.epfl.ch/spc/gdat/-/merge_requests/114#note_104015): (+10 comments)https://gitlab.epfl.ch/spc/gdat/-/issues/42set up imas tests on spcimas2021-10-07T07:36:21ZFederico Feliciset up imas tests on spcimashttps://gitlab.epfl.ch/spc/gdat/-/issues/41NBI fraction2023-03-09T16:39:48ZOlivier SauterNBI fraction@vallar To check line 93 of TCV_IMAS/tcv_get_ids_nbi.m:
`i_frac = i_frac.*1./repmat(sum(i_frac,2), 1, 3);`
I am astonished there are digits instead of "size(xxx,yy)" for example as before. This comes after your (old) merge request for ...@vallar To check line 93 of TCV_IMAS/tcv_get_ids_nbi.m:
`i_frac = i_frac.*1./repmat(sum(i_frac,2), 1, 3);`
I am astonished there are digits instead of "size(xxx,yy)" for example as before. This comes after your (old) merge request for the nbi fraction: old branch: fix_ids_curr_fracMatteo Vallarmatteo.vallar@epfl.chMatteo Vallarmatteo.vallar@epfl.chhttps://gitlab.epfl.ch/spc/gdat/-/issues/39summary field in ids structure2021-03-08T16:45:34ZMatteo Vallarmatteo.vallar@epfl.chsummary field in ids structure@osauter
I loaded a matlab ids structure from lacs, with the following command
`data=tcv2ids(64770, 'trialindx',3);`
Then I moved the structure I got to the iter cluster (you can find it in '/home/ITER/vallarm/public/data_64770_ids_t...@osauter
I loaded a matlab ids structure from lacs, with the following command
`data=tcv2ids(64770, 'trialindx',3);`
Then I moved the structure I got to the iter cluster (you can find it in '/home/ITER/vallarm/public/data_64770_ids_trialindx3.mat')
In that structure, there is the field "summary" and when I do ids2database it is complaining because it cannot write that field. I will remove it from the structure as quick-fix, but is it possible to set it as "not ids name" as is done with the following?
```
these subfields are not ids names, so not used: core_profiles_description ec_launchers_description equilibrium_description magnetics_description nbi_description params_tcv2ids pf_active_description summary_description tf_description thomson_scattering_description wall_description
```https://gitlab.epfl.ch/spc/gdat/-/issues/38Follow-up from "fix TCV ids for version 3.31"2021-02-17T16:51:59ZOlivier SauterFollow-up from "fix TCV ids for version 3.31"The following discussion from !88 should be addressed:
- [ ] @osauter started a [discussion](https://gitlab.epfl.ch/spc/gdat/-/merge_requests/88#note_81899): (+10 comments)
> @vallar Can you test on ITER IO the full data get and w...The following discussion from !88 should be addressed:
- [ ] @osauter started a [discussion](https://gitlab.epfl.ch/spc/gdat/-/merge_requests/88#note_81899): (+10 comments)
> @vallar Can you test on ITER IO the full data get and write, I put the branch "fix TCV ids for version 3.31" for the "official gdat" in my public. I added the CXRS links
>
> Once checked I'll add the ids_empty of the most recent version so it will work from the lacs as well
>
> @amerle @vallar It had "wrong" default structure even when running it at ITER-IO because there was an ```if exist('ids_list') else use IDS_list``` which was not working as expected because matlab was thinking the first ids_list was misprinted and asking if IDS_list should be used... I am now checking ```if exist('IDS_list')``` (this is historical we can soon, may be already now, forget about the former ids_list function)Antoine MerleAntoine Merlehttps://gitlab.epfl.ch/spc/gdat/-/issues/36Changes of core_profiles/profiles_1d/grid/area to core_profiles/profiles_1d/g...2021-02-08T10:27:38ZMatteo Vallarmatteo.vallar@epfl.chChanges of core_profiles/profiles_1d/grid/area to core_profiles/profiles_1d/grid/surface@osauter @amerle
I think the name of a `core_profiles/profiles_1d/grid/area` changed to `core_profiles/profiles_1d/grid/surface` in latest imas version.
If I load `IMAS/3.27.0-4.7.1`, everything works smoothly
```
$ module load MATL...@osauter @amerle
I think the name of a `core_profiles/profiles_1d/grid/area` changed to `core_profiles/profiles_1d/grid/surface` in latest imas version.
If I load `IMAS/3.27.0-4.7.1`, everything works smoothly
```
$ module load MATLAB
$ module load IMAS
$ matlab-nodesktop
>> ids=load('/home/ITER/vallarm/public/68587_eq_coreprof.mat');
>> run ~/../sautero/public/matlab/startup.m
>> [ids_put_status] = ids2database(68587, 001, 001, ids);
core_profiles ec_launchers equilibrium magnetics nbi pf_active summary tf thomson_scattering wall Warning: IDS_list not available, quick fix introducing list of ids available for TCV: b
> In ids2database (line 92)
these subfields are not ids names, so not used: core_profiles_description equilibrium_description params_tcv2ids
problems in putting data in database:
Error using ids_put
Unable to get field surface from input structure
... in field profiles_1d/grid/surface
... in IDS core_profiles
Error in ids2database (line 126)
ids_put(idx,[ids_to_get '/' num2str(occurence)],ids2put.(ids_to_get));
```https://gitlab.epfl.ch/spc/gdat/-/issues/34Job Failed #1769412020-11-30T14:30:41ZFederico FeliciJob Failed #176941The nightly tests are failing for `gdat` since the latest change, see e.g.
Job [#176941](https://gitlab.epfl.ch/spc/gdat/-/jobs/176941) failed for 7a655c5fff0b3be51366f9ecd6c6f3877b7482cb:
Try to run the tests locally to see whether th...The nightly tests are failing for `gdat` since the latest change, see e.g.
Job [#176941](https://gitlab.epfl.ch/spc/gdat/-/jobs/176941) failed for 7a655c5fff0b3be51366f9ecd6c6f3877b7482cb:
Try to run the tests locally to see whether they fail locally as well or whether it's an issue with the test setup.
You can see the script the test is trying to run on line 44 of the test log:
https://gitlab.epfl.ch/spc/gdat/-/jobs/176941#L44Olivier SauterOlivier Sauter