Unverified Commit 18fcd31d authored by jayd1860's avatar jayd1860 Committed by GitHub
Browse files

Merge changes to make DataTree work better standalone with new processing examples (#149) 

* v1.55.0

-- Change to stimulus loading to load from events .tsv file if it is present. It would override SNIRF stim field and StimEditGUI. If not present then default to SNIRF stim and StimEditGUI for stim loading and editing.

* v1.55.1

-- Fix some bugs with stim export to events TSV files.
-- Add documentation to DesignNotes.ppt describing the new stim loading (from events TSV files) and exporting (to events TSV files) feature.
-- Add option to Snirf2Tsv.m to find and delete all events TSV files

* v1.56.0

-- Fix issue in Snirf.GetDataTimeSeriesMatrix() with incorrect conversion to aux matrix when aux sample rate is different than data sample rate.
-- Move events TSV loading menu option from StimEditGUI.fig to MainGUI.fig per Davis's request
-- Fix bug in moving old output derived data folder homerOutput to new BIDS style derivative/homer.

* v1.56.1

-- One more bug fix in SnirfClass.GetAuxDataMatrix() where the case of if aux has same time base as data is not being handles properly.
-- Added config option "Quiet Mode" to be able to run processing without progress bars being displayed so it doesn't constantly take control of the mouse and one could work in parallel in other windows without interference.

* v1.56.2

-- Added automatic stim loading from TSV file (if changes were made or new file created) when clicking on run. 
-- Change to SnirfFile2Tsv.m: if snirf stim field is empty then generate empty TSV file with only the default column names instead not generating any file at all as was done before.
-- Change the way we detect if stim changes were made to acquisition file.
-- Minor change to readTsv.m to automatically complete input file name without an extention with a '.tsv' extension.
-- Add DeleteTsv.m for diagnostic purposes.

* v1.57.0

-- Make config changes made through Settings-->Edit App Configuration menu take effect immediately after saving so that one does not have to restart Homer3.

-- Fix bug loading stims: Change order at init time when DataTreeClass.SetConditions() is called; before or after loading groupResult.mat. This order was changed several months ago (I believe in version 1.38.2) to be SetConditions then load groupResults. The problem with this is that it groupResults then overwrites the correct condition initialization for the whole group if groupResults happened to have unsaved condition changes that differ from SetConditions. This may be inconsistent with the correct condition set generated by SetConditions. One solution could be that TreeNode.Copy() groupResults method SHOULD NOT BE copying conditions - i.e, TreeNode.CondNames field. The other solution - the one used here - is change the order back to calling SetConditions() after copying groupResults. Although I have to try to remember why that change to caLL SetConditions first was made in the first place IN v1.38.2. It seems like a mistaken solution to address some sort of issue which the commit message does not explain.

-- Clean up: delete unused and obsolete files.

* v1.58.0

-- Add back showing pruned channels in PlotProbeGUI. Also there's a new ability to display non-HRF time course in PlotProbeGUI. If user is displaying non-HRF data then a warning pops up informing about this. You can opt out of receiving with every element selection.

-- Standardized datatype names in methods GetDataTimeSeries() and GetMeasurementList(), across different levels in DataTreeClass, to be better able to use these methods in scripts.

-- Made small change to hmrR_PreprocessIntensity_Negative per Meryem's request.

-- Fix display issue in manually pruning channels with it not always being displayed correctly with a dotted line in the probe axes. This is because of lack of clarity with currently selected wavelength with only the currently selected wavelength measurement being toggled. To make things simple when selecting a channel for pruning measurements from ALL of the wavelengths are toggled simultaneously. Also to clarify, the meaning of manual pruning is always to the raw/OD timecourse data even when it is NOT selected in the Plot Select Data

-- Fix small issue in syncSubmodules when copying peerless files in non-existent folders in the destination the folder needs to be created first (that is, copyfile won't do it for you).

* v1.59.0

-- Fix retrieving inactive channels in hmrR_PCAFilter.m using the new mlAct supporting user functions.
-- Update design doc to document new loading and exporting of stim from and to BIDS events TSV files.

* Make standalone DataTree work by supplying missing cell2str.m function to shared library

* v1.60.0

-- Include startup functions in setpaths to be able initialize search paths for DataTree.
-- Add examples folder with runProcessingStream.m on how to download and run processing stream in DataTree standalone.

* v1.61.0

-- Add initialization script to run after downloading DataTree which downloads dependencies Utils and FuncRegistry and runs setpaths for ease of standalone operation
-- Add submodules .gitmodules file to DataTree library to stream line process of downloading dependencies when running DataTree standalone
-- Provide plotting method for TreeNodeClass elements to run from command line
-- Fix typos in .gitmodules and add FuncRegistry as a third submodule/library that Homer3 uses
-- Provide LoadRunPlotExamples.m example of how to use DataTree from scratch.
-- Add missing description of input argument mlAct in hmrR_PCAFilter.m

* v1.61.1

-- Fix typo in DataTree/.gitmodules FuncRegistry entry. This was fixed in the parent (Homer3) .gitmodules but not DataTree.
-- In syncSubmodules.m add .gitmodules to the list of file types to search for when syncing parent and submodules

* v1.62.0

-- Sync with changes in DataTree and Utils shared libraries:

  DataTree:

      1. Rename LoadRunPlotExamples to LoadRunPlotExample1.m and LoadRunPlotExample2.m - LoadRunPlotExample1 uses TreeNodeClass methods to get access to data and time and plot them, while LoadRunPlotExample2.m does the same thing BUT accesses snirf objects directly without any methods and then plots them directly in the script to show user how snirf objects work. Add example data that goes with it Example4_twNI.

      2. Add another example EditStimExample.m which shows how to export and edit stim and conditions. Add example data that goes with it SubjDataSample.

      3. Fix bugs in StimClass methods for editing stims (adding, deleting, and changing condition of) based on an inclusive time points vector.

      4. Fix run/plot examples and remove the download and initialization parts as that is unnecessary and does not quite makes sense since you have to first download the repo before accessing the LoadRunPlotExamples.m script.  This part can simply be described in the comments.

      5. Simplify initialization by moving downloading of submodules to setpaths and get rid of initialize.m

   Utils:

      1. Add capability to readTsv.m to be able to load TSV file that is separated by spaces instead of tabs if it is can be clearly determined that spaces are separators (instead of say names of conditions). If spaces are detected then the file is written to correctly contain tab separators as is appropriate for the .tsv format.

      2. Add namespace DataTreeClass to setGuiFonts.m so that fonts can be set in a figure when DataTree standalone is running

      3. Return figure handle of MenuBox.m so that it can be repositioned after calling it
parent 5f538f6a

.gitmodules

+4 −1
Original line number Diff line number Diff line 
[submodule "Utils/Shared"]

	path = Utils/Shared

	url = https://github.com/BUNPC/Utils

[submodule "Group/DataTree"]

[submodule "DataTree"]

	path = DataTree

	url = https://github.com/BUNPC/DataTree

[submodule "FuncRegistry"]

	path = FuncRegistry

	url = https://github.com/BUNPC/FuncRegistry

DataTree/.gitmodules

0 → 100644
+6 −0
Original line number Diff line number Diff line 
[submodule "Utils"]

	path = Utils

	url = https://github.com/BUNPC/Utils

[submodule "FuncRegistry"]

	path = FuncRegistry

	url = https://github.com/BUNPC/FuncRegistry

DataTree/AcquiredData/Snirf/StimClass.m

+29 −57
Original line number Diff line number Diff line
@@ -464,17 +464,8 @@ classdef StimClass < FileLoadSaveClass 
            if length(amp) < length(tPts)

                amp = [amp; ones(length(tPts)-length(amp),1)];

            end            

            

            if isempty(obj.data)

                obj.data = [tPts, duration, amp, more];

            else

                obj.data(end+1,:) = [tPts, duration, amp, more];                

                end

            if isempty(obj.states)

                obj.states = [tPts, ones(length(tPts),1)];

            else

                obj.states(end+1,:) = [tPts, ones(length(tPts),1)];

            end

            obj.data = [obj.data; tPts, duration, amp, more];

            obj.states = [obj.states; tPts, ones(length(tPts),1)];

        end
@@ -486,14 +477,9 @@ classdef StimClass < FileLoadSaveClass 
            if ~exist('amp','var')

                amp = 1;

            end

            for ii=1:length(tPts)

                k = find( abs(obj.data(:,1)-tPts(ii)) < obj.errmargin );

                if isempty(k)

                    continue;

                end

            k = GetIncludedStimIdxs(obj, tPts);

            obj.data(k,3) = amp;            

        end

        end



        

        % ----------------------------------------------------------------------------------
@@ -504,14 +490,9 @@ classdef StimClass < FileLoadSaveClass 
            if ~exist('state','var')

                state = 1;

            end

            for ii=1:length(tPts)

                k = find( abs(obj.states(:,1)-tPts(ii)) < obj.errmargin );

                if isempty(k)

                    continue;

                end

            k = GetIncludedStimIdxs(obj, tPts);

            obj.states(k,2) = state;

        end

        end

        

        

        % ----------------------------------------------------------------------------------
@@ -543,6 +524,22 @@ classdef StimClass < FileLoadSaveClass 
        end

        

        

        % -------------------------------------------------------

        function k = GetIncludedStimIdxs(obj, tPts)

            k = [];

            if length(tPts)<2

                errmargin = obj.errmargin;

            else

                errmargin = mean(diff(tPts));

            end

            for ii = 1:size(obj.data,1)

                if ~isempty(find( abs(obj.data(ii,1)-tPts) < errmargin))

                    k = [k; ii];

                end

            end

        end        

        

        

        % ----------------------------------------------------------------------------------

        function SetDuration(obj, duration, tPts)

            if isempty(obj.data)
@@ -554,13 +551,7 @@ classdef StimClass < FileLoadSaveClass 
            if ~exist('tPts','var')

                tPts = obj.data(:,1);

            end

            k=[];

            for ii=1:length(tPts)

                k(ii) = find( abs(obj.data(:,1)-tPts(ii)) < obj.errmargin );

                if isempty(k)

                    continue;

                end

            end

            k = GetIncludedStimIdxs(obj, tPts);

            obj.data(k,2) = duration;

        end
@@ -574,10 +565,7 @@ classdef StimClass < FileLoadSaveClass 
            if ~exist('tPts','var')

                tPts = obj.data(:,1);

            end

            k = [];

            for ii=1:length(tPts)

                k(ii) = find( abs(obj.data(:,1)-tPts(ii)) < obj.errmargin );

            end            

            k = GetIncludedStimIdxs(obj, tPts);

            duration = obj.data(k,2);

        end
@@ -593,13 +581,7 @@ classdef StimClass < FileLoadSaveClass 
            if ~exist('tPts','var')

                tPts = obj.data(:,1);

            end

            k=[];

            for ii=1:length(tPts)

                k(ii) = find( abs(obj.data(:,1)-tPts(ii)) < obj.errmargin );

                if isempty(k)

                    continue;

                end

            end

            k = GetIncludedStimIdxs(obj, tPts);

            obj.data(k,3) = amps;

        end
@@ -613,10 +595,7 @@ classdef StimClass < FileLoadSaveClass 
            if ~exist('tPts','var')

                tPts = obj.data(:,1);

            end

            k = [];

            for ii=1:length(tPts)

                k(ii) = find( abs(obj.data(:,1)-tPts(ii)) < obj.errmargin );

            end

            k = GetIncludedStimIdxs(obj, tPts);

            amps = obj.data(k,3);

        end
@@ -629,11 +608,7 @@ classdef StimClass < FileLoadSaveClass 
            

            % Find all stims for any conditions which match the time points and 

            % delete them from data. 

            k = [];

            j = [];

            for ii=1:length(tPts)

                k = [k, find( abs(obj.data(:,1)-tPts(ii)) < obj.errmargin )];

            end

            k = GetIncludedStimIdxs(obj, tPts);

            obj.data(k,:) = [];

            obj.states(k,:) = [];

        end
@@ -648,10 +623,7 @@ classdef StimClass < FileLoadSaveClass 
            

            % Find all stims for any conditions which match the time points and 

            % flip their states

            k = [];

            for ii=1:length(tPts)

                k = [k, find( abs(obj.states(:,1)-tPts(ii)) < obj.errmargin )];

            end

            k = GetIncludedStimIdxs(obj, tPts);

            obj.states(k,2) = -1*obj.states(k,2);

        end

DataTree/Examples/EditStimExample.m

0 → 100644
+120 −0
Original line number Diff line number Diff line 
function EditStimExample(dataSetDir)



%

%   Syntax:

%       EditStimExample(dataSetDir)

%

%   Description:

%       This script shows how to export and edit stim and conditions. This script does the following:

%

%       1. If starting from scratch, first download the DataTree repo from

%          github

%

%           a) git clone  -b <branch_name>   https://github.com/<username>/DataTree  <local_path>/DataTree  

%           b) cd <local_path>/DataTree

%           c) setpaths

%

%       2. Change folder to dataSetDir

%       3. Load data set

%       4. Plot raw data with stim for current element 

%       5. Export stim for current element if the events TSV file does not

%          exist

%       6. Re-plot raw data with stim for current element showing edited stim

%          and conditions

%

%   Examples:

%       cd <local_path>/DataTree

%       EditStimExample('./Examples/SubjDataSample')

%





%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Parse arguments

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

if ~exist('dataSetDir','var')

    f = which('EditStimExample');

    dataSetDir = [fileparts(f), '/SubjDataSample']; 

end





%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% a) Change current folder to a data set folder, 

% b) Load dataset

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

if ~exist(dataSetDir,'dir')

    mkdir(dataSetDir);

end

cd(dataSetDir) 

if exist([dataSetDir, '/derivatives']','dir')

    rmdir([dataSetDir, '/derivatives'], 's')

end

dataTree = DataTreeClass();

if dataTree.IsEmpty()

    MenuBox('No data set was loaded',{'OK'});

    return

end





%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% This step is NOT essential, but is a good idea in order to

% set up logging and load config settings, something that 

% would normally be done by the parent GUI. Since we are 

% running data tree standalone initilize logger an config 

% settings here instead of through parent gui like Homer3 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

global logger

global cfg



logger = Logger('DataTreeClass');

cfg = ConfigFileClass();



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% 1. Plot raw data for current element for channels 1 and 2

% 2. Export stim for the current element

% 3. Edit stim in the events TSV file

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

obj = dataTree.currElem;

obj.Load();

obj.Plot('raw',[1,2]);

obj.ExportStim();



pause(2);



EditEventsTsvFile_InMatlabEditor(obj, dataSetDir);



obj.ReloadStim();

obj.Plot('raw',[1,2]);





%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Close open log file

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

obj.logger.Close();











% -------------------------------------------------------------------------

function editorTab = EditEventsTsvFile_InMatlabEditor(obj, dataSetDir)

% Get file name of data file

filenameData = [dataSetDir, '/', obj.GetFilename()];

[~,f] = fileparts(filenameData);



% From data file name get events TSV file and load in matlab editor

filenameEvents = [dataSetDir, '/', f, '_events.tsv'];



edit(filenameEvents);

editorTabs = matlab.desktop.editor.getAll;



% Search for editor tab containing loaded file and make it active

for ii = 1:length(editorTabs)

    if pathscompare(editorTabs(ii).Filename, filenameEvents)

        break

    end

end

editorTab = editorTabs(ii);

editorTab.makeActive;

MenuBox('Please edit TSV stim file and save it, then click the ''OK'' button.');

DataTree/Examples/LoadRunPlotExample1.m

0 → 100644
+115 −0
Original line number Diff line number Diff line 
function LoadRunPlotExample1(dataSetDir)



%

%   Syntax:

%       LoadRunPlotExample1(dataSetDir)

%

%   Description:

%       This script does the following:

%

%       1. If starting from scratch, first download the DataTree repo from

%          github

%

%           a) git clone  -b <branch_name>   https://github.com/<username>/DataTree  <local_path>/DataTree  

%           b) cd <local_path>/DataTree

%           c) setpaths

%

%       2. Change folder to dataSetDir

%       3. Load data set

%       4. Plot raw data for various dataTree elements

%       5. Run processing stream

%       6. Plot HRF for various dataTree elements

%

%   Examples:

%       cd <local_path>/DataTree

%       LoadRunPlotExample1('./Examples/Example4_twNI')

%







%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Parse arguments

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

if ~exist('dataSetDir','var')

    f = which('LoadRunPlotExample1');

    dataSetDir = [fileparts(f), '/Example4_twNI']; 

end





%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% a) Change current folder to a data set folder, 

% b) Load dataset

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

if ~exist(dataSetDir,'dir')

    mkdir(dataSetDir);

end

cd(dataSetDir) 

if exist([dataSetDir, '/derivatives']','dir')

    rmdir([dataSetDir, '/derivatives'], 's')

end

dataTree = DataTreeClass();

if dataTree.IsEmpty()

    MenuBox('No data set was loaded',{'OK'});

    return

end







%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% This step is NOT essential, but is a good idea in order to

% set up logging. Since we are running data tree standalone 

% initilize logger here which would have been done by parent 

% gui like Homer3 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

global logger

logger = Logger('DataTreeClass');







%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Plot raw data for current element for channels 6 nd 7

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

obj = dataTree.currElem;

obj.Load();

obj.Plot('raw',[6,7]);

pause(2);







%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Plot raw data for 3rd run of the 1st session, 3rd subject,  

% and 1st group for channels 6 nd 7

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

obj = dataTree.groups(1).subjs(2).sess(1).runs(3);

obj.Load();

obj.Plot('raw',[6,7]);

pause(2);







%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Run processing stream for current element and plot 

% concentration HRF for channels 2,3 and 4

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

obj = dataTree.currElem;

obj.Calc();

obj.Plot('conc hrf',[2,3,4]);

pause(2);







%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Run processing stream for the first group and plot it's

% concentration HRF for channels 2,3 and 4

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

obj = dataTree.groups(1);

obj.Calc();

obj.Plot('conc hrf', [2,3,4]);







%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Close open log files

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

obj.logger.Close();

CRA Git | Maintained and supported by SUSTech CRA and CCSE