Skip to content

Implement thresholding in graph reconstructors - #93

Merged
tlarock merged 7 commits into
netsiphd:masterfrom
sdmccabe:master
Feb 13, 2019
Merged

tlarock merged 7 commits into
netsiphd:masterfrom
sdmccabe:master

Conversation

@sdmccabe

Copy link
Copy Markdown
Collaborator

Resolves #38. #57 introduced utilities for thresholding matrices in various ways, but they were largely not used. This incorporates them into the existing graph reconstructors where feasible.

There are a few reconstructors that do not threshold. For example, the minimum spanning tree does not threshold (because it is a minimum spanning tree). The one edge case here (which I did not touch) is convergent cross-mapping, which thresholds a matrix of p-values (i.e., a significance test). This could, in principle, be covered by the threshold utility function, but I don't think it's entirely appropriate to do so. (Giving the method the ability to set the significance level to get a fixed average degree seems to me more problematic than dropping low correlations to get a fixed average degree, but opinions may vary here.)

As a side effect of implementing this feature, the fit() method has been changed across all implementations (including the base reconstructor) to take arbitrary named keyword arguments through **kwargs. This allows us to pass threshold parameters freely, even to non-thresholding methods (where the parameters are ignored), which makes using this feature much easier in practice (see the change to netrdExplorer as an illustration of why).

As a side effect, this PR also adds tests for the thresholding functions and cleans up a number of the imports to netrd.utilities.

Finally, I'll note that some of the language used in the different implementations differs; e.g., matrix versus weights. Instead of addressing that in code review of this PR I would suggest raising an issue and PR-ing that separately.

@sdmccabe

Copy link
Copy Markdown
Collaborator Author

This is a big PR, so we should definitely talk things through and carefully examine what we're doing before merging this.

This commit adds the ability to pass `binary=True` to `**kwargs` to have
thresholding functions return the binary matrix instead of the thresholded, but
still continuous, weight matrix.

@tlarock tlarock left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed in the meeting 2/13, @sdmccabe will later add warnings in some places, particularly when the thresholding is called with default parameters (since they should be set per-dataset).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Proposing a consistent interface for thresholding matrices

2 participants