Skip to content
Snippets Groups Projects
Commit 71150227 authored by Arin Wongprommoon's avatar Arin Wongprommoon
Browse files

fix(postproc): detrend process only removes columns with all NaNs

WHY IS THIS CHANGE NEEDED?:
- detrend process overzealously removes columns if input dataframes
  have NaNs near the beginning or end.
- this is because by default, df.dropna() uses how="any", i.e. it drops
  columns with at least one NaN

HOW DOES THE CHANGE SOLVE THE PROBLEM?:
- drops columns with at least one NaN is not the intention -- the
  intention is to remove column filled with NaNs created by rolling
  window operations.
- so, specified the how argument

WHAT SIDE EFFECTS DOES THIS CHANGE HAVE?:
- i honestly don't know why i didn't catch it in the 1.5 years this
  process existed -- this seems so obvious.
- this is a deprecated signal processing post-process (i use the
  butterworth filter instead), so i don't expect it to break things much

EVIDENCE THAT COMMIT WORKS:
parent d638b430
No related branches found
No related tags found
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment