Skip to content
Snippets Groups Projects

[WIP] Overhaul statistics computation

Closed Vicentini Filippo requested to merge pv/stats into master

Created by: PhilipVinc

This PR rewrites the logic computing the statistics of our Stats class in order to use single-pass algorithms that need traverse the data only once.

With this PR, once a Stats object is created, it can be Stats.merge(other_stats) with another object in order to combine the indicators.

This is a requirement for properly computing such statistical diagnostics when working with minibatches while keeping memory requirements contained.

My idea is to compute the tau_corr and error of the mean by using the variance among the chains if there is a sufficient number of chains, otherwise switch to the variance among blocks.

Interestingly, this implementation requires 1 less MPI call then before, so it might be marginally faster in MPI contexts when computing many observables because it reduces the number of sync points.

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
Please register or sign in to reply
Loading