Causation Entropy

User Guide

  • Tutorials
    • Interactive Notebooks
    • Basic Usage
    • Simple Example

API Reference

  • Network Discovery
    • Main Discovery Function
      • discover_network()
    • Method Implementations
      • standard_optimal_causation_entropy()
      • alternative_optimal_causation_entropy()
      • information_lasso_optimal_causation_entropy()
      • lasso_optimal_causation_entropy()
    • Selection Algorithms
      • standard_forward()
      • alternative_forward()
      • backward()
    • Statistical Testing
      • shuffle_test()
    • Linear Algebra Utilities
      • correlation_log_determinant()
    • Statistical Utilities
      • auc()
      • Compute_TPR_FPR()
    • Visualization
      • roc_curve()
  • Information Theory
    • Main Functions
      • conditional_mutual_information()
      • gaussian_conditional_mutual_information()
    • Nonparametric Estimators
      • kde_conditional_mutual_information()
      • knn_conditional_mutual_information()
      • geometric_knn_conditional_mutual_information()
    • Distribution-Specific Estimators
      • poisson_conditional_mutual_information()
    • Entropy Functions
      • l2dist()
      • hyperellipsoid_check()
      • kde_entropy()
      • geometric_knn_entropy()
      • poisson_entropy()
      • poisson_joint_entropy()
    • Mutual Information Functions
      • gaussian_mutual_information()
      • kde_mutual_information()
      • knn_mutual_information()
      • geometric_knn_mutual_information()
  • Synthetic Datasets
    • Dynamical Systems
      • logistic_map()
      • logisic_dynamics()
      • linear_stochastic_gaussian_process()
    • Coupled Oscillators
      • poisson_coupled_oscillators()
    • Module Contents
      • logistic_map()
      • logisic_dynamics()
      • linear_stochastic_gaussian_process()
      • poisson_coupled_oscillators()
  • Linear Algebra Utilities
    • correlation_log_determinant()
    • subnetwork()
    • companion_matrix()
  • Plotting Utilities
    • roc_curve()
    • plot_causal_network()
  • Statistical Utilities
    • auc()
    • Compute_TPR_FPR()

Theory

  • Theoretical Guide
    • Glossary
      • Mathematical Notation
      • Abbreviations
    • Standard Causal Entropy
      • Mathematical Foundation
      • Algorithm Description
        • Phase 1: Forward Selection with Initial Conditioning
        • Phase 2: Backward Elimination
      • Key Properties
        • Initial Conditioning Benefits
        • Conditioning Set Evolution
      • Information-Theoretic Interpretation
      • Advantages and Limitations
        • Advantages
        • Limitations
      • Implementation Considerations
        • Hyperparameter Selection
        • Information Estimator Choice
      • Example Implementation
      • Comparison with Alternative Methods
      • Theoretical Connections
      • Conclusion
    • Alternative Causal Entropy
      • Mathematical Foundation
      • Algorithm Description
        • Phase 1: Forward Selection without Initial Conditioning
        • Phase 2: Backward Elimination
      • Key Algorithmic Differences
        • Conditioning Set Evolution
      • Information-Theoretic Interpretation
        • First Selection: Marginal Mutual Information
        • Subsequent Selections: Conditional Uniqueness
      • Advantages and Limitations
        • Advantages
        • Limitations
      • When to Use Alternative oCSE
        • Recommended Scenarios
      • Comparison with Standard oCSE
      • Implementation Considerations
        • Hyperparameter Differences
      • Example Implementation
      • Diagnostic Analysis
        • Selection Order Analysis
        • Conditional MI Comparison
      • Theoretical Implications
        • Model Selection Perspective
        • Connection to Feature Selection
      • Conclusion
    • Information-Theoretic LASSO
      • Mathematical Foundation
        • Traditional LASSO Objective
        • Information-Theoretic Extension
      • Information-Theoretic Weights
        • Base Weights
        • Adaptive Weighting
        • Significance-Based Weighting
      • Algorithmic Approaches
        • Approach 1: Weighted LASSO with Information Weights
        • Approach 2: Information-Guided Regularization Path
        • Approach 3: Iterative Information-LASSO
      • Implementation Framework
        • Two-Stage Implementation
        • Adaptive Implementation
      • Theoretical Properties
        • Sparsity-Information Tradeoff
        • Consistency Properties
        • Oracle Properties
      • Advantages and Limitations
        • Advantages
        • Limitations
      • Hyperparameter Selection
        • Cross-Validation for \(\lambda\)
        • Information Criteria
      • Comparison with Standard oCSE
      • Use Case Guidelines
        • When to Use Info-LASSO
        • When to Avoid Info-LASSO
      • Example Application
      • Future Directions
        • Research Opportunities
        • Implementation Improvements
      • Conclusion
    • Pure LASSO Methods
      • Mathematical Foundation
        • Standard LASSO Formulation
        • Causal Discovery Context
      • Causal Interpretation
        • Edge Detection
        • Network Construction
      • Regularization Parameter Selection
        • Cross-Validation Approach
        • Information Criteria
        • Stability Selection
      • Implementation Approaches
        • Standard LASSO Implementation
      • Advanced LASSO Variants
        • Adaptive LASSO
        • Group LASSO for Temporal Structure
        • Elastic Net
      • Theoretical Properties
        • Consistency and Oracle Properties
        • Conditions for Consistency
      • Advantages and Limitations
        • Advantages
        • Limitations
      • Comparison with Information-Theoretic Methods
      • When to Use LASSO Methods
        • Recommended Scenarios
        • Avoid When
      • Best Practices
        • Preprocessing
        • Model Selection
        • Post-Processing
      • Example Analysis
      • Integration with oCSE Framework
      • Future Directions
        • Research Areas
        • Methodological Improvements
      • Conclusion
    • Information Theory
      • Fundamental Concepts
        • Entropy
        • Joint and Conditional Entropy
      • Mutual Information
        • Definition and Properties
        • Conditional Mutual Information
      • Information-Theoretic Measures for Causality
        • Transfer Entropy
        • Causation Entropy
      • Estimation Methods
        • Parametric Estimators
        • Non-Parametric Estimators
      • Estimator Comparison
      • Advanced Information Measures
        • Partial Information Decomposition
        • Information Geometry
      • Multivariate Extensions
        • Multivariate Mutual Information
        • Information Networks
      • Practical Considerations
        • Sample Size Requirements
        • Bias and Variance Tradeoffs
      • Statistical Testing
        • Permutation Tests
        • Bootstrap Confidence Intervals
        • Multiple Testing Correction
      • Applications in Causal Discovery
        • Variable Selection
        • Conditional Independence Testing
        • Network Structure Learning
      • Future Directions
        • Emerging Methods
      • Conclusion
    • Statistical Foundations
      • Hypothesis Testing Framework
        • Null and Alternative Hypotheses
        • Test Statistics and Distributions
      • Permutation Testing
        • Theoretical Foundation
        • Permutation Strategies
        • Statistical Properties
      • Sequential Testing in oCSE
        • Forward Selection Testing
        • Backward Elimination Testing
      • Bootstrap Methods
        • Bootstrap Confidence Intervals
        • Bootstrap-based Variable Selection
      • Theoretical Guarantees
        • Consistency Properties
        • Estimation Error Bounds
        • High-Dimensional Theory
      • Power Analysis
        • Theoretical Power
        • Sample Size Calculations
      • Robustness and Sensitivity
        • Robustness to Outliers
        • Model Misspecification
        • Sensitivity Analysis
      • Practical Guidelines
        • Sample Size Requirements
        • Significance Level Selection
      • Diagnostic Procedures
        • Model Checking
        • Stability Analysis
      • Future Directions
        • Methodological Advances
        • Computational Improvements
      • Conclusion
    • Overview
    • Mathematical Foundation
    • Key Principles
    • Next Steps

Links

  • GitHub Repository
  • PyPI Package
Causation Entropy
  • Overview: module code

All modules for which code is available

  • causationentropy.core.discovery
  • causationentropy.core.information.conditional_mutual_information
  • causationentropy.core.information.entropy
  • causationentropy.core.information.mutual_information
  • causationentropy.core.linalg
  • causationentropy.core.plotting
  • causationentropy.core.stats
  • causationentropy.datasets.synthetic

© Copyright 2025, Kevin Slote.

Built with Sphinx using a theme provided by Read the Docs.