Ensemble Methods
The ensemble module provides ensemble learning algorithms.
Base Ensemble
Base classes for ensemble methods.
This module provides the foundation for building ensemble models that combine multiple estimators to improve predictive performance.
- class fit.ensemble.base.BaseEnsemble(n_estimators: int = 10, random_state: int | None = None)[source]
-
Base class for all ensemble methods.
Warning: This class should not be used directly. Use derived classes instead.
- __init__(n_estimators: int = 10, random_state: int | None = None)[source]
Initialize base ensemble.
- Parameters:
n_estimators – Number of estimators in the ensemble
random_state – Random state for reproducibility
- fit(X: ndarray | Tensor, y: ndarray | Tensor) BaseEnsemble[source]
Fit the ensemble.
- Parameters:
X – Training data
y – Target values
- Returns:
Self for method chaining
- class fit.ensemble.base.VotingClassifier(estimators: List[tuple], voting: str = 'hard', weights: List[float] | None = None)[source]
Bases:
BaseEnsembleVoting classifier for combining multiple classification models.
Examples
>>> from fit.ensemble import VotingClassifier >>> from fit.simple.models import MLP >>> >>> # Create base estimators >>> estimators = [ ... ('mlp1', MLP([4, 8, 3])), ... ('mlp2', MLP([4, 16, 3])), ... ('mlp3', MLP([4, 12, 3])) ... ] >>> >>> # Create voting classifier >>> ensemble = VotingClassifier(estimators, voting='soft') >>> ensemble.fit(X_train, y_train) >>> predictions = ensemble.predict(X_test)
- __init__(estimators: List[tuple], voting: str = 'hard', weights: List[float] | None = None)[source]
Initialize voting classifier.
- Parameters:
estimators – List of (name, estimator) tuples
voting – Voting strategy (‘hard’ or ‘soft’)
weights – Sequence of weights for the estimators
- class fit.ensemble.base.BaggingClassifier(base_estimator=None, n_estimators: int = 10, max_samples: int | float = 1.0, random_state: int | None = None)[source]
Bases:
BaseEnsembleBagging classifier that fits multiple models on bootstrap samples.
Examples
>>> from fit.ensemble import BaggingClassifier >>> from fit.simple.models import MLP >>> >>> # Create bagging classifier >>> ensemble = BaggingClassifier( ... base_estimator=MLP([4, 8, 3]), ... n_estimators=10, ... random_state=42 ... ) >>> ensemble.fit(X_train, y_train) >>> predictions = ensemble.predict(X_test)
- __init__(base_estimator=None, n_estimators: int = 10, max_samples: int | float = 1.0, random_state: int | None = None)[source]
Initialize bagging classifier.
- Parameters:
base_estimator – Base estimator to fit on bootstrap samples
n_estimators – Number of estimators in the ensemble
max_samples – Number/fraction of samples to draw for each estimator
random_state – Random state for reproducibility
Bagging
Boosting
Boosting ensemble methods.
This module implements boosting algorithms like AdaBoost that sequentially fit weak learners and combine them into a strong learner.
- class fit.ensemble.boosting.AdaBoostClassifier(base_estimator=None, n_estimators: int = 50, learning_rate: float = 1.0, random_state: int | None = None)[source]
Bases:
BaseEnsembleAdaBoost classifier implementation.
AdaBoost fits a sequence of weak learners on repeatedly modified versions of the data. The predictions from all of them are then combined through a weighted majority vote.
Examples
>>> from fit.ensemble import AdaBoostClassifier >>> from fit.simple.models import MLP >>> >>> # Create AdaBoost classifier >>> ada = AdaBoostClassifier( ... base_estimator=MLP([4, 2]), ... n_estimators=50, ... learning_rate=1.0 ... ) >>> ada.fit(X_train, y_train) >>> predictions = ada.predict(X_test)
- __init__(base_estimator=None, n_estimators: int = 50, learning_rate: float = 1.0, random_state: int | None = None)[source]
Initialize AdaBoost classifier.
- Parameters:
base_estimator – Base estimator to boost
n_estimators – Maximum number of estimators
learning_rate – Learning rate shrinks the contribution of each classifier
random_state – Random state for reproducibility
- fit(X: ndarray | Tensor, y: ndarray | Tensor) AdaBoostClassifier[source]
Build a boosted classifier from the training set.
- Parameters:
X – Training data
y – Target values
- Returns:
Self for method chaining
- class fit.ensemble.boosting.GradientBoostingClassifier(n_estimators: int = 100, learning_rate: float = 0.1, max_depth: int = 3, random_state: int | None = None)[source]
Bases:
BaseEnsembleGradient Boosting classifier.
This implementation is simplified and focuses on the core concept of gradient boosting for educational purposes.
Examples
>>> from fit.ensemble import GradientBoostingClassifier >>> >>> # Create gradient boosting classifier >>> gb = GradientBoostingClassifier( ... n_estimators=100, ... learning_rate=0.1, ... max_depth=3 ... ) >>> gb.fit(X_train, y_train) >>> predictions = gb.predict(X_test)
- __init__(n_estimators: int = 100, learning_rate: float = 0.1, max_depth: int = 3, random_state: int | None = None)[source]
Initialize Gradient Boosting classifier.
- Parameters:
n_estimators – Number of boosting stages
learning_rate – Learning rate shrinks contribution of each tree
max_depth – Maximum depth of individual regression estimators
random_state – Random state for reproducibility
- fit(X: ndarray | Tensor, y: ndarray | Tensor) GradientBoostingClassifier[source]
Fit the gradient boosting model.
- Parameters:
X – Training data
y – Target values
- Returns:
Self for method chaining
- predict(X: ndarray | Tensor) ndarray[source]
Predict class labels for samples in X.
- Parameters:
X – Input data
- Returns:
Predicted class labels
- class fit.ensemble.boosting.SimpleBoostingClassifier(base_estimator=None, n_estimators: int = 10, random_state: int | None = None)[source]
Bases:
BaseEnsembleSimplified boosting classifier for educational purposes.
This is a basic implementation that demonstrates the core concepts of boosting without the complexity of AdaBoost or Gradient Boosting.
Examples
>>> from fit.ensemble import SimpleBoostingClassifier >>> from fit.simple.models import MLP >>> >>> # Create simple boosting classifier >>> boost = SimpleBoostingClassifier( ... base_estimator=MLP([4, 2]), ... n_estimators=10 ... ) >>> boost.fit(X_train, y_train) >>> predictions = boost.predict(X_test)