Fairness aware counterfactuals for subgroups (FACTS) detector.
FACTS is an efficient, model-agnostic, highly parameterizable, and
explainable framework for evaluating subgroup fairness through
counterfactual explanations [#FACTS23]_.
This class is a wrapper for the various methods exposed by the
FACTS framework.
References
Parameters:
clf (sklearn.base.BaseEstimator) – A trained and ready to use
classifier, implementing method predict(X), where X is
the matrix of features; predictions returned by predict(X)
are either 0 or 1. In other words, fitted scikit-learn
classifiers.
prot_attr (str) – the name of the column that represents the
protected attribute.
categorical_features (list(str), optional) – the list of categorical
features. The default is to choose (dynamically, inside fit) the
columns of the dataset with types “object” or “category”.
freq_itemset_min_supp (float, optional) – minimum support for all the runs
of the frequent itemset mining algorithm (specifically, FP Growth).
We mine frequent itemsets to generate candidate subpopulation groups and candidate actions.
For more information, see paper [#FACTS23]_.
Defaults to 10%.
feature_weights (dict(str, float), optional) – the weights for each feature. Used in the calculation
of the cost of a suggested change. Specifically, the term corresponding to each feature is
multiplied by this weight.
Defaults to 1, for all features.
feats_allowed_to_change (list(str), optional) – if provided, only
allows these features to change value in the suggested recourses.
Default: no frozen features.
Note: providing both feats_allowed_to_change and
feats_not_allowed_to_change is currently treated as an error.
feats_not_allowed_to_change (list(str), optional) – if provided,
prevents these features from changing at all in any given
recourse.
Default: no frozen features.
Note: providing both feats_allowed_to_change and
feats_not_allowed_to_change is currently treated as an error.
Examines generated subgroups and calculates the top_count most
unfair ones, with respect to the chosen metric.
Stores the final groups in instance variable self.top_rules and the
respective subgroup costs in self.subgroup_costs (or self.unfairness
for the “fair-tradeoff” metric).
Parameters:
metric (str, optional) –
one of the following choices
”equal-effectiveness”
”equal-choice-for-recourse”
”equal-effectiveness-within-budget”
”equal-cost-of-effectiveness”
”equal-mean-recourse”
”fair-tradeoff”
Defaults to “equal-effectiveness”.
For explanation of each of those metrics, refer either to the
paper [#FACTS23]_ or the demo_FACTS notebook.
viewpoint (str, optional) –
“macro” or “micro”. Refers to the
notions of “macro viewpoint” and “micro viewpoint” defined
in section 2.2 of the paper [#FACTS23]_.
As a short explanation, consider a set of actions A and a
subgroup (cohort / set of individuals) G. Metrics with the
macro viewpoint interpretation are constrained to always apply
one action from A to the entire G, while metrics with the micro
interpretation are allowed to give each individual in G the
min-cost action from A which changes the individual’s class.
Note that not all combinations of metric and viewpoint are
valid, e.g. “Equal Choice for Recourse” only has a macro
interpretation.
Defaults to “macro”.
sort_strategy (str, optional) –
one of the following choices
”max-cost-diff-decr”: simply rank the groups in descending order according to the unfairness metric.
”max-cost-diff-decr-ignore-forall-subgroups-empty”: ignore groups for which we have no available actions whatsoever.
”max-cost-diff-decr-ignore-exists-subgroup-empty”: ignore groups for which at least one protected subgroup has no available actions.
Defaults to “max-cost-diff-decr”.
top_count (int, optional) – the number of subpopulation groups that
the algorithm will keep.
Defaults to 10.
filter_sequence (List[str], optional) –
List of various filters
applied on the groups and / or actions. Available filters are:
”remove-contained”: does not show groups which are subsumed by other shown groups. By “subsumed” we mean that the group is defined by extra feature values, but those values are not changed by any action.
”remove-below-thr-corr”: does not show actions which are below the given effectiveness threshold. Refer also to the documentation of parameter phi below.
”remove-above-thr-cost”: does not show action that cost more than the given cost budget. Refer also to the documentation of parameter c below.
”keep-rules-until-thr-corr-reached”:
”remove-fair-rules”: do not show groups which do not exhibit bias.
”keep-only-min-change”: for each group shown, show only the suggested actions that have minimum cost, ignore the others.
Defaults to [].
phi (float, optional) – effectiveness threshold. Real number in [0, 1].
Applicable for “equal-choice-for-recourse” and
“equal-cost-of-effectiveness” metrics. For these two metrics, an
action is considered to achieve recourse for a subpopulation group
if at least phi % of the group’s individuals achieve recourse.
Defaults to 0.5.
c (float, optional) – cost budget. Real number. Applicable for
“equal-effectiveness-within-budget” metric. Specifies the maximum
cost that can be payed for an action (by the individual, by a
central authority etc.)
Defaults to 0.5.
Calculates subpopulation groups, actions and respective effectiveness
Parameters:
X (DataFrame) – Dataset given as a pandas.DataFrame. As in
standard scikit-learn convention, it is expected to contain one
instance per row and one feature / explanatory variable per
column (labels not needed, we already have an ML model).
verbose (bool) – whether to print intermediate messages and progress bar. Defaults to True.
Raises:
ValueError – feats_allowed_to_change and feats_not_allowed_to_change
cannot be given simultaneously.
Prints a nicely formatted report of the results (subpopulation groups
and recourses) discovered by the bias_scan method.
Parameters:
population_sizes (dict(str, int), optional) – Number of individuals that
are given the negative prediction by the model, for each subgroup.
If given, it is included in the report together with some
coverage percentages.
missing_subgroup_val (str, optional) – Optionally specify a value of the
protected attribute which denotes that it is missing and should not be
included in the printed results.
Defaults to “N/A”.
show_subgroup_costs (bool, optional) – Whether to show the costs assigned
to each protected subgroup.
Defaults to False.
show_action_costs (bool, optional) – Whether to show the costs assigned
to each specific action.
Defaults to False.
show_cumulative_plots (bool, optional) – If true, shows, for each subgroup,
a graph of the effectiveness cumulative distribution, as it is
called in [#FACTS23]_.
Defaults to False.
show_bias (str, optional) – Specify which value of the protected
attribute corresponds to the subgroup against which we want to find
unfairness. Mainly useful for when the protected attribute is not
binary (e.g. race).
Defaults to None.
correctness_metric (bool, optional) – if True, the metric is considered
to quantify utility, i.e. the greater it is for a group, the
more beneficial it is for the individuals of the group.
Defaults to False.
metric_name (str, optional) – If given, it is added to the the printed
message for unfairness in a subpopulation group, i.e. the method
prints “Bias against females due to <metric_name>”.
Raises:
RuntimeError – if costs for groups and subgroups are empty. Most
likely the bias_scan method was not run.
Identify the subgroups with the most difficulty achieving recourse.
FACTS is an efficient, model-agnostic, highly parameterizable, and
explainable framework for evaluating subgroup fairness through
counterfactual explanations [#FACTS23]_.
Note
This function is a wrapper to run the FACTS framework from start to
finish. Its purpose is to provide an API which is both closer to the
detectors API and more succinct.
For more options and greater control (including the option to cache
some intermediate results and then apply more than one metric fast),
consider using the FACTS class.
References
Parameters:
X (DataFrame) – Dataset given as a pandas.DataFrame. As in
standard scikit-learn convention, it is expected to contain one
instance per row and one feature / explanatory variable per
column (labels not needed, we already have an ML model).
clf (sklearn.base.BaseEstimator) – A trained and ready to use
classifier, implementing method predict(X), where X is
the matrix of features; predictions returned by predict(X)
are either 0 or 1. In other words, fitted scikit-learn
classifiers.
prot_attr (str) – the name of the column that represents the
protected attribute.
metric (str, optional) –
one of the following choices
”equal-effectiveness”
”equal-choice-for-recourse”
”equal-effectiveness-within-budget”
”equal-cost-of-effectiveness”
”equal-mean-recourse”
”fair-tradeoff”
Defaults to “equal-effectiveness”.
For explanation of each of those metrics, refer either to the
paper [#FACTS23]_ or the demo_FACTS notebook.
categorical_features (list(str), optional) – the list of categorical
features. The default is to choose (dynamically, inside fit) the
columns of the dataset with types “object” or “category”.
freq_itemset_min_supp (float, optional) –
minimum support for all the runs
of the frequent itemset mining algorithm (specifically, FP Growth).
We mine frequent itemsets to generate candidate subpopulation groups and candidate actions.
For more information, see paper [#FACTS23]_.
Defaults to 10%.
feature_weights (dict(str, float), optional) – the weights for each feature. Used in the calculation
of the cost of a suggested change. Specifically, the term corresponding to each feature is
multiplied by this weight.
Defaults to 1, for all features.
feats_allowed_to_change (list(str), optional) – if provided, only
allows these features to change value in the suggested recourses.
Default: no frozen features.
Note: providing both feats_allowed_to_change and
feats_not_allowed_to_change is currently treated as an error.
feats_not_allowed_to_change (list(str), optional) – if provided,
prevents these features from changing at all in any given
recourse.
Default: no frozen features.
Note: providing both feats_allowed_to_change and
feats_not_allowed_to_change is currently treated as an error.
viewpoint (str, optional) –
“macro” or “micro”. Refers to the
notions of “macro viewpoint” and “micro viewpoint” defined
in section 2.2 of the paper [#FACTS23]_.
As a short explanation, consider a set of actions A and a
subgroup (cohort / set of individuals) G. Metrics with the
macro viewpoint interpretation are constrained to always apply
one action from A to the entire G, while metrics with the micro
interpretation are allowed to give each individual in G the
min-cost action from A which changes the individual’s class.
Note that not all combinations of metric and viewpoint are
valid, e.g. “Equal Choice for Recourse” only has a macro
interpretation.
Defaults to “macro”.
sort_strategy (str, optional) –
one of the following choices
”max-cost-diff-decr”: simply rank the groups in descending order according to the unfairness metric.
”max-cost-diff-decr-ignore-forall-subgroups-empty”: ignore groups for which we have no available actions whatsoever.
”max-cost-diff-decr-ignore-exists-subgroup-empty”: ignore groups for which at least one protected subgroup has no available actions.
Defaults to “max-cost-diff-decr”.
top_count (int, optional) – the number of subpopulation groups that
the algorithm will keep.
Defaults to 1, i.e. returns the most biased group.
phi (float, optional) – effectiveness threshold. Real number in [0, 1].
Applicable for “equal-choice-for-recourse” and
“equal-cost-of-effectiveness” metrics. For these two metrics, an
action is considered to achieve recourse for a subpopulation group
if at least phi % of the group’s individuals achieve recourse.
Defaults to 0.5.
c (float, optional) – cost budget. Real number. Applicable for
“equal-effectiveness-within-budget” metric. Specifies the maximum
cost that can be payed for an action (by the individual, by a
central authority etc.)
Defaults to 0.5.
verbose (bool, optional) – whether to print intermediate messages and
progress bar. Defaults to True.
print_recourse_report (bool, optional) – whether to print a detailed
and annotated report of the most biased groups to stdout. If False,
the most biased groups are only computed and returned.
Defaults to False.
show_subgroup_costs (bool, optional) – Whether to show the costs assigned
to each protected subgroup.
Defaults to False.
show_action_costs (bool, optional) – Whether to show the costs assigned
to each specific action.
Defaults to False.
is_correctness_metric (bool, optional) – if True, the metric is considered
to quantify utility, i.e. the greater it is for a group, the
more beneficial it is for the individuals of the group.
Defaults to False.
Returns:
the most biased groups as a list of pairs. In each pair, the first element is the group description as a dict. The second element is the value of the chosen unfairness metric for this group.
Prints a report detailing the recourses and fairness assessment for a given set of rules.
In our work, we generally refer to this representation as “Comparative Subgroup Counterfactuals”.
Parameters:
rules (Dict[Predicate, Dict[str, Tuple[float, List[Tuple[Predicate, float, float]]]]]) – The collection of rules with associated recourses, correctness, and costs.
population_sizes (Optional[Dict[str, int]], optional) – A dictionary specifying the population sizes for the protected and unprotected populations.
If provided, it is added to the coverage statistics included in the report.
Defaults to None.
missing_subgroup_val (str, optional) – In the case of records with missing values for the protected attribute (e.g. sex), this parameter
determines whether the “group” of individuals with unknown protected attribute will be shown
in the report or not. In other words, the group with this name is excluded altogether from the report.
Defaults to “N/A”.
subgroup_costs (Optional[Dict[Predicate, Dict[str, float]]], optional) – A dictionary specifying the aggregate costs for each subgroup in each rule.
If provided, the costs of recourses will be included in the report.
Defaults to None.
show_subgroup_costs (bool, optional) – Indicates whether to display the subgroup costs in the report.
Only applicable when subgroup_costs is provided.
Defaults to False.
show_then_costs (bool, optional) – Indicates whether to display the counterfactual costs of recourses in the report.
Defaults to False.
show_cumulative_plots (bool, optional) – Indicates whether to display the cumulative effectiveness plots in the report.
Defaults to False.
show_bias (Optional[str], optional) – Specifies the biased subgroup to highlight in the report.
Only applicable when subgroup_costs is provided.
Defaults to None.
correctness_metric (bool, optional) – Indicates whether the metric to use for assessing bias is correctness or cost metric.
If False, it is a cost metric and the group with maximum value is the biased one.
If True, it is a correctness metric and the group with minimum value is the biased one.
Only applicable when subgroup_costs is provided.
Defaults to False.
metric_name (str, optional) – The name of the fairness metric used to assess bias.
Only applicable when subgroup_costs is provided.
Defaults to “Equal Effectiveness”.
Deletes fair rules from the given set of rules based on subgroup costs.
Parameters:
rulesbyif (Dict[Predicate, Dict[str, Tuple[float, List[Tuple[Predicate, float]]]]]) – Dictionary mapping predicates to a dictionary of group IDs and associated cost and correctness tuples.
subgroup_costs (Dict[Predicate, Dict[str, float]]) – Dictionary mapping predicates to a dictionary of group IDs and their corresponding subgroup costs.
Returns:
Dictionary containing the remaining rules after deleting the fair rules.
Filters the rules to remove the contained rules based on simple containment criteria.
Parameters:
rulesbyif (Dict[Predicate, Dict[str, Tuple[float, List[Tuple[Predicate, float]]]]]) – Dictionary mapping predicates to a dictionary of group IDs and associated cost and correctness tuples.
Returns:
Filtered rules after removing the contained rules based on simple containment criteria.
rulesbyif (Dict[Predicate, Dict[str, Tuple[float, List[Tuple[Predicate, float]]]]]) – Dictionary mapping predicates to a dictionary of group IDs and associated cost and correctness tuples.
params (ParameterProxy, optional) – Parameter proxy object containing parameter values for calculating the feature change. Defaults to ParameterProxy().
Returns:
Dictionary containing the filtered rules based on the minimum change criterion.
Filters rules based on cumulative correctness threshold.
Parameters:
rulesbyif (Dict[ Predicate, Dict[str, Tuple[float, List[Tuple[Predicate, float, float]]]] ]) – Dictionary mapping predicates to a dictionary of group IDs and associated cost, correctness, and cumulative cost tuples.
threshold (float, optional) – Threshold value for the cumulative correctness. Rules with a cumulative correctness value greater than the threshold are kept. Defaults to 0.5.
Returns:
Dictionary containing the filtered rules based on the cumulative correctness threshold.
rulesbyif (Dict[ Predicate, Dict[str, Tuple[float, List[Tuple[Predicate, float, float]]]] ]) – Dictionary mapping predicates to a dictionary of group IDs and associated cost, correctness, and cumulative cost tuples.
threshold (float, optional) – Threshold value for the cumulative cost. Rules with a cumulative cost value less than or equal to the threshold are kept.
Defaults to 0.5.
Returns:
Dictionary containing the filtered rules based on the cumulative cost threshold.
rulesbyif (Dict[Predicate, Dict[str, Tuple[float, List[Tuple[Predicate, float]]]]]) – Dictionary mapping predicates to a dictionary of group IDs and associated cost and correctness tuples.
threshold (float, optional) – The threshold value for filtering the rules based on correctness. Defaults to 0.5.
Returns:
Filtered rules based on the correctness threshold.
Sorts the triples using the Kolmogorov-Smirnov test to measure unfairness.
Parameters:
rulesbyif (Dict[ Predicate, Dict[str, Tuple[float, List[Tuple[Predicate, float, float]]]] ]) – Dictionary mapping predicates to a dictionary of group IDs and associated cost, correctness, and predicate tuples.
affected_population_sizes (Dict[str, int]) – Dictionary mapping group IDs to their respective affected population sizes.
Returns:
A tuple containing a sorted list of triples and a dictionary mapping predicates to their unfairness scores.
Sorts the triples by maximum cost difference with generic options to handle NaN, infinity, and secondary objectives.
Parameters:
rulesbyif (Dict[Predicate, Dict[str, Tuple[float, List[Tuple[Predicate, float]]]]]) – Dictionary mapping predicates to a dictionary of group IDs and associated cost and predicate pairs.
ignore_nans (bool, optional) – Flag indicating whether to ignore NaN values in the cost difference. Defaults to False.
ignore_infs (bool, optional) – Flag indicating whether to ignore infinity values in the cost difference. Defaults to False.
secondary_objectives (List[str], optional) – List of secondary objectives to include in the sorting criteria. Defaults to an empty list.
Returns:
Sorted list of triples with the associated maximum cost difference.
Calculate the correctness of recourse actions for each subgroup in a list of if-then rules.
Parameters:
ifthens_withsupp (List[Tuple[Predicate, Predicate, Dict[str, float]]]) – List of if-then rules with their support values.
affected_by_subgroup (Dict[str, DataFrame]) – Dictionary where keys are subgroup names and values are DataFrames representing affected individuals in each subgroup.
sensitive_attribute (str) – Name of the sensitive attribute in the dataset.
model (ModelAPI) – The model used for making predictions.
verbose (bool) – whether to print progress bar. Defaults to True.
Returns:
List of tuples containing the if-then rule, its support values, and the correctness of recourse actions for each subgroup.
Calculate cumulative correctness and costs for all if-then rules.
Parameters:
rulesbyif – A dictionary containing if-clause predicates as keys and a nested
dictionary as values. The nested dictionary contains subgroup names as
keys, and tuples of coverage and a list of then-clause predicates with
their corresponding correctness values as values.
X – The DataFrame containing the data.
model – The model API used for prediction.
sensitive_attribute – The name of the sensitive attribute in the data.
verbose – whether to print intermediate messages and progress bar. Defaults to True.
Returns:
A dictionary with if-clause predicates as keys. Each if-clause predicate
maps to a nested dictionary where subgroup names are the keys, and tuples
of coverage and a list of updated then-clause predicates with their
cumulative correctness values and costs are the values.
Compute cumulative correctness and cost for all rules.
Parameters:
rulesbyif – A dictionary containing if-clause predicates as keys and a nested
dictionary as values. The nested dictionary contains subgroup names as
keys, and tuples of coverage and a list of then-clause predicates with
their corresponding correctness as values.
X – The input DataFrame.
model – The model API.
sensitive_attribute – The name of the sensitive attribute in the DataFrame.
A dictionary containing if-clause predicates as keys and a nested dictionary as
values. The nested dictionary contains subgroup names as keys, and a list of
tuples representing the cumulative correctness and cost of the then-clauses.
Group rules based on the If clauses instead of protected subgroups.
Parameters:
rules (List[Tuple[Predicate, Predicate, Dict[str, float], Dict[str, float]]]) – List of tuples containing the if-then rules, coverage metrics, and correctness metrics.
Returns:
Dictionary containing the rules grouped by the If clauses.
Constructs a dictionary of feature change functions based on the input parameters.
Parameters:
X (DataFrame) – The input DataFrame containing the data.
num_cols (List[str]) – A list of column names representing the numeric features.
cate_cols (List[str]) – A list of column names representing the categorical features.
ord_cols (List[str]) – _description_
feature_weights (Dict[str, int]) – A dictionary mapping feature names to their corresponding weights.
num_normalization (bool, optional) – A flag indicating whether to normalize numeric features. Default is False.
feats_to_normalize (Optional[List[str]], optional) – A list of column names specifying the numeric features to be normalized.
If None, all numeric features will be normalized. Default is None.
Returns:
A dictionary mapping feature names to the corresponding
feature change functions.
Builds a dictionary of feature change functions based on the provided lists of numerical
and categorical columns, along with the weights for each feature.
Parameters:
num_cols (List[str]) – List of numerical column names.
cate_cols (List[str]) – List of categorical column names.
feature_weights (Dict[str, int]) – Dictionary mapping feature names to their weights.