divide_dataset

divide_dataset(dataset: Dataset, division: List[float] | Tuple[float, ...] | Dict[str, float]) → List[Dataset] | DatasetDict

Divide the dataset according to the division.

The division support varying number of splits, which you can name. The splits are created from the beginning of the dataset.

Parameters:

• dataset (Dataset) – Dataset to be divided.

• division (Union[List[float], Tuple[float, ...], Dict[str, float]]) – Configuration specifying how the dataset is divided. Each fraction has to be >0 and <=1. They have to sum up to at most 1 (smaller sum is possible).

Returns:

divided_dataset – If division is List or Tuple then List[Dataset] is returned else if division is Dict then DatasetDict is returned.

Return type:

Union[List[Dataset], DatasetDict]

Examples

Use divide_dataset with division specified as a list.

>>> from flwr_datasets import FederatedDataset
>>> from flwr_datasets.utils import divide_dataset
>>>
>>> fds = FederatedDataset(dataset="mnist", partitioners={"train": 100})
>>> partition = fds.load_partition(0)
>>> division = [0.8, 0.2]
>>> train, test = divide_dataset(dataset=partition, division=division)

Use divide_dataset with division specified as a dict.

>>> from flwr_datasets import FederatedDataset
>>> from flwr_datasets.utils import divide_dataset
>>>
>>> fds = FederatedDataset(dataset="mnist", partitioners={"train": 100})
>>> partition = fds.load_partition(0)
>>> division = {"train": 0.8, "test": 0.2}
>>> train_test = divide_dataset(dataset=partition, division=division)
>>> train, test = train_test["train"], train_test["test"]