data

Fill in a module description here

---

source

Sampler

 Sampler (ds:Iterable[int], shuffle:bool=False)

A custom sampler class.

Args: ds (Iterable[int]): Iterable of indices. shuffle (bool): Whether to shuffle the indices.

Example: >>> x = range(10) >>> sampler = Sampler(x, shuffle=True)

---

source

BatchSampler

 BatchSampler (sampler:__main__.Sampler, bs:int, drop_last:bool=False)

A custom batch sampler class.

Args: sampler (Sampler): The sampler to use. bs (int): Batch size. drop_last (bool): Whether to drop the last batch if it is smaller than the batch size.

Example: >>> x = range(10) >>> sampler = Sampler(x, shuffle=True) >>> batch_sampler = BatchSampler(sampler, bs=3)

---

source

Dataset

 Dataset (transforms:Optional[List]=None)

An abstract class representing a :class:Dataset.

All datasets that represent a map from keys to data samples should subclass it. All subclasses should overwrite: __getitem__, supporting fetching a data sample for a given key. __len__, which is expected to return the size of the dataset.

---

source

DataLoader

 DataLoader (dataset:__main__.Dataset, batch_size:int=1,
             shuffle:bool=True, sampler:__main__.Sampler=None,
             batch_sampler:__main__.BatchSampler=None, num_workers:int=0,
             collate_fn:<built-infunctioncallable>=None,
             drop_last:bool=False)

A custom data loader class.

Args: ds (Dataset): The dataset to load. bs (int): Batch size.

Example: >>> dataloader = DataLoader(dataset, batch_size)

---

source

collate

 collate (b)

X = init.rand(100, 10)
Y = init.randb(X.shape[0])
X.shape, Y.shape

((100, 10), (100,))

class MiDataset(Dataset):

    def __init__(self, x, y):
        self.x = x
        self.y = y

    def __len__(self) -> int:
        return self.x.shape[0]

    def __getitem__(self, i: int):
        return self.x[i], self.y[i]

ds = MiDataset(X,Y)

len(ds)

100

ds[:10]

(minima.Tensor(
 [[0.835655 0.355682 0.341312 0.877118 0.964982 0.161331 0.871119 0.653924 0.112947 0.995913]
  [0.5362   0.935408 0.452379 0.046165 0.030116 0.169965 0.570334 0.426115 0.03146  0.132633]
  [0.163104 0.91955  0.635856 0.995347 0.993711 0.460345 0.894598 0.273158 0.124523 0.119298]
  [0.638094 0.598965 0.745832 0.370086 0.670154 0.071052 0.32124  0.806154 0.743922 0.123329]
  [0.946646 0.726528 0.724754 0.568195 0.282958 0.240825 0.941848 0.742167 0.750217 0.275818]
  [0.513921 0.476381 0.050706 0.543432 0.484169 0.318996 0.580121 0.343467 0.814567 0.417969]
  [0.753226 0.521739 0.940589 0.336658 0.906802 0.777629 0.836282 0.378091 0.259586 0.798654]
  [0.341872 0.972311 0.902511 0.193453 0.089093 0.564276 0.213413 0.927187 0.968153 0.307819]
  [0.749471 0.773822 0.469319 0.862515 0.167192 0.390072 0.942944 0.92225  0.460263 0.55879 ]
  [0.925903 0.233595 0.177896 0.976798 0.043768 0.61924  0.939949 0.875439 0.068964 0.574139]]),
 minima.Tensor(
 [False  True  True  True False False False  True  True  True]))

Export

import nbdev; nbdev.nbdev_export()