yann_utils package

Submodules

yann_utils.chunking module

Utilities for chunking objects into bytes and reconstructing them.

yann_utils.chunking.chunk_obj(obj: Any, chunk_size_mb: int) → Iterable[bytes]

Chunk an object into bytes of a given size.

yann_utils.chunking.get_chunks(dir: str) → Iterable[str]

Get the chunks of an object from a directory.

yann_utils.chunking.get_numeric_suffix(file_name: str) → None

Get the numeric suffix of a file name.

yann_utils.chunking.persist_chunks(chunks: Iterable[bytes], dir: str) → None

Persist chunks to disk and return directory.

yann_utils.chunking.reconstruct_obj(chunks: Iterable[bytes]) → Any

Reconstruct an object from its chunks of bytes.

yann_utils.phd module

yann_utils.utils module

General utility functions.

class yann_utils.utils.Cache

Bases: object

Persistent and automated caching of objects.

load(name: str) → Any

locs: List[str] = []

os = <module 'os' (frozen)>

save(obj: Any, name: str) → None

yann_utils.utils.clusters_to_cmap(clusters: List[int]) → List[Any]

yann_utils.utils.compose(*functions) → Any

yann_utils.utils.enumerate_flipped(xs: List[E]) → Generator[Tuple[E, int], None, None]

>>> list(enumerate_flipped(["a", "b", "c"]))
[('a', 0), ('b', 1), ('c', 2)]

yann_utils.utils.flatten(list_o_list: List[List[T]]) → List[T]

>>> flatten([[1, 2], [3, 4]])
[1, 2, 3, 4]

yann_utils.utils.invert_dict(d: Dict[A, List[B]]) → Dict[B, A]

Convert dictionary of lists to flat dictionary with values as keys.

yann_utils.utils.list_accessor(objects: List[O], field: str) → List[A]

>>> list_accessor([{"a": 1}, {"a": 2}], "a")
[1, 2]

yann_utils.utils.list_set(xs: List[O]) → List[O]

>>> list_set([1, 2, 3, 1, 2, 3])
[1, 2, 3]

yann_utils.utils.multiprocess_apply(func, args, n_jobs: int = 8) → List[Any]

Macro for multicore iteration.

yann_utils.utils.reassign_dict(ds: Dict[Any, Any], values: List[Any]) → Dict[Any, Any]

>>> reassign_dict({"a": 1, "b": 2}, [2, 1])
{'a': 2, 'b': 1}

yann_utils.utils.set_seed(seed)

yann_utils.utils.time_elapsed(t0: float) → float

Return time elapsed since t0 in ms.

yann_utils.utils.timer(func)

Simple timer decorator which outputs execution time in ms.

Module contents

Top-level package for Yann’s Utilities.