train.ChunkedRandomSampler Class Reference

Inheritance diagram for train.ChunkedRandomSampler:

Collaboration diagram for train.ChunkedRandomSampler:

Public Member Functions
	__init__ (self, data_source, chunk_size=1_000_000)
	__iter__ (self)
	__len__ (self)

Public Attributes
	num_samples = len(data_source)
	chunk_size = chunk_size
	num_chunks = math.ceil(self.num_samples / self.chunk_size)

Detailed Description

Shuffle sample order in large chunks to favor sequential disk I/O.

Constructor & Destructor Documentation

__init__()

train.ChunkedRandomSampler.__init__	(		self,
			data_source,
			chunk_size = 1_000_000 )

Member Function Documentation

__iter__()

train.ChunkedRandomSampler.__iter__

(

self

)

__len__()

train.ChunkedRandomSampler.__len__

(

self

)

Member Data Documentation

chunk_size

train.ChunkedRandomSampler.chunk_size = chunk_size

num_chunks

train.ChunkedRandomSampler.num_chunks = math.ceil(self.num_samples / self.chunk_size)

num_samples

train.ChunkedRandomSampler.num_samples = len(data_source)

---

The documentation for this class was generated from the following file:

• training/train.py